Real-time normal calculation method for multi-line laser point cloud

Abstract

The invention relates to a real-time normal calculation method for a multi-line laser point cloud. The method comprises the following steps: step 1, calibrating a cross laser surface equation; step 2, cross laser line image processing; 3, calculating a direction vector; step 4, calculating a direction vector; and step 5, calculating the normal vector of the laser line. The method has the beneficial effects that the normal vector calculation of the laser center point can be completed only according to the calibrated laser surface equation, the internal reference matrix of the camera and the distortion parameter matrix, and the neighborhood point of the laser center does not need to be additionally calculated; the three-dimensional coordinate of the laser center neighborhood point can be directly obtained from the processed cross laser line image, the calculation speed of the normal vector is greatly increased, the normal real-time calculation of the cross laser line can be realized, and a guarantee is provided for the subsequent display of a three-dimensional point cloud model.

Description

Technical Field

[0001] This invention relates to the field of multi-line laser 3D scanning, and specifically to a method for real-time normal calculation of multi-line laser point clouds. Background Technology

[0002] Handheld 3D scanners acquire point cloud data by projecting multiple intersecting laser lines and using a camera to capture the intersections of these lasers with the object's surface. During a 3D scan, the user can only obtain a local set of intersecting laser line point cloud data at a time. Therefore, it is necessary to continuously adjust the scanning position to gradually cover the entire object being measured in order to complete the scan. To help the user judge the scanning progress and identify missing areas, the point cloud model of the current object needs to be displayed in real time on the scanning interface. By observing the point cloud model on the scanning interface, the user can adjust the scanning position and perform targeted scanning of missing parts, ultimately obtaining a complete 3D point cloud model of the object.

[0003] However, point cloud coordinates alone can only reflect spatial distribution and cannot intuitively represent the shape and structure of an object. Without normal vectors, the point cloud will appear as a messy pile of dots on the interface, lacking a sense of three-dimensionality and lighting effects, making it difficult for users to identify the object's true surface and geometric features. Therefore, normal vectors must be calculated in real time during point cloud generation to provide surface orientation information, enabling rendering to represent bumps, layers, and realistic lighting and shadows. This also provides intuitive feedback to the operator, allowing for timely adjustments to the scanning position during the scanning process, ensuring a complete and high-quality 3D model is obtained.

[0004] During handheld scanning, a high frame rate is required to improve scanning accuracy and speed. To observe the 3D point cloud model of an object in real time, it's necessary to acquire the 3D coordinates and normal vectors of the intersecting laser line point cloud in each frame. Most existing point cloud normal vector calculation methods are based on neighborhood relationships. For example, in the paper "A Real-Time Calculation Method for Normal Vectors in 3D Scanned Point Clouds," a KD-tree is used to search for neighborhood points in the point cloud, and then principal component analysis is used to fit a plane to these neighborhood points, using the plane's normal vector as the normal vector for each point in the point cloud. While neighborhood-based methods can calculate the point cloud normal vectors relatively accurately, the time spent searching for neighborhood points is significant, typically making it impossible to acquire the normal vectors of every frame in real time. Furthermore, handheld 3D scanners can only acquire one set of intersecting laser line point clouds at a single scanning position, and the neighborhood points of the point cloud are generally on a single line, making it impossible to accurately obtain the point cloud's normal vectors. Summary of the Invention

[0005] In summary, to overcome the shortcomings of the prior art, the technical problem to be solved by this invention is to provide a real-time normal vector calculation method for multi-line laser point clouds. This method can quickly find neighboring points without building a KD tree or octree, and it does not require plane fitting. It only needs to calculate the normal vector of the point cloud based on the pre-calibrated laser surface equation and the three-dimensional coordinates of the point cloud. The calculation speed is fast, and the normal vector of the point cloud of the intersecting laser lines in the current frame can be obtained in real time, ensuring the accuracy and continuity of the subsequent display of the three-dimensional point cloud model of the object.

[0006] The technical solution of this invention to solve the above-mentioned technical problems is as follows: A real-time normal calculation method for multi-line laser point clouds, comprising the following steps:

[0007] Step 1: By taking multiple sets of calibration board images with and without intersecting laser lines, the camera parameters are calibrated and the three-dimensional coordinates of the laser center are calculated. Finally, the equations of 13 intersecting laser surfaces are obtained by fitting a quadratic surface model.

[0008] Step 2: Based on the laser parameters corresponding to the laser surface equation obtained in Step 1, control the device to acquire the scanned image, and after distortion correction, filtering, binarization and contour segmentation processing, obtain the contour image of a single laser line;

[0009] Step 3: Based on the camera parameter calibration in Step 1, the laser surface equation obtained in Step 1, and the single laser line contour image obtained in Step 2, first extract the sub-pixel coordinates of the laser center and solve for the three-dimensional coordinates of the laser center. Then, calculate the direction vector using the partial derivatives of the laser surface equation. ;

[0010] Step 4: Based on the single laser line profile output in Step 2 and the three-dimensional coordinates of the laser center obtained in Step 3, store the coordinates row by row, determine the neighborhood points, fit a straight line, and then normalize the direction vector to obtain the direction vector. ;

[0011] Step 5, using the direction vector obtained in Step 3 The direction vector obtained in step 4 Perform a cross product operation, and combine it with the three-dimensional coordinates of the laser center obtained in step 3 to complete the normal vector orientation and point cloud model display, thereby realizing real-time normal vector calculation of multi-line laser point clouds.

[0012] Based on the above technical solution, the present invention can be further improved as follows:

[0013] Furthermore, step 1 specifically includes:

[0014] Step 1.1: Place the calibration board in the field of view of the left and right dual cameras, control the laser to project the cross-left and cross-right laser lines, and simultaneously trigger the camera to capture the calibration board image containing the laser lines; turn on the fill light and capture the calibration board image without the laser lines at the same position; move the calibration board to different positions and repeat the above operation to obtain multiple sets of calibration board images with and without laser lines.

[0015] Step 1.2: The calibration board image without laser lines obtained in Step 1.1 is processed by filtering, binarization, contour extraction, contour filtering, sub-pixel point calculation, and sub-pixel contour fitting to obtain and sort the sub-pixel center coordinates of each marker circle on the calibration board. The sub-pixel center coordinates of each marker circle are then associated with its corresponding three-dimensional coordinates. The sub-pixel center coordinates and corresponding three-dimensional coordinates of the left and right cameras are input into the camera calibration function to obtain the camera intrinsic parameter matrix, distortion parameter matrix, and extrinsic parameter matrix corresponding to each calibration position.

[0016] Step 1.3: Calculate the three-dimensional coordinates of the laser center;

[0017] Step 1.4: The three-dimensional coordinates of the laser center obtained in Step 1.3 are... The lasers are categorized, and then the coordinates of the three-dimensional laser center points belonging to the same laser surface are substituted into the quadratic surface model. By performing quadratic surface fitting, the following quadratic surface equation for the laser surface is obtained:

[0018] ;

[0019] in, Denotes the coefficients of the equation for a quadratic surface. Represents the three-dimensional coordinates of all laser centers on the same laser plane, where , This represents the number of points used to fit the quadratic surface; by solving the above system of equations using the least squares method, the equation coefficients of the 13 intersecting left and right laser surfaces, i.e., the laser surface equations, are obtained.

[0020] Furthermore, step 1.3 specifically includes:

[0021] Step 1.3.1: Based on the calibration board image containing laser lines obtained in Step 1.1, the center pixel coordinates of the intersecting laser lines are calculated using the gray-scale centroid method. ;

[0022] Step 1.3.2: Based on the camera extrinsic parameter matrix obtained in Step 1.2, the three-dimensional coordinates of the center of the marker circle on the calibration plate are transformed to the camera coordinate system using the following formula:

[0023] ;

[0024] in, This represents the three-dimensional coordinates of the center of the circle in the camera coordinate system. This represents the three-dimensional coordinates of the center of the circle in the world coordinate system. This represents the rotation matrix between the camera coordinate system and the world coordinate system corresponding to the current pose of the calibration board. This represents the translation matrix between the camera coordinate system and the world coordinate system corresponding to the current pose of the calibration board;

[0025] Step 1.3.3: Convert the three-dimensional coordinates of the circle center. Substitute into the plane equation Let C = 1. We obtain the following equation:

[0026] ;

[0027] in , The number of centers of the fitted plane is represented by the value 'x'. The equations are then solved using the least squares method to obtain the equations of the calibration plate plane in the camera coordinate system.

[0028] Step 1.3.4, based on the camera intrinsic parameter matrix obtained in Step 1.2 The center pixel coordinates of the laser line obtained in step 1.3.1 The normalized coordinates of the laser center are calculated using the following formula. :

[0029] ;

[0030] Step 1.3.5: Based on the 3D coordinates of the camera center (0,0,0) and the normalized coordinates of the laser center obtained in Step 1.3.4... Establish the parametric equation of the line passing through the two points as follows:

[0031] ;

[0032] Solving the parametric equations of the above lines simultaneously with the plane equations of the calibration plate, we obtain:

[0033] ;

[0034] Solving the above equation yields the following results: Then Substituting the above linear parametric equations, we obtain the three-dimensional coordinates of the laser center pixel in the camera coordinate system. .

[0035] Furthermore, step 2 specifically includes:

[0036] Step 2.1, acquire scanned images: Based on the laser parameters corresponding to the laser surface equation obtained in Step 1, control the laser to alternately project the laser beams of the left and right intersecting lines according to the frequency, and simultaneously trigger the camera to capture deformed images containing laser lines; turn on the supplementary light intermittently, and simultaneously trigger the camera to capture images without laser lines, repeat the operation, and output multiple sets of scanned images containing laser lines and those without laser lines;

[0037] Step 2.2, Image preprocessing: First, perform distortion correction on the laser line image obtained in Step 2.1 and the camera distortion parameter matrix obtained in Step 1.2, then perform Gaussian filtering for noise reduction, and finally binarize to filter background noise, to obtain a preprocessed image that retains only the laser line data.

[0038] Step 2.3, Contour Extraction and Segmentation: Extract the laser line contours from the preprocessed image obtained in Step 2.2, segment each contour separately, and obtain a contour image containing only a single laser line.

[0039] Furthermore, step 3 specifically includes:

[0040] Step 3.1: Based on the single laser line contour image obtained in Step 2, extract the sub-pixel coordinates (u,v) of the laser center using the gray-scale centroid method or the Steger method, and output the sub-pixel coordinates of the laser center.

[0041] Step 3.2: Using the laser center sub-pixel coordinates (u,v) obtained in Step 3.1, the camera intrinsic parameter matrix obtained in Step 1.2, and the laser surface quadratic equation obtained in Step 1.4, the following system of equations is established:

[0042] ;

[0043] in, Indicates the first The quadratic surface coefficients of each laser surface, , and They represent and Focal length of direction, and They represent and The coordinates of the principal point in the direction are used to solve the above system of equations to obtain the three-dimensional coordinates of the laser center. ;

[0044] Step 3.3: Using the equation of the laser surface quadratic curve obtained in Step 1.4 and the coordinates of the three-dimensional laser center point obtained in Step 3.2... Calculate the direction vector .

[0045] Furthermore, step 3.3 specifically includes:

[0046] Step 3.3.1, take the first result obtained in step 1.4... The equation of the quadratic surface of the laser surface is transformed into an implicit function form: ;

[0047] Step 3.3.2, calculate the implicit function from step 3.3.1 respectively. , and Taking the partial derivatives in the direction yields the following partial derivative equation:

[0048] ;

[0049] Step 3.3.3: Obtain the three-dimensional coordinates of the laser center point obtained in Step 3.2. Substituting these equations into the partial derivative equations obtained in step 3.3.2, the direction vector is calculated. .

[0050] Furthermore, step 4 specifically includes:

[0051] Step 4.1: Based on the single laser line contour image obtained in Step 2 and the three-dimensional coordinates of the laser center obtained in Step 3.2, store the three-dimensional coordinates of all laser centers on the laser line row by row, and output an ordered set of three-dimensional coordinates of the laser centers.

[0052] Step 4.2: Using the set of three-dimensional coordinates of the laser center obtained in Step 4.1, determine the current position of the laser center point: if it is located at the beginning of the laser line, continue traversing... 1 neighboring point; if it is in the middle, traverse to both ends respectively. 1. Neighboring points; if it is at the end, traverse backwards. The output includes the current laser center point and its neighboring points. The three-dimensional coordinates of the neighboring points;

[0053] Step 4.3: Fit a straight line and normalize it using the 3D coordinates of the neighborhood points obtained in Step 4.2 to obtain the direction vector. .

[0054] Furthermore, step 4.3 specifically includes:

[0055] Step 4.3.1: Substitute the three-dimensional coordinates of the neighborhood points obtained in Step 4.2 into the following parametric equation of the line:

[0056] ;

[0057] in, Represent the three-dimensional coordinates of a point on a straight line; obtain the direction vector of the line through fitting. ;

[0058] Step 4.3.2, regarding the direction vector of the line Normalize to obtain the direction vector. .

[0059] Furthermore, step 5 specifically includes:

[0060] Step 5.1: Input the direction vector obtained in Step 3. The direction vector obtained in step 4 Through cross product operation Solving for the normal vector of the current laser center point yields the solution. ;

[0061] Step 5.2, based on the normal vector obtained in Step 5.1 Using the three-dimensional coordinates of the laser center obtained in step 3, the normal vector is determined based on the viewpoint of the scanning interface. The direction is redirected; the 3D coordinates of the intersecting laser point cloud are combined with the redirected normal vector. This yields a real-time display of the 3D point cloud model of the scanned object.

[0062] The beneficial effects of this invention are: the normal vector of the laser center point can be calculated simply by using the calibrated laser surface equation, the camera's intrinsic parameter matrix, and the distortion parameter matrix. This method eliminates the need to calculate the neighboring points of the laser center separately, and can directly obtain the three-dimensional coordinates of the laser center's neighboring points from the processed image of the intersecting laser lines, greatly accelerating the calculation speed of the normal vector. It can realize the real-time calculation of the normal vector of the intersecting laser lines, providing a guarantee for the subsequent display of the three-dimensional point cloud model. Attached Figure Description

[0063] Figure 1 This is a flowchart of the present invention;

[0064] Figure 2 This is a schematic diagram of the system structure of the present invention;

[0065] Figure 3 This is a diagram showing the result of image binarization.

[0066] Figure 4 Image showing the contour extraction results;

[0067] Figure 5 This is the binarized result of the cross-left line;

[0068] Figure 6 The result of extracting a single contour of the intersecting left line;

[0069] Figure 7 This is the binarized result of the right-crossing line;

[0070] Figure 8This is the result of extracting a single contour from the intersecting right lines.

[0071] The attached diagram lists the components represented by each number as follows:

[0072] 1. Left camera, 2. Right camera, 3. Cross 13-line laser. Detailed Implementation

[0073] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0074] like Figure 1 and 2 As shown, this invention proposes a real-time normal calculation method for multi-line laser point clouds in a handheld 3D scanner consisting of two cameras (left camera 1 and right camera 2) and a cross 13-line laser 3, which includes the following steps:

[0075] Step 1, Calibrate the equations of the cross laser surfaces: By taking multiple sets of images of the calibration board with and without cross laser lines, the camera parameters are calibrated and the three-dimensional coordinates of the laser center are calculated. Finally, the equations of 13 cross laser surfaces are obtained by fitting a quadratic surface model, which provides core parameters and equation basis for subsequent image processing and vector calculation.

[0076] Step 1.1, Acquiring Calibration Images: Place the calibration board within the field of view of the left and right cameras. Control the laser to project intersecting left and right laser lines onto the calibration board, simultaneously triggering the cameras to capture images of the calibration board containing the laser lines. Turn on the fill light and capture images of the calibration board without laser lines at the same location. Move the calibration board to different positions and repeat the above operation to obtain multiple sets of calibration board images with and without laser lines. Specifically: First, place the calibration board within the field of view of the left and right cameras. Control the laser to project intersecting left and right laser lines onto the calibration board, and simultaneously trigger the cameras to capture images of the calibration board containing the intersecting left and right lines. Then, turn on the fill light and capture images of the calibration board without laser lines. Continuously change the orientation of the calibration board and repeat the above operation at each calibration position, sequentially capturing multiple sets of calibration board images with and without laser lines until the camera and the cross-laser surface equation calibration images are acquired.

[0077] Step 1.2, Camera Parameter Calibration: The calibration board image without laser lines obtained in Step 1.1 is processed through filtering, binarization, contour extraction, contour filtering, sub-pixel point calculation, and sub-pixel contour fitting to obtain and sort the sub-pixel center coordinates of each marker circle on the calibration board. The sub-pixel center coordinates of each marker circle are then associated with its corresponding 3D coordinates. The sub-pixel center coordinates and corresponding 3D coordinates of the left and right cameras are input into the camera calibration function to obtain the camera intrinsic parameter matrix, distortion parameter matrix, and extrinsic parameter matrix corresponding to each calibration position. Specifically, the calibration board image without laser lines needs to be processed first, including image filtering, image binarization, and so on. Figure 3 As shown), contour extraction ( Figure 4 (As shown), contour filtering, contour sub-pixel point calculation, sub-pixel contour fitting, and other operations are performed to obtain the sub-pixel center coordinates of each marker circle on the calibration board. Then, the marker circles are sorted to link the sub-pixel center coordinates of each marker circle with its corresponding three-dimensional coordinates. By inputting all the sub-pixel center coordinates of the left and right cameras and their corresponding three-dimensional coordinates into the camera calibration function, the intrinsic parameter matrix, distortion parameter matrix, and extrinsic parameter matrix corresponding to each calibration position of the left and right cameras can be obtained respectively.

[0078] Step 1.3, calculate the three-dimensional coordinates of the laser center:

[0079] Step 1.3.1: Based on the calibration board image containing laser lines obtained in Step 1.1, the center pixel coordinates of the intersecting laser lines are calculated using the gray-scale centroid method. .

[0080] Step 1.3.2: Based on the camera extrinsic parameter matrix obtained in Step 1.2, the three-dimensional coordinates of the center of the marker circle on the calibration plate are transformed to the camera coordinate system using the following formula:

[0081] .

[0082] in, This represents the three-dimensional coordinates of the center of the circle in the camera coordinate system. This represents the three-dimensional coordinates of the center of the circle in the world coordinate system. This represents the rotation matrix between the camera coordinate system and the world coordinate system corresponding to the current pose of the calibration board. This represents the translation matrix between the camera coordinate system and the world coordinate system corresponding to the current pose of the calibration board.

[0083] Step 1.3.3: Convert the three-dimensional coordinates of the circle center. Substitute into the plane equation Let C = −1, we get the following equation:

[0084] .

[0085] in , This represents the number of centers of the fitted plane. The above system of equations is then solved using the least squares method to obtain the equation of the calibration plate plane in the camera coordinate system.

[0086] Step 1.3.4, based on the camera intrinsic parameter matrix obtained in Step 1.2 The center pixel coordinates of the laser line obtained in step 1.3.1 The normalized coordinates of the laser center are calculated using the following formula. :

[0087] .

[0088] Step 1.3.5: Based on the 3D coordinates of the camera center (0,0,0) and the normalized coordinates of the laser center obtained in Step 1.3.4... Establish the parametric equation of the line passing through the two points as follows:

[0089] .

[0090] Solving the parametric equations of the above lines simultaneously with the plane equations of the calibration plate, we obtain:

[0091] .

[0092] Solving the above equation yields the following results: Then Substituting the above linear parametric equations, we obtain the three-dimensional coordinates of the laser center pixel in the camera coordinate system. .

[0093] Step 1.4, Fitting the Laser Surface Equation: In practical applications, due to limitations in laser manufacturing processes, the projected intersecting multi-line lasers are not straight lines but curves. Therefore, a planar model cannot be used to calibrate the intersecting laser surface; instead, a quadratic surface is used. The three-dimensional coordinates of the laser center obtained in Step 1.3.5 are then used... The lasers are categorized, and then the coordinates of the three-dimensional laser center points belonging to the same laser surface are substituted into the quadratic surface model. By performing quadratic surface fitting, the following quadratic surface equation for the laser surface is obtained:

[0094] .

[0095] in, Denotes the coefficients of the equation for a quadratic surface. Represents the three-dimensional coordinates of all laser centers on the same laser plane, where , This represents the number of points used to fit the quadratic surface. Solving the above system of equations using the least squares method yields the equation coefficients for the 13 intersecting left and right laser surfaces, i.e., the laser surface equations.

[0096] Step 2, Cross-laser line image processing: Based on the laser parameters corresponding to the laser surface equation obtained in Step 1, the control device acquires the scanned image, and after distortion correction, filtering, binarization and contour segmentation processing, the contour image of a single laser line is obtained.

[0097] Step 2.1, Acquiring Scanned Images: Based on the laser parameters corresponding to the laser surface equation obtained in Step 1, the laser is controlled to alternately project intersecting left / right laser lines at a specific frequency, simultaneously triggering the camera to capture deformed images containing the laser lines. The supplementary lighting is activated intermittently, simultaneously triggering the camera to capture images without laser lines. This process is repeated to output multiple sets of scanned images containing and without laser lines. In actual scanning, the laser, supplementary lighting, and camera need to be controlled to operate at a certain frequency. For example, the laser is first controlled to project a set of intersecting left-side laser lines, simultaneously triggering the left and right cameras to capture images of the intersecting left lines. Then, the supplementary lighting is activated, simultaneously triggering the left and right cameras to capture images without laser lines. Next, the laser is controlled to project intersecting right-side laser lines, simultaneously triggering the left and right cameras to capture images of the intersecting right lines. Then, the supplementary lighting is activated again, simultaneously triggering the left and right cameras to capture images without laser lines. The images acquired in the above process are considered as a set, and this process is repeated sequentially.

[0098] Step 2.2, Image Preprocessing: The laser line image obtained in Step 2.1 and the camera distortion parameter matrix obtained in Step 1.2 are first subjected to distortion correction, then Gaussian filtering for noise reduction, and finally binarization to filter background noise, resulting in a preprocessed image that retains only the laser line data. After obtaining the cross-laser line image, distortion correction, Gaussian filtering, and binarization are first performed on the image. Then, contour extraction is performed on the binarized image to obtain multiple laser line contours.

[0099] Step 2.3, Contour Extraction and Segmentation: Extract the laser line contours from the preprocessed image obtained in Step 2.2, and segment each contour individually to obtain contour images containing only a single laser line. Each contour is segmented separately from the image, and each contour image contains only one laser line. Subsequently, only the laser center point needs to be extracted from each contour image individually to ensure the continuity of the laser line and accelerate the acquisition of the laser center neighborhood points. The binarization results of the intersecting left and right lines and the contour extraction results of a single laser line are shown below. Figure 5-8 As shown.

[0100] Step 3, calculate the direction vector Based on the camera parameter calibration in step 1, the laser surface equation obtained in step 1, and the single laser line contour image obtained in step 2, the sub-pixel coordinates of the laser center are first extracted and the three-dimensional coordinates of the laser center are solved. Then, the direction vector is calculated using the partial derivatives of the laser surface equation. .

[0101] Step 3.1, Extract sub-pixel coordinates of the laser center: Based on the single laser line contour image obtained in Step 2, use the gray-scale centroid method or Steger method to extract the sub-pixel coordinates (u,v) of the laser center and output the sub-pixel coordinates of the laser center.

[0102] Step 3.2, Solve for the 3D coordinates of the laser center: Using the sub-pixel coordinates (u,v) of the laser center obtained in Step 3.1, the camera intrinsic parameter matrix obtained in Step 1.2, and the quadratic surface equation of the laser surface obtained in Step 1.4, solve for the following system of equations:

[0103] .

[0104] in, Indicates the first The quadratic surface coefficients of each laser surface, , and They represent and Focal length of direction, and They represent and The coordinates of the principal point in the direction are used to solve the above system of equations to obtain the three-dimensional coordinates of the laser center. .

[0105] Step 3.3: Using the equation of the laser surface quadratic curve obtained in Step 1.4 and the coordinates of the three-dimensional laser center point obtained in Step 3.2... Calculate the direction vector :

[0106] Step 3.3.1, take the first result obtained in step 1.4... The equation of the quadratic surface of the laser surface is transformed into an implicit function form: .

[0107] Step 3.3.2, calculate the implicit function from step 3.3.1 respectively. , and Taking the partial derivatives in the direction yields the following partial derivative equation:

[0108] .

[0109] Step 3.3.3: Obtain the three-dimensional coordinates of the laser center point obtained in Step 3.2. Substituting these equations into the partial derivative equations obtained in step 3.3.2, the direction vector is calculated. .

[0110] Step 4, calculate the direction vector Based on the single laser line profile output in step 2 and the three-dimensional coordinates of the laser center obtained in step 3, the coordinates are stored row by row and neighborhood points are determined. After fitting a straight line, the direction vector is normalized to obtain the direction vector. .

[0111] Step 4.1, Store laser center coordinates: Based on the single laser line contour image obtained in Step 2 and the laser center 3D coordinates obtained in Step 3.2, store the 3D coordinates of all laser centers on the laser line row by row, and output an ordered set of laser center 3D coordinates.

[0112] Step 4.2, Determine Neighboring Points: Using the set of 3D coordinates of the laser center obtained in Step 4.1, determine the position of the current laser center point: if it is located at the beginning of the laser line, traverse forward... 1. If a point is in the middle, traverse towards both ends. 1. If it's at the end, traverse backwards. The output includes the current laser center point and its neighboring points. The three-dimensional coordinates of the neighboring points.

[0113] Step 4.3: Fit a straight line and normalize it using the 3D coordinates of the neighborhood points obtained in Step 4.2 to obtain the direction vector. :

[0114] Step 4.3.1: Substitute the three-dimensional coordinates of the neighborhood points obtained in Step 4.2 into the following parametric equation of the line:

[0115] .

[0116] in, This represents the three-dimensional coordinates of a point on a straight line. The direction vector of the line is obtained through fitting. .

[0117] Step 4.3.2, regarding the direction vector of the line Normalize to obtain the direction vector. .

[0118] Step 5: Calculate the normal vector of the laser line. The direction vector obtained through step 3 The direction vector obtained in step 4 Perform a cross product operation, and combine it with the three-dimensional coordinates of the laser center obtained in step 3 to complete the normal vector orientation and point cloud model display, thereby realizing real-time normal vector calculation of multi-line laser point clouds.

[0119] Step 5.1, Calculate the normal vector Based on the direction vector obtained in step 3 The direction vector obtained in step 4 Through cross product operation Solving for the normal vector of the current laser center point yields the solution. Based on the equation of the quadratic surface of the laser obtained in step 1.4 and the three-dimensional coordinates of the neighborhood points of the laser center, the two direction vectors corresponding to each laser center point can be obtained. and Due to the direction vector This represents the plane normal direction of the neighborhood where the current laser point is located, while the direction vector... This represents the tangent direction of the current laser point. It can be seen that neither direction vector can represent the change in the laser line's height direction, while the height direction of the laser line is perpendicular to the direction vector. and Therefore, the direction vector and The cross product yields the normal vector of the current laser center point. .

[0120] Step 5.2, based on the normal vector obtained in Step 5.1 Using the three-dimensional coordinates of the laser center obtained in step 3, the normal vector is determined based on the viewpoint of the scanning interface. The direction is redirected. The 3D coordinates of the intersecting laser point cloud are combined with the redirected normal vector. This yields a real-time display of the 3D point cloud model of the scanned object.

[0121] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for real-time normal calculation of multi-line laser point clouds, characterized by comprising the following steps: Step 1: By taking multiple sets of calibration board images with and without intersecting laser lines, the camera parameters are calibrated and the three-dimensional coordinates of the laser center are calculated. Finally, the equations of 13 intersecting laser surfaces are obtained by fitting a quadratic surface model. Step 2: Based on the laser parameters corresponding to the laser surface equation obtained in Step 1, control the device to acquire the scanned image, and after distortion correction, filtering, binarization and contour segmentation processing, obtain the contour image of a single laser line; Step 3: Based on the camera parameter calibration in Step 1, the laser surface equation obtained in Step 1, and the single laser line contour image obtained in Step 2, first extract the sub-pixel coordinates of the laser center and solve for the three-dimensional coordinates of the laser center. Then, calculate the direction vector using the partial derivatives of the laser surface equation. ; Step 4: Based on the single laser line profile output in Step 2 and the three-dimensional coordinates of the laser center obtained in Step 3, store the coordinates row by row, determine the neighborhood points, fit a straight line, and then normalize the direction vector to obtain the direction vector. ; Step 5, using the direction vector obtained in Step 3 The direction vector obtained in step 4 Perform a cross product operation, and combine it with the three-dimensional coordinates of the laser center obtained in step 3 to complete the normal vector orientation and point cloud model display, thereby realizing real-time normal vector calculation of multi-line laser point clouds.

2. The real-time normal calculation method for multi-line laser point clouds according to claim 1, characterized in that, Step 1 specifically includes: Step 1.1: Place the calibration board in the field of view of the left and right dual cameras, control the laser to project the cross-left and cross-right laser lines, and simultaneously trigger the camera to capture the calibration board image containing the laser lines; turn on the fill light and capture the calibration board image without the laser lines at the same position; move the calibration board to different positions and repeat the above operation to obtain multiple sets of calibration board images with and without laser lines. Step 1.2: The calibration board image without laser lines obtained in Step 1.1 is processed by filtering, binarization, contour extraction, contour filtering, sub-pixel point calculation, and sub-pixel contour fitting to obtain and sort the sub-pixel center coordinates of each marker circle on the calibration board. The sub-pixel center coordinates of each marker circle are then associated with its corresponding three-dimensional coordinates. The sub-pixel center coordinates and corresponding three-dimensional coordinates of the left and right cameras are input into the camera calibration function to obtain the camera intrinsic parameter matrix, distortion parameter matrix, and extrinsic parameter matrix corresponding to each calibration position. Step 1.3: Calculate the three-dimensional coordinates of the laser center; Step 1.4: The three-dimensional coordinates of the laser center obtained in Step 1.3 are... The lasers are categorized, and then the coordinates of the three-dimensional laser center points belonging to the same laser surface are substituted into the quadratic surface model. By performing quadratic surface fitting, the following quadratic surface equation for the laser surface is obtained: ； in, Denotes the coefficients of the equation for a quadratic surface. Represents the three-dimensional coordinates of all laser centers on the same laser plane, where , This represents the number of points used to fit the quadratic surface; by solving the above system of equations using the least squares method, the equation coefficients of the 13 intersecting left and right laser surfaces, i.e., the laser surface equations, are obtained.

3. The real-time normal calculation method for multi-line laser point clouds according to claim 2, characterized in that, Step 1.3 specifically includes: Step 1.3.1: Based on the calibration board image containing laser lines obtained in Step 1.1, the center pixel coordinates of the intersecting laser lines are calculated using the gray-scale centroid method. ; Step 1.3.2: Based on the camera extrinsic parameter matrix obtained in Step 1.2, the three-dimensional coordinates of the center of the marker circle on the calibration plate are transformed to the camera coordinate system using the following formula: ； in, This represents the three-dimensional coordinates of the center of the circle in the camera coordinate system. This represents the three-dimensional coordinates of the center of the circle in the world coordinate system. This represents the rotation matrix between the camera coordinate system and the world coordinate system corresponding to the current pose of the calibration board. This represents the translation matrix between the camera coordinate system and the world coordinate system corresponding to the current pose of the calibration board; Step 1.3.3: Convert the three-dimensional coordinates of the circle center. Substitute into the plane equation Let C = 1. We obtain the following equation: ； in , The number of centers of the fitted plane is represented by the value 'x'. The equations are then solved using the least squares method to obtain the equations of the calibration plate plane in the camera coordinate system. Step 1.3.4, based on the camera intrinsic parameter matrix obtained in Step 1.2 The center pixel coordinates of the laser line obtained in step 1.3.1 The normalized coordinates of the laser center are calculated using the following formula. : ； Step 1.3.5: Based on the 3D coordinates of the camera center (0,0,0) and the normalized coordinates of the laser center obtained in Step 1.3.4... Establish the parametric equation of the line passing through the two points as follows: ； Solving the parametric equations of the above lines simultaneously with the plane equations of the calibration plate, we obtain: ； Solving the above equation yields the following results: Then Substituting the above linear parametric equations, we obtain the three-dimensional coordinates of the laser center pixel in the camera coordinate system. .

4. The real-time normal calculation method for multi-line laser point clouds according to claim 2, characterized in that, Step 2 specifically includes: Step 2.1, acquire scanned images: Based on the laser parameters corresponding to the laser surface equation obtained in Step 1, control the laser to alternately project the laser beams of the left and right intersecting lines according to the frequency, and simultaneously trigger the camera to capture deformed images containing laser lines; turn on the supplementary light intermittently, and simultaneously trigger the camera to capture images without laser lines, repeat the operation, and output multiple sets of scanned images containing laser lines and those without laser lines; Step 2.2, Image preprocessing: First, perform distortion correction on the laser line image obtained in Step 2.1 and the camera distortion parameter matrix obtained in Step 1.2, then perform Gaussian filtering for noise reduction, and finally binarize to filter background noise, to obtain a preprocessed image that retains only the laser line data. Step 2.3, Contour Extraction and Segmentation: Extract the laser line contours from the preprocessed image obtained in Step 2.2, segment each contour separately, and obtain a contour image containing only a single laser line.

5. The real-time normal calculation method for multi-line laser point clouds according to claim 2, characterized in that, Step 3 specifically includes: Step 3.1: Based on the single laser line contour image obtained in Step 2, extract the sub-pixel coordinates (u,v) of the laser center using the gray-scale centroid method or the Steger method, and output the sub-pixel coordinates of the laser center. Step 3.2: Using the laser center sub-pixel coordinates (u,v) obtained in Step 3.1, the camera intrinsic parameter matrix obtained in Step 1.2, and the laser surface quadratic equation obtained in Step 1.4, the following system of equations is established: ； in, Indicates the first The quadratic surface coefficients of each laser surface, , and They represent and Focal length of direction, and They represent and The coordinates of the principal point in the direction are used to solve the above system of equations to obtain the three-dimensional coordinates of the laser center. ; Step 3.3: Using the equation of the laser surface quadratic curve obtained in Step 1.4 and the coordinates of the three-dimensional laser center point obtained in Step 3.2... Calculate the direction vector .

6. The real-time normal calculation method for multi-line laser point clouds according to claim 5, characterized in that, Step 3.3 specifically includes: Step 3.3.1, take the first result obtained in step 1.4... The equation of the quadratic surface of the laser surface is transformed into an implicit function form: ; Step 3.3.2, calculate the implicit function from step 3.3.1 respectively. , and Taking the partial derivatives in the direction yields the following partial derivative equation: ； Step 3.3.3: Obtain the three-dimensional coordinates of the laser center point obtained in Step 3.

2. Substituting these equations into the partial derivative equations obtained in step 3.3.2, the direction vector is calculated. .

7. The real-time normal calculation method for multi-line laser point clouds according to claim 5, characterized in that, Step 4 specifically includes: Step 4.1: Based on the single laser line contour image obtained in Step 2 and the three-dimensional coordinates of the laser center obtained in Step 3.2, store the three-dimensional coordinates of all laser centers on the laser line row by row, and output an ordered set of three-dimensional coordinates of the laser centers. Step 4.2: Using the set of three-dimensional coordinates of the laser center obtained in Step 4.1, determine the current position of the laser center point: if it is located at the beginning of the laser line, continue traversing... 1 neighboring point; if it is in the middle, traverse to both ends respectively.

1. Neighboring points; if it is at the end, traverse backwards. The output includes the current laser center point and its neighboring points. The three-dimensional coordinates of the neighboring points; Step 4.3: Fit a straight line and normalize it using the 3D coordinates of the neighborhood points obtained in Step 4.2 to obtain the direction vector. .

8. The real-time normal calculation method for multi-line laser point clouds according to claim 7, characterized in that, Step 4.3 specifically includes: Step 4.3.1: Substitute the three-dimensional coordinates of the neighborhood points obtained in Step 4.2 into the following parametric equation of the line: ； in, Represent the three-dimensional coordinates of a point on a straight line; obtain the direction vector of the line through fitting. ; Step 4.3.2, regarding the direction vector of the line Normalize to obtain the direction vector. .

9. The real-time normal calculation method for multi-line laser point clouds according to any one of claims 1 to 8, characterized in that, Step 5 specifically includes: Step 5.1, based on the direction vector obtained in step 3 The direction vector obtained in step 4 Through cross product operation Solving for the normal vector of the current laser center point yields the solution. ; Step 5.2, based on the normal vector obtained in Step 5.1 Using the three-dimensional coordinates of the laser center obtained in step 3, and the normal vector based on the viewpoint of the scanning interface... The direction is redirected; the 3D coordinates of the intersecting laser point cloud are combined with the redirected normal vector. This yields a real-time display of the 3D point cloud model of the scanned object.

Images