Online three-dimensional measurement method, system and storage medium based on laser triangulation machine vision

Abstract

The application discloses a laser triangulation-based machine vision online three-dimensional measurement method and system and a storage medium, and particularly relates to the field of machine vision three-dimensional measurement, and comprises the following steps: controlling a laser projection intelligent sensing element to make the same line laser irradiate a measured workpiece, a first fixed reference surface and a second fixed reference surface at the same time, and collecting the same frame laser image; based on computer vision, center points of the workpiece laser stripe, the first reference laser stripe and the second reference laser stripe are extracted, the two reference laser stripe center points are converted into imaging rays, and the imaging rays are respectively intersected with the first fixed reference surface and the second fixed reference surface to obtain a workpiece stripe point set, a first reference point set and a second reference point set; the application collects the workpiece and the two-side fixed reference stripes in the same frame, iteratively calculates the current laser plane based on double residual error weighting and coplanar constraint, and completes the workpiece stripe triangular inversion by using the current laser plane, so that the problem that the laser plane drift is calculated as the workpiece three-dimensional size change is solved.

Description

Technical Field

[0001] This invention relates to the field of machine vision 3D measurement technology, and more specifically, to a machine vision online 3D measurement method, system, and storage medium based on laser triangulation. Background Technology

[0002] Online 3D measurement using laser triangulation typically begins with the geometric calibration of the camera and laser. The imaging position of the laser line in the camera image is then converted into the spatial coordinates of the measured surface, thereby continuously obtaining the 3D contour of the workpiece. Existing equipment generally determines the laser plane position during the installation and commissioning phase, and continuously uses the same calibration result for triangulation during online measurement. In continuous conveying and inspection scenarios of metal sheets, profiles, or machined parts, lasers and cameras need to operate for extended periods, and the measurement process cannot be frequently interrupted for recalibration. Simultaneously, the equipment is affected by laser temperature rise, mounting bracket vibration, and changes in mechanical stress. Under these conditions, even with slight changes in the laser projection direction, the camera can still obtain continuous and clear laser stripes, and the stripe center extraction results will not show obvious anomalies. However, the actual laser plane has deviated from its calibrated position. If the system continues to intersect the original laser plane with the camera's imaging ray, it will directly calculate the change in the laser plane as a change in the workpiece surface height. This manifests as a slow shift in height value during continuous measurement of the same standard part, or a positional deviation in the overall three-dimensional contour even without workpiece deformation. Furthermore, subsequent image filtering and point cloud processing cannot distinguish this deviation from the actual dimensional changes. Therefore, the technical problem to be solved by this application is: how to determine the actual position of the laser plane at each acquisition without interrupting online measurement, and avoid the laser plane change being incorrectly converted into the three-dimensional size change of the workpiece being measured. Summary of the Invention

[0003] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide an online three-dimensional measurement method, system, and storage medium based on laser triangulation machine vision. The method involves acquiring the workpiece and fixed reference stripes on both sides in the same frame, iteratively obtaining the current laser plane based on double residual weighting and coplanar constraints, and using the current laser plane to complete the triangulation inversion of the workpiece stripes, thereby solving the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: an online 3D measurement method based on laser triangulation machine vision, comprising: S1. Control the laser projection intelligent sensing element to simultaneously irradiate the workpiece under test, the first fixed reference surface and the second fixed reference surface with the same line of laser, and acquire the same frame of laser image. S2. Based on computer vision, extract the center points of the workpiece laser stripe, the first reference laser stripe, and the second reference laser stripe. Convert the center points of the two reference laser stripes into imaging rays and intersect them with the first fixed reference surface and the second fixed reference surface respectively to obtain the workpiece stripe point set, the first reference point set, and the second reference point set. S3. Fit the first reference line and the second reference line to the first reference point set and the second reference point set respectively. Use the calibrated laser plane as the candidate laser plane. Form a double residual based on the distance of each reference point to its reference line and the candidate laser plane. Update the weight of each reference point using Student-t variational Bayes. S4. With the first reference line and the second reference line being coplanar as a constraint, the updated laser plane is obtained by using the Lagrange multiplier according to the reference point weights. Based on the updated laser plane, the double residual, the reference point weights, the first reference line and the second reference line are updated until the intersection line displacement of the laser plane in two adjacent rounds with the two fixed reference planes does not exceed the sub-pixel resolution of the stripe center point, thus obtaining the current laser plane. S5. Convert the workpiece stripe point set into camera imaging rays, intersect with the current laser plane to form the current three-dimensional contour, arrange the current three-dimensional contours according to the laser image acquisition order, and obtain the three-dimensional measurement results.

[0005] In a preferred embodiment, S1 includes: S11. The surfaces of the two planar target plates fixed on both sides of the workpiece being measured and facing the camera are respectively used as the first fixed reference surface and the second fixed reference surface. The planar position of the corresponding fixed reference surface in the camera coordinate system is determined by three non-collinear calibration points on each fixed reference surface. S12. Convert the outer edge points of the two fixed reference surfaces and the two endpoints of the workpiece to the laser projection coordinate system, take the two points at the two ends of the angle between the laser emission points to determine the line laser unfolding angle, and control the laser projection intelligent sensing element to project the same line laser according to the unfolding angle. S13. The line laser intersects with the first fixed reference surface, the surface of the workpiece being measured, and the second fixed reference surface to form a first reference laser stripe, a workpiece laser stripe, and a second reference laser stripe, respectively. A single exposure by a camera captures a laser image containing the three laser stripes in the same frame.

[0006] In a preferred embodiment, S2 includes: S21. Read the pixel grayscale values ​​of the first reference laser stripe, the workpiece laser stripe, and the second reference laser stripe column by column. Sort the pixels in each column in descending order of grayscale value. When the grayscale values ​​are the same, sort them in ascending order of pixel ordinate. Take the first pixel in the sorted list as the peak point. Take the first pixel on the top and bottom of the peak point whose grayscale value is not greater than the adjacent pixel as the upper boundary point and the lower boundary point. Linearly interpolate the background grayscale between the two boundary points. Calculate the weighted average of the pixel ordinate with the positive difference between the pixel grayscale value and the background grayscale value as the weight. This will give the first reference center point set, the workpiece stripe point set, and the second reference center point set, respectively. S22. Convert the center points of the first reference center point set and the second reference center point set into imaging rays passing through the camera optical center according to the camera intrinsic parameters, and obtain the first reference ray set and the second reference ray set respectively. S23. Determine the equations of the first and second reference planes using three non-collinear calibration points on the first and second fixed reference planes respectively. Obtain the first reference point set by intersecting the first reference ray set with the first reference plane equation. Obtain the second reference point set by intersecting the second reference ray set with the second reference plane equation.

[0007] In a preferred embodiment, S3 includes: S31. Calculate the mean coordinates and the coordinate covariance matrix of the first reference point set and the second reference point set respectively. Using the mean coordinates as the passing point and the eigenvector of the first eigenvalue in the covariance matrix as the direction, obtain the first reference line and the second reference line, and use the calibration laser plane as the candidate laser plane. S32. Calculate the perpendicular distance from each reference point to its reference line and the normal distance to the candidate laser plane to form a double residual. Calculate the covariance matrix of all double residuals. Multiply the transpose of the double residuals, the inverse of the covariance matrix, and the double residuals in sequence to obtain the Mahalanobis distance of each reference point.

[0008] In a preferred embodiment, S3 further includes: S33. Using Student-t variational Bayes, the Mahalanobis distance of each reference point is substituted into the Student-t Gaussian-Gamma mixture representation. The weight of each reference point is obtained by dividing the shape parameter of the gamma posterior by the rate parameter. Then, the weighted mean of the first reference point set and the weighted covariance matrix are calculated according to the reference point weights. The first reference line and the second reference line are updated with the eigenvector of the weighted mean of the point and the eigenvalue of the first eigenvalue in the weighted covariance matrix as the direction. S34. Calculate the vertical distance of each reference point using the updated first and second reference lines, and form a new double residual with the normal distance of each reference point to the candidate laser plane. Update the covariance matrix, Mahalanobis distance, reference point weights, first and second reference lines in sequence until the difference in the ordinates of the first and second reference lines projected onto the same frame of laser image in two adjacent rounds does not exceed the sub-pixel resolution of the stripe center point. Output the first reference line, second reference line, and reference point weights.

[0009] In a preferred embodiment, S4 includes: S41. Take the unit direction vectors of the first reference line and the second reference line, and mark the spatial coordinates of the first reference point set and the second reference point set as the reference point coordinates. Multiply the reference point weight by the reference point and substitute it into the equation of the laser plane to be solved, and sum the squares of the absolute values ​​as the plane cost. The unit normal vector of the laser plane to be solved and the inner product of the two unit direction vectors are zero and the magnitude of the unit normal vector is 1 to form a coplanar constraint. S42. Multiply the plane cost and the three coplanar constraints by Lagrange multipliers respectively and sum them. Take the partial derivatives of the unit normal vector, plane intercept, and the three Lagrange multipliers of the laser plane to be determined and set the partial derivatives to zero. Solve the equations to obtain the unit normal vector and plane intercept, and form the updated laser plane.

[0010] In a preferred embodiment, S4 further includes: S43. Multiply the square of the normal distance from the reference point to the updated laser plane by the reference point weight, sum the results, divide by the sum of the reference point weights, and take the square root to obtain the first plane residual and the second plane residual. Calculate the first line residual and the second line residual in the same way using the square of the perpendicular distance from the reference point to the first reference line and the second reference line, respectively. If the first plane residual is not greater than the first line residual and the second plane residual is not greater than the second line residual, retain the updated laser plane. If the first plane residual is greater than the first line residual and the second plane residual is not greater than the second line residual, update the first reference point weight and the first reference line using the perpendicular distance and normal distance from the first reference point. If the second plane residual is greater than the second line residual and the first plane residual is not greater than the first line residual, update the second reference point weight and the second reference line using the perpendicular distance and normal distance from the second reference point. Otherwise, update the two sets of reference point weights, the first reference line, and the second reference line respectively, and calculate the updated laser plane again. S44. The retained updated laser plane is intersected with the first fixed reference plane and the second fixed reference plane to form two spatial intersection lines. The two spatial intersection lines are projected onto the same frame of laser image, and the absolute value of the difference between the current projection vertical coordinate and the previous projection vertical coordinate under the same pixel horizontal coordinate is used as the intersection line displacement. When the intersection line displacement of the two spatial intersection lines does not exceed the resolution of the vertical coordinate of the fringe center point, the updated laser plane is used as the current laser plane. Otherwise, the updated laser plane is obtained again with the updated reference point weight, the first reference line and the second reference line.

[0011] In a preferred embodiment, S5 includes: S51. Convert the coordinates of each pixel in the workpiece stripe point set into the ray direction vector of the camera coordinate system through the inverse matrix of the camera intrinsic parameters. Take the camera optical center as the ray starting point. Use the inner product of the current laser plane unit normal vector and the ray direction vector as the intersection denominator. Use the negative value of the sum of the inner product of the current laser plane intercept, the unit normal vector and the camera optical center coordinates as the intersection numerator. Divide the intersection numerator by the intersection denominator to obtain the ray parameter. Then, add the ray starting point coordinates to the product of the ray parameter and the ray direction vector to obtain the spatial point. S52. When the intersection denominator is non-zero and the ray parameter is positive, retain the space points; when the intersection denominator is zero, remove the corresponding workpiece stripe points; when the intersection denominator is non-zero and the ray parameter is not positive, remove the corresponding space points; arrange the retained space points in ascending order of the pixel x-coordinate of the workpiece stripe points to form the current 3D contour. S53. Using the sequence of acquisition of laser images in the same frame as the contour number, arrange each current three-dimensional contour in ascending order of the contour number, and maintain the spatial point arrangement order within each current three-dimensional contour to obtain the three-dimensional measurement result.

[0012] In a preferred embodiment, the laser triangulation-based machine vision online 3D measurement system includes: The same-frame acquisition module is used to control the laser projection intelligent sensing element so that the same line of laser simultaneously illuminates the workpiece under test, the first fixed reference surface and the second fixed reference surface, and acquires the same-frame laser image. The ray intersection module is used to extract the center points of the workpiece laser stripes, the first reference laser stripe, and the second reference laser stripe based on computer vision. It converts the center points of the two reference laser stripes into imaging rays and intersects them with the first fixed reference surface and the second fixed reference surface, respectively, to obtain the workpiece stripe point set, the first reference point set, and the second reference point set. The residual weighting module is used to fit the first reference line and the second reference line to the first reference point set and the second reference point set respectively, with the calibrated laser plane as the candidate laser plane, and to form double residuals based on the distance of each reference point to its reference line and the candidate laser plane, and to update the weights of each reference point using Student-t variational Bayes. The plane update module is used to obtain the updated laser plane by using the coplanarity of the first reference line and the second reference line as constraints and the Lagrange multiplier according to the reference point weights. Based on the updated laser plane, the double residual, the reference point weights, the first reference line and the second reference line are updated until the intersection line displacement of the laser plane in two adjacent rounds with the two fixed reference planes does not exceed the sub-pixel resolution of the stripe center point, thus obtaining the current laser plane. The contour reconstruction module is used to convert the workpiece stripe point set into camera imaging rays, intersect with the current laser plane to form the current three-dimensional contour, arrange the current three-dimensional contours according to the laser image acquisition order, and obtain the three-dimensional measurement results.

[0013] In a preferred embodiment, the laser triangulation-based machine vision online 3D measurement storage medium includes: the storage medium storing a computer program, which, when executed by a processor, implements the various steps of the 3D measurement method as described above.

[0014] The technical effects and advantages of this invention are as follows: 1. By simultaneously exposing the line laser to the workpiece and the fixed reference surfaces on both sides at the same exposure time, and fitting the reference line with two sets of reference points to constrain and obtain the current laser plane, and then using the current laser plane to participate in the intersection of the workpiece stripe imaging rays, the three-dimensional coordinate calculation of each frame uses the actual laser geometric position of the frame, thereby reducing the situation where the laser plane offset caused by laser temperature rise and support vibration is converted into workpiece height change, and avoiding the need to rely on line stop recalibration to correct such deviations; 2. A double residual is formed by the two distances from the reference point to the reference line and the candidate laser plane. The weight of the reference point is calculated by Student-t variational Bayes. This reduces the participation of the reference point that deviates from the two geometric relationships in the fitting of the reference line, and reduces the pulling effect of local reflection and fringe center shift on the position of the reference line. 3. By jointly solving the coplanar constraint of the first and second reference lines and the weighted plane cost of the reference points, the updated laser plane is simultaneously constrained by the directions of the reference lines on both sides and the spatial positions of the two sets of reference points, reducing the plane direction offset caused by relying solely on reference data from one side. 4. By comparing the residuals of the first and second planes with their respective line residuals, only the reference point weight and reference line on that side are updated when there is a deviation on one side. When there is a deviation on both sides, the data on both sides are updated synchronously to reduce the propagation of anomalies in the reference fringes on one side to the reference results on the other side. 5. By projecting the intersection line formed by the updated laser plane and the two fixed reference planes onto the same frame image, and comparing the intersection line displacement with the resolution of the ordinate of the fringe center point, the iteration termination is determined, so that the plane iteration stop position is connected with the actual extraction accuracy of the fringe center. 6. By judging the denominator and ray parameters of the intersection between the workpiece imaging ray and the current laser plane, spatial points without unique intersection points and those located in the reverse extension direction of the camera are eliminated, reducing invalid intersection results from entering the current three-dimensional contour. Attached Figure Description

[0015] Figure 1 This is a roadmap for the online three-dimensional measurement technology of the present invention.

[0016] Figure 2 This is a schematic diagram of the online three-dimensional measurement system of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Refer to the instruction manual appendix Figure 1-2 The present invention provides a machine vision-based online 3D measurement method using laser triangulation, comprising: S1. Control the laser projection intelligent sensing element to simultaneously irradiate the workpiece under test, the first fixed reference surface and the second fixed reference surface with the same line of laser, and acquire the same frame of laser image. In this embodiment, before the workpiece to be measured enters the measurement position, the spatial positions of two fixed reference surfaces in the camera coordinate system are first established. Then, the projection range of the line laser is determined according to the two fixed reference surfaces and the lateral coverage range of the workpiece to be measured in the measurement section, so that the same line laser can cross the first fixed reference surface, the surface of the workpiece to be measured and the second fixed reference surface in one projection. The camera records laser stripes formed in three regions during a single exposure, providing a spatial reference at the same acquisition moment for subsequent inversion of the current laser plane using two fixed reference surfaces. The camera coordinate system has the camera optical center as its origin, the optical axis as the Zc axis, and the horizontal and vertical directions of the imaging plane determine the Xc and Yc axes, respectively. The laser projection coordinate system has the laser emission point as its origin, and its rotation matrix and translation vector with the camera coordinate system are obtained during the equipment installation and calibration phase and remain unchanged during measurement. This implementation process includes the following steps: S11. Fix a planar target plate on each side of the motion channel of the workpiece being measured. Use the flat surfaces of the two planar target plates facing the camera as the first fixed reference surface and the second fixed reference surface, respectively. Set three fixed and non-collinear calibration points on each planar target plate. The center of the calibration point is obtained in the three-dimensional coordinates of the camera coordinate system through camera calibration during the installation and calibration stage. For any fixed reference surface, form a first vector with the coordinate difference between the first calibration point and the second calibration point, and form a second vector with the coordinate difference between the first calibration point and the third calibration point. Perform a cross product of the first vector and the second vector and normalize to obtain the unit normal vector of the fixed reference surface. Then substitute the coordinates of the first calibration point into the plane equation to obtain the plane intercept. Determine the planar position of the fixed reference surface in the camera coordinate system by the unit normal vector and the plane intercept. Perform the above operations on the first fixed reference surface and the second fixed reference surface respectively, and save the coordinates of the three calibration points, the unit normal vector and the plane intercept respectively for use in finding the intersection of the imaging rays of the reference laser stripe. S12. Within the measurement section, the intersection points of the boundary line of the first fixed reference surface away from the workpiece and the measurement section, and the intersection points of the boundary line of the second fixed reference surface away from the workpiece and the measurement section are respectively taken as the outer edge points of the two fixed reference surfaces. When the workpiece is positioned laterally by the positioning fixture, the intersection points of the left and right boundaries of the workpiece determined by the fixture and the measurement section are taken as the two endpoints of the workpiece. The coordinates of the two endpoints in the camera coordinate system are calculated from the lateral dimensions in the product specifications and the fixture reference position. The translation vector between the camera coordinate system and the laser projection coordinate system is successively subtracted from the camera coordinates of the two outer edge points and the two endpoints of the workpiece, and then multiplied by the inverse of the rotation matrix to obtain the coordinates of the four points in the laser projection coordinate system. The four emission direction angles are calculated using the vectors pointing from the laser emission point to the four points respectively. The direction angles are sorted by value, and the angle between the two emission direction vectors of the first and last items in the sort is calculated. The obtained angle is used as the line laser deployment angle. The laser projection intelligent sensing element consists of a line laser emitter, a deployment optics and a drive control interface. The drive control interface adjusts the deployment optics according to the obtained deployment angle so that the line laser covers the measurement section between the first and last items of the direction angle, thereby covering the two fixed reference surfaces and the workpiece to be measured between them. S13. The laser projection intelligent sensing element maintains the linear laser projection according to the obtained linear laser unfolding angle. The linear laser intersects with the first fixed reference surface to form the first reference laser stripe, intersects with the surface of the workpiece to form the workpiece laser stripe, and intersects with the second fixed reference surface to form the second reference laser stripe. The camera's imaging range simultaneously covers the first fixed reference surface, the surface of the workpiece being measured, and the second fixed reference surface. During the same exposure, it receives the reflected light from three laser stripes, forming a single-frame laser image. Based on the lateral projection range of the two planar target plates and the workpiece measurement area onto the camera's imaging surface, the single-frame laser image is divided into a first reference area, a workpiece area, and a second reference area. The first reference laser stripe, the workpiece laser stripe, and the second reference laser stripe are located in the three areas respectively, providing an image range for subsequent column-by-column reading of pixel grayscale and extraction of the center points of the three types of stripes. A single exposure allows the three laser stripes to be recorded from the same acquisition time, avoiding the time misalignment between the reference stripe and the workpiece stripe caused by changes in the laser plane position when using different images. In the above manner, two planar target plates provide spatial references with fixed positions. The line laser deployment angle is calculated from the four boundary points of the actual measurement section, so that the line laser passes through two fixed reference surfaces and the workpiece being measured simultaneously in one projection. The camera then obtains three laser stripes in one exposure, so that the subsequent reference point calculation and workpiece three-dimensional inversion are established in the same laser projection state at the same moment. In practical applications: For example, the metal profile being measured is located at the center of the conveying fixture, with a planar target plate installed on each side. First, the coordinates of three calibration points of each of the two planar target plates in the camera coordinate system are obtained, and the plane equations of the two fixed reference surfaces are calculated. Then, four emission direction angles are calculated from the outer edge points of the two planar target plates and the intersection points of the left and right boundaries of the profile with the measurement section. The direction angles are sorted at both ends to form the line laser unfolding angle. After the laser projection intelligent sensing element projects according to the obtained angle, a laser line passes through the first fixed reference surface, the profile surface and the second fixed reference surface in sequence. The camera obtains the first reference laser stripe, the workpiece laser stripe and the second reference laser stripe in one exposure. Subsequent processing directly extracts the three types of stripes from the same frame laser image.

[0019] S2. Based on computer vision, extract the center points of the workpiece laser stripe, the first reference laser stripe, and the second reference laser stripe. Convert the center points of the two reference laser stripes into imaging rays and intersect them with the first fixed reference surface and the second fixed reference surface respectively to obtain the workpiece stripe point set, the first reference point set, and the second reference point set. In this embodiment, the center points of the first reference region, the workpiece region, and the second reference region in the same frame of laser image are extracted respectively. Then, the center points obtained from the two reference regions are converted from two-dimensional pixel positions into spatial imaging rays, and the intersection points of the rays are calculated using the known spatial positions of the two fixed reference surfaces. Thus, the first and second reference laser stripes are converted into a first and a second set of reference points in the camera coordinate system. In this stage, only the center point coordinates of the workpiece laser stripes in the image are retained for subsequent intersection with the current laser plane. The two reference laser stripes are first restored to their spatial coordinates using the fixed reference surfaces to provide input for reference line fitting. This implementation process includes the following steps: S21. Read gray values ​​column by column along the horizontal coordinate of pixels in the first reference area, the workpiece area and the second reference area respectively. The pixels in each column are arranged in descending order of gray value. When the gray values ​​are the same, they are arranged in ascending order of vertical coordinate. The vertical coordinate of the first pixel in the sorting is taken as the vertical coordinate of the peak point. Based on the downward increase of the vertical coordinate of the image, move pixel by pixel from the peak point to the direction of decreasing vertical coordinate. When the gray value of the current pixel is not greater than the gray value of the adjacent pixel outside, the current pixel is taken as the upper boundary point. The lower boundary point is obtained in the same way from the peak point in the direction of increasing ordinate. If no boundary point is obtained after moving to the boundary of the image area in any direction, no center point is generated in this column. Let the ordinate and gray value of the upper boundary point be vu and Iu, and the ordinate and gray value of the lower boundary point be vd and Id. For pixels with ordinate v between vu and vd, the background gray value is calculated as Iu + (Id - Iu) × (v - vu) / (vd - vu). When the pixel gray value is greater than the background gray value, the difference between the two is used as the pixel weight. Otherwise, the pixel weight is zero. The ordinate of the stripe center is obtained by multiplying the ordinate of each pixel by the pixel weight and dividing the sum of the pixel weights. This sum is then combined with the abscissa of the pixels in this column to form the center point. When the sum of pixel weights is zero, no center point of this column is generated. The above operation is performed on the three image regions respectively to obtain the first reference center point set, the workpiece stripe point set and the second reference center point set. The gray-level centroid is calculated after boundary gray-level linear interpolation to reduce the offset of the stripe center position by the slowly changing background brightness of the target plate or workpiece surface. S22. Each center point in the first and second reference center point sets is denoted as pixel coordinates (u, v), forming the transpose of the homogeneous pixel vector [u, v, 1], and multiplied by the inverse of the camera intrinsic parameter matrix K to obtain the ray direction vector in the camera coordinate system; the ray direction vector is divided by its own magnitude to obtain the unit ray direction vector, with the camera optical center O as the ray origin, and O+tq representing the imaging ray, where q is the unit ray direction vector and t is the ray parameter. The first reference center point set is transformed point by point to form the first reference ray set, and the second reference center point set is transformed point by point to form the second reference ray set; the two ray sets retain the original pixel horizontal coordinates and point order of each center point, so that the subsequent spatial reference points and the image center points are maintained point by point. S23. For the three non-collinear calibration points A1, B1, and C1 on the first fixed reference plane, the cross product vector of B1-A1 and C1-A1 is normalized to obtain the first plane unit normal vector n1, and the inner product of -n1 and A1 is used to obtain the first plane intercept d1, forming the first reference plane equation where the inner product of n1 and spatial coordinate X plus d1 equals zero; the second fixed reference plane uses the three non-collinear calibration points A2, B2, and C2 to obtain the second plane unit normal vector n2 and the second plane intercept d2 according to the same operation, forming the second reference plane equation. For the first reference ray O+tq, the sum of the first plane intercept and the inner product of n1 and O is negative as the numerator, and the inner product of n1 and q is used as the denominator. The two are divided to obtain the ray parameter t, and then t is substituted back into O+tq to obtain the first reference point; The second reference ray and the second reference plane equation are used to obtain the second reference point. When the denominator is zero, the imaging ray and the fixed reference plane do not have a unique intersection point, and the corresponding center point is not added to the reference point set. When the denominator is non-zero but the ray parameter is not positive, the intersection point is located in the direction of backward extension of the camera optical center, and is also not added to the reference point set. The remaining intersection points form the first reference point set and the second reference point set, respectively, for subsequent fitting of the first reference line and the second reference line. Through the above processing, the two-dimensional center positions of the three types of laser stripes are first determined by the same gray-scale center calculation method. Then, the three-dimensional coordinates of only two reference laser stripes are recovered by using the spatial geometric relationship of the fixed reference plane, so that the calculation of the reference points does not depend on the current laser plane, thereby avoiding circular references before the current laser plane is obtained. The workpiece stripe point set maintains two-dimensional pixel coordinates, and three-dimensional inversion is performed after the current laser plane is obtained. In practical applications: When the profile under test is located between two planar target plates, the camera obtains three laser stripes in a single exposure. The system searches for peak points and upper and lower boundary points in the three image regions column by column. After background grayscale interpolation and weighted calculation, three sets of center points are obtained. The first reference center point and the second reference center point are then converted into two sets of imaging rays by the camera's intrinsic parameters. The two sets of imaging rays intersect with the plane equations of the two planar target plates respectively, and finally obtain the first reference point set and the second reference point set. The workpiece stripe point set retains the pixel coordinates and is used in the subsequent three-dimensional contour calculation after solving the current laser plane.

[0020] S3. Fit the first reference line and the second reference line to the first reference point set and the second reference point set respectively. Use the calibrated laser plane as the candidate laser plane. Form a double residual based on the distance of each reference point to its reference line and the candidate laser plane. Update the weight of each reference point using Student-t variational Bayes. In this embodiment, the first reference point set and the second reference point set are respectively from two fixed planar target plates. The spatial points in the same reference point set are ideally distributed on the intersection line of the line laser and the corresponding fixed reference surface. First, the first reference line and the second reference line are obtained according to the spatial distribution of the two sets of reference points. Then, the degree of deviation of each reference point from its reference line and from the candidate laser plane is measured at the same time, and the two distances form a double residual. The double residuals are converted into reference point weights using Student-t variational Bayes, which reduces the influence of reference points that deviate from the two geometric relationships in subsequent reference line fitting. The double residuals are then recalculated based on the updated reference lines until the position change of the reference lines in the image reaches the sub-pixel resolution of the fringe center point itself. This process includes the following steps: S31. For the first set of reference points, let the number of reference points be N1. Summate the three-dimensional coordinates of each reference point coordinate by coordinate and divide by N1 to obtain the first coordinate mean. Subtract the first coordinate mean from the coordinates of each reference point to obtain the three-dimensional difference vector. Summate the product of each three-dimensional difference vector and its own transpose and divide by N1 to obtain the first coordinate covariance matrix. Perform eigenvalue decomposition on the first coordinate covariance matrix, sort by eigenvalue values, take the eigenvector of the eigenvalue with the highest value and normalize it as the direction of the first reference line. Use the first coordinate mean as the first reference line passing through the point to obtain the first reference line. The second set of reference points is calculated in the same way to obtain the second coordinate mean, the second coordinate covariance matrix and the second reference line. At least two reference points with different spatial coordinates are retained in each set of reference points to ensure that the reference line has a definite direction. The calibration laser plane is obtained during the equipment installation and calibration phase: when the equipment is stationary, the same line of laser light is used to illuminate two fixed reference planes. Initial reference points are obtained by using the same spatial intersection method as the first and second reference point sets. The average coordinates of all initial reference points are used as the plane's passing point. The difference vector obtained by subtracting the average coordinates from the initial reference point coordinates constitutes a three-dimensional covariance matrix. The eigenvector of the least significant eigenvalue is taken as the plane's unit normal vector. The plane intercept is calculated from the plane's passing point. The resulting plane is saved as the calibration laser plane. When processing the current laser image in the same frame, the calibration laser plane is used as a candidate laser plane to provide a fixed comparison benchmark for the plane deviation of the reference points. S32. For any reference point P in the first reference point set, take the cross product of P minus the vector obtained by the first reference line passing through the point and the unit direction vector of the first reference line, and take the magnitude of the cross product vector as the perpendicular distance from P to the first reference line; the reference points in the second reference point set are calculated in the same way using the second reference line. The candidate laser plane is written as the inner product of the unit normal vector n and the spatial coordinate P plus the plane intercept d, which is equal to zero. Then, take the absolute value of the inner product of n and P plus d as the normal distance from the reference point to the candidate laser plane. The perpendicular distance to the same reference point is used as the first component and the normal distance as the second component to form a two-dimensional column vector, resulting in the double residual of the reference point. Since both ideal geometric deviations aim for zero, this implementation uses the zero vector as the basis for calculating the double residual. All double residuals of the first and second reference point sets are multiplied by their transposes, and the resulting two-dimensional matrices are summed and divided by the total number of double residuals to obtain the double residual covariance matrix. For any double residual, its transpose, the inverse of the double residual covariance matrix, and the double residual are multiplied sequentially, and the resulting scalar is used as the Mahalanobis distance of the reference point. When the determinant of the double residual covariance matrix is ​​non-zero, it is directly inverted. When all double residuals are zero, the weights of each reference point are set to 1, and the iteration of this stage ends. When the double residuals are not all zero but the determinant is zero, the reference point weights of the previous valid round are used in this frame. When this situation occurs in the first round, the weights of each reference point are set to 1 to avoid the calculation interruption caused by the non-invertibility of the matrix. S33. Let the dimension of the double residuals be 2, and take the degree of freedom of the Student-t distribution as the dimension of the double residuals plus 2, that is, the degree of freedom is 4. When using the Student-t Gaussian-Gamma mixture representation, for the i-th reference point with Mahalanobis distance δi, the gamma posterior shape parameter is calculated as the sum of the degree of freedom and the dimension of the double residuals divided by 2, that is, 3. The gamma posterior rate parameter is calculated as the sum of the degree of freedom and δi divided by 2, that is, (4+δi) / 2. Then, the reference point weight is obtained by dividing the shape parameter by the rate parameter, that is, the reference point weight is 6 / (4+δi). Thus, as the Mahalanobis distance increases, the reference point weight decreases according to the relationship of the denominator increasing. There is no need to set an outlier discrimination value separately. For the first reference point set, the coordinates of each reference point are multiplied by their respective reference point weights and summed. Then, the sum is divided by the total weight of the reference points in the first reference point set to obtain the first weighted coordinate mean. Subtract the first weighted mean coordinate from the coordinates of each reference point to obtain the difference vector. Multiply the product of the difference vector and its transpose by the reference point weight, sum all the results, and divide by the sum of the reference point weights of the first reference point set to obtain the first weighted covariance matrix. Update the first reference line with the eigenvector of the first weighted mean coordinate passing through the point and the first eigenvalue of the first weighted covariance matrix as the direction. Update the second reference line of the second reference point set according to the same operation. For example, when a reference laser stripe is affected by metal reflection and a reference point deviates from the main stripe, the reference point also has a larger vertical distance or plane normal distance. The Mahalanobis distance increases accordingly, and the weight of the reference point decreases, which weakens the pulling effect of the point on the weighted mean coordinate and the direction of the reference line. S34. Calculate the vertical distance from each reference point in the first reference point set to the first reference line using the updated first reference line. Calculate the vertical distance from each reference point in the second reference point set to the second reference line using the updated second reference line. The normal distance from each reference point to the candidate laser plane is still calculated according to the candidate laser plane equation. Reassemble the new vertical distance and normal distance into a two-dimensional double residual. Form a double residual covariance matrix with the zero vector as the reference from all the new double residuals. Then, calculate the Mahalanobis distance, Student-t gamma posterior shape parameter, rate parameter, and reference point weights in sequence. Recalculate the first weighted coordinate mean, the second weighted coordinate mean, the two weighted covariance matrices, the first reference line, and the second reference line using the new reference point weights. After each round of reference line updates, determine two projection points on the reference line by subtracting the unit direction vector from the weighted coordinate mean of the reference line and adding the unit direction vector to the two spatial points. Project the two projection points onto the same frame laser image according to the camera intrinsic parameters and connect them to form the reference line projection. Calculate the absolute value of the difference between the ordinate of the first reference line projection in the current round and the first reference line projection in the previous round at the x-coordinate of each pixel in the first reference center point set. Calculate the absolute value of the difference between the ordinate of the second reference line projection in the same way at the x-coordinate of each pixel in the second reference center point set. For the first comparison, use the first and second reference lines obtained in S31 as the reference lines for the previous round. The sub-pixel resolution of the stripe center point is taken as the quantization step size of the center point ordinate output in S21. For example, when the center point ordinate is retained to 0.01 pixels, the sub-pixel resolution is 0.01 pixels. When the absolute value of the difference between the two sets of ordinates does not exceed this sub-pixel resolution, stop updating and output the first reference line, the second reference line, and the weights of each reference point; otherwise, use the results of this round to proceed to the next round of double residual and reference point weight calculation. Through the above processing, whether a reference point participates in the reference line fitting is no longer determined by a single distance or an artificially set outlier boundary, but by the Mahalanobis distance formed by the two geometric quantities of the reference point's deviation from the reference line and the candidate laser plane, and then transformed into reference point weights by the Student-t posterior relationship; after the reference point weights change the reference line, the reference line participates in the next round of vertical distance calculation, so that the influence of local offset points on the reference line position is reduced round by round, while reference points that conform to the geometric relationship of the line laser maintain their main role, and finally stop relying on the sub-pixel quantization capability of the stripe center point itself, avoiding the iteration accuracy from deviating from the image measurement capability; In practical applications: When a small number of center points in the first reference laser stripe shift to one side due to local high-brightness reflection, while the second reference laser stripe remains normal, the first reference line obtained from the first round of fitting will be pulled by the shifted points; these points produce a larger vertical distance in the double residual, and combined with their normal distance relative to the candidate laser plane, they form a larger Mahalanobis distance. After the Student-t posterior calculation, a lower reference point weight is obtained. In the next round, the first reference line is re-determined with the weighted coordinate mean and weighted covariance matrix, gradually returning to the spatial direction formed by the main reference point; when the changes in the ordinates of the two reference lines relative to the previous round in the same frame of laser image have both fallen into the sub-pixel resolution of the stripe center point, the two reference lines and reference point weights are output, and the calculation of updating the laser plane is entered.

[0021] S4. With the first reference line and the second reference line being coplanar as a constraint, the updated laser plane is obtained by using the Lagrange multiplier according to the reference point weights. Based on the updated laser plane, the double residual, the reference point weights, the first reference line and the second reference line are updated until the intersection line displacement of the laser plane in two adjacent rounds with the two fixed reference planes does not exceed the sub-pixel resolution of the stripe center point, thus obtaining the current laser plane. In this embodiment, the laser plane at the current acquisition time is determined by the first reference line, the second reference line, and the reference point weights. The directions of the first and second reference lines are used to limit the spatial attitude of the laser plane to be determined, and the reference point weights are used to reduce the influence of the offset reference points on the plane position. After obtaining the updated laser plane, the dispersion of the two sets of reference points to the plane and reference lines is compared respectively. Based on this, it is decided whether to retain the plane, update the reference lines on one side or both sides. Finally, the position change of the intersection line of the updated laser plane and the two fixed reference planes in the same frame of laser image is used to determine whether the iteration has ended. This implementation process includes the following steps: S41. Take the normalized direction vectors of the first and second reference lines as u1 and u2, respectively. Denote the coordinates of the reference points in the first and second reference point sets as Pi, and the weight of each reference point as wi. Represent the laser plane to be determined as the inner product of the unit normal vector n and the spatial coordinate P plus the plane intercept d, which equals zero. Since the magnitude of n is 1, the normal distance from the reference point Pi to the laser plane to be determined is |the inner product of n and Pi + d|. Multiply wi by the square of this normal distance and sum it over the two sets of reference points to obtain the plane cost. At the same time, set the inner product of n and u1 to zero and the inner product of n and u2 to zero, so that the directions of the two reference lines are both located in the direction space of the laser plane to be determined. Set the inner product of n and itself to 1 to eliminate the scale uncertainty of the plane equation. The plane cost is used to determine the plane position that the two sets of reference points approach together. The two directional constraints are used to prevent the low weight points from causing the laser plane to be determined to deviate from the spatial direction of the two reference lines. S42. Taking the plane cost as the main term, multiply the inner product of n and u1, the inner product of n and u2, and the inner product of n and itself minus 1 by the first, second, and third Lagrange multipliers respectively, and then add them together to form the Lagrange function; take the partial derivative with respect to the plane intercept d and set it to zero, and we can get that d is equal to the negative value of the inner product of the weighted coordinate mean of the reference point and n, where the weighted coordinate mean of the reference point is obtained by multiplying all the coordinates of the reference point by wi, summing them, and then dividing by the sum of all wi. Subtract the weighted coordinate mean of the reference point from the coordinates of each reference point to obtain the weighted centrifugal coordinates, and multiply the product of each weighted centrifugal coordinate with its own transpose by wi and then sum them to form the weighted scatter matrix; After taking the partial derivative with respect to n, we obtain the product of the weighted scatter matrix and n, u1, u2, and the equation composed of three Lagrange multipliers. We then solve this equation simultaneously with two directional constraints and a unit modulus constraint to obtain the unit normal vector n. We then calculate the plane intercept d from the aforementioned weighted coordinate mean to form the updated laser plane. When the unit normal vector has both positive and negative representations, the first round uses the unit normal vector of the candidate laser plane as the directional reference, and subsequent rounds use the unit normal vector of the updated laser plane from the previous round as the directional reference. When the inner product of the unit normal vector and the directional reference in this round is negative, we multiply the unit normal vector and the plane intercept in this round by negative one to ensure that adjacent rounds use the same normal direction representation. S43. For the first set of reference points, calculate the normal distance from each reference point to the updated laser plane, multiply the square of the normal distance by the reference point weight, sum the results, divide by the sum of the reference point weights in the first set of reference points, and take the square root to obtain the first plane residual; calculate the first line residual by calculating the perpendicular distance from each reference point to the first reference line using the same weighted root mean square method; calculate the second plane residual and the second line residual for the second set of reference points respectively. When the first plane residual is not greater than the first line residual and the second plane residual is not greater than the second line residual, it indicates that the dispersion of the two sets of reference points on the updated laser plane does not exceed their own dispersion on the line, and the updated laser plane is retained. When the first plane residual is greater than the first line residual and the second plane residual is not greater than the second line residual, the perpendicular distance from the first reference point to the first reference line and the normal distance to the updated laser plane are combined to form a new first set of double residuals. The two sets of current double residuals are used to form a zero-centered second-order matrix and calculate the Mahalanobis distance. Then, the weight of the first reference point is updated according to the reference point weight 6 / (4+Mahalanobis distance). The first weighted coordinate mean, the first weighted covariance matrix and the first reference line are recalculated with the new first reference point weight. The weight of the second reference point and the second reference line remain unchanged. When the second plane residual is greater than the second line residual and the first plane residual is not greater than the first line residual, the same operation is used to update only the weight of the second reference point and the second reference line. In other cases, when both plane residuals are greater than their respective line residuals, the weights of the two reference points, the first reference line and the second reference line are updated with the two new double residuals respectively. The updated two reference lines and reference point weights are used to re-form the plane cost and coplanar constraints to obtain the next round of updated laser plane. The above four results cover all combinations of the comparison of the two sets of residuals, and no additional empirical judgment value is set. S44. For the retained updated laser plane, solve the equations simultaneously with the first fixed reference plane and the second fixed reference plane. The cross product of the unit normal vectors of the two planes is used to obtain the spatial intersection direction. Then, solve the equations of the two planes simultaneously with the auxiliary equations perpendicular to the intersection direction to obtain a spatial point on the intersection line. The spatial point and the intersection direction form the first spatial intersection line and the second spatial intersection line, respectively. If the cross product of the unit normal vectors of the updated laser plane and any fixed reference plane is zero, then a unique spatial intersection line cannot be formed in this round. No intersection line displacement determination is performed. The reference point weights and the two reference lines are updated again using the double residuals formed by the updated laser plane in this round. For cases where a spatial intersection line can be formed, the first spatial intersection line is projected onto the laser image in the same frame. The absolute value of the difference between the vertical coordinate of the current round projection and the vertical coordinate of the previous round projection is calculated at the pixel horizontal coordinates contained in the first reference center point set. The value with the first value is taken as the first intersection line displacement. The second spatial intersection line is obtained at the x-coordinate of the pixels contained in the second reference center point set in the same way. In the first comparison, the projection intersection lines formed by the candidate laser plane and the two fixed reference planes are used as the results of the previous round. In subsequent rounds, the projection intersection lines formed by the updated laser plane retained in the previous round are used as the comparison benchmark. The resolution of the y-coordinate of the fringe center point is taken as the quantization step size used when calculating the y-coordinate of the fringe center. If the first intersection line displacement and the second intersection line displacement do not exceed the quantization step size, the updated laser plane of this round is taken as the current laser plane; otherwise, the updated laser plane is obtained again using the reference point weights of this round, the first reference line, and the second reference line. Through the above process, the two reference lines first restrict the spatial direction of the laser plane, and the reference point weights then participate in the solution of the plane position, so that local fringe offsets will not directly pull the laser plane with the same effect. After the laser plane is updated, the first plane residual and the second plane residual are cross-compared with the line residuals of the two sets of reference lines, so that the anomaly of the reference fringe on one side only triggers the update of the weights and reference lines on one side. Only when both sides deviate at the same time will the two sets of reference data be updated synchronously. Finally, the intersection of the projections on the two fixed reference planes is used to jointly judge the plane change, and the resolution of the ordinate of the fringe center point is used as the stopping boundary, so that the iteration accuracy is connected with the image center extraction capability. In practical applications: For example, local reflections on the first plane target plate may cause several points in the first reference point set to deviate, while the second reference point set remains stable. After obtaining the updated laser plane, the residual of the first plane may be greater than the residual of the first line, while the residual of the second plane may not be greater than the residual of the second line. In this case, only the reference point weights of the offset points in the first reference point set are reduced and the first reference line is recalculated, while the second reference line remains unchanged. After obtaining the laser plane again, its projection intersections with the two fixed reference planes are checked until the changes in the ordinates of the two intersection lines relative to the previous round fall within the resolution of the ordinates of the fringe center point. The resulting plane is then used as the current laser plane for calculating the three-dimensional contour of the workpiece.

[0022] S5. Convert the workpiece stripe point set into camera imaging rays, intersect with the current laser plane to form the current three-dimensional contour, arrange the current three-dimensional contours according to the laser image acquisition order, and obtain the three-dimensional measurement results. In this embodiment, after the current laser plane has been obtained, the two-dimensional center point of the workpiece stripe point set is restored to spatial coordinates point by point. Each workpiece stripe point first forms an imaging ray passing through the camera optical center according to the camera imaging relationship, and then the intersection point of the imaging ray and the current laser plane is calculated. Workpiece stripe points that cannot intersect with the current laser plane in front of the camera are discarded. The remaining spatial points are arranged into a single frame of three-dimensional contour according to the lateral order of the original workpiece laser stripes. Then, the obtained three-dimensional contours are arranged according to the acquisition order of the laser images in the same frame to form a three-dimensional measurement result. This implementation process includes the following steps: S51. For any pixel coordinate (u, v) in the workpiece stripe point set, first use the same camera distortion parameters as the first reference center point set and the second reference center point set to perform pixel distortion removal, and then multiply the transpose of the distorted homogeneous pixel vector [u, v, 1] by the inverse of the camera intrinsic parameter matrix K to obtain the ray direction vector q in the camera coordinate system. Taking the camera optical center O as the starting point of the ray, the imaging ray is represented as O+tq, where t is the ray parameter. The current laser plane is represented as the inner product of the unit normal vector n and the spatial coordinate P plus the plane intercept d, which is equal to zero. Substituting O+tq into the equation of the current laser plane, we get t=-(d+n·O) / (n·q). Where n·q is used as the denominator of the intersection calculation and -(d+n·O) is used as the numerator of the intersection calculation. When the denominator of the intersection calculation is non-zero, the ray parameter t is obtained by dividing the numerator of the intersection calculation by the denominator of the intersection calculation. Then, t is substituted into O+tq, and the resulting spatial coordinates are the spatial points of the workpiece stripe points on the current laser plane. This operation uses the current laser plane obtained in this frame to complete the trigonometric inversion. Therefore, when the position of the laser plane changes during online measurement, the spatial coordinates of the workpiece are recalculated with the current laser plane instead of continuing to use the calibrated laser plane. S52. For each workpiece stripe point, judge the intersection result: when the intersection denominator is non-zero and the ray parameter t is positive, the intersection point is located at the positive position of the camera optical center along the imaging ray, and the obtained spatial point is retained; when the intersection denominator is zero, the imaging ray is parallel to the current laser plane or located in the current laser plane, and the unique intersection point cannot be determined by the ray, and the workpiece stripe point that generated the ray is discarded. When the denominator of the intersection is non-zero and the ray parameter t is not positive, the intersection point is located at the camera optical center or the reverse extension direction of the imaging ray. The resulting spatial points are discarded. The retained spatial points inherit the pixel abscissa u of the original workpiece stripe points and are arranged in ascending order of u to form the current three-dimensional contour. If a pixel column is discarded due to invalid intersection, no additional points are added between adjacent spatial points. This avoids adding positions that have not been obtained by laser triangulation to the current three-dimensional contour. For example, if the imaging ray formed by a point of the workpiece stripe is parallel to the current laser plane, even if there are valid spatial points in the two adjacent columns, the invalid intersection point is not replaced by interpolation of adjacent points. S53. Each time the camera completes an exposure including the first reference laser stripe, the workpiece laser stripe, and the second reference laser stripe, it assigns an incremental contour number to the same frame of laser images according to the order of acquisition, and makes the current three-dimensional contour obtained from the same frame of laser images inherit the same contour number; arranges each current three-dimensional contour according to the incremental contour number, while maintaining the arrangement order of the spatial points inside each current three-dimensional contour according to the incremental pixel horizontal coordinate of the workpiece stripe points, and obtains the three-dimensional measurement result. The three-dimensional measurement results are composed of the current three-dimensional contour formed at each acquisition time in sequence, so that the lateral spatial point sequence within a single frame and the acquisition sequence between multiple frames are preserved respectively, which is used to characterize the three-dimensional contour change process of the workpiece as it passes through the measurement position. Through the above processing, the three-dimensional coordinates of each workpiece stripe point are obtained from the actual intersection of the camera imaging ray and the current laser plane in the current frame. The results of non-unique intersection points or located in the camera's reverse space are excluded by calculating the intersection denominator and ray parameters. The three-dimensional contour is formed according to the horizontal point order of the workpiece laser stripes within a single frame, and the multiple frames are organized according to the exposure acquisition order, so that the two-dimensional stripe extraction, the current laser plane solution and the three-dimensional result output form a closed data chain. In practical applications: When the profile under test passes the measurement position, the camera sequentially acquires multiple laser images of the same frame. For each image, the current laser plane at the acquisition time is first obtained based on two reference laser stripes. Then, the workpiece stripe points are back-projected point by point into imaging rays and intersected with the current laser plane. For example, if a frame acquires 1200 workpiece stripe points, and 1196 points satisfy the condition that the intersection denominator is non-zero and the ray parameter is positive, then 1196 spatial points are retained and arranged according to the original pixel horizontal coordinates to form the current three-dimensional contour of the frame, and the remaining 4 points are directly discarded. Subsequent frames are arranged according to the exposure order, and finally, a three-dimensional measurement result composed of each current three-dimensional contour is obtained.

[0023] Furthermore, the laser triangulation-based machine vision online 3D measurement system includes: The same-frame acquisition module is used to control the laser projection intelligent sensing element so that the same line of laser simultaneously illuminates the workpiece under test, the first fixed reference surface and the second fixed reference surface, and acquires the same-frame laser image. The ray intersection module is used to extract the center points of the workpiece laser stripes, the first reference laser stripe, and the second reference laser stripe based on computer vision. It converts the center points of the two reference laser stripes into imaging rays and intersects them with the first fixed reference surface and the second fixed reference surface, respectively, to obtain the workpiece stripe point set, the first reference point set, and the second reference point set. The residual weighting module is used to fit the first reference line and the second reference line to the first reference point set and the second reference point set respectively, with the calibrated laser plane as the candidate laser plane, and to form double residuals based on the distance of each reference point to its reference line and the candidate laser plane, and to update the weights of each reference point using Student-t variational Bayes. The plane update module is used to obtain the updated laser plane by using the coplanarity of the first reference line and the second reference line as constraints and the Lagrange multiplier according to the reference point weights. Based on the updated laser plane, the double residual, the reference point weights, the first reference line and the second reference line are updated until the intersection line displacement of the laser plane in two adjacent rounds with the two fixed reference planes does not exceed the sub-pixel resolution of the stripe center point, thus obtaining the current laser plane. The contour reconstruction module is used to convert the workpiece stripe point set into camera imaging rays, intersect with the current laser plane to form the current three-dimensional contour, arrange the current three-dimensional contours according to the laser image acquisition order, and obtain the three-dimensional measurement results.

[0024] Furthermore, the laser triangulation-based machine vision online 3D measurement storage medium includes: the storage medium storing a computer program, which, when executed by a processor, implements the various steps of the 3D measurement method described above.

[0025] Working principle: In this embodiment, a planar target plate is fixed on each side of the movement channel of the workpiece being measured. The surfaces of the two planar target plates facing the camera are used as the first fixed reference surface and the second fixed reference surface. The planar position of each fixed reference surface in the camera coordinate system is determined by three non-collinear calibration points on each fixed reference surface. Then, the linear laser unfolding angle is determined according to the outer edge points of the two fixed reference surfaces and the direction angles of the endpoints of the workpiece being measured relative to the laser emission point. This allows the same linear laser to cross the first fixed reference surface, the surface of the workpiece being measured, and the second fixed reference surface in one exposure. The camera obtains the first reference laser stripe, the workpiece laser stripe, and the second reference laser stripe in one exposure. The grayscale values ​​of the three laser stripes are read column by column. The center points of the stripes are obtained by peak point search, upper and lower boundary determination, background grayscale linear interpolation and grayscale positive difference weighted calculation. The workpiece laser stripes form the workpiece stripe point set. The center points of the two reference laser stripes are converted into imaging rays passing through the camera optical center by the camera intrinsic parameters. Then, they are intersected with the first fixed reference surface and the second fixed reference surface respectively to obtain the first reference point set and the second reference point set. Since the two sets of reference points are derived from the actual intersections of the line laser and two fixed reference surfaces, the first and second reference lines are first fitted based on the coordinate covariance matrix of the two sets of reference points. Then, a double residual is formed by the perpendicular distance from the reference point to its reference line and the normal distance to the candidate laser plane. The Mahalanobis distance of the double residual is converted into reference point weights through Student-t variational Bayes, so that the role of the reference points caused by reflection and fringe center shift in the reference line fitting decreases with the degree of deviation. The reference point weights are then used to calculate the weighted coordinate mean and weighted covariance matrix to update the two reference lines until the position changes of the two reference lines in the same frame of laser image enter the sub-pixel resolution range of the fringe center point. In this way, two spatial reference lines that are less affected by local anomalies are obtained first, and then the laser plane is solved, avoiding the overall plane deflection caused by unilateral fringe shift when directly using all reference points to fit the plane. After the weights of the two reference lines and reference points are determined, the geometric constraint is that both reference lines are located within the laser plane to be determined. The weights of the reference points are multiplied by the square of the normal distance from the reference point to the laser plane to be determined and then accumulated to form the plane cost. Constraints are established with the dot product of the unit normal vector of the laser plane to be determined with the unit direction vectors of the two reference lines being zero and the magnitude of the unit normal vector being 1. The unit normal vector and plane intercept of the updated laser plane are obtained by simultaneously solving the Lagrange multipliers. After the updated laser plane is obtained, it is not directly used as the measurement plane. Instead, the plane residuals of the first and second reference point sets relative to the updated laser plane are calculated separately and cross-judged with the line residuals of the two sets of reference points relative to their respective reference lines. If neither of the two sets of planar residuals exceeds the residual of the current line, the updated laser plane is retained. If only one set exceeds the residual, the double residuals are re-formed using the reference points of the current set, and the weights of the reference points and reference lines of the current set are updated. If both sets exceed the residual, the weights of the two sets of reference points and reference lines are updated simultaneously, and the laser plane is re-obtained. The retained updated laser plane is intersected with the two fixed reference planes and projected onto the laser image of the same frame. The change in the ordinate of the intersection line of two adjacent projections at the same pixel x-coordinate is used to determine whether the plane has stabilized. If the displacement of the two intersection lines does not exceed the ordinate resolution of the fringe center point, the updated laser plane is determined as the current laser plane. Otherwise, the reference point weighting, reference line updating, and plane solving are continued. After obtaining the current laser plane, each pixel in the workpiece stripe point set is converted into a camera imaging ray through the camera intrinsic parameter inverse matrix. The spatial intersection points of the imaging ray and the current laser plane are obtained. Spatial points with non-zero denominators and positive ray parameters are retained. The current three-dimensional contour is formed according to the pixel abscissa order of the workpiece stripe points. During the movement of the workpiece being measured, the camera sequentially acquires multiple laser images of the same frame. Each image independently acquires the current laser plane and the current three-dimensional contour at the acquisition time according to the same process. Then, the current three-dimensional contours are arranged according to the order of image acquisition to form the three-dimensional measurement result. For example, during the online transport of metal profiles, when the laser temperature rises or the mounting bracket vibrates, causing the linear laser projection plane to shift, the reference laser stripes on the target plates on both sides will synchronously reflect this change. The system first uses two sets of reference points to correct the reference line, then uses coplanar constraints to inversely calculate the new laser plane, and finally uses the new laser plane to calculate the spatial coordinates of the profile stripes. Therefore, the change in the laser projection direction is used to correct the trigonometric inversion geometric relationship, rather than being directly used as the height change of the profile surface in the three-dimensional measurement results.

[0026] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A machine vision-based online 3D measurement method based on laser triangulation, characterized in that, include: S1. Control the laser projection intelligent sensing element to simultaneously irradiate the workpiece under test, the first fixed reference surface and the second fixed reference surface with the same line of laser, and acquire the same frame of laser image. S2. Based on computer vision, extract the center points of the workpiece laser stripe, the first reference laser stripe, and the second reference laser stripe. Convert the center points of the two reference laser stripes into imaging rays and intersect them with the first fixed reference surface and the second fixed reference surface respectively to obtain the workpiece stripe point set, the first reference point set, and the second reference point set. S3. Fit the first reference line and the second reference line to the first reference point set and the second reference point set respectively. Use the calibrated laser plane as the candidate laser plane. Form a double residual based on the distance of each reference point to its reference line and the candidate laser plane. Update the weight of each reference point using Student-t variational Bayes. S4. With the first reference line and the second reference line being coplanar as a constraint, the updated laser plane is obtained by using the Lagrange multiplier according to the reference point weights. Based on the updated laser plane, the double residual, the reference point weights, the first reference line and the second reference line are updated until the intersection line displacement of the laser plane in two adjacent rounds with the two fixed reference planes does not exceed the sub-pixel resolution of the stripe center point, thus obtaining the current laser plane. S5. Convert the workpiece stripe point set into camera imaging rays, intersect with the current laser plane to form the current three-dimensional contour, arrange the current three-dimensional contours according to the laser image acquisition order, and obtain the three-dimensional measurement results.

2. The online 3D measurement method based on laser triangulation machine vision according to claim 1, characterized in that: S1 includes: S11. The surfaces of the two planar target plates fixed on both sides of the workpiece being measured and facing the camera are respectively used as the first fixed reference surface and the second fixed reference surface. The planar position of the corresponding fixed reference surface in the camera coordinate system is determined by three non-collinear calibration points on each fixed reference surface. S12. Convert the outer edge points of the two fixed reference surfaces and the two endpoints of the workpiece to the laser projection coordinate system, take the two points at the two ends of the angle between the laser emission points to determine the line laser unfolding angle, and control the laser projection intelligent sensing element to project the same line laser according to the unfolding angle. S13. The line laser intersects with the first fixed reference surface, the surface of the workpiece being measured, and the second fixed reference surface to form a first reference laser stripe, a workpiece laser stripe, and a second reference laser stripe, respectively. A single exposure by a camera captures a laser image containing the three laser stripes in the same frame.

3. The online 3D measurement method based on laser triangulation machine vision according to claim 2, characterized in that: S2 includes: S21. Read the pixel grayscale values ​​of the first reference laser stripe, the workpiece laser stripe, and the second reference laser stripe column by column. Sort the pixels in each column in descending order of grayscale value. When the grayscale values ​​are the same, sort them in ascending order of pixel ordinate. Take the first pixel in the sorted list as the peak point. Take the first pixel on the top and bottom of the peak point whose grayscale value is not greater than the adjacent pixel as the upper boundary point and the lower boundary point. Linearly interpolate the background grayscale between the two boundary points. Calculate the weighted average of the pixel ordinate with the positive difference between the pixel grayscale value and the background grayscale value as the weight. This will give the first reference center point set, the workpiece stripe point set, and the second reference center point set, respectively. S22. Convert the center points of the first reference center point set and the second reference center point set into imaging rays passing through the camera optical center according to the camera intrinsic parameters, and obtain the first reference ray set and the second reference ray set respectively. S23. Determine the equations of the first and second reference planes using three non-collinear calibration points on the first and second fixed reference planes respectively. Obtain the first reference point set by intersecting the first reference ray set with the first reference plane equation. Obtain the second reference point set by intersecting the second reference ray set with the second reference plane equation.

4. The online 3D measurement method based on laser triangulation machine vision according to claim 3, characterized in that: S3 includes: S31. Calculate the mean coordinates and the coordinate covariance matrix of the first reference point set and the second reference point set respectively. Using the mean coordinates as the passing point and the eigenvector of the first eigenvalue in the covariance matrix as the direction, obtain the first reference line and the second reference line, and use the calibration laser plane as the candidate laser plane. S32. Calculate the perpendicular distance from each reference point to its reference line and the normal distance to the candidate laser plane to form a double residual. Calculate the covariance matrix of all double residuals. Multiply the transpose of the double residuals, the inverse of the covariance matrix, and the double residuals in sequence to obtain the Mahalanobis distance of each reference point.

5. The online 3D measurement method based on laser triangulation machine vision according to claim 4, characterized in that: S3 further includes: S33. Using Student-t variational Bayes, the Mahalanobis distance of each reference point is substituted into the Student-t Gaussian-Gamma mixture representation. The weight of each reference point is obtained by dividing the shape parameter of the gamma posterior by the rate parameter. Then, the weighted mean of the first reference point set and the weighted covariance matrix are calculated according to the reference point weights. The first reference line and the second reference line are updated with the eigenvector of the weighted mean of the point and the eigenvalue of the first eigenvalue in the weighted covariance matrix as the direction. S34. Calculate the vertical distance of each reference point using the updated first and second reference lines, and form a new double residual with the normal distance of each reference point to the candidate laser plane. Update the covariance matrix, Mahalanobis distance, reference point weights, first and second reference lines in sequence until the difference in the ordinates of the first and second reference lines projected onto the same frame of laser image in two adjacent rounds does not exceed the sub-pixel resolution of the stripe center point. Output the first reference line, second reference line, and reference point weights.

6. The online 3D measurement method based on laser triangulation machine vision according to claim 5, characterized in that: S4 includes: S41. Take the unit direction vectors of the first reference line and the second reference line, and mark the spatial coordinates of the first reference point set and the second reference point set as the reference point coordinates. Multiply the reference point weight by the reference point and substitute it into the equation of the laser plane to be solved, and sum the squares of the absolute values ​​as the plane cost. The unit normal vector of the laser plane to be solved and the inner product of the two unit direction vectors are zero and the magnitude of the unit normal vector is 1 to form a coplanar constraint. S42. Multiply the plane cost and the three coplanar constraints by Lagrange multipliers respectively and sum them. Take the partial derivatives of the unit normal vector, plane intercept, and the three Lagrange multipliers of the laser plane to be determined and set the partial derivatives to zero. Solve the equations to obtain the unit normal vector and plane intercept, and form the updated laser plane.

7. The online 3D measurement method based on laser triangulation machine vision according to claim 6, characterized in that: S4 further includes: S43. Multiply the square of the normal distance from the reference point to the updated laser plane by the reference point weight, sum the results, divide by the sum of the reference point weights, and take the square root to obtain the first plane residual and the second plane residual. Calculate the first line residual and the second line residual in the same way using the square of the perpendicular distance from the reference point to the first reference line and the second reference line, respectively. If the first plane residual is not greater than the first line residual and the second plane residual is not greater than the second line residual, retain the updated laser plane. If the first plane residual is greater than the first line residual and the second plane residual is not greater than the second line residual, update the first reference point weight and the first reference line using the perpendicular distance and normal distance from the first reference point. If the second plane residual is greater than the second line residual and the first plane residual is not greater than the first line residual, update the second reference point weight and the second reference line using the perpendicular distance and normal distance from the second reference point. Otherwise, update the two sets of reference point weights, the first reference line, and the second reference line respectively, and calculate the updated laser plane again. S44. The retained updated laser plane is intersected with the first fixed reference plane and the second fixed reference plane to form two spatial intersection lines. The two spatial intersection lines are projected onto the same frame of laser image, and the absolute value of the difference between the current projection vertical coordinate and the previous projection vertical coordinate under the same pixel horizontal coordinate is used as the intersection line displacement. When the intersection line displacement of the two spatial intersection lines does not exceed the resolution of the vertical coordinate of the fringe center point, the updated laser plane is used as the current laser plane. Otherwise, the updated laser plane is obtained again with the updated reference point weight, the first reference line and the second reference line.

8. The online 3D measurement method based on laser triangulation machine vision according to claim 7, characterized in that: S5 includes: S51. Convert the coordinates of each pixel in the workpiece stripe point set into the ray direction vector of the camera coordinate system through the inverse matrix of the camera intrinsic parameters. Take the camera optical center as the ray starting point. Use the inner product of the current laser plane unit normal vector and the ray direction vector as the intersection denominator. Use the negative value of the sum of the inner product of the current laser plane intercept, the unit normal vector and the camera optical center coordinates as the intersection numerator. Divide the intersection numerator by the intersection denominator to obtain the ray parameter. Then, add the ray starting point coordinates to the product of the ray parameter and the ray direction vector to obtain the spatial point. S52. When the intersection denominator is non-zero and the ray parameter is positive, retain the space points; when the intersection denominator is zero, remove the corresponding workpiece stripe points; when the intersection denominator is non-zero and the ray parameter is not positive, remove the corresponding space points; arrange the retained space points in ascending order of the pixel x-coordinate of the workpiece stripe points to form the current 3D contour. S53. Using the sequence of acquisition of laser images in the same frame as the contour number, arrange each current three-dimensional contour in ascending order of the contour number, and maintain the spatial point arrangement order within each current three-dimensional contour to obtain the three-dimensional measurement result.

9. A machine vision online 3D measurement system based on laser triangulation, characterized in that, include: The same-frame acquisition module is used to control the laser projection intelligent sensing element so that the same line of laser simultaneously illuminates the workpiece under test, the first fixed reference surface and the second fixed reference surface, and acquires the same-frame laser image. The ray intersection module is used to extract the center points of the workpiece laser stripes, the first reference laser stripe, and the second reference laser stripe based on computer vision. It converts the center points of the two reference laser stripes into imaging rays and intersects them with the first fixed reference surface and the second fixed reference surface, respectively, to obtain the workpiece stripe point set, the first reference point set, and the second reference point set. The residual weighting module is used to fit the first reference line and the second reference line to the first reference point set and the second reference point set respectively, with the calibrated laser plane as the candidate laser plane, and to form double residuals based on the distance of each reference point to its reference line and the candidate laser plane, and to update the weights of each reference point using Student-t variational Bayes. The plane update module is used to obtain the updated laser plane by using the coplanarity of the first reference line and the second reference line as constraints and the Lagrange multiplier according to the reference point weights. Based on the updated laser plane, the double residual, the reference point weights, the first reference line and the second reference line are updated until the intersection line displacement of the laser plane in two adjacent rounds with the two fixed reference planes does not exceed the sub-pixel resolution of the stripe center point, thus obtaining the current laser plane. The contour reconstruction module is used to convert the workpiece stripe point set into camera imaging rays, intersect with the current laser plane to form the current three-dimensional contour, arrange the current three-dimensional contours according to the laser image acquisition order, and obtain the three-dimensional measurement results.

10. A machine vision-based online 3D measurement storage medium using laser triangulation, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the various steps of the three-dimensional measurement method as described in any one of claims 1-8.

Images