Method and device for measuring object surface shape based on structured light

Abstract

The present invention relates to a method for measuring the surface shape of an object based on structured light, which includes the following steps: Step 1: constructing a measuring device for three-dimensional measurement of the surface shape of an object; Step 2: determining the optical axis setting angle of a single-line structured light emitter in the measuring device; Step 3: calibrating the measuring device and using the measuring device to obtain a laser stripe image of the surface of the object; Step 4: extracting the center point of the laser stripe in the image, and obtaining three-dimensional point cloud data of the surface of the object to be measured through coordinate transformation. The present invention uses multi-line structured light to assist in calibrating the light plane parameters of the single-line structured light, avoiding multiple calibrations of the single-line structured light. Compared with using a displacement sensor to position the single-line structured light, the present invention improves the point cloud density and measurement accuracy; the method of matching spatial points to spatial planes through a distance matrix reduces the algorithm complexity and improves the measurement speed; the measuring device of the present invention is conveniently installed at the end of a robotic arm to perform three-dimensional surface shape measurement of large components.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional measurement, and in particular to a method for measuring the surface shape of an object based on structured light and a measuring device thereof. Background Art

[0002] Line structured light detection technology is one of the most common methods for measuring object geometric parameters and achieving 3D reconstruction. Due to its non-contact, large-scale, fast, high-precision, stable algorithm, and simple structure, it has been widely used in industrial inspection.

[0003] In commonly used line structured light 3D measurement systems, the relative position of the structured light emitter and camera is fixed. From the camera's captured image, the center point of the laser stripe where the structured light intersects the surface of the object being measured is extracted. Combining the camera's intrinsic parameters with the structured light's light plane equation, the 3D coordinates of the intersecting surface shape in the camera coordinate system are determined. A mature solution for camera calibration is already available. Therefore, the core issue in calibrating the entire 3D measurement system is determining the light plane equation of the line structured light in the camera coordinate system.

[0004] In traditional line-structured light 3D measurement systems, the structured light emitter is fixed in position relative to the camera coordinate system. Line-structured light emitters can have single-line, triple-line, or multi-line configurations. However, regardless of the configuration, because the structured light emitter is fixed relative to the camera, the camera coordinate system differs from the world coordinate system. This means that only a single or limited number of lines of 3D data can be measured on the surface of the object being measured, resulting in a relatively sparse surface shape. To achieve more intensive measurements, at least one of the sensor and the object under test must be in relative motion.

[0005] Line structured light 3D measurement systems measure the surface of a test object by adjusting the relative position of the structured light emitter within the camera coordinate system, while the object under test is fixed and the sensor is in relative motion. In actual measurement, the motion parameters of the structure must be determined using high-precision encoders, fixed targets, or the characteristics of the test object itself. This allows the corresponding transformation matrix to be calculated from known transformation relationships and complete the entire measurement process. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, the measurement method proposed by the present invention uses multi-line structured light to assist in calibrating the light plane parameters of single-line structured light, thereby improving the density of the point cloud and measurement accuracy. A method for matching spatial points to spatial planes via a distance matrix is ​​also employed, reducing algorithm complexity and improving measurement speed in actual use. The measuring device proposed by the present invention is suitable for measuring the surface shape of large components when measuring the surface shape of an object to be measured, and can be conveniently installed at the end of a robotic arm to perform surface shape measurements on large components.

[0007] To achieve the above objectives, the present invention adopts a solution: a method for measuring the surface shape of an object based on structured light, which uses multi-line structured light to assist in determining the light plane parameters of single-line structured light and optimizes the parameters of the measurement device, thereby improving the measurement accuracy from two aspects. The method includes the following steps:

[0008] Step 1: Build a measuring device for three-dimensional measurement of the surface shape of an object;

[0009] The measuring device is a device that uses structured light and a camera to achieve three-dimensional measurement of the surface shape of an object; the main components of the measuring device include: a camera, a single-line structured light emitter, a multi-line structured light emitter, and a linear module;

[0010] Step 2: Determine the optical axis setting angle of the single-line structured light emitter in the measurement device;

[0011] The set angle is the angle between the optical axis of the single-line structured light emitter and the baseline;

[0012] Step 21: Determine the effect of the setting angle on depth resolution;

[0013] Establish the conversion relationship between the coordinates of the point on the surface to be measured projected by the single-line structured light in the camera coordinate system and the coordinates of the projection point on the camera imaging plane in the camera coordinate system. The depth resolution obtained by taking the partial derivative of z with respect to x' is as follows:

[0014]

[0015] Where: represents the depth resolution; f represents the focal length of the camera; d represents the distance between the optical center of the single-line structured light emitter and the optical center of the camera; θ represents the angle between the optical axis of the single-line structured light emitter and the baseline; represents the partial derivative of the horizontal coordinate of the projection point of the single-line structured light projected onto the surface to be measured on the camera imaging plane in the camera coordinate system; cot represents the cotangent function; x' represents the horizontal coordinate of the projection point of the single-line structured light projected onto the surface to be measured on the camera imaging plane in the camera coordinate system;

[0016] Step 22: Determine the effect of the setting angle on the accuracy of laser stripe center extraction;

[0017] Analyze and extract the deviation e between the theoretical derivative calculated value and the actual differential calculated value of the laser stripe center u =u c -u', laser stripe center extraction error e u The method to obtain is as follows:

[0018]

[0019] Where: eu represents the laser stripe center extraction error; u c represents the theoretical derivative calculated value of the laser stripe center; u' represents the actual differential calculated value of the laser stripe center; G u Represents the first-order derivative of the light intensity distribution law; G uu Represents the second-order derivative of the light intensity distribution law; u0 represents the horizontal coordinate of the pixel with the maximum light intensity in the width direction of the laser stripe; g'(u0) represents the first-order difference at the position of maximum light intensity; g"(u0) represents the second-order difference at the position of maximum light intensity;

[0020] Step 23: Determine the effect of the setting angle on the accuracy of the measuring device and obtain the value of the setting angle;

[0021] Get the depth resolution determined in step 1 and the laser stripe center extraction error e determined in step 22 u , the method for obtaining the comprehensive error e of the measuring device is as follows:

[0022]

[0023] Where: e represents the comprehensive error of the measuring device;

[0024] The computer draws a comprehensive error curve with the angle between the optical axis of the single-line structured light emitter and the baseline, that is, the optical axis setting angle θ of the single-line structured light emitter as the horizontal coordinate and the comprehensive error e of the measuring device as the vertical coordinate. The angle θ corresponding to the minimum value of the comprehensive error curve is taken from the graph as the value used when the measurement system is installed;

[0025] Step 3: Calibrate the measuring device and use it to obtain the laser stripe image on the object surface;

[0026] Adjust the structural parameters of the measurement system; use the measurement device to obtain an image of the planar target and calibrate the light plane parameters of the structured light; turn on the measurement device to scan the object to be measured and obtain the laser stripe image on the object surface;

[0027] Step 4: Extract the center point of the laser stripe in the image and obtain the three-dimensional point cloud data of the surface of the object to be measured through coordinate transformation;

[0028] Step 41: Extract the laser stripe intersections in the image and perform light plane matching on the laser stripe intersections in the image; construct a distance matrix between each laser stripe intersection and the light plane of the single-line structured light; set a threshold and perform a cyclic search to match the light plane of the multi-line structured light to which each laser stripe intersection belongs;

[0029] Step 42: Use the matched laser stripe intersection points to determine the light plane equation of the single-line structured light, and obtain three-dimensional data of the shape of the object to be measured through coordinate transformation;

[0030] Step 43: The three-dimensional coordinates of the center point of the single-line structured light projection laser stripe in the image in the camera coordinate system can all be calculated, that is, the dense point cloud data of all areas on the surface of the object being measured that are scanned by the single-line structured light are obtained, and the three-dimensional measurement of the surface shape of the object being measured is completed.

[0031] Preferably, the conversion relationship between the coordinates of the point on the surface to be measured projected by the single-line structured light in the camera coordinate system and the coordinates of the projection point on the camera imaging plane in the camera coordinate system in step 21 is specifically as follows:

[0032] According to the coordinate transformation relationship of camera imaging, the coordinates of the point where the single-line structured light is projected onto the surface to be measured in the camera coordinate system are x, y, z, and the coordinates of the projection point on the camera imaging plane in the camera coordinate system are x', y', f. The transformation relationship between the two coordinate systems is as follows:

[0033]

[0034] Where: x, y, and z represent the horizontal coordinate, vertical coordinate, and coordinate along the optical axis of the point projected by the single-line structured light onto the surface to be measured in the camera coordinate system, respectively; x' and y' represent the horizontal and vertical coordinates of the projection point of the single-line structured light onto the surface to be measured on the camera imaging plane, respectively.

[0035] Preferably, the analysis in step 22 extracts the deviation e between the theoretical derivative calculated value of the laser stripe center and the actual differential calculated value. u =u c -u', specifically:

[0036] The theoretical derivative calculated value u of the laser stripe center c The specific calculation process is as follows:

[0037] When the projected plane is perpendicular to the optical axis of the single-line structured light emitter, the light intensity distribution on the laser stripe presents a standard Gaussian distribution in the width direction, as shown below:

[0038]

[0039] Where: G(t) represents the light intensity value; I' represents the light intensity amplitude; σ represents the standard deviation of the light intensity value; t represents the offset value relative to the center of the laser stripe; e represents the natural logarithm;

[0040] When the projected plane is not perpendicular to the optical axis of the single-line structured light emitter and the acute angle between them is θ, the light intensity distribution law G(u,θ) of the laser stripe in the width direction related to the angle θ is obtained through geometric analysis and coordinate system transformation. First, the first-order derivative G of the light intensity distribution law G(u,θ) of the laser stripe in the width direction is obtained. u(u,θ) and the second-order derivative G uu (u,θ); and then the Taylor expansion of the light intensity distribution at the pixel point with the maximum light intensity in the width direction of the laser stripe in the image is obtained as follows:

[0041]

[0042] Where: u represents the horizontal coordinate of any position of the laser stripe in the image;

[0043] Taking partial derivative of u in the above formula, we get:

[0044]

[0045] Where: represents the first-order partial derivative of the Taylor expansion of the light intensity distribution with respect to u;

[0046] Maximum light intensity Equal to 0, the method for obtaining the first light intensity maximum position is derived from the above formula as follows:

[0047]

[0048] The specific calculation process of the actual differential calculated value u' of the laser stripe center is as follows:

[0049] The expressions of the first-order difference g'(u0) and the second-order difference g"(u0) at the position of maximum light intensity are as follows:

[0050]

[0051] Use g'(u0) and g”(u0) instead of G respectively u (u0) and G uu (u0), the Taylor expansion of the light intensity distribution at u0 is as follows:

[0052]

[0053] Where: G(u) represents the light intensity value at any position in the image; G(u0) represents the maximum light intensity among all pixels in the image;

[0054] Derivative G(u) with respect to u, and make The actual differential value u' of the laser stripe center is calculated as follows:

[0055]

[0056] Preferably, the measuring device is calibrated in step 3, and the laser stripe image of the object surface is obtained using the measuring device, specifically:

[0057] Step 31: Acquire an image of the planar target;

[0058] First, the measuring device including the linear module, camera, multi-line structured light emitter, and single-line structured light emitter is fixed on a bracket to ensure stability when the system is working; secondly, a planar target is fixed in a suitable position so that the camera can fully capture the image of the planar target, and both the multi-line structured light and the single-line structured light can be projected onto the planar target; then, the multi-line structured light emitter is turned on, and the camera is used to capture the image while the single-line structured light emitter is turned off; then, the camera is used to capture the image again with the multi-line structured light turned off and the single-line structured light turned on; finally, the multi-line structured light emitter and the single-line structured light emitter are turned off, and the camera is used to capture the image for the third time to complete this round of image capture;

[0059] By changing the pose of the planar target J times through the above steps, J groups of planar target images with different poses are collected. Each group contains 3 images, namely the projection of multi-line structured light on the planar target, the projection of single-line structured light on the planar target, and the background image of the planar target. The pose of the calibration plate at different poses relative to the camera is defined as T j (j=1,2,...,J);

[0060] The three images in the j-th posture are defined as a group of images as follows:

[0061]

[0062] Where: IMG j Indicates that the calibration plate posture is at T j The three images in the state are defined as a group of images; IMG m,j Indicates that the calibration plate posture is at T j Projection of multi-line structured light on a flat target captured in the state; IMG s,j Indicates that the calibration plate posture is at T j The projection image of single-line structured light on a flat target taken in the state; IMG b,j Indicates that the calibration plate posture is at T j Plane target background image taken in the state; j represents the sequence number of each group of images; J represents the number of image groups; T j Indicates the pose definition of the calibration plate with different poses relative to the camera;

[0063] Step 32: Use the image of the planar target to calibrate the light plane parameters of the structured light:

[0064] Get image IMG s,j and image IMG m,jExtract the center point of the laser stripe of the single-line structured light and the center point of the multi-line structured light, use these points to calibrate the light plane equations of the single-line structured light and the multi-line structured light in the camera coordinate system, and obtain the light plane equation π of the multi-line structured light in the camera coordinate system i (i=1,2,...n) and the light plane equation π0 of the single-line structured light at the initial position;

[0065] Step 33: Use a measuring device to measure the object to be tested, and collect an image of the surface of the object to be tested with laser stripes;

[0066] Adjust the structural parameters of the measurement system, place the object to be measured, and adjust the installation position of the single-line structured light emitter based on the angle between the single-line structured light and the baseline calculated in step 2, that is, the preferred value of the optical axis setting value θ of the single-line structured light emitter; fix the measuring device on the bracket to ensure its stability during operation, and place the object to be measured in the working area of ​​the measurement system;

[0067] Step 34: Start the measurement system to scan the object to be measured, and obtain an image of the object to be measured with laser stripes;

[0068] Turn on the multi-line structured light emitter to project structured light onto the surface of the object to be measured to form multiple horizontal laser stripes, and turn on the single-line structured light emitter to project single-line structured light onto the surface of the object to be measured to form a longitudinal laser stripe; turn on the linear module, and under the drive of the linear module, the single-line structured light makes a horizontal linear motion, so that the longitudinal laser stripes formed by the single-line structured light projected onto the surface of the object to be measured are horizontally scanned across the surface of the object to be measured; turn on the camera, and periodically capture images of the object to be measured with laser stripes in the process of the single-line structured light scanning the surface of the object to be measured, and the longitudinal laser stripes in each image are located at different positions.

[0069] Preferably, the light plane of the multi-line structured light to which each laser stripe intersection belongs is matched in step 41, specifically:

[0070] Step 411: Construct the distance matrix Dis; when matching a laser stripe intersection to different light planes π i When (i=1,2,...n), the distance between it and the light plane π0 will change. The distance is represented by each element in a matrix Dis, as shown below:

[0071]

[0072] Where: Dis represents the distance matrix; M represents the total number of laser stripe intersections found in a single image; N represents the total number of light planes emitted by the multi-line structured light emitter in the image; d m,n Indicates matching the mth intersection point to the light plane π nWhen π0 is the distance between it and the light plane; m represents the number of the laser stripe intersection found in a single image; n represents the number of the light planes emitted by the multi-line structured light emitter in the image;

[0073] The mth intersection point is matched to the light plane π n When the distance d between it and the light plane π0 m,n The method to obtain is as follows:

[0074]

[0075] Where: A0, B0, C0 represent the first, second and third plane equation parameters of the light plane π0; Indicates that when the mth intersection point matches the light plane π n When , its horizontal coordinate, vertical coordinate and coordinate along the optical axis in the camera coordinate system;

[0076] The mth intersection point is matched to the light plane π n When , its coordinates in the camera coordinate system Calculated by the following formula:

[0077]

[0078] Where: u m and v m A represents the horizontal and vertical coordinates of the mth laser stripe intersection in the image coordinate system; n 、B n 、C n and D n Represent the light plane π n Parameters of the first, second, third and fourth light planes; c x Indicates the horizontal offset of the pixel coordinate origin; c y Indicates the vertical offset of the pixel coordinate origin; α indicates the scaling factor of the pixel coordinate system in the horizontal direction; β indicates the scaling factor of the pixel coordinate system in the vertical direction;

[0079] Step 412: using the geometric relationship between the intersection points of the laser stripes on the light plane of the single-line structured light, setting a threshold and performing a cyclic search to match the laser stripe intersection point to the light plane of the multi-line structured light to which it belongs;

[0080] The geometric properties of a plane determine that the distances from all points on the same plane to the plane are equal and zero. Therefore, the distances from the laser stripe intersections in each image to the single-line structured light plane are theoretically equal and zero. However, in practice, due to the error in extracting the center points of the laser stripes, the distances from the intersections to the single-line structured light plane are close but not equal. Therefore, a threshold Td is set. If the difference in the distances from two intersections to the single-line structured light plane is less than Td, the pair of intersections are judged to be in the same plane; otherwise, the pair of intersections are judged to be in two different planes.

[0081] After calculating the element values ​​in the matrix Dis, enter the loop and use the elements in the first row of the matrix Subtract the elements in the second row of the matrix Dis in column order Determine whether and Make this difference less than Td, if it exists, save it and And continue to search the next line Determine whether and The difference between them is less than Td, and the process is repeated until the last row is found. make where [n1,n2,…,n M ] is the correct light plane number of each intersection point. So far, each laser stripe intersection point in an image is matched to the light plane of the multi-line structured light to which it belongs.

[0082] Preferably, the step 42 of obtaining the three-dimensional data of the shape of the object to be measured by coordinate transformation is as follows:

[0083] Since the camera captures the planar target while the single-line structured light emitter is moving, each image corresponds to the single-line structured light emitter at a different position. Among the four light plane parameters A0, B0, C0, and D0 of the single-line structured light at different positions, only D0 changes. To obtain the light plane parameter D0 corresponding to each image, after matching the laser stripe intersection in each image to the correct light plane, the three-dimensional coordinates of the laser stripe intersection in each image in the camera coordinate system are calculated. The calculation formula is as follows:

[0084]

[0085] Where: x m 、y m and z m Respectively represent the horizontal coordinate, vertical coordinate and optical axis coordinate of the laser stripe intersection in the camera coordinate system; u m and v mRespectively represent the horizontal and vertical coordinates of the intersection point in the image coordinate system; A m 、B m 、C m and D m They respectively represent the first, second, third and fourth parameters of the light plane of the multi-line structured light to which the intersection point belongs;

[0086] After calculating the three-dimensional coordinates of the laser stripe intersection in each image within the camera coordinate system, the light plane parameter D0 of the single-line structured light is obtained using fitting. Since the light plane parameters A0, B0, and C0 of the single-line structured light emitter do not change during linear movement, the light plane parameters of all single-line structured lights at each shooting moment are obtained after calculating D0.

[0087] According to the camera coordinate transformation model and the light plane equation of single-line structured light, the three-dimensional coordinates of the center point of the longitudinal laser stripe in the image are calculated. The calculation formula is as follows:

[0088]

[0089] Where: x p 、y p and z p u respectively represent the horizontal coordinate, vertical coordinate and optical axis coordinate of the center point of the longitudinal laser stripe in the camera coordinate system; p and v p represents the horizontal and vertical coordinates of the laser center point in the image coordinate system; A0, B0, C0 and D0 represent the first, second, third and fourth parameters of the light plane of the single-line structured light.

[0090] A second aspect of the present invention provides a measuring device capable of implementing the aforementioned method for measuring the surface shape of an object based on structured light, capable of achieving three-dimensional measurement of the surface shape of an object. The device includes a camera, a single-line structured light emitter, a multi-line structured light emitter, a linear module, a central mounting frame, a single-line structured light mounting base, a multi-line structured light mounting base, a front housing, and a rear housing.

[0091] The camera is fixedly mounted on the top protruding frame of the central mounting frame through a threaded connection, collects images of the surface of the object to be measured and the structured light, and performs three-dimensional measurement of points on the surface of the object to be measured with the aid of the structured light;

[0092] The single-line structured light emitter is first fixedly mounted on the single-line structured light mounting base through a threaded connection. The single-line structured light mounting base is then fixedly mounted on the moving end of the linear module through a threaded connection so that the single-line structured light emitter and the single-line structured light emitter move linearly with the moving end of the linear module. The single-line structured light emitter is a moving component in the surface shape three-dimensional measurement device, assisting the camera in performing three-dimensional measurement.

[0093] The multi-line structured light emitter is first fixedly mounted on the multi-line structured light mounting base through a threaded connection. The multi-line structured light mounting base then secures itself and the multi-line structured light emitter to the protruding frame at the bottom end of the central mounting frame through an interference fit. The multi-line structured light replaces the encoder displacement sensor to position the single-line structured light and is a key component for improving measurement accuracy.

[0094] The fixed end of the linear module is fixedly mounted on the top flat part of the central mounting frame through a threaded connection;

[0095] The central mounting frame is the basis for assembling all components into a whole, and other components are fixed to the central mounting frame by direct installation or with the help of the mounting frame;

[0096] The rear housing is fixedly mounted on the rear end of the central mounting frame through a threaded connection;

[0097] The front shell is fixedly connected to the rear shell by a threaded connection, enclosing the entire central mounting frame and the components installed thereon for protection. There is a hole in the front shell at the camera installation position to expose the camera lens, and there are corresponding transparent windows at the installation positions of the multi-line structured light emitter and the single-line structured light emitter for the structured light to pass through.

[0098] Compared with the prior art, the present invention has the following beneficial effects:

[0099] (1) The measurement method of the present invention uses multi-line structured light to assist in calibrating the light plane parameters of single-line structured light, thereby avoiding multiple calibrations of single-line structured light. Compared with using encoders and other displacement sensors to locate single-line structured light, the application of this method improves the point cloud density and measurement accuracy.

[0100] (2) The measurement method involved in the present invention uses a method of matching spatial points to spatial planes through a distance matrix, which reduces the algorithm complexity and improves the measurement speed in actual use;

[0101] (3) The measuring device of the present invention can measure the surface shape of an object to be measured without moving the object to be measured, and is applicable to the surface shape measurement of large components. The overall size of the measuring device is small, and it is convenient to be installed at the end of a robotic arm to measure the surface shape of large components. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] Figure 1 This is a flow chart of a method for measuring the surface shape of an object based on structured light according to an embodiment of the present invention;

[0103] Figure 2 Schematic diagram of the optical axis setting angle of the single-line structured light according to an embodiment of the present invention;

[0104] Figure 3 This is a schematic diagram of the structured light projection effect according to an embodiment of the present invention;

[0105] Figure 4 Schematic diagram of the overall structure of the measuring device according to an embodiment of the present invention;

[0106] Figure 5 This is a schematic diagram of the structure of the measuring device after the front shell is removed according to an embodiment of the present invention.

[0107] Reference numerals:

[0108] 1. Camera; 2. Central mounting bracket; 3. Multi-line structured light mounting base; 4. Multi-line structured light emitter; 5. Single-line structured light emitter; 6. Single-line structured light mounting base; 7. Linear module; 8. Rear housing; 9. Front housing. DETAILED DESCRIPTION

[0109] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0110] The method for measuring the surface shape of an object based on structured light proposed in an embodiment of the present invention uses multi-line structured light to assist in calibrating the light plane parameters of a single-line structured light. Compared with using a displacement sensor such as an encoder to locate a single-line structured light, the method improves the density of the point cloud and the measurement accuracy. The method of matching spatial points to spatial planes through a distance matrix improves the measurement speed in actual use. Figure 1 The flowchart of the method for measuring the surface shape of an object based on structured light according to an embodiment of the present invention is shown. The measuring device involved in the embodiment of the present invention can be conveniently installed at the end of a robotic arm to measure the surface shape of large components. The description of the device in the embodiment proves that the device can measure the surface shape of an object.

[0111] An embodiment of the present invention provides a method for measuring the surface shape of an object based on structured light. To demonstrate the applicability of the present invention, the method is applied to an example. The method specifically includes the following steps:

[0112] S1: Build a measuring device for three-dimensional measurement of the surface shape of an object;

[0113] The measuring device refers to a device that uses structured light to achieve three-dimensional measurement of the surface shape of an object; the measuring device includes: a camera, a single-line structured light emitter, a multi-line structured light emitter and a linear module.

[0114] S2: Determine the optical axis setting angle of the single-line structured light emitter in the measuring device;

[0115] The setting angle is the angle between the optical axis of the single-line structured light emitter and the baseline; Figure 2The figure shows a schematic diagram of the angle between the optical axis and the baseline of the single-line structured light in an embodiment of the present invention. The baseline is perpendicular to the optical axis of the camera and parallel to the movement direction of the single-line structured light emitter. The angle between the optical axis and the baseline will affect the measurement accuracy of the system.

[0116] S21: Determine the effect of the setting angle on the depth resolution;

[0117] Establish the conversion relationship between the coordinates of the point on the surface to be measured projected by the single-line structured light in the camera coordinate system and the coordinates of the projection point on the camera imaging plane in the camera coordinate system. According to the coordinate conversion relationship of the camera imaging, the coordinates of the point on the surface to be measured projected by the single-line structured light in the camera coordinate system are (x, y, z) T , the coordinates of the projection point on the camera imaging plane in the camera coordinate system are (x', y', f) T , the transformation relationship between the two coordinate systems is as follows:

[0118]

[0119] Where: x, y, and z represent the horizontal coordinate, vertical coordinate, and coordinate along the optical axis of the point projected by the single-line structured light onto the surface to be measured in the camera coordinate system, respectively; x' and y' represent the horizontal and vertical coordinates of the projection point of the single-line structured light onto the surface to be measured on the camera imaging plane, respectively.

[0120] The depth resolution obtained by taking the partial derivative of z with respect to x' is as follows:

[0121]

[0122] Where: represents the depth resolution; f represents the focal length of the camera; d represents the distance between the optical center of the single-line structured light emitter and the optical center of the camera; θ represents the angle between the optical axis of the single-line structured light emitter and the baseline; represents the partial derivative of the horizontal coordinate of the projection point of the single-line structured light projected onto the surface to be measured on the camera imaging plane in the camera coordinate system; cot represents the cotangent function; x' represents the horizontal coordinate of the projection point of the single-line structured light projected onto the surface to be measured on the camera imaging plane in the camera coordinate system.

[0123] S22: Determine the effect of the setting angle on the accuracy of laser stripe center extraction;

[0124] Analyze and extract the deviation e between the theoretical derivative calculated value and the actual differential calculated value of the laser stripe center u =u c -u', the theoretical derivative calculated value u at the center of the laser stripe c The specific calculation process is as follows:

[0125] This case uses the Steger algorithm to extract the center point of the laser stripe. The algorithm extracts the point of maximum intensity within the laser stripe image as the center point of the laser stripe. When using the Steger algorithm to extract the maximum intensity point of the laser stripe, the intensity distribution pattern of the laser stripe along the width of the projection surface must be determined. When the projection plane is perpendicular to the optical axis of the single-line structured light emitter, the intensity distribution along the laser stripe exhibits a standard Gaussian distribution along the width, as shown below:

[0126]

[0127] Where: G(t) represents the light intensity value; I' represents the light intensity amplitude; σ represents the standard deviation of the light intensity value; t represents the offset value relative to the center of the laser stripe; and e represents the natural logarithm.

[0128] When the projected plane is not perpendicular to the optical axis of the single-line structured light emitter and the acute angle between them is θ, the light intensity distribution law G(u,θ) of the laser stripe in the width direction related to the angle θ is obtained through geometric analysis and coordinate system transformation. First, the first-order derivative G of the light intensity distribution law G(u,θ) of the laser stripe in the width direction is obtained. u (u,θ) and the second-order derivative G uu (u,θ); and then the Taylor expansion of the light intensity distribution at the pixel point with the maximum light intensity in the width direction of the laser stripe in the image is obtained as follows:

[0129]

[0130] Where: u represents the horizontal coordinate of any position of the laser stripe in the image.

[0131] Taking partial derivative of u in the above formula, we get:

[0132]

[0133] Where: Represents the first-order partial derivative of the Taylor expansion of the light intensity distribution with respect to u.

[0134] Maximum light intensity Equal to 0, the method for obtaining the first light intensity maximum position is derived from the above formula as follows:

[0135]

[0136] The specific calculation process of the actual differential calculated value u' of the laser stripe center is as follows:

[0137] The expressions of the first-order difference g'(u0) and the second-order difference g"(u0) at the position of maximum light intensity are as follows:

[0138]

[0139] Use g'(u0) and g”(u0) instead of G respectively u (u0) and G uu (u0), the Taylor expansion of the light intensity distribution at u0 is as follows:

[0140]

[0141] Where: G(u) represents the light intensity value at any position in the image; G(u0) represents the maximum light intensity among all pixels in the image.

[0142] Derivative G(u) with respect to u, and make The actual differential value u' of the laser stripe center is calculated as follows:

[0143]

[0144] Laser stripe center extraction error e u The method to obtain is as follows:

[0145]

[0146] Where: e u represents the laser stripe center extraction error; u c represents the theoretical derivative calculated value of the laser stripe center; u' represents the actual differential calculated value of the laser stripe center; G u Represents the first-order derivative of the light intensity distribution law; G uu represents the second-order derivative of the light intensity distribution law; u0 represents the horizontal coordinate of the pixel point with maximum light intensity in the width direction of the laser stripe; g'(u0) represents the first-order difference at the position of maximum light intensity; g"(u0) represents the second-order difference at the position of maximum light intensity.

[0147] S23: determining the influence of the setting angle on the accuracy of the measuring device and obtaining a value of the setting angle;

[0148] Get the depth resolution determined by S1 and the laser stripe center extraction error e determined by S22 u , the method for obtaining the measurement device error e is as follows:

[0149]

[0150] Where: e represents the error of the measuring device.

[0151] The computer draws a comprehensive error curve with the optical axis setting angle θ of the single-line structured light transmitter as the horizontal coordinate and the measurement device error e as the vertical coordinate. The θ value corresponding to the minimum value of the comprehensive error curve is taken from the graph as the value used when the measurement system is installed.

[0152] S3: Calibrate the measuring device and use the measuring device to obtain a laser stripe image on the object surface;

[0153] Use the measuring device to obtain the image of the planar target and calibrate the light plane parameters of the structured light; adjust the structural parameters of the measuring system, turn on the measuring device to scan the object to be measured, and obtain the laser stripe image on the object surface; Figure 3 The figure shows a schematic diagram of the structured light projection effect of an embodiment of the present invention. Multi-line structured light is projected onto the surface of the object to be measured to form multiple horizontal laser stripes, and single-line structured light is projected onto the surface of the object to be measured to form a longitudinal laser stripe.

[0154] S31: Acquire an image of a planar target;

[0155] First, the measuring device including the linear module, camera, multi-line structured light emitter and single-line structured light emitter is fixed on a bracket to ensure stability when the system is working; secondly, a planar target is fixed in a suitable position so that the camera can completely capture the image of the planar target, and both the multi-line structured light and the single-line structured light can be projected onto the planar target; then, the multi-line structured light emitter is turned on, and the camera is used to capture the image while the single-line structured light emitter is turned off; then, the camera is used to capture the image again with the multi-line structured light turned off and the single-line structured light turned on; finally, the multi-line structured light emitter and the single-line structured light emitter are turned off, and the camera is used to capture the image for the third time to complete this round of image capture.

[0156] By changing the pose of the planar target J times through the above steps, J groups of planar target images with different poses are collected. Each group contains 3 images, namely the projection of multi-line structured light on the planar target, the projection of single-line structured light on the planar target, and the background image of the planar target. The pose of the calibration plate at different poses relative to the camera is defined as T j (j=1,2,...,J).

[0157] The three images in the j-th posture are defined as a group of images as follows:

[0158]

[0159] Where: IMG j Indicates that the calibration plate posture is at T j The three images in the state are defined as a group of images; IMG m,j Indicates that the calibration plate posture is at T j Projection of multi-line structured light on a flat target captured in the state; IMG s,j Indicates that the calibration plate posture is at T j The projection image of single-line structured light on a flat target taken in the state; IMGb,j Indicates that the calibration plate posture is at T j Plane target background image taken in the state; j represents the sequence number of each group of images; J represents the number of image groups; T j Indicates the pose definition of the calibration plate in different poses relative to the camera.

[0160] S32: Calibrate the light plane parameters of structured light using an image of a planar target:

[0161] Get image IMG s,j and image IMG m,j The center points of the laser stripes of single-line structured light and multi-line structured light are extracted from the image, and the light plane equations of single-line structured light and multi-line structured light in the camera coordinate system are calibrated using these points to obtain the light plane equation π of the multi-line structured light in the camera coordinate system. i (i=1,2,...n) and the light plane equation π0 of the single-line structured light at the initial position.

[0162] S33: Using a measuring device to measure the object to be tested, and collecting an image of the surface of the object to be tested with laser stripes;

[0163] Adjust the structural parameters of the measurement system, place the object to be measured, and adjust the installation position of the single-line structured light emitter according to the preferred value of the optical axis setting angle θ of the single-line structured light calculated in S2 above; fix the measuring device on the bracket to ensure its stability during operation, and place the object to be measured in the working area of ​​the measurement system.

[0164] S34: Turn on the measuring system to scan the object to be measured, and obtain an image of the object to be measured with laser stripes;

[0165] Turn on the multi-line structured light emitter to project structured light onto the surface of the object to be measured to form multiple horizontal laser stripes, and turn on the single-line structured light emitter to project single-line structured light onto the surface of the object to be measured to form a longitudinal laser stripe; turn on the linear module, and under the drive of the linear module, the single-line structured light makes a horizontal linear motion, so that the longitudinal laser stripes formed by the single-line structured light projected onto the surface of the object to be measured are horizontally scanned across the surface of the object to be measured; turn on the camera, and periodically capture images of the object to be measured with laser stripes in the process of the single-line structured light scanning the surface of the object to be measured, and the longitudinal laser stripes in each image are located at different positions.

[0166] S4: Extract the center point of the laser stripe in the image and obtain the three-dimensional point cloud data of the surface of the object to be measured through coordinate transformation;

[0167] S41: Extract the laser stripe intersections in the image and perform light plane matching on the laser stripe intersections in the image; construct a distance matrix between each intersection and the light plane of the single-line structured light, set a threshold and perform a cyclic search to match the light plane of the multi-line structured light to which each laser stripe intersection belongs.

[0168] S411: Construct the distance matrix Dis; when a laser stripe intersection is matched to different light planes π i When (i=1,2,...n), the distance between it and the light plane π0 will change. The distance is represented by each element in a matrix Dis, as shown below:

[0169]

[0170] Where: Dis represents the distance matrix; M represents the total number of laser stripe intersections found in a single image; N represents the total number of light planes emitted by the multi-line structured light emitter in the image; d m,n Indicates matching the mth intersection point to the light plane π n When π0 is the distance between it and the light plane; m is the number of the laser stripe intersection found in a single image; n is the number of the light planes emitted by the multi-line structured light emitter in the image.

[0171] The mth intersection point is matched to the light plane π n When the distance d between it and the light plane π0 m,n The method to obtain is as follows:

[0172]

[0173] Where: A0, B0, C0 represent the first, second and third plane equation parameters of the light plane π0; Indicates that when the mth intersection point matches the light plane π n , its horizontal coordinate, vertical coordinate and coordinate along the optical axis in the camera coordinate system.

[0174] The mth intersection point is matched to the light plane π n When , its coordinates in the camera coordinate system Calculated by the following formula:

[0175]

[0176] Where: u m and v m A represents the horizontal and vertical coordinates of the mth laser stripe intersection in the image coordinate system; n 、B n 、C n and D n Represent the light plane π n Parameters of the first, second, third and fourth light planes; c x Indicates the horizontal offset of the pixel coordinate origin; c yIt represents the vertical offset of the pixel coordinate origin; α represents the scaling factor of the pixel coordinate system in the horizontal direction; β represents the scaling factor of the pixel coordinate system in the vertical direction.

[0177] S412: Using the geometric relationship between the intersections of the laser stripes on the light plane of the single-line structured light, a threshold is set and a cyclic search is performed to match the laser stripe intersection to the light plane of the multi-line structured light to which it belongs.

[0178] The geometric properties of the plane determine that the distances from all points on the same plane to the plane are equal and zero. Therefore, the distances from the intersection of the laser stripes in each image to the single-line structured light plane are theoretically equal and zero. However, in practice, due to the error in extracting the center point of the laser stripes, the distances from the intersection to the single-line structured light plane are close but not equal. Therefore, a threshold Td is set. If the difference in the distances from two intersection points to the single-line structured light plane is less than Td, the pair of intersection points are judged to be in the same plane; otherwise, the pair of intersection points are judged to be in two different planes.

[0179] After calculating the element values ​​in the matrix Dis, enter the loop and use the elements in the first row of the matrix Subtract the elements in the second row of the matrix Dis in column order Determine whether and Make this difference less than Td, if it exists, save it and And continue to search the next line Determine whether and The difference between them is less than Td, and the process is repeated until the last row is found. make where [n1,n2,…,n M ] is the correct light plane number of each intersection point. So far, each laser stripe intersection point in an image is matched to the light plane of the multi-line structured light to which it belongs.

[0180] S42: Use the matched laser stripe intersection points to determine the light plane equation of the single-line structured light, and obtain the three-dimensional data of the shape of the object to be measured through coordinate transformation. Since the camera shoots the planar target when the single-line structured light emitter is moving, each image corresponds to the single-line structured light emitter at a different position. Among the four light plane parameters A0, B0, C0, and D0 of the single-line structured light at different positions, only D0 will change. To obtain the light plane parameter D0 corresponding to each image, after matching the laser stripe intersection points in each image to the correct light plane, calculate the three-dimensional coordinates of the laser stripe intersection points in each image in the camera coordinate system. The calculation formula is as follows:

[0181]

[0182] Where: x m 、y m and z m Respectively represent the horizontal coordinate, vertical coordinate and optical axis coordinate of the laser stripe intersection in the camera coordinate system; u m and v m A represents the horizontal and vertical coordinates of the mth laser stripe intersection in the image coordinate system; m 、B m 、C m and D m They respectively represent the first, second, third and fourth parameters of the light plane of the multi-line structured light to which the intersection point belongs.

[0183] After calculating the three-dimensional coordinates of the intersection of the laser stripes in each image in the camera coordinate system, the Ransac algorithm is used to fit the light plane parameter D0 of the single-line structured light. Since the light plane parameters A0, B0, and C0 of the single-line structured light emitter do not change during the straight-line movement, the light plane parameters of all single-line structured lights at each shooting moment are obtained after calculating D0.

[0184] According to the camera coordinate transformation model and the light plane equation of single-line structured light, the three-dimensional coordinates of the center point of the longitudinal laser stripe in the image are calculated. The calculation formula is as follows:

[0185]

[0186] Where: x p 、y p and z p u respectively represent the horizontal coordinate, vertical coordinate and optical axis coordinate of the center point of the longitudinal laser stripe in the camera coordinate system; p and v p represents the horizontal coordinate and vertical coordinate of the longitudinal laser center point in the image coordinate system; A0, B0, C0 and D0 represent the first, second, third and fourth parameters of the light plane of the single-line structured light.

[0187] S43: The three-dimensional coordinates of the center point of the single-line structured light projection laser stripe in the image in the camera coordinate system can all be calculated, that is, the dense point cloud data of all areas on the surface of the object being measured that are scanned by the single-line structured light are obtained, and the surface shape feature measurement of the object being measured is completed.

[0188] The second aspect of the present invention proposes a measuring device for measuring the surface shape of an object based on a method of measuring the surface shape of an object using structured light, which can realize three-dimensional measurement of the surface shape of an object. The device includes a camera 1, a single-line structured light emitter 5, a multi-line structured light emitter 4, a linear module 7, a central mounting frame 2, a single-line structured light mounting base 6, a multi-line structured light mounting base 3, a front housing 9, and a rear housing 8; Figure 4 FIG. 1 is a schematic diagram of the overall structure of a measuring device according to an embodiment of the present invention; Figure 5 The figure shows the structure diagram of the measuring device after the front shell is removed according to the embodiment of the present invention. The components are centered on the central mounting frame 2 and are directly or indirectly connected thereto. The front shell 9 and the rear shell 8 envelop the outside.

[0189] The camera 1 is fixedly mounted on the top protruding frame of the central mounting frame 2 through a threaded connection, collects images of the surface of the object to be measured and the structured light, and performs three-dimensional measurement of points on the surface of the object to be measured with the assistance of the structured light.

[0190] The single-line structured light emitter 5 is first fixedly mounted on the single-line structured light mounting base 6 through a threaded connection. The single-line structured light mounting base 6 then fixes itself and the single-line structured light emitter 5 on the moving end of the linear module 7 through a threaded connection so that the single-line structured light emitter 5 and the single-line structured light emitter 5 move linearly with the moving end of the linear module 7. The single-line structured light emitter 5 is the only moving component in the surface shape three-dimensional measurement device, assisting the camera 1 in performing three-dimensional measurement.

[0191] The multi-line structured light emitter 4 is first fixedly mounted on the multi-line structured light mounting base 3 through a threaded connection. The multi-line structured light mounting base 3 then fixes itself and the multi-line structured light emitter 4 on the protruding frame at the bottom end of the central mounting frame 2 through an interference fit. The role of the multi-line structured light is to replace the encoder displacement sensor to position the single-line structured light, which is the key to improving measurement accuracy.

[0192] The fixed end of the linear module 7 is fixedly mounted on the top flat part of the central mounting frame 2 through a threaded connection.

[0193] The central mounting frame 2 is the basis for assembling various components into a whole. Other components are fixed on the central mounting frame 2 by direct installation or with the help of the mounting frame.

[0194] The rear housing 8 is fixedly mounted on the rear end of the central mounting frame 2 by means of threaded connection.

[0195] The front housing 9 is fixedly connected to the rear housing 8 by a threaded connection, enclosing the entire central mounting frame 2 and the components mounted thereon for protection. There is a hole in the front housing 9 at the installation position of the camera 1 to expose the lens of the camera 1, and there are corresponding transparent windows at the installation positions of the multi-line structured light emitter 4 and the single-line structured light emitter 5 for the structured light to pass through.

[0196] In summary, the method of measuring the surface shape of an object based on structured light and the prediction results of its measurement device in this case have proven to be very effective.

[0197] (1) The measurement method involved in the embodiment of the present invention uses multi-line structured light to assist in calibrating the light plane parameters of single-line structured light, avoiding multiple calibrations of single-line structured light. Compared with using displacement sensors such as encoders to locate single-line structured light, the calculation results in the embodiment prove that the application of this method improves the point cloud density and measurement accuracy.

[0198] (2) The measurement method involved in the embodiment of the present invention uses a method of matching spatial points to spatial planes through a distance matrix, which reduces the complexity of the algorithm and improves the measurement speed in actual use; the embodiment proves that this method has good application effect.

[0199] (3) The measuring device involved in the embodiment of the present invention does not need to move the object to be measured when measuring the surface shape of the object to be measured, and can be used for measuring the surface shape of large components; the overall size of the measuring device is small, and it is convenient to be installed at the end of a robotic arm to measure the surface shape of large components; the introduction of the device in the embodiment proves that the device can measure the surface shape of an object.

[0200] The embodiments described above are merely descriptions of preferred implementations of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the scope of protection determined by the claims of the present invention.

Claims

1. A method for measuring the surface shape of an object based on structured light, characterized in that: Using multi-line structured light to assist in determining the light plane parameters of single-line structured light includes the following steps: Step 1: Build a measuring device for three-dimensional measurement of the surface shape of an object; The measuring device includes a camera, a single-line structured light emitter, a multi-line structured light emitter and a linear module; Step 2: Determine the optical axis setting angle of the single-line structured light emitter in the measurement device; The optical axis setting angle is the angle between the optical axis of the single-line structured light emitter and the baseline; Step 21: Determine the effect of the setting angle on depth resolution; Establish the conversion relationship between the coordinates of the point on the surface to be measured projected by the single-line structured light in the camera coordinate system and the coordinates of the projection point on the camera imaging plane in the camera coordinate system. The depth resolution obtained by taking the partial derivative of z with respect to x' is as follows: Where: represents the depth resolution; f represents the focal length of the camera; d represents the distance between the optical center of the single-line structured light emitter and the optical center of the camera; θ represents the angle between the optical axis of the single-line structured light emitter and the baseline; represents the partial derivative of the horizontal coordinate of the projection point of the single-line structured light projected onto the surface to be measured on the camera imaging plane in the camera coordinate system; cot represents the cotangent function; x' represents the horizontal coordinate of the projection point of the single-line structured light projected onto the surface to be measured on the camera imaging plane in the camera coordinate system; Step 22: Determine the effect of the setting angle on the accuracy of laser stripe center extraction; Analyze and extract the deviation e between the theoretical derivative calculated value and the actual differential calculated value of the laser stripe center u =u c -u', laser stripe center extraction error e u The method to obtain is as follows: Where: e u represents the laser stripe center extraction error; u c represents the theoretical derivative calculated value of the laser stripe center; u' represents the actual differential calculated value of the laser stripe center; G u Represents the first-order derivative of the light intensity distribution law; G uu Represents the second-order derivative of the light intensity distribution law; u0 represents the horizontal coordinate of the pixel with the maximum light intensity in the width direction of the laser stripe; g'(u0) represents the first-order difference at the position of maximum light intensity; g"(u0) represents the second-order difference at the position of maximum light intensity; Step 23: Determine the effect of the optical axis setting angle of the single-line structured light on the accuracy of the measuring device and obtain the value of the setting angle; Get the depth resolution determined in step 21 and the laser stripe center extraction error e determined in step 22 u , the method for obtaining the comprehensive error e of the measuring device is as follows: Where: e represents the comprehensive error of the measuring device; The computer draws a comprehensive error curve with the optical axis setting angle θ of the single-line structured light as the horizontal coordinate and the comprehensive error e of the measuring device as the vertical coordinate. The angle θ corresponding to the minimum value of the comprehensive error curve is taken from the graph as the value used when the measurement system is installed; Step 3: Calibrate the measuring device and use it to obtain the laser stripe image on the object surface; Adjust the structural parameters of the measurement system; use the measurement device to obtain an image of the planar target and calibrate the light plane parameters of the structured light; turn on the measurement device to scan the object to be measured and obtain the laser stripe image on the object surface; Step 4: Extract the center point of the laser stripe in the image and obtain the three-dimensional point cloud data of the surface of the object to be measured through coordinate transformation; Step 41: Extract laser stripe intersections in the image and perform light plane matching on the laser stripe intersections in the image; construct a distance matrix between the laser stripe intersections and the light planes of the single-line structured light; set a threshold and perform a cyclic search to match the light plane of the multi-line structured light to which each laser stripe intersection belongs; Step 42: Use the matched laser stripe intersection points to determine the light plane equation of the single-line structured light, and obtain three-dimensional data of the shape of the object to be measured through coordinate transformation; Step 43: Obtain the three-dimensional coordinates of the center point of the single-line structured light projected laser stripe in the image in the camera coordinate system, that is, obtain dense point cloud data of all areas on the surface of the object being measured that are scanned by the single-line structured light, and complete the three-dimensional measurement of the surface shape of the object being measured.

2. The method for measuring the surface shape of an object based on structured light according to claim 1, characterized in that: The conversion relationship between the coordinates of the point on the surface to be measured projected by the single-line structured light in the camera coordinate system and the coordinates of the projection point on the camera imaging plane in the camera coordinate system in step 21 is specifically established as follows: According to the coordinate transformation relationship of camera imaging, the coordinates of the point where the single-line structured light is projected onto the surface to be measured in the camera coordinate system are x, y, z, and the coordinates of the projection point on the camera imaging plane in the camera coordinate system are x', y', f. The transformation relationship between the two coordinate systems is as follows: Where: x, y, and z represent the horizontal coordinate, vertical coordinate, and coordinate along the optical axis of the point projected by the single-line structured light onto the surface to be measured in the camera coordinate system, respectively; x' and y' represent the horizontal and vertical coordinates of the projection point of the single-line structured light onto the surface to be measured on the camera imaging plane, respectively.

Images