A Multi-Sensor-Based Method and System for Measuring the Surface Dimensions of Automotive Parts

Abstract

This invention belongs to the field of mechanical dimension measurement and relates to a method for measuring the dimensions of automotive parts using multi-sensor fusion. The invention utilizes a two-dimensional camera to acquire RGB images and obtains the part's contour from the images. A laser rangefinder is used to obtain the distance between the camera lens and the part. The contour dimensions are obtained through inverse camera imaging principles. Based on the part's contour and dimensions, path planning is performed, and a moving displacement platform guides a point-spectral confocal sensor to measure the surface dimensions and obtain a point cloud of the part's surface. Since the data obtained using the point-spectral confocal sensor can reach the micrometer level, the processing of the point cloud mainly addresses noise points that are close to the main point cloud data or mixed in with the effective data point set. This invention employs bilateral filtering to process such noise points. This solves the cost and accuracy problems in automotive part dimension measurement.

Description

Technical Field

[0001] In the field of mechanical dimension measurement, specifically, it involves methods and systems for measuring the surface dimensions of automotive parts based on multiple sensors. Background Technology

[0002] Automotive parts are manufactured through molding and forming processes, followed by machining to meet dimensional and surface finish requirements. Therefore, slight dimensional errors can occur in some cases, necessitating more detailed inspection of these parts. Furthermore, due to the wide variety of automotive parts, the inspection system needs to be universally applicable.

[0003] In existing technologies, three-dimensional sensors are typically used in vehicle manufacturing to measure the surface profile of automotive parts. These sensors are used to accurately measure the surface of automotive parts. Three-dimensional intelligent sensors are mainly used in painting workshops for quality inspection of the exterior paint surface of automobiles. However, existing three-dimensional intelligent sensors are inconvenient to install and fix during use. The height and direction of the sensor cannot be adjusted as needed, thus failing to guarantee all-round detection of the part's surface profile. Furthermore, the sensors are unstable when moving, which can easily cause certain errors. Summary of the Invention

[0004] The purpose of this invention is to propose a method and system for measuring the surface dimensions of automotive parts based on multiple sensors, so as to solve one or more technical problems existing in the prior art, and at least provide a beneficial option or create conditions.

[0005] A multi-sensor-based method for measuring the surface dimensions of automotive parts, the method comprising the following steps:

[0006] Step 1: Calibrate the 2D camera and point spectral confocal sensor, and input the automotive parts;

[0007] Step 2: Use a 2D camera to acquire RGB images of the car parts, and use the adaptive Canny operator to process the images to initially extract the outer contours of the parts;

[0008] Step 3: Use b-spline interpolation to compensate for the outer contour of the automotive parts to obtain the final outer contour;

[0009] Step 4: Based on the distance between the camera lens and the car part obtained from the laser displacement sensor, solve the camera imaging formula to obtain the outer contour dimensions of the car part;

[0010] Step 5: Combine the final outer contour and the outer contour dimensions of the automotive parts, and use the Hilbert curve to generate the scan path;

[0011] Step 6: Move the car part using the micro-motion platform according to the scanning path obtained in Step 5, and use a point spectral confocal sensor to scan the car part;

[0012] Step 7: Use bilateral filtering to process the point cloud of the automotive parts to ensure the accuracy of the point cloud.

[0013] Furthermore, in step 2, RGB images of the car parts are acquired using a 2D camera, and edge detection is performed on the images using the Sobel operator to obtain an initial contour image. Based on the edge point information of the image, a guided filtering algorithm is used to blur the background and enhance the details of the car parts to achieve the effect of background blurring. The Canny algorithm is used to perform edge detection on the preprocessed image to obtain the outer contour image of the parts with some breakpoints.

[0014] Furthermore, in step 3, the obtained contour is compensated using the b-spline interpolation method, as follows:

[0015] S31: Perform equidistant resampling on the outer contour of the part, calculate the curvature distribution of each discrete point according to the following formula, and set the average curvature value as the curvature critical value to screen out the key points of the contour:

[0016]

[0017] Where, ρ i Let d be the radius of curvature at the i-th data point. i-1 d i and d i+1 Let k be the radius of the arc passing through 3 adjacent data points, and k be the curvature. i It is the reciprocal of the radius of curvature;

[0018] S32: Perform centripetal parameterization on the selected contour key points:

[0019]

[0020] Among them, Q k These are the coordinates of the kth key point. These are the parameters of the k-th keypoint, where k = 1, 2, ..., n-1;

[0021] S33: The node vector of the b-spline curve is determined using the average value of the keypoint parameters.

[0022]

[0023] S34: Generate an initial b-spline interpolation curve C(u) using the selected key points, and control the vertex P. i and b-spline basis functions N i,p (u) Establish relationship:

[0024]

[0025] Furthermore, in step 4, the distance between the camera lens and the car part is obtained using a laser rangefinder, and the dimensions of the car part's outline are calculated. This specifically includes the following steps:

[0026] S41: Obtain the imaging scale relationship of the two-dimensional camera to get tanα=h / I, where I is the distance from the lens to the imaging area and h is the height of the imaging area;

[0027] S42: The actual dimensions of automotive parts can be obtained through proportional relationships. L is the distance from the lens to the car part, and v is the size of the car part in the imaging area;

[0028] S43: Based on the deviation m between the laser rangefinder measurement point and the camera focus, the optimized formula for calculating the actual size of automotive parts is as follows:

[0029] Further, in step 6, the micro-motion platform is moved using the measurement path obtained in step 5, and the moving speed v of the micro-motion platform is calculated based on the step size and the measurement frequency of the point spectrum confocal sensor;

[0030] Micrometer-level point cloud data of the surface of automotive parts is constructed by using the distance information z of the sampling points obtained from the point spectral confocal sensor and the two-dimensional position information (x, y) of each sampling point.

[0031] Furthermore, in step 7, the point cloud is processed using bilateral filtering, and the specific steps are as follows:

[0032] S71: Use KD-trees to establish topological relationships for point cloud data, assuming the target test point cloud is T. si Its domain point is T sm T si K nearest neighbors T sm This is called the K-domain, denoted as N. K (T n );

[0033] S72: The normal vector of a point is estimated using the least squares method, and the formula is as follows:

[0034]

[0035] Where β is the Gaussian weighting parameter, d E This is the Euclidean distance between the local plane and the origin.

[0036] S73: Calculate the correlation weight function ω of the bilateral filter. c Let ω be the smoothing filter weight function. s This is the feature preservation weight function, and its formula is as follows:

[0037]

[0038] Where, σ c Let σ be the neighborhood radius. s n is the standard deviation of the nearest neighbor. si and n sm It is the normal vector between the target point and its neighboring points;

[0039] S74: After obtaining the weight values, the bilateral filter factor is calculated using the following formula:

[0040]

[0041] S75: Calculate the new point cloud coordinates using the following formula:

[0042] T' = T si +μ s ·n si .

[0043] A multi-sensor-based automotive parts surface dimension measurement system, the system includes: a vision inspection platform, a camera, a laser displacement sensor, a point spectral confocal sensor, a micro-motion platform, and a machine vision processing system. The vision inspection platform includes: a platform support plate, a fixture, a 2D / 3D camera, and a bracket.

[0044] The machine vision processing system consists of equipment with image detection and point cloud clustering capabilities.

[0045] The beneficial effects of this invention are as follows: This invention utilizes a two-dimensional camera to acquire RGB images, and obtains the outline of a part from the images. A laser rangefinder is used to obtain the distance between the camera lens and the part. The outline dimensions are obtained through the principle of inverse camera imaging. Path planning is performed based on the part outline and dimensions. A moving displacement platform guides a point spectral confocal sensor to measure the surface dimensions and obtain a point cloud of the part surface. Since the data obtained using the point spectral confocal sensor can reach the micrometer level, the processing of the point cloud mainly deals with noise points that are close to the main point cloud data or mixed in with the effective data point set. This invention uses bilateral filtering to process such noise points. This solves the cost and accuracy problems of automotive part dimension measurement. Attached Figure Description

[0046] The above and other features of the present invention will become more apparent from the detailed description of the embodiments shown in conjunction with the accompanying drawings. In the accompanying drawings, the same reference numerals denote the same or similar elements. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort. In the drawings:

[0047] Figure 1 The flowchart shown is a method for measuring the surface dimensions of automotive parts based on multiple sensors.

[0048] Figure 2 The image shows a Hilbert curve of a multi-sensor-based method for measuring the surface dimensions of automotive parts. Detailed Implementation

[0049] The following will provide a clear and complete description of the concept, specific structure, and technical effects of the present invention in conjunction with the embodiments and accompanying drawings, so as to fully understand the purpose, solution, and effects of the present invention. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0050] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0051] This invention utilizes RGB images acquired by a camera to quickly determine the contour of automotive parts. A laser displacement sensor acquires the distance between the 2D camera and the part. The outer contour dimensions of the part are calculated by inversely applying the camera imaging formula, and then the precise surface dimensions of the automotive part are obtained using the laser displacement sensor. For the problem of automotive part measurement path planning, an intelligent algorithm can be incorporated to plan the optimal measurement sequence. Main implementation process:

[0052] like Figure 1 As shown, a multi-sensor-based method for measuring the surface dimensions of automotive parts includes the following steps:

[0053] Step 1: Calibrate the 2D camera and point spectral confocal sensor, and input the automotive parts;

[0054] Step 2: Use a 2D camera to acquire RGB images of the car parts, and use the adaptive Canny operator to process the images to initially extract the outer contours of the parts;

[0055] Step 3: Use b-spline interpolation to compensate for the outer contour of the automotive parts to obtain the final outer contour;

[0056] Step 4: Based on the distance between the camera lens and the car part obtained from the laser displacement sensor, solve the camera imaging formula to obtain the outer contour dimensions of the car part;

[0057] Step 5: Combine the final outer contour and the outer contour dimensions of the automotive parts, and use the Hilbert curve to generate the scan path;

[0058] Step 6: Move the car part using the micro-motion platform according to the scanning path obtained in Step 5, and use a point spectral confocal sensor to scan the car part;

[0059] Step 7: Use bilateral filtering to process the point cloud of the automotive parts to ensure the accuracy of the point cloud.

[0060] Furthermore, in step 2, RGB images of the car parts are acquired using a 2D camera, and edge detection is performed on the images using the Sobel operator to obtain an initial contour image. Based on the edge point information of the image, a guided filtering algorithm is used to blur the background and enhance the details of the car parts to achieve the effect of background blurring. The Canny algorithm is used to perform edge detection on the preprocessed image to obtain the outer contour image of the parts with some breakpoints.

[0061] Furthermore, in step 3, the obtained contour is compensated using the b-spline interpolation method, as follows:

[0062] S31: Perform equidistant resampling on the outer contour of the part, calculate the curvature distribution of each discrete point according to the following formula, and set the average curvature value as the curvature critical value to screen out the key points of the contour:

[0063]

[0064] Where, ρ i Let d be the radius of curvature at the i-th data point. i-1 d i and d i+1 Let k be the radius of the arc passing through 3 adjacent data points, and k be the curvature. i It is the reciprocal of the radius of curvature;

[0065] S32: Perform centripetal parameterization on the selected contour key points:

[0066]

[0067] Among them, Q k These are the coordinates of the kth key point. These are the parameters of the k-th keypoint, where k = 1, 2, ..., n-1;

[0068] S33: The node vector of the b-spline curve is determined using the average value of the keypoint parameters.

[0069]

[0070] S34: Generate an initial b-spline interpolation curve C(u) using the selected key points, and control the vertex P. i and b-spline basis functions N i,p (u) Establish relationship:

[0071]

[0072] Furthermore, in step 4, the distance between the camera lens and the car part is obtained using a laser rangefinder, and the dimensions of the car part's outline are calculated. This specifically includes the following steps:

[0073] S41: Obtain the imaging scale relationship of the two-dimensional camera to get tanα=h / I, where I is the distance from the lens to the imaging area and h is the height of the imaging area;

[0074] S42: The actual dimensions of automotive parts can be obtained through proportional relationships. L is the distance from the lens to the car part, and v is the size of the car part in the imaging area;

[0075] S43: Based on the deviation m between the laser rangefinder measurement point and the camera focus, the optimized formula for calculating the actual size of automotive parts is as follows:

[0076] Preferably, such as Figure 2 As shown, various fractal curves are generated using the Lindenmayer system, the principle of which is that the system can be described as a triplet.<V,P,ω> The initial string ω is the starting point for string transformation in the L system. The system transformation rule P is applied to the initial string multiple times, and finally a very long string is generated. The fractal curve can be drawn by drawing the graphic operation assigned to each character.

[0077] <V,P,ω> The definitions of each element in the code are as follows:

[0078] V = {ZF G+-}, where Z,+,- are constants, and F,G are variables;

[0079] Initial state ω = F;

[0080] Production rules

[0081] The geometric definitions of each element in V when plotting Hilbert curves are as follows:

[0082] Z: Move forward the set distance to draw a straight line segment;

[0083] +: Changes the direction of the drawn line segment, rotating it 90° counterclockwise compared to the drawing direction of the previous line segment;

[0084] -: Change the direction of the drawn line segment, rotating it 90° clockwise compared to the drawing direction of the previous line segment;

[0085] F: Has no geometric meaning; is used for string substitution in production rule P.

[0086] G: Has no geometric meaning; is used for string replacement in production rule P.

[0087] Based on the part's outer contour dimensions and the table showing the correspondence between iteration count, step size, and side length, the number of iterations and step size for measuring the part are calculated, and the measurement path is generated.

[0088] Further, in step 6, the micro-motion platform is moved using the measurement path obtained in step 5, and the moving speed v of the micro-motion platform is calculated based on the step size and the measurement frequency of the point spectrum confocal sensor;

[0089] Micrometer-level point cloud data of the surface of automotive parts is constructed by using the distance information z of the sampling points obtained from the point spectral confocal sensor and the two-dimensional position information (x, y) of each sampling point.

[0090] Furthermore, in step 7, the point cloud is processed using bilateral filtering, and the specific steps are as follows:

[0091] S71: Use KD-trees to establish topological relationships for point cloud data, assuming the target test point cloud is T. si Its domain point is T sm T si K nearest neighbors T sm This is called the K-domain, denoted as N. K (T n );

[0092] S72: The normal vector of a point is estimated using the least squares method, and the formula is as follows:

[0093]

[0094] Where β is the Gaussian weighting parameter, d E This is the Euclidean distance between the local plane and the origin.

[0095] S73: Calculate the correlation weight function ω of the bilateral filter. c Let ω be the smoothing filter weight function. s This is the feature preservation weight function, and its formula is as follows:

[0096]

[0097] Where, σ c Let σ be the neighborhood radius. s n is the standard deviation of the nearest neighbor. si and n smIt is the normal vector between the target point and its neighboring points;

[0098] S74: After obtaining the weight values, the bilateral filter factor is calculated using the following formula:

[0099]

[0100] S75: Calculate the new point cloud coordinates using the following formula:

[0101] T' = T si +μ s ·n si .

[0102] A multi-sensor-based automotive parts surface dimension measurement system, the system includes: a vision inspection platform, a camera, a laser displacement sensor, a point spectral confocal sensor, a micro-motion platform, and a machine vision processing system. The vision inspection platform includes: a platform support plate, a fixture, a 2D / 3D camera, and a bracket.

[0103] The machine vision processing system consists of equipment with image detection and point cloud clustering capabilities.

[0104] The camera can be a common two-dimensional camera, the laser displacement sensor can be a Keyence IL-1000 series CMOS laser displacement sensor, the point spectral confocal sensor can be a STIL CL2 type spectral confocal sensor, and the micro-motion platform can be a Hanno MN-XY series precision micro-motion platform.

[0105] Although the invention has been described in considerable detail and particularly with regard to several of the described embodiments, it is not intended to limit itself to any of these details or embodiments or any particular embodiment, thereby effectively covering the intended scope of the invention. Furthermore, the invention has been described above with respect to embodiments foreseeable by the inventors in order to provide a useful description, and non-substantial modifications to the invention that have not yet been foreseen may still represent equivalent modifications.

Claims

1. A method for measuring the surface dimensions of automotive parts based on multiple sensors, characterized in that, The method includes the following steps: Step 1: Calibrate the 2D camera and point spectral confocal sensor, and input the automotive parts; Step 2: Use a 2D camera to acquire RGB images of the car parts, and use the adaptive Canny operator to process the images to initially extract the outer contours of the parts; Step 3: Use b-spline interpolation to compensate for the outer contour of the automotive parts to obtain the final outer contour; Step 4: Based on the distance between the camera lens and the car part obtained from the laser displacement sensor, solve the camera imaging formula to obtain the outer contour dimensions of the car part; Step 5: Combine the final outer contour and the outer contour dimensions of the automotive parts, and use the Hilbert curve to generate the scan path; Step 6: Move the micro-motion platform using the measurement path obtained in Step 5, and calculate the moving speed v of the micro-motion platform based on the step size and the measurement frequency of the point spectrum confocal sensor; Micrometer-level point cloud data of the automotive part surface is constructed based on the distance information z of the sampling points obtained by the point spectral confocal sensor and the two-dimensional position information (x, y) of each sampling point. Step 7: Process the point cloud of the automotive parts using bilateral filtering to ensure point cloud accuracy; In step 7, bilateral filtering is used to process the point cloud. The specific steps are as follows: S71: Use KD-trees to establish topological relationships for point cloud data, assuming the target test point cloud is... Its neighborhood points are , K nearest neighbors This is called the K-neighborhood, denoted as ; S72: The normal vector of a point is estimated using the least squares method, and the formula is as follows: ； in, The Gaussian weight parameters are... The Euclidean distance between the local plane and the origin; S73: Calculate the correlation weight function of the bilateral filter. The smoothing filter weight function is... This is the feature preservation weight function, and its formula is as follows: ； in, The neighborhood radius, The standard deviation of the nearest neighbor points. and It is the normal vector between the target point and its neighboring points; S74: After obtaining the weight values, the bilateral filter factor is calculated using the following formula: ； S75: Calculate the new point cloud coordinates using the following formula: 。 2. The method for measuring the surface dimensions of automotive parts based on multiple sensors according to claim 1, characterized in that, In step 2, RGB images of the car parts are acquired using a 2D camera. The Sobel operator is used to perform edge detection on the images to obtain an initial contour image. Based on the edge point information of the images, a guided filtering algorithm is used to blur the background and enhance the details of the car parts to achieve a background blurring effect. The Canny algorithm is used to perform edge detection on the preprocessed image to obtain the outer contour image of the parts with some breaks.

3. The method for measuring the surface dimensions of automotive parts based on multiple sensors according to claim 1, characterized in that, In step 3, the obtained contour is compensated using the b-spline interpolation method, as follows: S31: Perform equidistant resampling on the outer contour of the part, calculate the curvature distribution of each discrete point according to the following formula, and set the average curvature value as the curvature critical value to screen out the key points of the contour: ； ； in, Let be the radius of curvature at the i-th data point. , and Let be the radius of the arc passing through 3 adjacent data points, and be the curvature. It is the reciprocal of the radius of curvature; S32: Perform centripetal parameterization on the selected contour key points: ； ； in, These are the coordinates of the kth key point. These are the parameters of the k-th keypoint, where k=1, 2, , n-1; S33: The node vector of the b-spline curve is determined using the average value of the keypoint parameters. ； ； S34: Generate an initial b-spline interpolation curve using the selected key points. By controlling the vertices and b-spline basis functions Establishing relationships: ； ； 。 4. The method for measuring the surface dimensions of automotive parts based on multiple sensors according to claim 1, characterized in that, In step 4, the distance between the camera lens and the car part is obtained using a laser rangefinder, and the dimensions of the car part's outline are calculated. This includes the following steps: S41: Obtain the imaging scale relationship of the two-dimensional camera as tanα=h / I, where I is the distance from the lens to the imaging area and h is the height of the imaging area; S42: The actual size V of the car part is obtained through proportional relationships. L is the distance from the lens to the car part, and V is the size of the car part in the imaging area; S43: There is a deviation m between the laser rangefinder's measurement point and the camera's focus. The optimized formula for calculating the actual dimensions of automotive parts is: V= .

Images