Brake shoe thickness measuring method and system based on 3D machine vision analysis

Abstract

The invention belongs to the technical field of brake shoe detection, and particularly relates to a brake shoe thickness measurement method and system based on 3D machine vision analysis, and the method comprises the steps: obtaining a reference point cloud picture and a reference envelope box of a brake shoe, and calibrating a thickness measurement direction on the brake shoe; obtaining a current point cloud picture of the brake shoe, registering the current point cloud picture with the reference point cloud picture, extracting a brake shoe region from the current point cloud picture by using the reference envelope box, and carrying out clustering denoising to obtain a to-be-detected brake shoe region point cloud; segmenting and clustering point clouds obtained by routing inspection; projecting the segmented and clustered point cloud to a two-dimensional plane, and carrying out ROI value calculation on the two-dimensional projection of the point cloud of the brake shoe region as an actual brake shoe point cloud; extracting a brake shoe plane point cloud from the actual brake shoe point cloud; and according to the calibrated thickness measurement direction, searching a point pair with the farthest distance in the brake shoe plane point cloud, and calculating the distance as the brake shoe thickness. The method is high in measurement precision, low in requirement for in-place shooting precision and capable of being deployed on a mobile robot.

Description

Technical Field

[0001] The present invention belongs to the technical field of brake shoe detection, and in particular relates to a brake shoe thickness measurement method and system based on 3D machine vision analysis. Background Art

[0002] During the train inspection process, the current mainstream solution for measuring brake shoe thickness is to extract the brake shoe edge line through the edge in the color image, and then obtain the brake shoe thickness through binocular vision or approximate methods. For this method, first of all, color images are strongly affected by light, and the brake shoe color of trains with a long running time is often very close to the wheel color, which can easily lead to problems such as unclear edges and inability to extract correctly. Secondly, because binocular sight or approximate methods are used, fixed-point cameras are generally required to be deployed (to ensure that the position does not change). In the actual train inspection process, cameras on robots are often required (that is, they cannot be fixed), so problems are easy to occur, and the thickness calculation accuracy using binocular vision or approximate methods is low, often at the centimeter level. Summary of the invention

[0003] Purpose of the invention: The purpose of the present invention is to address the deficiencies in the prior art and provide a brake shoe thickness measurement method and system based on 3D machine vision analysis, which has high accuracy and is easy to deploy.

[0004] Technical solution: The brake shoe thickness measurement method based on 3D machine vision analysis of the present invention includes a calibration step and a detection step; The calibration step comprises: S1 uses the robot to obtain the reference point cloud and reference envelope box of the brake shoe, and calibrates the thickness measurement direction on the brake shoe; The detection step comprises: S2. Using the robot to obtain the current point cloud image of the brake shoe, aligning the current point cloud image with the reference point cloud image, extracting the brake shoe area from the current point cloud image using the reference envelope box, and performing clustering denoising to obtain the point cloud of the brake shoe area to be detected; S3, segmenting and clustering the point cloud obtained from the inspection; S4, projecting the segmented and clustered point cloud onto a two-dimensional plane, and calculating the region of interest (ROI) value with the two-dimensional projection of the brake shoe area point cloud, and selecting the point cloud corresponding to the maximum ROI value as the actual brake shoe point cloud; S5, extracting brake shoe plane point cloud from the actual brake shoe point cloud; S6. According to the calibrated thickness measurement direction, find the point pair with the longest distance in the brake shoe plane point cloud, and calculate their distance as the brake shoe thickness.

[0005] To further improve the above technical solution, S1 includes: S1.1. The robot moves to the brake shoe inspection point, faces the brake shoe, and takes visible light images and depth images; S1.2, marking the brake shoe position in the visible light image to obtain the coordinates of the marking frame; S1.3, extracting the depth information of the corresponding area from the depth image according to the coordinates of the annotation box, and converting it into a reference point cloud map; S1.4. Calculate the envelope box of the reference point cloud image as the reference envelope box; S1.5. Mark two points on the brake shoe to determine the direction of thickness measurement.

[0006] Further, the S2 includes: S2.1. The robot moves to a new inspection point, faces the brake shoe, takes a depth map, and generates a current point cloud map. S2.2, downsampling the reference point cloud image and the current point cloud image obtained in S1; S2.2. Use the ICP registration algorithm to register the downsampled point cloud image. The ICP algorithm obtains the corresponding point pairs between the source point cloud and the target point cloud, constructs a rotation and translation matrix based on the corresponding point pairs, and uses the obtained matrix to transform the source point cloud to the coordinate system of the target point cloud. It estimates the error function of the transformed source point cloud and the target point cloud. If the error function value is greater than the threshold, the above operation is iterated until the given error requirement is met.

[0007] S2.3, extract the rotation matrix P from the transformation matrix obtained by ICP registration, and use the inverse matrix of P to rotate the current point cloud image to the same coordinate system as the reference point cloud image; S2.4. Use the KNN clustering algorithm based on Euclidean distance to cluster the brake shoe area point cloud, remove outliers, and select the largest category as the brake shoe point cloud area to be detected. The KNN algorithm clustering includes: (1) selecting a data point as the center point of the first cluster; (2) using the spatial Euclidean distance as the calculation value to calculate the distance from all other data points to the center point; (3) selecting the K data points closest to the center point and assigning them to the cluster; repeating steps (2) and (3) until all data points are assigned to a cluster.

[0008] Furthermore, the S2.2 includes the following steps: (1) for each point in the reference point cloud map, find the nearest point in the current point cloud map to form a corresponding point pair; (2) construct a rotation and translation matrix based on the corresponding point pair, including a rotation matrix and a translation vector; (3) apply the rotation and translation matrix constructed based on the corresponding point pair in the current point cloud to make it coincide with the reference point cloud as much as possible; (4) estimate the error function between the current point cloud and the reference point cloud after transformation. If the error function value is greater than a threshold, iterate steps (1) to (3) until the given error requirement is met.

[0009] Furthermore, the S2.4 includes the following steps: (1) selecting a data point as the center point of the first cluster; (2) using the spatial Euclidean distance as the calculation value, calculating the distances from all other data points to the center point; (3) selecting the K data points closest to the center point and assigning them to the cluster; repeating steps (2) and (3) until all data points are assigned to a cluster.

[0010] Furthermore, in S3, the brake shoe area point cloud is clustered based on a clustering algorithm based on color differences of visible light images and depth differences of point cloud images.

[0011] Furthermore, the brake shoe area point cloud obtained by S2 is projected onto a 2D plane to obtain a two-dimensional bounding box; the point cloud after segmentation and clustering by S3 is projected onto a 2D plane to obtain a point cloud bounding box, and the ROI value is calculated with the two-dimensional bounding box obtained by S2, and the point cloud with the maximum ROI value is used as the actual brake shoe point cloud.

[0012] Furthermore, the brake shoe plane point cloud is extracted by the plane extraction algorithm. The plane extraction uses the classic RANSEC plane extraction: randomly select 3 points and calculate the sampling plane formed by the 3 points; calculate the distance from all points to the plane according to the sampling plane, and divide the points into inner points and outer points according to the parameter (distance threshold); count the number of inner points and outer points, and update the maximum number of iterations; repeat the above process until the stop condition is reached.

[0013] Further, S6 includes: using S2 to obtain the thickness measurement direction as the measurement direction, segmenting along the direction perpendicular to the measurement direction, measuring the difference value in the measurement direction in the segment, taking the largest difference value as the length of the segment, sorting all the segments by length, and taking 2 / 3 as the brake shoe thickness.

[0014] A brake shoe thickness measurement system for implementing the above method comprises: a robot, a depth camera, a processor and a memory, wherein the memory stores a computer program, and the above method steps are implemented when the program is executed by the processor.

[0015] Beneficial effects: Compared with the prior art, the advantages of the present invention are: 1) the 3D point cloud measurement accuracy is high, which can reach the millimeter level; 2) the segmentation uses color information and depth information, and is less affected by lighting; 3) the requirements for the accuracy of in-place shooting are not high, and it can be deployed on a mobile robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of the implementation of the method in the present invention; Figure 2 It is a schematic diagram of the calibration process; Figure 3 It is a schematic diagram of the brake shoe extraction area; Figure 4It is a schematic diagram of the segmented point cloud color block; Figure 5 It is a schematic diagram of the point cloud after plane extraction; Figure 6 is a schematic diagram of thickness measurement. DETAILED DESCRIPTION

[0017] The technical solution of the present invention is described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the embodiments.

[0018] Embodiment 1: The present invention extracts brake shoe point cloud through color change and depth change, and extracts brake shoe thickness based on point cloud information.

[0019] like Figure 1 In the process shown, S1 is the calibration step, and S2-S6 are the actual inspection steps.

[0020] S1: The robot moves to the brake shoe inspection point, faces the brake shoe, takes a visible light image and a depth image, and marks the position of the brake shoe in the visible light image. The position of the marking box is { }, The X-axis and Y-axis coordinates of the upper left corner and the lower right corner of the annotation box are respectively. Using the annotation box information, the depth of the corresponding position of the envelope box in the depth map can be extracted and converted into a point cloud map as the reference point cloud map. By calculating the envelope box of the reference point cloud map { } , The minimum values ​​of the X-axis, Y-axis, and Z-axis in the point cloud, and the maximum values ​​of the X-axis, Y-axis, and Z-axis, respectively. This envelope box will be used as the reference envelope box, and two points { }{ }, the direction of the connected line is used as the direction of brake shoe thickness measurement, see Figure 2 .

[0021] S2: The robot moves to the brake shoe, faces the brake shoe, and takes a 3D point cloud image. Due to the instability of the robot, it is impossible to ensure that the two point clouds are in the same perspective, so the reference point cloud obtained by S1 is used to align with the current point cloud. Considering the number of point clouds, the two point cloud images are first downsampled, and then the alignment algorithm uses ICP (Iterative Closet Point). Considering stability, the point cloud of S1 is rotated to the point cloud of step 2. Let the rotation matrix be P, and the current point cloud is rotated to the point cloud of step 2 using the inverse matrix of P. The brake shoe area in the current point cloud is framed by the reference envelope box obtained in step 1. Considering the noise problem, KNN clustering based on Euclidean distance is used to remove outliers, and the largest class is used as the brake shoe area to be detected, see Figure 3 .

[0022] S3: Use the color difference of color images and the depth difference of point cloud images to segment and cluster the point cloud information obtained from the inspection, see Figure 4 .

[0023] S3.1 Data preparation Point cloud data acquisition: Get point cloud data, including the spatial coordinates of each point And the corresponding color information (such as RGB value).

[0024] Data preprocessing: Filter the point cloud and remove outliers (e.g., through statistical filtering) to improve the quality of subsequent processing.

[0025] S3.2 Feature extraction Feature definition: Define a feature vector for each point, including position coordinates and color information: ; Feature normalization: Normalize the features to make the influence of different features consistent.

[0026] S3.3, parameter selection: Set two main parameters of the DBSCAN algorithm: : The maximum distance of the neighborhood, which defines the distance threshold between points.

[0027] minPts: The minimum number of points required to form a cluster.

[0028] S3.4. DBSCAN algorithm execution Construct an adjacency graph: For each point, calculate all points in its neighborhood (those with a distance less than point).

[0029] To calculate the distance based on the feature vector (position and color), you can use the Euclidean distance: Mark core points: traverse all points and identify core points (number of points in the neighborhood ≥ minPts).

[0030] Clustering process: Starting from the core point, mark the points that belong to the same cluster: mark all points in the neighborhood as the same cluster; repeat this process for the points in the neighborhood until all reachable points are marked.

[0031] Processing boundary points and noise points: Mark unlabeled core points and points in their neighborhood as boundary points; mark isolated points (points not labeled by any cluster) as noise points.

[0032] S3.5 Results Analysis Number of clusters: Calculate the number of clusters and the number of points in each cluster in the clustering results.

[0033] Cluster visualization: Use visualization tools to display point cloud segmentation results for easy analysis and verification.

[0034] Cluster feature extraction: Extract features (such as center position, shape, volume, etc.) from each cluster to provide information for subsequent processing.

[0035] S3.6 Post-processing Cluster merging or splitting: Merge similar clusters or split too small clusters as needed and output the results.

[0036] S4: Project the brake shoe area point cloud obtained in step 2 to the 2D plane, calculate the envelope box according to the projection information, project the segmented and clustered point clouds obtained in step 3 to the 2D plane respectively, obtain a list of boxes, calculate the Region of Interest (ROI) value with the box obtained by projecting the point cloud to the 2D plane in step 2, and take the point cloud with the maximum ROI value as the actual brake shoe point cloud.

[0037] S5: Brake shoe plane point cloud through plane extraction algorithm, see Figure 5 The plane extraction uses the classic RANSEC plane extraction: randomly select 3 points and calculate the sampling plane formed by the 3 points; calculate the distance from all points to the plane according to the sampling plane, and divide the points into inner points and outer points according to the parameter (distance threshold); count the number of inner points and outer points, and update the maximum number of iterations; repeat the above process until the stopping condition is reached.

[0038] S6: Use the thickness measurement direction obtained in step 1 as the measurement direction, divide the segment along the perpendicular direction of the measurement direction, measure the difference value in the measurement direction in the segment, take the largest difference value as the length of the segment, sort all the segments by length, and take 2 / 3 as the brake shoe thickness. By taking the value at 2 / 3, it is helpful to obtain a more representative thickness, and at the same time, it can effectively reduce the impact of abnormal values ​​in the measurement process on the final thickness, ensuring that the measured thickness is more stable and reliable.

[0039] S6.1. Calculate the vector direction of the measurement direction using the two marked points according to the markings in S1. Normalize the direction vector: , yes for subsequent processing.

[0040] S6.2. Segment processing Set the segment length, assuming that each segment is L (can be adjusted as needed), segment along the measurement direction, and define the starting and ending points of the segment. Assume that the measurement length is D and the number of segments is : ; S6.3. Depth Difference Calculation After plane extraction, the Z axis of each segment is considered to be consistent. Each segment is traversed and the Xmin, Ymin, Xmax, and Ymax of each segment are calculated. The depth difference value of each segment is:

[0041] S6.4. Thickness value Store the depth difference values ​​of all segments in the list depth_differences; Sort depth difference values ; Take the length after sorting The final thickness is taken as Calculate the index: ; Value: .

[0042] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes in form and details may be made without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A brake shoe thickness measurement method based on 3D machine vision analysis, characterized in that: It includes a calibration step and a detection step; The calibration step comprises: S1. Use the robot to obtain the reference point cloud and reference envelope box of the brake shoe, and calibrate the thickness measurement direction on the brake shoe; The detection step comprises: S2. Use the robot to inspect the brake shoe, obtain the current point cloud image of the brake shoe, align the current point cloud image with the reference point cloud image, extract the brake shoe area from the current point cloud image using the reference envelope box, and perform clustering and denoising to obtain the point cloud of the brake shoe area to be inspected; S3, segment and cluster the current point cloud image obtained in S2; S4, projecting the segmented and clustered point cloud onto a two-dimensional plane, and calculating the ROI value with the two-dimensional projection of the brake shoe area point cloud, and selecting the point cloud corresponding to the maximum ROI value as the actual brake shoe point cloud; S5, extracting brake shoe plane point cloud from the actual brake shoe point cloud; S6. According to the calibrated thickness measurement direction, find the point pair with the longest distance in the brake shoe plane point cloud, and calculate their distance as the brake shoe thickness.

2. The brake shoe thickness measurement method based on 3D machine vision analysis according to claim 1 is characterized in that: The S1 includes: S1.

1. The robot moves to the brake shoe inspection point, faces the brake shoe, and takes visible light images and depth images; S1.2, marking the brake shoe position in the visible light image to obtain the coordinates of the marking frame; S1.3, extracting the depth information of the corresponding area from the depth image according to the coordinates of the annotation box, and converting it into a reference point cloud map; S1.

4. Calculate the envelope box of the reference point cloud image as the reference envelope box; S1.

5. Mark two points in the brake shoe visible light image, obtain the position information of the two points in space through the corresponding depth map information, and determine the thickness measurement direction.

3. The brake shoe thickness measurement method based on 3D machine vision analysis according to claim 1 is characterized in that: The S2 includes: S2.

1. The robot moves to a new inspection point, faces the brake shoe, takes a depth map, and generates a current point cloud map. S2.2, downsampling the reference point cloud image and the current point cloud image obtained in S1; S2.2, use ICP registration algorithm to register the downsampled point cloud image; S2.3, extract the rotation matrix P from the transformation matrix obtained by ICP registration, and use the inverse matrix of P to rotate the current point cloud image to the same coordinate system as the reference point cloud image; S2.

4. Use the KNN clustering algorithm based on Euclidean distance to cluster the brake shoe area point cloud, remove outliers, and select the largest category as the brake shoe point cloud area to be detected.

4. The brake shoe thickness measurement method based on 3D machine vision analysis according to claim 3 is characterized in that: The S2.2 comprises the following steps: (1) For each point in the reference point cloud, find the nearest point in the current point cloud to form a corresponding point pair; (2) Construct a rotation and translation matrix based on corresponding point pairs, including a rotation matrix and a translation vector; (3) Apply the rotation and translation matrix based on the corresponding point pairs in the current point cloud to make it coincide with the reference point cloud as much as possible; (4) Estimate the error function between the current point cloud and the reference point cloud after transformation. If the error function value is greater than the threshold, iterate steps (1) to (3) until the given error requirement is met.

5. The brake shoe thickness measurement method based on 3D machine vision analysis according to claim 3 is characterized in that: The S2.4 includes the following steps: (1) selecting a data point as the center point of the first cluster; (2) using the spatial Euclidean distance as the calculation value, calculating the distances from all other data points to the center point; (3) selecting the K data points closest to the center point and assigning them to the cluster; repeating steps (2) and (3) until all data points are assigned to a cluster.

6. The brake shoe thickness measurement method based on 3D machine vision analysis according to claim 5, characterized in that: In S3, the point cloud of the brake shoe area is clustered using a clustering algorithm based on the color difference of the visible light image and the depth difference of the point cloud image.

7. The brake shoe thickness measurement method based on 3D machine vision analysis according to claim 1, characterized in that: The brake shoe area point cloud obtained by S2 is projected onto a 2D plane to obtain a two-dimensional bounding box; the point cloud after segmentation and clustering in S3 is projected onto a 2D plane to obtain a point cloud bounding box, and the ROI value is calculated with the box obtained by projecting the point cloud in S2 onto the 2D plane, and the point cloud with the maximum ROI value is used as the actual brake shoe point cloud.

8. The brake shoe thickness measurement method based on 3D machine vision analysis according to claim 1, characterized in that: Brake shoe plane point cloud using plane extraction algorithm.

9. The brake shoe thickness measurement method based on 3D machine vision analysis according to claim 1, characterized in that: The S6 comprises: using the thickness measurement direction obtained by S2 as the measurement direction, segmenting along the direction perpendicular to the measurement direction, measuring the difference value in the measurement direction in the segment, taking the largest difference value as the length of the segment, sorting all the segments by length, and taking 2 / 3 as the brake shoe thickness.

10. A brake shoe thickness measurement system for implementing the method according to any one of claims 1 to 9, characterized in that: include: A robot, a depth camera, a processor and a memory, wherein the memory stores a computer program, which, when executed by the processor, implements the method steps described in any one of claims 1 to 9.

Images