Method and system for three-dimensional detection of residual thickness of locomotive brake pad based on heterogeneous coordinate depth sampling

Abstract

The locomotive brake pad residual thickness three-dimensional detection method and system based on heterogeneous coordinate depth sampling disclosed by the application acquire a color image, an original depth image and a camera internal parameter of brake pad friction material to be detected, perform target segmentation on the color image to obtain a binary mask of the brake pad friction material, and generate a dense measurement depth map aligned with pixels of the color image; then, the main shaft direction and the thickness scanning direction of the brake pad are determined according to the binary mask, two side boundary pixel points are determined, and the corresponding safe depth value of the position or region is read; finally, the outer boundary pixel coordinates and the internal safe depth value are combined, the Euclidean distance between the three-dimensional boundary points on the two sides is calculated as the residual thickness of a single scanning line, the thickness values of multiple scanning lines are abnormally removed and statistically fused, and the residual thickness of the brake pad friction material is output, so that the influence of edge depth missing, depth completion edge smoothing artifacts and brake pad inclined posture on thickness measurement can be reduced.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, robot vision inspection, and industrial 3D measurement technology, and more specifically, to a 3D detection method and system for the remaining thickness of locomotive brake pads based on heterogeneous coordinate depth sampling. Background Technology

[0002] Brake pads are critical wear components in train braking systems, and the remaining thickness of their friction material directly affects braking performance, maintenance cycles, and operational safety. In maintenance scenarios, traditional manual caliper measurements rely on manual approach and contact measurement of the brake pads, resulting in low efficiency, insufficient repeatability, and difficulty in integrating with automated maintenance processes. With the development of robot vision and RGB-D imaging equipment, using a robot's end-effector camera for non-contact measurement of brake pads has become a feasible approach.

[0003] However, in actual locomotive brake pad inspection scenarios, the edges of brake pad friction materials are often subject to complex factors such as wear, chamfering, contamination, dark areas, and metal reflections, which leads to the following shortcomings in vision-based thickness measurement.

[0004] 1. Physical depth sensors lack stability in edge regions. Structured light, binocular, or active depth cameras are prone to depth loss, flying spots, or boundary depth jitter at object edges, reflective areas, dark areas, and locations of abrupt changes in foreground and background depth. This can lead to unreliable 3D coordinates when reading depth values ​​directly from pixels at the edge of the sensor.

[0005] 2. Simple two-dimensional pixel measurement is insufficient to compensate for spatial pose errors. Some vision measurement methods detect the two sides of the gate and then multiply the two-dimensional pixel distance by a calibrated scale to obtain the remaining thickness. These methods assume that the gate measurement surface is approximately parallel to the imaging plane. When the robot's end effector pose is skewed, or when the gate is tilted relative to the camera, a deviation will occur between the two-dimensional projected distance and the actual three-dimensional thickness.

[0006] 3. Depth completion models may produce smooth transition artifacts at absolute edges. Existing depth completion networks can generate dense depth maps from color images and sparse or incomplete depth maps. However, at the boundary between the gate and the background, the depth values ​​output by the model may mix the depth information of the foreground and background, forming a smooth transition region. If the depth value is read directly from the outer boundary of the segmentation mask, this transition depth may be mistakenly identified as the true edge depth of the gate.

[0007] 4. Using circumscribed rectangles or bounding boxes as measurement boundaries is not well-suited for irregular edges. The visible area of ​​the brake pad may have local gaps, end chamfers, obstructions, or irregular contours. If the width of the rectangle is used directly as the thickness reference, it cannot accurately reflect the physical distance between the actual friction material boundaries. Summary of the Invention

[0008] To address at least one of the aforementioned technical problems, this invention proposes a three-dimensional detection method and system for the remaining thickness of locomotive brake pads based on heterogeneous coordinate depth sampling.

[0009] The first aspect of this invention provides a three-dimensional detection method for the remaining thickness of a locomotive brake pad based on heterogeneous coordinate depth sampling, comprising the following steps:

[0010] Acquire a color image of the friction material of the brake pad of the locomotive under test, an original depth image registered with the color image, and camera intrinsic parameters;

[0011] The color image is segmented to obtain a binary mask corresponding to the friction material of the locomotive brake pad under test;

[0012] A dense metric depth map aligned with the pixels of the color image is generated based on the color image and the original depth image;

[0013] The principal axis direction of the brake pad friction material of the locomotive under test and the thickness scanning direction orthogonal to the principal axis direction are determined according to the binary mask, and multiple scanning lines are generated along the principal axis direction within the effective measurement range;

[0014] For each scan line, determine the first boundary pixel and the second boundary pixel formed by the intersection of the scan line and the real boundary of the binary mask;

[0015] Using the first boundary pixel and the second boundary pixel as references, offset by a preset distance along the direction pointing into the binary mask, the first safe sampling position and the second safe sampling position are obtained;

[0016] In the dense metric depth map, the first safe depth value and the second safe depth value corresponding to the first safe sampling position and the second safe sampling position are read respectively;

[0017] The pixel coordinates of the first boundary pixel are combined with the first safe depth value, and the pixel coordinates of the second boundary pixel are combined with the second safe depth value. The first three-dimensional boundary point and the second three-dimensional boundary point are obtained by back-projection based on the camera intrinsic parameters.

[0018] Calculate the three-dimensional Euclidean distance between the first three-dimensional boundary point and the second three-dimensional boundary point, and use it as the measurement value of the remaining thickness of the brake pad friction material corresponding to the scan line;

[0019] Outlier removal and statistical fusion are performed on the remaining thickness measurements corresponding to multiple scan lines to obtain the final remaining thickness of the brake pad friction material of the locomotive under test.

[0020] In a preferred embodiment of the present invention, the color image and the original depth image are acquired by an RGB-D camera installed on the robot end effector, and the robot end effector drives the RGB-D camera to a preset shooting pose of the brake pad of the locomotive under test.

[0021] In a preferred embodiment of the present invention, target segmentation of the color image includes: inputting the color image into an instance segmentation network to obtain a candidate mask, performing connected component analysis on the candidate mask, and retaining the connected components corresponding to the main body region of the brake pad friction material according to the area, position or shape constraints of the connected components to obtain a binary mask.

[0022] In a preferred embodiment of the present invention, the dense metric depth map is generated according to a depth completion network, which takes at least two of the following as inputs: the color image, the original depth image, and a depth validity mask for characterizing the validity of the original depth, and outputs a dense depth map with physical scale and aligned with the pixels of the color image.

[0023] In a preferred embodiment of the present invention, the deep completion network includes a deep completion network based on a convolutional neural network, a visual Transformer, a cross-attention mechanism, or a mask depth modeling mechanism.

[0024] In a preferred embodiment of the present invention, the principal axis direction is obtained by performing minimum bounding rectangle fitting or principal component analysis on the binary mask; the effective measurement interval is determined by removing the end chamfered regions, occlusion regions or unstable boundary regions at both ends of the binary mask in the principal axis direction.

[0025] In a preferred embodiment of the present invention, the preset distance is a fixed pixel distance or an adaptive pixel distance; the adaptive pixel distance is based on at least one of the following: the bandwidth of the original depth invalid pixels in the boundary neighborhood, the depth gradient bandwidth of the dense metric depth map, or the pixel span of the current scan line within the binary mask.

[0026] In a preferred embodiment of the present invention, when reading the first safe depth value and the second safe depth value, directional local windows are constructed with the corresponding safe sampling positions as the center. The size of the directional local window along the main axis is larger than the size along the thickness scanning direction, and the effective sampling pixels in the directional local window are constrained to fall within the erosion area of ​​the binary mask. The median, truncated mean, or weighted median of the effective depth values ​​in the directional local window is used as the corresponding safe depth value.

[0027] In a preferred embodiment of the present invention, when performing outlier removal and statistical fusion, the median and median absolute deviation of the remaining thickness measurements corresponding to multiple scan lines are calculated, and thickness measurements with a deviation from the median greater than a preset multiple of the median absolute deviation are removed; the average, median or weighted average of the remaining thickness measurements are calculated to obtain the final remaining thickness.

[0028] A second aspect of the present invention provides a three-dimensional detection system for the remaining thickness of locomotive brake pads based on heterogeneous coordinate depth sampling, which is applied to the three-dimensional detection method for the remaining thickness of locomotive brake pads, including:

[0029] The data acquisition module is used to acquire color images, raw depth images, and camera intrinsic parameters of the friction material of the brake pads of the locomotive under test.

[0030] A mask extraction module is used to perform target segmentation on the color image to obtain a binary mask;

[0031] A depth reconstruction module is used to generate a dense metric depth map based on the color image and the original depth image;

[0032] The scan line generation module is used to determine the main axis direction and thickness scanning direction based on the binary mask, and generate multiple scan lines;

[0033] The heterogeneous sampling module is used to determine the boundary pixel points on the true boundary of the binary mask and read the safe depth value after offset along the internal direction in the dense metric depth map.

[0034] The 3D measurement module is used to perform 3D back projection based on the pixel coordinates of boundary pixels, safe depth values, and camera intrinsic parameters, and to calculate the 3D Euclidean distance.

[0035] The statistical fusion module is used to remove outliers and statistically fuse the remaining thickness measurements of multiple scan lines to obtain the final remaining thickness.

[0036] The technical solution of the present invention has the following advantages over the prior art:

[0037] 1. This invention adopts a heterogeneous coordinate depth sampling mechanism of "outer layer boundary taking, inner layer depth taking", that is, the two-dimensional pixel coordinates are taken from the true outer boundary of the binary mask, while the depth value is taken from the safe position or safe area after offset along the internal direction. In this way, while maintaining the geometric span of the boundary, the influence of edge depth loss, flying points and depth completion smoothing artifacts on the measurement results is reduced.

[0038] 2. This invention does not use the smallest bounding rectangle or enclosing box as the actual measurement boundary, but only as the directional prior of the main axis direction and the scanning direction. The actual thickness endpoint is determined by the intersection of the scan line and the true boundary of the binary mask, thus it can adapt to local irregular edges and tilted gate images.

[0039] 3. This invention combines the boundary pixel coordinates with the safety depth value and performs three-dimensional back projection, and calculates the three-dimensional Euclidean distance in the camera coordinate system, which can reduce the two-dimensional projection measurement error introduced by the attitude tilt of the gate relative to the camera imaging plane.

[0040] 4. This invention obtains the final thickness result through multi-scanline measurement and robust statistical fusion, which can reduce the impact of end chamfers, local occlusion, segmentation burrs or abnormalities of a single scanline on the final result. Attached Figure Description

[0041] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, some of the drawings in the following description are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is a module architecture diagram of the three-dimensional detection system for the remaining thickness of locomotive brake pads according to the present invention;

[0043] Figure 2 This is a flowchart illustrating the overall process of the three-dimensional detection method for the remaining thickness of locomotive brake pads according to the present invention.

[0044] Figure 3 This is a schematic diagram of the heterogeneous coordinate depth sampling mechanism of the present invention;

[0045] Figure 4 This is a schematic diagram illustrating the invention of generating multiple scan lines based on the main axis direction and skipping the end chamfer region;

[0046] Figure 5 This is a schematic diagram of the directional window midpoint sampling for the security depth of the present invention. Detailed Implementation

[0047] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0048] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0049] Example 1

[0050] See Figures 1-5 As shown, this invention proposes a three-dimensional detection system for the remaining thickness of a locomotive brake pad based on heterogeneous coordinate depth sampling. The system can be deployed on an industrial computer, an edge computing node within a robot control cabinet, a workstation, or a server, and communicates with an RGB-D camera installed on the robot's end effector. The RGB-D camera can be an Intel RealSense camera, or other structured light, binocular, ToF, or depth cameras capable of outputting both color and depth images.

[0051] The system includes a data acquisition module 101, a mask extraction module 102, a depth reconstruction module 103, a scan line generation module 104, a heterogeneous sampling module 105, a three-dimensional measurement module 106, a statistical fusion module 107, and a result output module 108. The modules are connected through data interfaces or control interfaces to form a complete gate plate remaining thickness detection link.

[0052] The data acquisition module 101 is used to acquire color images, raw depth images, and camera intrinsic parameters of the friction material of the brake pads of the locomotive under test. The color images and raw depth images are registered in a pixel coordinate system, or mapped to the same image coordinate system through camera calibration parameters and extrinsic parameter transformation. The camera intrinsic parameters include focal lengths f_x and f_y, and principal point coordinates c_x and c_y.

[0053] The mask extraction module 102, connected to the data acquisition module 101, is used to segment the color image to obtain a binary mask corresponding to the friction material of the brake pad under test. This module can use YOLO series instance segmentation models, semantic segmentation models, or other models that can output target region masks. To reduce the influence of background noise and small error regions, the mask extraction module 102 can also perform connected component analysis on the candidate masks, retaining the connected component with the largest area or that meets the preset shape and position constraints as the main region of the brake pad friction material.

[0054] The depth reconstruction module 103, connected to the data acquisition module 101, is used to generate a dense metric depth map based on the color image and the original depth image. The original depth image may contain depth holes, edge flypoints, or noise points. The depth reconstruction module 103 completes and refines the original depth using a depth completion network, outputting a dense depth map that is pixel-aligned with the color image and retains its physical scale. In one specific embodiment, the depth completion network can call the LingBot-Depth open-source model for inference; in other embodiments, a depth completion model based on convolutional neural networks, visual Transformers, cross-attention mechanisms, or masked depth modeling mechanisms can also be used. It should be noted that this invention is not limited to a specific model name; the key is to utilize the dense metric depth map in conjunction with a subsequent heterogeneous sampling mechanism for 3D thickness measurement.

[0055] The scan line generation module 104 is used to determine the principal axis direction and thickness scanning direction of the brake pad friction material based on the binary mask. Specifically, it can perform minimum bounding rectangle fitting on the binary mask, using the long side direction of the rotation rectangle as the principal axis direction and the short side direction or a direction orthogonal to the principal axis direction as the thickness scanning direction. Alternatively, it can perform principal component analysis on the foreground pixels of the mask, using the first principal component direction as the principal axis direction and the second principal component direction as the thickness scanning direction.

[0056] The heterogeneous sampling module 105 is used to perform boundary localization and safe depth reading for each scan line. This module first determines the boundary pixels on both sides of the true boundary of the binary mask; then, instead of directly reading the depth at the boundary pixels, it offsets the depth by a preset distance along the direction pointing inward of the mask to obtain the safe sampling position or safe sampling area, and reads the corresponding safe depth value in the dense metric depth map.

[0057] The three-dimensional measurement module 106 is used to combine the pixel coordinates of the boundary pixel points with the corresponding safe depth values, and perform back projection using the camera intrinsic parameters to obtain the three-dimensional boundary points on both sides; then, it calculates the three-dimensional Euclidean distance between the three-dimensional boundary points on both sides to obtain the remaining thickness of the brake friction material corresponding to a single scan line.

[0058] The statistical fusion module 107 is used to remove outliers and perform statistical fusion on the remaining thickness measurements obtained from multiple scan lines. The result output module 108 is used to output the final remaining thickness, the thickness values ​​of each scan line, outlier scan line markers, and optional visualization of the measurement results.

[0059] Example 2

[0060] like Figure 2 As shown, this embodiment, based on embodiment 1, provides a three-dimensional detection method for the remaining thickness of locomotive brake pads based on heterogeneous coordinate depth sampling, including the following steps.

[0061] Step S201: Multimodal data acquisition and registration. The robot's end effector moves the RGB-D camera to a preset shooting pose in front of the brake pad of the locomotive under test, acquiring a color image I_rgb containing the friction material of the brake pad and a raw depth image D_raw, while simultaneously reading the camera intrinsic parameter matrix K. If the color image and the depth image are not in the same coordinate system, the raw depth image is mapped to the color image coordinate system according to the camera calibration relationship.

[0062] Step S202: Extraction of brake pad friction material mask. Input the color image I_rgb into the instance segmentation network to obtain candidate masks for the brake pad friction material region. Perform binarization, connected component analysis, and necessary morphological opening and closing operations on the candidate masks to remove small noise regions in the background, and obtain the binary mask M of the main body of the brake pad friction material.

[0063] Step S203: Dense Measurement Depth Map Reconstruction. The color image I_rgb, the original depth image D_raw, and an optional depth effective mask V are input into the depth completion network to generate a dense measurement depth map D_dense. The depth effective mask V is used to mark the effective and invalid depth pixels in the original depth image. The depth completion network uses texture, edge, and semantic information from the color image to guide the repair of hole regions in the original depth image, thereby obtaining a dense depth map without obvious holes and possessing physical scale.

[0064] Step S204: Principal Axis Analysis and Scan Line Generation. A minimum bounding rectangle is fitted to the binary mask M to obtain the long and short side directions of the rectangle. The long side direction is defined as the principal axis direction t, and the direction orthogonal to the principal axis direction is defined as the thickness scan direction n. Multiple scan lines L_i are generated at equal intervals along the principal axis direction within the effective measurement range of the binary mask. In one specific embodiment, the number of scan lines is set to fifteen, and the chamfered areas at both ends are skipped along the principal axis direction to reduce the impact of end chamfers, occlusion, or contour instability on the measurement results.

[0065] Step S205: Heterogeneous coordinate depth sampling. For example... Figure 3 As shown, for any scan line L_i, find the two intersection points of the scan line and the true outer boundary of the mask on the binary mask M, and take them as the first boundary pixel point P_edge1=(u_1,v_1) and the second boundary pixel point P_edge2=(u_2,v_2), respectively. Then, offset N_i pixels from the two boundary pixel points in the direction pointing into the mask to obtain the first safe sampling position P_safe1 and the second safe sampling position P_safe2.

[0066] In the above steps, P_edge1 and P_edge2 are used to determine the two-dimensional span between the actual outer contours of the brake pad friction material, and P_safe1 and P_safe2 are used to read depth values ​​from the dense metric depth map D_dense. Thus, the source of pixel coordinates and the source of depth values ​​are decoupled within the same scan line local area; that is, boundary coordinates are taken from the outer edge, and depth is taken from the inward-shrinking safe region.

[0067] Step S206: 3D Backprojection and Slice Thickness Measurement. Read the safe depth values ​​Z_safe1 and Z_safe2 at P_safe1 and P_safe2 respectively. Then combine the pixel coordinates of P_edge1 with Z_safe1, and combine the pixel coordinates of P_edge2 with Z_safe2, performing backprojection using camera intrinsics. For any point P=(u,v) and its safe depth Z, its 3D coordinates Q=(X,Y,Z) are calculated according to the following formula:

[0068] X=(u-c_x)Z / f_x, Y=(v-c_y)Z / f_y, Z=Z.

[0069] Thus, the first three-dimensional boundary point Q_1 and the second three-dimensional boundary point Q_2 are obtained. The remaining thickness d_i corresponding to a single scan line is calculated according to the following formula:

[0070] d_i = ||Q_1 - Q_2 ||_2.

[0071] Step S207: Outlier Removal and Thickness Fusion Output. Outlier removal is performed on the set of remaining thickness measurements {d_i} obtained from all scan lines. The average, median, or weighted average of the remaining thickness values ​​is calculated to obtain the final remaining thickness d_final.

[0072] Example 3

[0073] This embodiment further explains the safe sampling distance and safe depth window.

[0074] In a basic implementation, the safe sampling distance N_i is a fixed pixel distance, such as any value from 3 to 10 pixels. Fixed pixel distances have the advantages of being simple to implement, computationally inexpensive, and easy to deploy in engineering, making them suitable for workstations where the camera mounting distance is relatively stable and the scale of the shutter in the image changes little.

[0075] In a preferred embodiment, the safe sampling distance N_i is an adaptive pixel distance. The invalid depth bandwidth B_raw in the original depth image D_raw can be calculated near the boundary pixels along the thickness scan direction, or the high depth gradient bandwidth B_grad in the dense metric depth map D_dense can be calculated, and the safe sampling distance can be determined based on B_raw and B_grad. Specifically, it can be calculated as follows:

[0076] N_i=clip(ceil(lambda·max(B_raw,B_grad)),N_min,N_max).

[0077] Where lambda is a magnification factor greater than or equal to 1, N_min is the minimum safe sampling distance, N_max is the maximum safe sampling distance, and clip indicates that the value is limited to a preset upper and lower limit range. In this way, if the depth hole or smooth transition area at the edge is wide, the safe sampling position will automatically move further inward; if the edge depth quality is good, the safe sampling distance will remain small to avoid excessive inward shrinkage that would reduce the representativeness of the depth.

[0078] In another adaptive approach, the safe sampling distance can be determined based on the pixel span w_i of the current scan line within the binary mask, for example, N_i = clip(ceil(beta·w_i), N_min, N_max), where beta is a scaling factor. This method is suitable for situations where changes in camera distance cause changes in the shutter image scale.

[0079] like Figure 5 As shown, to further suppress single-point depth noise, the safe depth value can be obtained without single-point reading, instead using the median of a directional local window centered on the safe sampling position. The directional local window R_i is centered on the safe sampling position P_safe, with a larger tangential radius r_t along the main axis direction t and a smaller normal radius r_n along the thickness scanning direction n. This is because edge depth artifacts mainly cross the foreground and background along the normal direction; if the normal window is too large, it can easily re-cover the boundary transition area. Meanwhile, the local surface depth along the main axis direction of the brake disc usually changes more gradually, allowing for a more robust window length.

[0080] In one specific embodiment, it can be made , A set of oriented window points P_safe+a·t+b·n is constructed to satisfy |a|≤r_t and |b|≤r_n. If there are pixels within the window that do not belong to the binary mask erosion region, r_n or r_t is reduced until all effective pixels in the window fall inside the mask, or only pixels falling within the mask erosion region are retained for depth statistics.

[0081] When calculating the window depth, pixels with empty depth values, depth values ​​exceeding the camera's range, or deviations from the initial median of the window greater than a threshold can be removed first. Then, the median, truncated mean, or weighted median of the remaining valid depth values ​​is taken as Z_safe. This window sampling mechanism is associated with the safe sampling distance N_i, which can automatically increase the safe indentation distance of the window when the edge artifact bandwidth is wide, while avoiding the window from crossing the real boundary again.

[0082] Example 4

[0083] This embodiment further explains the scan line selection and outlier removal.

[0084] like Figure 4 As shown, after performing principal axis direction analysis on the binary mask M, the system generates fifteen scan lines along the principal axis direction. To avoid the influence of end chamfers or boundary damage areas on the measurement, the system can remove a certain proportion of the area at both ends along the principal axis direction, for example, removing 5% to 20% of the length at each end, generating scan lines only within the effective middle section. This proportion can be preset according to the gate structure, camera angle, and on-site occlusion conditions.

[0085] For any scan line, if the number of intersections between the scan line and the binary mask is less than two, the distance between the intersections is less than a preset threshold, the corresponding safe sampling position falls outside the mask, or the number of effective depth pixels within the safe sampling window is less than a preset number, then the scan line is determined to be invalid and discarded.

[0086] After measuring the thickness of all valid scan lines, a robust outlier removal rule based on the median absolute deviation can be used. Let the set of valid thickness measurements be {d_i}, with the median being m, and MAD = median(|d_i - m|). When MAD is greater than 0, retain thickness values ​​that satisfy |d_i - m| ≤ k · 1.4826 · MAD, where k can be 2.5 or 3. When MAD equals 0 or the number of valid scan lines is small, the interquartile range rule or standard deviation rule can be used as an alternative.

[0087] The final remaining thickness d_final can be the arithmetic mean, median, or weighted average of the retained thickness values ​​after determining the weights based on the number of effective scan line depths, window depth variance, and mask boundary stability.

[0088] Example 5

[0089] This embodiment illustrates the operational process of the present invention on a robot end effector. After the robot moves to the vicinity of the brake pad of the locomotive under test, the end-effector RGB-D camera acquires an image of the brake pad; the system generates a color image and a raw depth image in a unified coordinate system based on the camera intrinsic parameters and registration relationship; subsequently, mask extraction, dense depth reconstruction, scan line generation, heterogeneous coordinate depth sampling, three-dimensional back projection, and statistical fusion are performed to output the remaining thickness of the brake pad friction material.

[0090] In one specific implementation, the RGB-D camera is an Intel RealSense camera, the depth completion model is the LingBot-Depth open-source model, the number of scan lines is fifteen, and the end chamfer area is skipped. The safe sampling distance can initially be a fixed value; when the on-site camera distance changes significantly, the width of the edge depth hole changes significantly, or the smooth transition of the model edge is severe, the adaptive safe sampling distance and the directional window mid-value sampling mechanism described in Example 3 are activated.

[0091] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0092] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to the above embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0093] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A three-dimensional detection method for the remaining thickness of a locomotive brake pad based on heterogeneous coordinate depth sampling, characterized in that, Includes the following steps: Acquire a color image of the friction material of the brake pad of the locomotive under test, an original depth image registered with the color image, and camera intrinsic parameters; The color image is segmented to obtain a binary mask corresponding to the friction material of the locomotive brake pad under test; A dense metric depth map aligned with the pixels of the color image is generated based on the color image and the original depth image; The principal axis direction of the brake pad friction material of the locomotive under test and the thickness scanning direction orthogonal to the principal axis direction are determined according to the binary mask, and multiple scanning lines are generated along the principal axis direction within the effective measurement range; For each scan line, determine the first boundary pixel and the second boundary pixel formed by the intersection of the scan line and the real boundary of the binary mask; Using the first boundary pixel and the second boundary pixel as references, offset by a preset distance along the direction pointing into the binary mask, the first safe sampling position and the second safe sampling position are obtained; In the dense metric depth map, the first safe depth value and the second safe depth value corresponding to the first safe sampling position and the second safe sampling position are read respectively; The pixel coordinates of the first boundary pixel are combined with the first safe depth value, and the pixel coordinates of the second boundary pixel are combined with the second safe depth value. The first three-dimensional boundary point and the second three-dimensional boundary point are obtained by back-projection based on the camera intrinsic parameters. Calculate the three-dimensional Euclidean distance between the first three-dimensional boundary point and the second three-dimensional boundary point, and use it as the measurement value of the remaining thickness of the brake pad friction material corresponding to the scan line; Outlier removal and statistical fusion are performed on the remaining thickness measurements corresponding to multiple scan lines to obtain the final remaining thickness of the brake pad friction material of the locomotive under test.

2. The three-dimensional detection method for the remaining thickness of locomotive brake pads based on heterogeneous coordinate depth sampling according to claim 1, characterized in that, The color image and the original depth image are acquired by an RGB-D camera installed on the robot's end effector. The robot's end effector drives the RGB-D camera to a preset shooting position for the brake pad of the locomotive under test.

3. The three-dimensional detection method for the remaining thickness of locomotive brake pads based on heterogeneous coordinate depth sampling according to claim 1, characterized in that, The target segmentation of the color image includes: inputting the color image into an instance segmentation network to obtain a candidate mask, performing connected component analysis on the candidate mask, and retaining the connected components corresponding to the main body region of the brake pad friction material according to the area, position or shape constraints of the connected components to obtain a binary mask.

4. The three-dimensional detection method for the remaining thickness of locomotive brake pads based on heterogeneous coordinate depth sampling according to claim 3, characterized in that, The dense metric depth map is generated based on a depth completion network, which takes at least two of the following as input: the color image, the original depth image, and a depth validity mask used to characterize the validity of the original depth, and outputs a dense depth map with physical scale and aligned with the pixels of the color image.

5. The three-dimensional detection method for the remaining thickness of locomotive brake pads based on heterogeneous coordinate depth sampling according to claim 1, characterized in that, The deep completion network includes deep completion networks based on convolutional neural networks, visual Transformers, cross-attention mechanisms, or mask depth modeling mechanisms.

6. The three-dimensional detection method for the remaining thickness of locomotive brake pads based on heterogeneous coordinate depth sampling according to claim 5, characterized in that, The principal axis direction is obtained by performing minimum bounding rectangle fitting or principal component analysis on the binary mask; the effective measurement interval is determined by removing the end chamfered regions, occlusion regions or unstable boundary regions at both ends of the binary mask in the principal axis direction.

7. The three-dimensional detection method for the remaining thickness of locomotive brake pads based on heterogeneous coordinate depth sampling according to claim 6, characterized in that, The preset distance is either a fixed pixel distance or an adaptive pixel distance; the adaptive pixel distance is based on at least one of the following: the bandwidth of the original depth invalid pixels in the boundary neighborhood, the depth gradient bandwidth of the dense metric depth map, or the pixel span of the current scan line within the binary mask.

8. The three-dimensional detection method for the remaining thickness of locomotive brake pads based on heterogeneous coordinate depth sampling according to claim 7, characterized in that, When reading the first and second safe depth values, directional local windows are constructed with the corresponding safe sampling positions as the center. The size of the directional local window along the main axis is larger than the size along the thickness scanning direction, and the effective sampling pixels in the directional local window are constrained to fall within the erosion area of ​​the binary mask. The median, truncated mean, or weighted median of the effective depth values ​​in the directional local window is used as the corresponding safe depth value.

9. The three-dimensional detection method for the remaining thickness of locomotive brake pads based on heterogeneous coordinate depth sampling according to claim 8, characterized in that, When performing outlier removal and statistical fusion, the median and median absolute deviation of the remaining thickness measurements corresponding to multiple scan lines are calculated, and thickness measurements with a deviation from the median greater than a preset multiple of the median absolute deviation are removed; the average, median, or weighted average of the remaining thickness measurements are calculated to obtain the final remaining thickness.

10. A three-dimensional detection system for the remaining thickness of a locomotive brake pad based on heterogeneous coordinate depth sampling, applied to the three-dimensional detection method for the remaining thickness of a locomotive brake pad as described in any one of claims 1-9, characterized in that, include: The data acquisition module is used to acquire color images, raw depth images, and camera intrinsic parameters of the friction material of the brake pads of the locomotive under test. A mask extraction module is used to perform target segmentation on the color image to obtain a binary mask; A depth reconstruction module is used to generate a dense metric depth map based on the color image and the original depth image; The scan line generation module is used to determine the main axis direction and thickness scanning direction based on the binary mask, and generate multiple scan lines; The heterogeneous sampling module is used to determine the boundary pixel points on the true boundary of the binary mask and read the safe depth value after offset along the internal direction in the dense metric depth map. The 3D measurement module is used to perform 3D back projection based on the pixel coordinates of boundary pixels, safe depth values, and camera intrinsic parameters, and to calculate the 3D Euclidean distance. The statistical fusion module is used to remove outliers and statistically fuse the remaining thickness measurements of multiple scan lines to obtain the final remaining thickness.

Images