Railway pantograph carbon slide plate wear detection method based on machine vision

Abstract

The invention discloses a railway pantograph carbon slide plate wear detection method based on machine vision, and the method comprises the steps: firstly carrying out the illumination enhancement of a pantograph image through employing an SCI model which introduces a channel attention mechanism, and carrying out the super-resolution reconstruction of the pantograph image after the illumination enhancement through employing a Real-ESRGAN model; secondly, an improved M3Net model is adopted to carry out pantograph carbon slide plate area positioning, an improved M3Net model encoder adopts a residual error post-standardization structure and logarithm interval continuous relative position deviation method, a decoder adopts an MIB module and an MAB module, and then a DexiNed model introducing a CA block and a space attention mechanism is adopted to carry out carbon slide plate abrasion edge detection; and finally, judging whether uneven wear exists or not according to a wear edge detection result. According to the railway pantograph carbon slide plate wear detection method disclosed by the invention, the problem of large wear detection error of the carbon slide plate under the condition of poor image quality or complex background in the existing method is solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of rail transit equipment state detection method, and particularly relates to a railway pantograph carbon slide plate wear detection method based on machine vision. BACKGROUND

[0002] With the expansion of the railway network scale and the development trend of high speed, the safety guarantee and operation reliability of railway facilities are facing severe challenges. As the core component of the electric power locomotive and motor train unit energy transmission system, the dynamic contact performance of the pantograph directly affects the operation safety and transportation efficiency of the train. Therefore, how to realize real-time monitoring and accurate evaluation of the state of the pantograph has become an important issue in the field of railway operation and maintenance.

[0003] As the core component of the electric power locomotive and motor train unit energy transmission system, the pantograph bears the key function of obtaining electric energy from the catenary to drive the train to run, and the carbon slide plate (also known as the contact plate) realizes electric energy transmission through continuous sliding contact with the contact wire. The health state of the carbon slide plate directly affects the dynamic stability of the pantograph-catenary system and the operation safety of the train. The carbon slide plate needs to withstand mechanical friction and electrical wear in long-term operation. When abnormal wear occurs, it is easy to cause abnormal vibration of the catenary, arc discharge and other faults, and in severe cases, it may cause line interruption or locomotive shutdown, which brings great hidden dangers to railway transportation. The non-contact detection method based on machine vision is the current research hotspot, but this technology is significantly restricted by complex working conditions: (1) The quality of the pantograph image collected is poor The acquisition of pantograph images usually needs to be completed synchronously in the process of train high-speed running, which is to realize real-time monitoring of the state of the pantograph and timely find abnormal conditions to ensure operation safety. However, this real-time and dynamic acquisition method faces many challenges. In high dynamic light environments such as train passing through tunnels, night driving or bad weather, the image often has problems such as insufficient brightness, glare and noise interference, which leads to edge blur, detail loss, and affects wear feature extraction and analysis. Restricted by the requirements of railway scene on equipment stability and cost, low-cost imaging equipment is actually used, which has deployment advantages but easily causes image quality to decrease, so that subtle wear features are covered by noise and the detection accuracy is reduced.

[0004] (2) The image background is complex Images of pantographs captured from the vehicle roof often contain complex background information (such as the base plate, support frame, and insulators). These background elements significantly overlap with the carbon strip in spatial distribution, making edge extraction from the original image extremely difficult. Furthermore, the edges formed by long-term wear on the carbon strip typically exhibit low contrast, while the metal frame of the pantograph, such as the pantograph itself, forms a distinct geometric outline in the image, with dramatic changes in edge grayscale, making its edges more pronounced than the worn edges of the carbon strip. Conventional image processing methods are prone to generating false edge features, reducing the accuracy of wear detection. Summary of the Invention

[0005] The purpose of this invention is to provide a machine vision-based method for detecting wear on carbon sliding plates of railway pantographs, which solves the problem of large errors in carbon sliding plate wear detection under conditions of poor image quality or complex background in existing methods.

[0006] The technical solution adopted in this invention is: a machine vision-based method for detecting wear on the carbon sliding plate of a railway pantograph. First, an SCI model incorporating a channel attention mechanism is used to enhance the illumination of the pantograph image. Then, a Real-ESRGAN model is used for super-resolution reconstruction of the enhanced pantograph image. Finally, an improved M... 3 Net model for pantograph carbon slider region localization, improved M 3 The Net model encoder employs a residual post-normalization structure and a log-interval continuous relative position deviation method. The decoder uses MIB and MAB modules. Then, the DexiNed model, which incorporates CA blocks and spatial attention mechanisms, is used to detect the wear edges of the carbon slide plate. Finally, the wear edge detection results are used to determine whether there is uneven wear.

[0007] The present invention is further characterized by including the following steps: Step 1: Add noise of varying degrees to the original pantograph image and perform cropping operations to generate an expanded dataset; Step 2: Input the pantograph image obtained in Step 1 into the SCI low-light enhancement model with channel attention mechanism, and output the image with enhanced illumination; Step 3: Input the image output from Step 2 into the Real-ESRGAN model to output a high-resolution image; Step 4: Use the Labelme tool to perform two types of annotations on the image output in Step 2: carbon skateboard segmentation mask and ground truth edge information. Step 5: Use the image output from Step 3 and the labels from Step 4 as input to train the improved M³Net localization model. The model encoder adopts a residual post-normalization structure and a log-interval continuous relative position deviation method. The decoder adopts MIB and MAB modules and outputs a mask of the pantograph carbon sliding plate area. Step 6: Using the carbon skateboard area mask output in Step 5, segment the carbon skateboard image based on the pixel value difference between the target area and the non-target area of ​​the mask. Step 7: Take the carbon skateboard image output in Step 6 and the edge annotation labels from Step 4 as input to train the improved DexiNed model. After each encoding block, a CA module is introduced into the model, and after the upsampling module, a spatial attention mechanism is introduced to output the carbon skateboard wear edge detection results. Step 8: Determine whether there is uneven wear based on the detection results output in Step 7.

[0008] Step 2 specifically includes the following steps: Step 2.1: Input the original pantograph image into the SCI model illumination learning module, learn the image illumination information through the weight-sharing parameterized operator, and generate residual terms and stage illumination output; Step 2.2: Input the output of Step 2.1 into the self-calibration module, connect the input of each stage with the original low-light image, explore the convergence behavior between stages through self-calibration mapping, and make the results of each stage converge to the same state. Step 2.3: The input channel attention denoising module of the output of step 2.2 first performs global pooling to convert each channel of the feature map into a scalar value; second, it generates channel weights to [0,1] by sequentially performing dimensionality reduction, ReLU activation, dimensionality increase, and Sigmoid normalization through a fully connected layer; finally, it multiplies the weights with the original feature map channel by channel to output the weighted feature map, which is the image after illumination enhancement.

[0009] Step 5 specifically includes the following steps: Step 5.1: Input the image output from Step 3 and the labels from Step 4 into the encoder and use the Swing Transformer with a residual post-normalization structure and log-interval continuous relative position deviation for feature extraction. Step 5.2: Input the multi-level features extracted in Step 5.1 into the decoder, and fuse them with the MAB global-window self-attention module through the MIB cross-attention mechanism to train the improved M³Net localization model, and output the pantograph carbon skateboard region mask.

[0010] The logarithmic interval continuous relative position deviation method in step 5.1 is to use a meta-network based on the relative coordinates ( x , y The deviation value is generated using the following formula:

[0011] Then, a feature map with geometric relationship encoding is output.

[0012] Step 7 specifically includes the following steps: Step 7.1: Pair the carbon skateboard image output in Step 6 with the edge annotation labels in Step 4, input it into the DexiNed network, and extract multi-level edge feature maps sequentially through the Dexi sub-blocks to complete the hierarchical extraction process of basic edge features. Step 7.2: Insert a CA module after the output of each encoding block of the Dexi network to perform orientation-aware enhancement operation on the current feature map: First, decouple the global average pooling into horizontal and vertical position encodings to generate an orientation-aware attention weight map; then multiply the weight map with the original feature map pixel by pixel to achieve dynamic enhancement of the real edge response and suppression of background noise. Step 7.3: The feature map after directional enhancement is fed into the upsampling module. The feature map resolution is gradually restored through the upsampling operation to generate edge feature images at various stages containing edge details at different scales, providing multi-scale input for subsequent feature fusion. Step 7.4: Insert a spatial attention mechanism after the upsampling module output and before multi-scale feature fusion: First, dynamically generate a spatial weight map, then adaptively adjust the fusion ratio of high-resolution features and low-resolution features according to the pixel position, and finally repair the edge breakage area and compensate for the geometric misalignment caused by downsampling through weighted fusion. Step 7.5: Integrate the fused multi-level edge features to generate and output the final carbon skateboard wear edge detection result image.

[0013] Step 8 specifically includes the following steps: Step 8.1: Extract the main curves based on the detection results output in Step 7 to obtain two main curves; Step 8.2: Calculate the cumulative arc length of the two main curves, sample a fixed number of points according to the principle of equal arc length, and then use B-spline interpolation with a smoothing factor to fit the sampled points to generate a continuous curve. Step 8.3: Calculate the vertical distance between each point on the skeleton curve and the corresponding point on the fitted curve. If the distance exceeds the preset threshold, it is determined to be uneven wear.

[0014] Step 8.1 specifically includes the following steps: Step 8.1.1: Binarize the input edge image, determine all non-white pixels as foreground regions, and use the Skeletonize algorithm to compress the lines into a single-pixel-width skeleton structure, preserving the topological features of the original edges; Step 8.1.2: Perform connected component labeling on the skeletonized image to identify all interconnected pixel sets and label each connected component as an independent candidate curve object; Step 8.1.3: Construct a pixel-level graph structure for each connected region: use pixels as nodes and establish edges connecting adjacent pixels; start the depth-first traversal algorithm from the endpoint with degree 1 to extract the complete ordered pixel path sequence; Step 8.1.4: Use a three-point sliding window to evaluate the local slope of each path: calculate the central difference slope value of three consecutive points; if the slope exceeds the preset threshold, determine the corresponding point as an interference point with geometric deviation and remove it, and break the non-main curve connection segment; Step 8.1.5: Sort all paths after removing interference in descending order of pixel count, and select the two longest paths as the main curves of the long side of the carbon skateboard.

[0015] The beneficial effects of this invention are as follows: This invention, a machine vision-based method for detecting wear on the carbon sliding plate of a railway pantograph, focuses on detecting the target edge of the carbon sliding plate from low-quality images to achieve wear detection. It enhances the image quality of low-quality pantograph images by improving both brightness and clarity to obtain high-quality images. Then, the carbon sliding plate region is segmented from the image, avoiding interference from complex backgrounds and the base plate. Finally, edge detection is performed on the segmented region, and non-uniform wear detection is performed based on the edge detection results. This invention can extract edge contours from low-quality pantograph images and perform non-uniform wear detection even without knowing the equipment parameters, simplifying the complexity of preliminary work and improving the accuracy level in the field of rail transit equipment condition monitoring. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the process framework of the machine vision-based method for detecting wear of carbon sliding plates in railway pantographs according to the present invention. Figure 2(a) is a schematic diagram of the process and effect of carbon skateboard region segmentation masking in this invention; Figure 2(b) is a schematic diagram of the process and effect of marking the wear edge of the carbon sliding plate according to the present invention; Figure 3 This is a schematic diagram of the pantograph image illumination enhancement model structure of the present invention; Figure 4 This is a schematic diagram of the positioning model structure of the pantograph carbon sliding plate of the present invention; Figure 5 This is a schematic diagram of the residual post-normalization method of the present invention; Figure 6 This is a schematic diagram of the pantograph carbon sliding plate wear edge detection model structure in this invention; Figure 7 This is a schematic diagram of the CA module structure in this invention; Figure 8 This is a visual comparison diagram of the fitted curve and the original skeleton curve in this invention. Detailed Implementation

[0017] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0018] Example 1 This invention provides a machine vision-based method for detecting wear on the carbon sliding plate of a railway pantograph. First, an SCI model incorporating a channel attention mechanism is used for pantograph image illumination enhancement, followed by super-resolution reconstruction using Real-ESRGAN with the enhanced pantograph image. Second, an improved M... 3 Net is used to locate the pantograph carbon slider area. The improved model uses M 3 The model uses Net as the basic framework, but employs a post-normalized structure and a log-interval continuous relative position deviation method in the encoder to alleviate model training instability and reduce the extrapolation ratio. Then, an improved Dexined algorithm is used for carbon skateboard wear edge detection. The improved model uses Dexined as the basic framework and introduces CA blocks and spatial attention mechanisms to improve detection accuracy. Finally, the presence of uneven wear is determined based on the wear edge detection results. Experimental data comes from publicly available datasets released by Kaggle, such as... Figure 1 As shown, the specific steps include: Step 1: Dataset Expansion. To expand the dataset, different levels of noise were added to the images, and cropping was performed. The final dataset contains 2400 pantograph images.

[0019] Step 2: Construct a pantograph image illumination enhancement model. Input the original pantograph image into the improved SCI low-light enhancement model to generate an image with enhanced illumination.

[0020] Step 3: Construct a super-resolution reconstruction model for the pantograph image. Input the pantograph image processed in Step 2 into Real-ESRGAN to process the image sharpness and generate a high-resolution image.

[0021] Step 4: Manually annotating the dataset. In this invention, the images processed in Step 2 were annotated in two ways using the Labelme open-source annotation tool developed based on Python: one is carbon skateboard segmentation mask, the annotation process and annotation effect are shown in Figure 2(a); the other is real edge annotation, the annotation process and annotation effect are shown in Figure 2(b).

[0022] Step 5: Construct the pantograph carbon skateboard positioning model. Use the image processed in Step 3 and the labels from Step 4 as input to train the pantograph carbon skateboard positioning model.

[0023] Step 6: Segment the carbon fiber skateboard image. Based on the mask of the carbon fiber skateboard region generated in Step 5, segment the carbon fiber skateboard using the pixel value difference between the target area and the non-target area of ​​the mask.

[0024] Step 7: Construct a pantograph carbon skid plate wear edge detection model. Use the carbon skid plate image segmented in Step 6 and the labels from Step 4 as input to train the pantograph carbon skid plate wear edge detection model.

[0025] Step 8: Detection of uneven wear on the carbon slide plate. Based on the edge detection results in Step 7, perform curve fitting and compare the fitted curve with the original edge curve to detect uneven wear.

[0026] Example 2 This invention provides a machine vision-based method for detecting wear on the carbon sliding plate of a railway pantograph, based on Example 1, such as... Figure 3 As shown, the preferred model in step 2 includes three modules: an illumination learning module, a self-calibration module, and a channel attention denoising module.

[0027] Step 2.1, in the lighting learning module, based on Retinex theory, by introducing parameters... mapping This allows us to learn the lighting information of the image. This provides a progressive perspective to simulate this task, with the basic unit shown in Equation 1.

[0028] (1) In the formula The input is a low-light image. express The residual term of the stage, express Phase-based illumination. Parameterized operators. We learned a simple residual representation between illuminated and low-light images, but the formula did not provide... The level number is marked because the lighting learning module uses a weight-sharing mechanism, meaning the same architecture is used in each level. and weight Compared to using a direct mapping between low-light images and illumination, the shared weight method can significantly reduce computational complexity while maintaining high performance and stability, which is especially important for resource-constrained environments or real-time applications.

[0029] Step 2.2: The self-calibration module plays a crucial role in the self-calibration illumination learning framework. Its core function is to converge the results of each stage to the same state. Therefore, this module connects the inputs of each stage (except the first stage) with the original weak light input (i.e., the input of the first stage), indirectly exploring the convergence behavior between stages. To this end, a self-calibration mapping is introduced. , used to represent the difference between the input of each stage and the input of the first stage. The principle of the self-calibration module can be expressed as Equation 2.

[0030] (2) In the formula , For use after calibration and conversion Input, For learnable parameters, For parameterized operators.

[0031] Step 2.3: The channel attention mechanism denoising module used in this invention first transforms the feature map of each channel into a scalar value through global pooling, laying the foundation for subsequent channel weight prediction. Then, it predicts the importance weight of each channel through two fully connected (FC) layers. The first FC layer performs dimensionality reduction, decreasing the dimension of the feature vector from C to C / r, where r is the dimensionality reduction ratio. Then, the ReLU activation function is used to introduce non-linearity, enhancing the model's expressive power. The second FC layer then increases the dimension back to C from C / r to match the original number of channels. Finally, the Sigmoid activation function normalizes the weight of each channel of the feature map to between 0 and 1. By multiplying these weights channel-by-channel with the original input feature map, weighted processing of the feature map is achieved.

[0032] Example 3 This invention provides a machine vision-based method for detecting wear on the carbon sliding plate of a railway pantograph, based on Example 1, such as... Figure 4 As shown, step 5 preferably includes the following steps: Step 5.1: The encoder part of this invention uses the Swin Transformer as its basic structure. In the original Swin Transformer encoder architecture, a normalization layer is embedded at the beginning of each block. Under this pre-normalization setting, the output value of each residual block is directly merged into the main branch, causing the activation amplitude of the main branch to become larger and larger as the network layers deepen. Since each layer directly accumulates the residual output of the previous layer, the amplitude difference between different layers gradually widens. This unbalanced amplitude change is one of the key factors leading to instability in the training process. To solve this problem, the encoder part in step 5 of this invention uses a residual post-normalization method instead of the original pre-normalization setting, and its structure is as follows: Figure 5 As shown.

[0033] In the original Swin Transformer encoder, the relative position bias (RPB) is a key component. It encodes geometric relationships by introducing an additional parameter, the bias term, into the self-attention, as shown in Equation 3.

[0034] (3) In the formula , is the relative position deviation matrix; , representing the query, key, and value matrix, respectively; For feature dimensions; This indicates the number of window patches. Relative positional bias plays a crucial role in various vision tasks by encoding the relative spatial configuration of visual elements, and is particularly critical in dense recognition tasks.

[0035] However, the relative positional bias is discrete, and the bias at each position is learned through parameters. When the model needs to be transferred to fine-tuning tasks with different window sizes, this discrete biasing method may lead to performance degradation. To enable the relative positional bias to be smoothly transferred between window resolutions, the encoder part of step 5 of the invention adopts a logarithmically spaced continuous relative positional bias method, which generates bias values ​​based on relative coordinates through a small meta-network. The core idea of ​​this method is to establish a more compact and efficient positional mapping relationship between different window sizes by encoding the coordinate values ​​in logarithmic form, thereby reducing the required extrapolation ratio, as expressed in Equation 4.

[0036] (4) Step 5.2, the decoder part of the present invention uses M 3 The Net decoder serves as the basic structure. To address the information loss problem during sampling, this invention employs MIB (Multi-Level Attention). This module introduces a cross-attention mechanism to achieve multi-level feature interaction, allowing high-level features to guide low-level features, strengthening the target region and eliminating information from non-target regions. To promote the aggregation of multi-level features in sequence form, a MAB (Multi-Level Attention) module is used, combining global self-attention and window self-attention to model the context at both global and local levels, further improving the local accuracy of the prediction map.

[0037] Example 4 This invention provides a machine vision-based method for detecting wear on carbon sliding plates of railway pantographs. Building upon Example 1, step 7 preferentially uses DexiNed as the basic network structure for wear edge detection, primarily due to its unique advantages in complex edge feature extraction and multi-scale information fusion. The DexiNed model requires no pre-trained weights, can be trained from scratch, and exhibits faster convergence and higher generalization ability during training. DexiNed mainly consists of two sub-blocks: the Dexi network and an upsampling module. The image is input into the Dexi network for feature extraction, and then the extracted feature maps are input into the upsampling module to generate edge feature images for each stage. Finally, these feature maps are fused to generate the final edge information image.

[0038] Traditional convolutional operations can easily introduce noise or cause edge position shifts when handling complex scenes, thus affecting detection accuracy. In particular, pantograph images suffer from uneven lighting and complex carbon skateboard surface textures, making it difficult for convolution to accurately distinguish between real edges and interfering noise, leading to misjudgments or omissions. To address this issue, this invention introduces a CA module after each encoding block of Dexined, enhancing detection accuracy by explicitly modeling the directional features of edges. The CA module captures the directional information of edges by decomposing global average pooling into horizontal and vertical position encodings. This decomposition mechanism aligns perfectly with the direction-sensitive localization requirements of edge detection tasks, enabling the feature maps output by each encoding module to generate direction-aware attention weights, dynamically enhancing the response of real edge pixels while suppressing background noise. In contrast, other attention mechanisms (such as the SE module) or local insertion schemes struggle to achieve globally consistent position enhancement, while the layer-by-layer deployment of CA ensures coherent optimization from details to contours, thereby achieving directional enhancement of edge features and noise suppression globally.

[0039] To address the issues of blurred edges, breaks, or positional shifts caused by scale differences and semantic misalignment during feature fusion at different resolutions, this invention introduces a spatial attention mechanism before feature fusion following the upsampling module. The spatial attention mechanism focuses on the importance allocation of the spatial dimensions of feature maps. By dynamically generating a spatial weight map, it guides the model to adaptively adjust the feature contribution ratio for each pixel position during fusion: in regions with clear edges but positional shifts, it enhances the local details of high-resolution features to correct the localization; in semantically clear regions with blurred edges, it relies on the overall contour information provided by low-resolution features. This dynamic selection is essentially a "feature fusion wizard," capable of capturing long-range spatial dependencies, such as the directional continuity of edge lines, forcing the model to focus on the connecting regions of broken edges or suppressing interference from textured backgrounds. When high-resolution features detect broken edge endpoints, while low-resolution features indicate the existence of continuous edges in the region, spatial attention enhances the weights around the endpoints, prompting the fused features to fill in the breaks. Simultaneously, this mechanism can also compensate for positional shifts caused by encoder downsampling—by learning the spatial correlation between adjacent feature maps, it automatically calibrates the pixel correspondences of features across scales, reducing edge misalignment caused by multiple pooling operations.

[0040] Example 5 This invention provides a machine vision-based method for detecting wear on the carbon sliding plate of a railway pantograph. Based on Example 1, step 8 preferably includes the following steps: Step 8.1: Extract the master curve based on the edge detection results from Step 7. The carbon skateboard edge detection results contain edge information for both the long and short sides. However, when performing uneven wear detection tasks, only the long edge of the carbon skateboard needs to be considered; the presence of the short edge may introduce interference and affect the accuracy of wear identification. Therefore, this invention proposes a master curve extraction method for carbon skateboard edge images with many boundary connection lines, based on a combination of skeletonization processing, graph structure modeling, and sliding window slope analysis. The core process is as follows: (1) Skeleton extraction: First, the input image is binarized, and all non-white areas are regarded as the foreground. Then, the Skeletonize algorithm is used to skeletonize the image, compressing the lines into a single-pixel width structure, thereby preserving the topological features of the image and simplifying subsequent processing.

[0041] (2) Connected Component Identification: All connected components in the skeleton graph are identified using the Connected Component Labeling method and treated as candidate curves. Each connected component may correspond to a main curve, an edge connection line, or noise.

[0042] (3) Graph structure construction and path extraction: For each connected region, a pixel-level graph structure is constructed, where each pixel is a graph node and edges are established between adjacent pixels. The ordered pixel path of the entire curve is extracted by starting from the endpoint with degree 1 using a depth-first traversal algorithm.

[0043] (4) For each path, a three-point sliding window is used to evaluate its local slope. Let three consecutive points in the path be... Then the local slope at that location is calculated using the central difference formula as follows: (5) If in the formula If the threshold is exceeded, the point is considered to be in a connection segment with an excessively steep slope and is removed. This operation can effectively break up interfering line segments that are connected to the main curve but deviate in geometric direction.

[0044] (5) Main curve selection: Sort all the remaining curve paths by length and select the first two as the main curves.

[0045] Step 8.2: To achieve accurate curve reconstruction and modeling, this invention employs a method combining equal arc length sampling and B-spline fitting. The visual comparison between the fitted curve and the original skeleton curve is shown below. Figure 8As shown. The core idea of ​​this method is to construct a continuous and differentiable smooth curve model based on discrete skeleton points. First, for the two longest skeleton curves retained after sliding window slope analysis, the cumulative arc length is calculated according to their pixel sequence. Then, a fixed number of sampling points are selected on each curve according to the principle of equal arc length, thereby ensuring that the sampling points are evenly distributed in geometric space and avoiding the impact of inconsistent local pixel density on fitting accuracy and stability. After completing the equal arc length sampling, B-spline interpolation is used to model the sampling points and generate the final continuous curve expression. To further improve the robustness of the fitting, a smoothing factor is introduced into the B-spline fitting to adjust the degree of curve fitting to the sampling points. The introduction of this smoothing factor can effectively suppress the interference of local sharp noise or small-amplitude concavities on the overall curve shape, thereby avoiding overfitting and ensuring that the fitted curve better conforms to the overall trend of the actual wear profile.

[0046] Step 8.3: Uneven Wear Detection. Based on the wear edge detection results and curve fitting results, calculate the vertical distance between each point on the skeleton curve and its corresponding point on the fitted curve to determine whether abnormal wear exists. When this distance exceeds a preset threshold, it can be determined that uneven wear exists in that area.

[0047] Example 6 This invention provides a machine vision-based method for detecting wear on the carbon sliding plate of a railway pantograph. The overall framework is as follows: Figure 1 As shown, firstly, an SCI model incorporating a channel attention mechanism is used for pantograph image illumination enhancement, and then Real-ESRGAN is used for super-resolution reconstruction of the enhanced pantograph image. Secondly, an improved M... 3 Net is used to locate the pantograph carbon slider area. The improved model uses M 3 The model uses Net as the basic framework, but employs a post-normalized structure and a log-interval continuous relative position deviation method in the encoder to alleviate instability during model training and reduce the extrapolation ratio. Then, an improved Dexined algorithm is used for carbon skateboard wear edge detection. The improved model uses Dexined as the basic framework and introduces CA blocks and spatial attention mechanisms to improve detection accuracy. Finally, the presence of uneven wear is determined based on the wear edge detection results. The following steps are followed: Step 1: Dataset Expansion. To expand the dataset, different levels of noise were added to the images, and cropping was performed. The final dataset contains 2400 pantograph images.

[0048] Step 2: Construct a pantograph image illumination enhancement model. Input the original pantograph image into the improved SCI low-light enhancement model to generate an image with enhanced illumination.

[0049] Step 2.1, in the lighting learning module, based on Retinex theory, by introducing parameters... mapping This allows us to learn the lighting information of the image. This provides a progressive perspective to simulate this task, with the basic unit shown in Equation 1.

[0050] (1) In the formula The input is a low-light image. express The residual term of the stage, express Phase-based illumination. Parameterized operators. We learned a simple residual representation between illuminated and low-light images, but the formula did not provide... The level number is marked because the lighting learning module uses a weight-sharing mechanism, meaning the same architecture is used in each level. and weight Compared to using a direct mapping between low-light images and illumination, the shared weight method can significantly reduce computational complexity while maintaining high performance and stability, which is especially important for resource-constrained environments or real-time applications.

[0051] Step 2.2: The self-calibration module plays a crucial role in the self-calibration illumination learning framework. Its core function is to converge the results of each stage to the same state. Therefore, this module connects the inputs of each stage (except the first stage) with the original weak light input (i.e., the input of the first stage), indirectly exploring the convergence behavior between stages. To this end, a self-calibration mapping is introduced. , used to represent the difference between the input of each stage and the input of the first stage. The principle of the self-calibration module can be expressed as Equation 2.

[0052] (2) In the formula , For use after calibration and conversion Input, For learnable parameters, For parameterized operators.

[0053] Step 2.3: The channel attention mechanism denoising module employed in this invention first transforms the feature map of each channel into a scalar value through global pooling, laying the foundation for subsequent channel weight prediction. Then, it predicts the importance weight of each channel through two fully connected (FC) layers. The first FC layer performs dimensionality reduction, decreasing the dimension of the feature vector from C to C / r, where r is the dimensionality reduction ratio. Then, the ReLU activation function is used to introduce non-linearity, enhancing the model's expressive power. The second FC layer then increases the dimension back to C from C / r to match the original number of channels. Finally, the Sigmoid activation function normalizes the weight of each channel of the feature map to between 0 and 1. By multiplying these weights channel-by-channel with the original input feature map, weighted processing of the feature map is achieved.

[0054] Step 3: Construct a super-resolution reconstruction model for the pantograph image. Input the pantograph image processed in Step 2 into Real-ESRGAN to process the image sharpness and generate a high-resolution image.

[0055] Real-ESRGAN's generator uses the same structure as ESRGASN, namely a deep network containing multiple residual dense blocks (RRDBs). While ESRGAN can generate super-resolution images, its structure is relatively complex and computationally demanding. To address this, Real-ESRGAN uses a pixel-unshuffle operation to increase the number of image channels while reducing image resolution, and then inputs the processed image into the network for super-resolution reconstruction.

[0056] Since Real-ESRGAN aims to solve the more complex degradation space problem than ESRGAN, its discriminator needs stronger discrimination capabilities to handle complex training outputs. Real-ESRGAN employs a U-Net structure with skip connections as its discriminator, which can determine the authenticity of individual generated pixels at the pixel level. U-Net's multi-scale feature extraction capabilities enable it to provide more detailed per-pixel feedback, thereby enhancing adversarial learning in image detail and ensuring that the generated images are realistic overall while maintaining attention to detail.

[0057] Step 4: Manually annotating the dataset. This invention performs two types of annotations on the images processed in Step 2: one is a carbon skateboard segmentation mask, and the other is a ground truth edge annotation. The annotation process and results are shown in Figure 2.

[0058] Step 4.1, in constructing the carbon skateboard segmentation mask, uses the Labelme open-source annotation tool developed in Python. This tool relies on the Qt framework's graphical user interface system. Its core function is to generate semantic segmentation labels through vector polygon annotations. Its technical principle lies in mapping user-defined geometric vertex coordinates to pixel-level masks. Labelme is used to annotate and generate JSON files, which are then converted into segmentation mask images.

[0059] Step 4.2: The edge detection dataset of this invention is also labeled using the Labelme tool. The core principle is to generate a binary edge mask aligned with the pixels of the original image through manual annotation. For the edge features of the image, the LineStrip mode is used to annotate the polyline path. Labelme is used to annotate and generate a JSON file, and then the JSON file is converted into an edge-annotated image.

[0060] Step 5: Construct the pantograph carbon skateboard positioning model. Use the image processed in Step 3 and the labels from Step 4 as input to train the pantograph carbon skateboard positioning model.

[0061] Step 5.1: The encoder part of this invention uses the Swin Transformer as its basic structure. In the original Swin Transformer encoder architecture, a normalization layer is embedded at the beginning of each block. Under this pre-normalization setting, the output value of each residual block is directly merged into the main branch, causing the activation amplitude of the main branch to become larger and larger as the network layers deepen. Since each layer directly accumulates the residual output of the previous layer, the amplitude difference between different layers gradually widens. This unbalanced amplitude change is one of the key factors leading to instability in the training process. To solve this problem, the encoder part in step 5 of this invention uses a residual post-normalization method instead of the original pre-normalization setting, and its structure is as follows: Figure 5 As shown.

[0062] In the original Swin Transformer encoder, the relative position bias (RPB) is a key component. It encodes geometric relationships by introducing an additional parameter, the bias term, into the self-attention, as shown in Equation 3.

[0063] (3) In the formula , is the relative position deviation matrix; , representing the query, key, and value matrix, respectively; For feature dimensions; This indicates the number of window patches. Relative positional bias plays a crucial role in various vision tasks by encoding the relative spatial configuration of visual elements, and is particularly critical in dense recognition tasks.

[0064] However, the relative positional bias is discrete, and the bias at each position is learned through parameters. When the model needs to be transferred to fine-tuning tasks with different window sizes, this discrete biasing method may lead to performance degradation. To enable the relative positional bias to be smoothly transferred between window resolutions, the encoder part of step 5 of the invention adopts a logarithmically spaced continuous relative positional bias method, which generates bias values ​​based on relative coordinates through a small meta-network. The core idea of ​​this method is to establish a more compact and efficient positional mapping relationship between different window sizes by encoding the coordinate values ​​in logarithmic form, thereby reducing the required extrapolation ratio, as expressed in Equation 4.

[0065] (4) Step 5.2, the decoder part of the present invention uses M 3 The Net decoder serves as the basic structure. To address the information loss problem during sampling, this invention employs MIB (Multi-Level Attention). This module introduces a cross-attention mechanism to achieve multi-level feature interaction, allowing high-level features to guide low-level features, strengthening the target region and eliminating information from non-target regions. To promote the aggregation of multi-level features in sequence form, a MAB (Multi-Level Attention) module is used, combining global self-attention and window self-attention to model the context at both global and local levels, further improving the local accuracy of the prediction map.

[0066] Step 6: Segment the carbon fiber skateboard image. Based on the mask of the carbon fiber skateboard region generated in Step 5, segment the carbon fiber skateboard using the pixel value difference between the target area and the non-target area of ​​the mask. Retain the white area corresponding to the segmentation mask in the image, and do not retain the other areas.

[0067] Step 7: Construct a pantograph carbon skid plate wear edge detection model. Use the carbon skid plate image segmented in Step 6 and the labels from Step 4 as input to train the pantograph carbon skid plate wear edge detection model.

[0068] This invention selects DexiNed as the basic network structure for wear edge detection, primarily based on its unique advantages in complex edge feature extraction and multi-scale information fusion. The DexiNed model requires no pre-trained weights, can be trained from scratch, and exhibits faster convergence speed and higher generalization ability during training. Figure 6As shown, DexiNed mainly consists of two sub-blocks: the Dexi network and the upsampling module. The image is input into the Dexi network for feature extraction, and then the extracted feature maps are input into the upsampling module to generate edge feature images at each stage. Finally, these feature maps are fused to generate the final edge information image.

[0069] Traditional convolutional operations are prone to introducing noise or causing edge position shifts when handling complex scenes, thus affecting detection accuracy. In particular, pantograph images suffer from uneven lighting and complex carbon skateboard surface textures, making it difficult for convolution to accurately distinguish between real edges and interfering noise, leading to misjudgments or omissions. To address this issue, this invention introduces a CA module after each Dexined encoding block, enhancing detection accuracy by explicitly modeling the directional features of the edges. The CA module captures the directional information of the edges by decomposing global average pooling into horizontal and vertical position encodings, and its structure is as follows: Figure 7 As shown, this decomposition mechanism aligns perfectly with the orientation-sensitive localization requirements of edge detection tasks. It enables the feature maps output by each encoding module to generate orientation-aware attention weights, dynamically enhancing the response of real edge pixels while suppressing background noise. In contrast, other attention mechanisms (such as the SE module) or local insertion schemes struggle to achieve globally consistent position enhancement, while the layer-by-layer deployment of CA ensures coherent optimization from details to contours, thereby achieving directional enhancement of edge features and noise suppression globally.

[0070] To address the issues of blurred edges, breaks, or positional shifts caused by scale differences and semantic misalignment during feature fusion at different resolutions, this invention introduces a spatial attention mechanism before feature fusion following the upsampling module. The spatial attention mechanism focuses on the importance allocation of the spatial dimensions of feature maps. By dynamically generating a spatial weight map, it guides the model to adaptively adjust the feature contribution ratio for each pixel position during fusion: in regions with clear edges but positional shifts, it enhances the local details of high-resolution features to correct the localization; in semantically clear regions with blurred edges, it relies on the overall contour information provided by low-resolution features. This dynamic selection is essentially a "feature fusion wizard," capable of capturing long-range spatial dependencies, such as the directional continuity of edge lines, forcing the model to focus on the connecting regions of broken edges or suppressing interference from textured backgrounds. When high-resolution features detect broken edge endpoints, while low-resolution features indicate the existence of continuous edges in the region, spatial attention enhances the weights around the endpoints, prompting the fused features to fill in the breaks. Simultaneously, this mechanism can also compensate for positional shifts caused by encoder downsampling—by learning the spatial correlation between adjacent feature maps, it automatically calibrates the pixel correspondences of features across scales, reducing edge misalignment caused by multiple pooling operations.

[0071] Step 8: Detection of uneven wear on the carbon slide plate. Based on the edge detection results in Step 7, perform curve fitting and compare the fitted curve with the original edge curve to detect uneven wear.

[0072] Step 8.1: Extract the master curve based on the edge detection results from Step 7. The carbon skateboard edge detection results contain edge information for both the long and short sides. However, when performing uneven wear detection tasks, only the long edge of the carbon skateboard needs to be considered; the presence of the short edge may introduce interference and affect the accuracy of wear identification. Therefore, this invention proposes a master curve extraction method for carbon skateboard edge images with many boundary connection lines, based on a combination of skeletonization processing, graph structure modeling, and sliding window slope analysis. The core process is as follows: (1) Skeleton extraction: First, the input image is binarized, and all non-white areas are regarded as the foreground. Then, the Skeletonize algorithm is used to skeletonize the image, compressing the lines into a single-pixel width structure, thereby preserving the topological features of the image and simplifying subsequent processing.

[0073] (2) Connected Component Identification: All connected components in the skeleton graph are identified using the Connected Component Labeling method and treated as candidate curves. Each connected component may correspond to a main curve, an edge connection line, or noise.

[0074] (3) Graph structure construction and path extraction: For each connected region, a pixel-level graph structure is constructed, where each pixel is a graph node and edges are established between adjacent pixels. The ordered pixel path of the entire curve is extracted by starting from the endpoint with degree 1 using a depth-first traversal algorithm.

[0075] (4) For each path, a three-point sliding window is used to evaluate its local slope. Let three consecutive points in the path be... Then the local slope at that location is calculated using the central difference formula as follows: (5) If in the formula If the threshold is exceeded, the point is considered to be in a connection segment with an excessively steep slope and is removed. This operation can effectively break up interfering line segments that are connected to the main curve but deviate in geometric direction.

[0076] (5) Main curve selection: Sort all the remaining curve paths by length and select the first two as the main curves.

[0077] Step 8.2: To achieve accurate curve reconstruction and modeling, this invention employs a method combining equal arc length sampling and B-spline fitting. The visual comparison between the fitted curve and the original skeleton curve is shown below. Figure 8As shown. The core idea of ​​this method is to construct a continuous and differentiable smooth curve model based on discrete skeleton points. First, for the two longest skeleton curves retained after sliding window slope analysis, the cumulative arc length is calculated according to their pixel sequence. Then, a fixed number of sampling points are selected on each curve according to the principle of equal arc length, thereby ensuring that the sampling points are evenly distributed in geometric space and avoiding the impact of inconsistent local pixel density on fitting accuracy and stability. After completing the equal arc length sampling, B-spline interpolation is used to model the sampling points and generate the final continuous curve expression. To further improve the robustness of the fitting, a smoothing factor is introduced into the B-spline fitting to adjust the degree of curve fitting to the sampling points. The introduction of this smoothing factor can effectively suppress the interference of local sharp noise or small-amplitude concavities on the overall curve shape, thereby avoiding overfitting and ensuring that the fitted curve better conforms to the overall trend of the actual wear profile.

[0078] Step 8.3: Uneven Wear Detection. Based on the wear edge detection results and curve fitting results, calculate the vertical distance between each point on the skeleton curve and its corresponding point on the fitted curve to determine whether abnormal wear exists. When this distance exceeds a preset threshold, it can be determined that uneven wear exists in that area.

Claims

1. A machine vision-based method for detecting wear on carbon sliding plates of railway pantographs, characterized in that, First, the SCI model incorporating a channel attention mechanism is used to enhance the illumination of the pantograph image. Then, the Real-ESRGAN model is used for super-resolution reconstruction of the enhanced pantograph image. Next, an improved M... 3 Net model for pantograph carbon slider region localization, improved M 3 The Net model encoder employs a residual post-normalization structure and a log-interval continuous relative position deviation method. The decoder uses MIB and MAB modules. Then, the DexiNed model, which incorporates CA blocks and spatial attention mechanisms, is used to detect the wear edges of the carbon slide plate. Finally, the wear edge detection results are used to determine whether there is uneven wear.

2. The method for detecting wear of carbon sliding plates of railway pantographs based on machine vision as described in claim 1, characterized in that, Includes the following steps: Step 1: Add noise of varying degrees to the original pantograph image and perform cropping operations to generate an expanded dataset; Step 2: Input the pantograph image obtained in Step 1 into the SCI low-light enhancement model with channel attention mechanism, and output the image with enhanced illumination; Step 3: Input the image output from Step 2 into the Real-ESRGAN model to output a high-resolution image; Step 4: Use the Labelme tool to perform two types of annotations on the image output in Step 2: carbon skateboard segmentation mask and ground truth edge information. Step 5: Use the image output from Step 3 and the labels from Step 4 as input to train the improved M³Net localization model. The model encoder adopts a residual post-normalization structure and a log-interval continuous relative position deviation method. The decoder adopts MIB and MAB modules and outputs a mask of the pantograph carbon sliding plate area. Step 6: Using the carbon skateboard area mask output in Step 5, segment the carbon skateboard image based on the pixel value difference between the target area and the non-target area of ​​the mask. Step 7: Take the carbon skateboard image output in Step 6 and the edge annotation labels from Step 4 as input to train the improved DexiNed model. After each encoding block, a CA module is introduced into the model, and after the upsampling module, a spatial attention mechanism is introduced to output the carbon skateboard wear edge detection results. Step 8: Determine whether there is uneven wear based on the detection results output in Step 7.

3. The method for detecting wear of carbon sliding plates of railway pantographs based on machine vision as described in claim 2, characterized in that, Step 2 specifically includes the following steps: Step 2.1: Input the original pantograph image into the SCI model illumination learning module, learn the image illumination information through the weight-sharing parameterized operator, and generate residual terms and stage illumination output; Step 2.2: Input the output of Step 2.1 into the self-calibration module, connect the input of each stage with the original low-light image, explore the convergence behavior between stages through self-calibration mapping, and make the results of each stage converge to the same state. Step 2.3: The input channel attention denoising module of the output of step 2.2 first performs global pooling to convert each channel of the feature map into a scalar value; second, it generates channel weights to [0,1] by sequentially performing dimensionality reduction, ReLU activation, dimensionality increase, and Sigmoid normalization through a fully connected layer; finally, it multiplies the weights with the original feature map channel by channel to output the weighted feature map, which is the image after illumination enhancement.

4. The method for detecting wear of carbon sliding plates of railway pantographs based on machine vision as described in claim 2, characterized in that, Step 5 specifically includes the following steps: Step 5.1: Input the image output from Step 3 and the labels from Step 4 into the encoder and use the Swing Transformer with a residual post-normalization structure and log-interval continuous relative position deviation for feature extraction. Step 5.2: Input the multi-level features extracted in Step 5.1 into the decoder, and fuse them with the MAB global-window self-attention module through the MIB cross-attention mechanism to train the improved M³Net localization model, and output the pantograph carbon skateboard region mask.

5. The method for detecting wear of carbon sliding plates of railway pantographs based on machine vision as described in claim 4, characterized in that, The logarithmic interval continuous relative position deviation method in step 5.1 is to use a meta-network based on relative coordinates ( x , y The deviation value is generated using the following formula: Then, a feature map with geometric relationship encoding is output.

6. The method for detecting wear of carbon sliding plates of railway pantographs based on machine vision as described in claim 2, characterized in that, Step 7 specifically includes the following steps: Step 7.1: Pair the carbon skateboard image output in Step 6 with the edge annotation labels in Step 4, input it into the DexiNed network, and extract multi-level edge feature maps sequentially through the Dexi sub-blocks to complete the hierarchical extraction process of basic edge features. Step 7.2: Insert a CA module after the output of each encoding block of the Dexi network to perform orientation-aware enhancement operation on the current feature map: First, decouple the global average pooling into horizontal and vertical position encodings to generate an orientation-aware attention weight map; then multiply the weight map with the original feature map pixel by pixel to achieve dynamic enhancement of the real edge response and suppression of background noise. Step 7.3: The feature map after directional enhancement is fed into the upsampling module. The feature map resolution is gradually restored through the upsampling operation to generate edge feature images at various stages containing edge details at different scales, providing multi-scale input for subsequent feature fusion. Step 7.4: Insert a spatial attention mechanism after the upsampling module output and before multi-scale feature fusion: First, dynamically generate a spatial weight map, then adaptively adjust the fusion ratio of high-resolution features and low-resolution features according to the pixel position, and finally repair the edge breakage area and compensate for the geometric misalignment caused by downsampling through weighted fusion. Step 7.5: Integrate the fused multi-level edge features to generate and output the final carbon skateboard wear edge detection result image.

7. The method for detecting wear of carbon sliding plates of railway pantographs based on machine vision as described in claim 2, characterized in that, Step 8 specifically includes the following steps: Step 8.1: Extract the main curves based on the detection results output in Step 7 to obtain two main curves; Step 8.2: Calculate the cumulative arc length of the two main curves, sample a fixed number of points according to the principle of equal arc length, and then use B-spline interpolation with a smoothing factor to fit the sampled points to generate a continuous curve. Step 8.3: Calculate the vertical distance between each point on the skeleton curve and the corresponding point on the fitted curve. If the distance exceeds the preset threshold, it is determined to be uneven wear.

8. The method for detecting wear of carbon sliding plates of railway pantographs based on machine vision as described in claim 7, characterized in that, Step 8.1 specifically includes the following steps: Step 8.1.1: Binarize the input edge image, determine all non-white pixels as foreground regions, and use the Skeletonize algorithm to compress the lines into a single-pixel-width skeleton structure, preserving the topological features of the original edges; Step 8.1.2: Perform connected component labeling on the skeletonized image to identify all interconnected pixel sets and label each connected component as an independent candidate curve object; Step 8.1.3: Construct a pixel-level graph structure for each connected region: use pixels as nodes and establish edges connecting adjacent pixels; start the depth-first traversal algorithm from the endpoint with degree 1 to extract the complete ordered pixel path sequence; Step 8.1.4: Use a three-point sliding window to evaluate the local slope of each path: calculate the central difference slope value of three consecutive points; if the slope exceeds the preset threshold, determine the corresponding point as an interference point with geometric deviation and remove it, and break the non-main curve connection segment; Step 8.1.5: Sort all paths after removing interference in descending order of pixel count, and select the two longest paths as the main curves of the long side of the carbon skateboard.

Images