Pantograph slide plate abnormal state detection method based on convolutional neural network

Abstract

The invention discloses a pantograph slide plate abnormal state detection method based on a convolutional neural network, and the method comprises the steps: obtaining a pantograph slide plate surface defect image, and carrying out the image preprocessing, and obtaining a data set; making a training sample set and a test sample set based on the data set; constructing a pantograph slide plate segmentation model based on a convolutional neural network and training the pantograph slide plate segmentation model; verifying whether the accuracy exceeds a threshold value or not, and obtaining a trained pantograph slide plate segmentation model; collecting a pantograph slide plate image in real time and carrying out image preprocessing; performing image segmentation by adopting the trained pantograph slide plate segmentation model to obtain a pantograph slide plate area image; and inputting the pantograph slide plate area image into the trained slide plate abnormal state detection network to obtain a pantograph slide plate abnormal state detection result. The method has the advantages that the detection precision is higher, the robustness is higher, and multiple abnormal types can be detected at the same time generally.

Description

Technical Field

[0001] This invention relates to the field of electric locomotive safety operation technology, specifically to a method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks. Background Technology

[0002] The pantograph is a crucial electrical device for electric locomotives to obtain power from the overhead contact line. The pantograph's contact plate, as a key component, directly contacts the contact line and plays a vital role in collecting current from it. During high-speed operation, excessive wear, defects, cracks, and other faults in the contact plate can affect the locomotive's normal power supply, causing pantograph-catenary faults. Therefore, real-time monitoring of the contact plate's condition is essential to ensure timely detection of abnormalities, timely replacement of the pantograph contact plate, and guaranteeing the safety and reliability of train operations.

[0003] Currently, the mainstream pantograph status detection methods at home and abroad can be divided into four categories according to their technical means: manual climbing detection, sensor-based detection, traditional image processing detection, and deep learning-based detection.

[0004] (1) Manual summit detection

[0005] Currently, some locomotive depots and subway depots in China still use traditional manual inspection methods. After an electric locomotive enters the depot and stops, maintenance personnel climb onto the roof and rely on their experience to inspect the carbon steel plates for cracks, missing pieces, grooves, burns, and abnormal wear between the carbon steel strips on the bow head. If any problems are found, the carbon steel plates should be repaired or replaced according to the work instructions. This manual inspection method relies excessively on the experience and judgment of maintenance personnel, often resulting in missed or incorrect inspections. Furthermore, manual inspection requires climbing onto the roof after the locomotive has entered the depot, stopped, and the power has been cut off, which undoubtedly increases workload and reduces efficiency.

[0006] (2) Sensor-based detection

[0007] Later, with the emergence of sensor technology, different types of sensor elements began to be applied to the field of pantograph sliding plate condition detection.

[0008] Existing sensor-based pantograph spool abnormality detection methods have achieved the detection of some anomalies and are more efficient than manual inspection. However, these methods are costly and have a narrow scope of application. Different types of sensors often need to be designed and installed for different anomalies, and not all pantographs are equipped with the necessary sensors.

[0009] (3) Detection based on traditional image processing techniques

[0010] Image processing technology uses cameras to capture images of the pantograph and its sliding plate, and then uses intelligent image recognition methods to detect and identify faults in the pantograph and its sliding plate.

[0011] The pantograph sliding plate anomaly detection method based on traditional image processing technology is lower in cost and has better real-time performance compared to manual climbing detection and sensor-based detection methods. However, its detection algorithm is complex and often requires a series of preprocessing operations. Furthermore, the detection effect is easily affected by factors such as lighting and noise, resulting in low detection accuracy and weak generalization ability.

[0012] (4) Detection based on deep learning technology

[0013] In recent years, deep learning convolutional neural networks have attracted much attention from scholars due to their high detection accuracy, strong generalization ability, and end-to-end learning capabilities.

[0014] Compared to traditional pantograph trolley anomaly detection methods, deep learning-based pantograph status detection methods not only offer higher detection accuracy and stronger robustness, but also typically can detect multiple anomaly types simultaneously. Secondly, image detection has a significant cost advantage; industrial cameras are cheaper than laser sensors. Finally, current vision technologies, primarily based on convolutional neural networks, have made significant progress in areas such as object detection and image segmentation, overcoming the shortcomings of traditional digital image processing algorithms, such as poor robustness in complex and changing environments, low detection efficiency, and high computational overhead. Compared to laser scanning 3D reconstruction and contact detection digital signal processing, these methods offer higher robustness and better real-time performance.

[0015] However, current research on deep learning-based pantograph detection, both domestically and internationally, mainly focuses on the pantograph head, horns, and fault classification. Research specifically targeting abnormal pantograph slider detection is scarce. There are two main reasons for this: ① In actual pantograph slider image acquisition, external light interference and camera metering inaccuracies often result in overexposure or underexposure, leading to poor image quality. This causes numerous missegmentations and missed segments when using traditional image processing techniques for pantograph image segmentation. Furthermore, the presence of shadows and other components interferes with edge detection, resulting in the detection of many irrelevant edge details. ② In real-world operating conditions, acquiring abnormal pantograph slider data is extremely difficult. Actual abnormal data is often very limited, making it impossible to accurately detect abnormal states using pantograph slider abnormality datasets. Summary of the Invention

[0016] The purpose of this invention is to provide a method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks. This method uses machine vision to capture pantograph images, combines deep learning technology with traditional image processing technology to segment the acquired images, extracts pantograph sliding plate images, and uses a sliding plate state detection network to determine whether the carbon sliding plate in the pantograph sliding plate image has cracks or abnormal wear. This method can effectively solve the technical problems existing in the background technology.

[0017] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0018] A method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks, the key of which includes the following steps:

[0019] Step 1: Obtain images of surface defects on the pantograph slider and perform image preprocessing to obtain a dataset;

[0020] Step 2: Create a training sample set and a test sample set based on the dataset, where the data in the training sample set is labeled;

[0021] Step 3: Construct a pantograph slide segmentation model based on a convolutional neural network, and train the pantograph slide segmentation model using a labeled training sample set;

[0022] Step 4: Use the test sample set to test the accuracy of the pantograph sliding plate segmentation model and determine whether the accuracy exceeds the threshold. If it does, obtain the trained pantograph sliding plate segmentation model. Otherwise, increase the number of samples in the training sample set or modify the parameters of the pantograph sliding plate segmentation model and retrain the pantograph sliding plate segmentation model until the accuracy exceeds the threshold and then end the training.

[0023] Step 5: Acquire images of the pantograph slider in real time and perform image preprocessing;

[0024] Step 6: Input the preprocessed pantograph region image into the trained pantograph slide segmentation model to perform image segmentation and obtain the pantograph slide region image.

[0025] Step 7: Input the pantograph skateboard area image into the trained skateboard abnormal state detection network to obtain the pantograph skateboard abnormal state detection results.

[0026] Furthermore, the image preprocessing described in steps 1 and 5 includes image normalization, image filtering, and image enhancement.

[0027] Furthermore, the specific steps of the image filtering process are as follows:

[0028] Step A1: Perform grayscale mapping preprocessing on the original image after image normalization to obtain a grayscale image;

[0029] Step A2: Perform median filtering on the grayscale image according to the median filtering algorithm to obtain a median-filtered image;

[0030] Step A3: Apply a mean filter to the median-filtered image to obtain the filtered median-mean-filtered image.

[0031] Furthermore, the specific steps of the median filtering algorithm are as follows:

[0032] A k×k window is used to process the grayscale image;

[0033] Calculate the average gray value of the pixels in the neighborhood of the center pixel within the filtering window;

[0034] Sort all pixels within the filtering window in descending order of their grayscale values;

[0035] Divide all pixels into several groups of pixels after sorting in descending order;

[0036] The median gray value of each pixel group is the median of the desired neighborhood.

[0037] Output the median-filtered image processed by the median filtering algorithm.

[0038] Furthermore, the specific steps of the image enhancement process are as follows:

[0039] Step B1: Input the image after the image filtering process;

[0040] Step B2: Convert the input image from RGB space to HSV space to obtain the hue component, lightness component, and saturation component;

[0041] Step B3: Perform adaptive weight calculation on the luminance component using the improved MSRCR algorithm to obtain the enhanced luminance component;

[0042] Step B4: Inversely transform the hue component, the enhanced luminance component, and the saturation component to the RGB space to obtain the enhanced image.

[0043] Furthermore, in step B3, the enhanced luminance components are obtained by adaptively calculating weights using the improved MSRCR algorithm as follows:

[0044] Within the [0,255] region, the pixels of the brightness component are classified into dark pixels, medium-bright pixels, and bright pixels using a standard normal distribution model;

[0045] Calculate the likelihood probabilities of the dark pixels, medium-bright pixels, and bright pixels respectively;

[0046] The weights of the dark pixels, the medium-bright pixels, and the bright pixels are calculated based on the likelihood probabilities of the dark pixels, the medium-bright pixels, and the bright pixels.

[0047] The luminance component is enhanced by amplifying each pixel based on the weights of the dark pixels, the medium-bright pixels, and the bright pixels, thus obtaining the enhanced luminance component.

[0048] Furthermore, the ratio of the number of samples in the training sample set to the number of samples in the test sample set is 8:2.

[0049] Furthermore, the process of creating the training sample set in step 2 is as follows:

[0050] Step C1: Based on the obtained dataset, automatically cut and extract the primary sample set using the sliding window algorithm;

[0051] Step C2: Extract features from each image in the primary sample set and classify them using a clustering algorithm. Remove the samples without pantographs in the primary sample set to obtain the intermediate sample set.

[0052] Step C3: Input the intermediate sample set into the semantic segmentation model in batches for iterative optimization training, and predict the samples after each iteration optimization, remove the erroneous samples in the intermediate sample set, and obtain the training sample set.

[0053] Furthermore, the pantograph slider segmentation model described in step 3 includes:

[0054] The encoder module is used to perform multiple convolutional pooling downsampling operations on the image to extract multi-scale primary features;

[0055] The decoder module is used to perform deconvolution upsampling on the multi-scale primary features extracted by the encoder module to extract multi-scale object-level semantic features;

[0056] The multi-scale feature deconvolution module is used to perform deconvolution operations on multi-scale primary features and multi-scale object-level semantic features respectively, so as to unify the feature map size;

[0057] The feature fusion module superimposes and fuses the two feature maps output by the multi-scale feature deconvolution module;

[0058] The category determination module is used to perform category determination on the superimposed and fused feature maps to achieve image segmentation and obtain the pantograph slider region image.

[0059] Furthermore, the skateboard anomaly detection network includes a feature extraction module, a dilated convolution module, an attention module, an upsampling module, and a convolution prediction module, wherein:

[0060] The feature extraction module is used to extract multi-scale features from the input pantograph area image to obtain multi-scale feature maps. The dilated convolution module is used to perform dilated convolution processing on the lowest-level feature map in the multi-scale feature map using a dilated spatial convolution pooling pyramid structure, and then input it into the attention module and the upsampling module. The attention module is used to perform iterative optimization processing on the feature maps of each scale except the lowest-level feature map in the multi-scale feature map and the corresponding scale feature maps output by the upsampling module using the erase attention method. The upsampling module is used to perform upsampling processing on the feature maps of different scales output by the dilated convolution module and the attention module. The convolution prediction module is used to perform convolution and prediction on the feature maps of different scales output by the dilated convolution module and the attention module to obtain pantograph sliding plate abnormality detection results based on different scales.

[0061] Furthermore, the loss function for the pantograph sliding plate abnormality detection result is:

[0062]

[0063] Where Loss0 is the supervised loss function at the original resolution. To compute the supervised loss function at scale i, where i ranges from {2, 3, ..., n}, and n is the total number of selected original resolution out-of-scale values, G... j G is the true label for pixel j. j =1 indicates a pixel representing an abnormal state of the pantograph slider, P j Let j be the probability that pixel j is in an abnormal state of the pantograph slider, and N be the total number of pixels in an image.

[0064] Furthermore, the attention module employs an erasure attention method to iteratively optimize the initial skateboard abnormality detection results. The specific process of the erasure attention method is as follows:

[0065] First, subtract the probability map of the multi-scale skateboard anomaly detection to be optimized from 0.5 to obtain an inverted weight matrix. Set the weights of the positions with values ​​less than 0 in the weight matrix to 0, erase the parts of the probability map of the multi-scale skateboard anomaly detection to be optimized with probabilities greater than 0.5, and then multiply the weight matrix by 2 to scale the weights to between 0 and 1 to obtain the output weight matrix.

[0066] The significant effects of this invention are:

[0067] 1. This invention uses machine vision to capture pantograph images, combines deep learning technology with traditional image processing technology, segments the acquired on-site images, extracts the pantograph sliding plate, and judges whether the carbon sliding plate has cracks or abnormal wear based on the sliding plate state detection network. Compared with the traditional pantograph sliding plate detection method, the detection method described in this invention has higher detection accuracy, stronger robustness, and can generally detect multiple abnormal types at the same time.

[0068] 2. Pantograph Skateboard Image Segmentation Based on Convolutional Neural Networks: Traditional image segmentation methods often result in numerous missegmentations and omissions. This invention relies on semantic segmentation technology and utilizes a convolutional neural network to construct an image segmentation model. This model accurately segments the pantograph skateboard, thus avoiding the large number of missegments and omissions that occur with traditional image segmentation methods. It also avoids the interference caused by shadows and other components on edge detection, which leads to the detection of many irrelevant edge information. This effectively improves the segmentation effect of pantograph skateboard images.

[0069] 3. By using deep learning (convolutional neural network) to construct different convolutional neural network models, the real-time detection of the pantograph's sliding plate status (whether cracks or abnormal wear have appeared) is achieved. This overcomes the shortcomings of traditional methods, such as poor robustness in complex and variable environments, low detection efficiency, and high computational overhead. Compared with laser scanning 3D reconstruction and contact detection digital signal processing, it has higher robustness and better real-time performance. Attached Figure Description

[0070] Figure 1 This is a flowchart of the method of the present invention;

[0071] Figure 2 This is a schematic diagram of the pantograph sliding plate segmentation model;

[0072] Figure 3 This is a schematic diagram of the structure of the skateboard abnormal state detection network. Detailed Implementation

[0073] The specific embodiments and working principles of the present invention will be further described in detail below with reference to the accompanying drawings.

[0074] like Figure 1 As shown, a method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks includes the following steps:

[0075] Step 1: Obtain images of surface defects on the pantograph slider and perform image preprocessing to obtain a dataset;

[0076] Due to the complex environment in which the pantograph acquires images, the acquired images of the locomotive pantograph are preprocessed. This image preprocessing includes image normalization, image filtering, and image enhancement; specifically:

[0077] The specific steps of the image filtering process are as follows:

[0078] Step A1: Perform grayscale mapping preprocessing on the original image after image normalization to obtain a grayscale image;

[0079] Step A2: Perform median filtering on the grayscale image according to the median filtering algorithm to obtain a median-filtered image. The specific steps of the median filtering algorithm are as follows:

[0080] A 3×3 window is used to process grayscale images;

[0081] Define each pixel in the 3×3 filter window as x1~x9. Calculate the average gray value of the pixels in the neighborhood of the center pixel x4 in the 3×3 filter window. If the average value is 0, then directly set the gray value of pixel x4 to 0.

[0082] All pixels x1 to x9 within the filter window are sorted in descending order of their grayscale values, and the sorted pixels are represented as y1 to y9.

[0083] Divide all pixels y1 to y9, after sorting in descending order, into three groups of pixels;

[0084] Calculate the median gray value y for each pixel group mid , which is the median of the neighborhood we are looking for;

[0085] Output the median-filtered image processed by the median filtering algorithm.

[0086] Step A3: Apply a mean filter to the median-filtered image to obtain the filtered median-mean-filtered image.

[0087] A combination of median filtering and mean filtering was used to denoise the pantograph image.

[0088] The specific steps of the image enhancement process are as follows:

[0089] Step B1: Input image I after the image filtering process;

[0090] Step B2: Convert the input image I from RGB space to HSV space to obtain the hue component H, the brightness component V0, and the saturation component S;

[0091] Step B3: Perform adaptive weight calculation on the luminance component V0 using the improved MSRCR algorithm to obtain the enhanced luminance component V1;

[0092] The specific process of the improved MSRCR algorithm is as follows:

[0093] Within the [0,255] region, the pixels of the brightness component are classified into dark pixels [0,85), medium-bright pixels [85,170), and bright pixels [170,255) using a standard normal distribution model.

[0094] Calculate the likelihood probabilities of the dark pixels, medium-bright pixels, and bright pixels respectively;

[0095] The weights α1, α2, and α3 of the dark pixels are calculated based on the likelihood probability P1 of the dark pixels, the likelihood probability P2 of the medium-bright pixels, and the likelihood probability P3 of the bright pixels.

[0096] The formula for calculating the weight is:

[0097] Among them, P i represents the likelihood probabilities of dark, medium-bright, and bright pixels;

[0098] The brightness component V1 is enhanced by weighting each pixel in the brightness component according to the weight α1 of the dark pixel, the weight α2 of the medium bright pixel, and the weight α3 of the bright pixel.

[0099] The formula for calculating the luminance component V1 is:

[0100] Among them, C i Color restoration factor at each scale, is the transpose of the pixel weights, f is the Gaussian filter at each scale, and * is the convolution operation.

[0101] Step B4: Inversely transform the hue component H, the enhanced luminance component V1, and the saturation component S into the RGB space to obtain the enhanced image I'.

[0102] Through the image filtering and enhancement processes described above, an improved MSRCR image enhancement algorithm is used after noise reduction to improve image quality degradation and detail loss caused by lighting conditions and haze. This overcomes the problems of overexposure or underexposure and poor image quality caused by external light interference and camera metering inaccuracies. By filtering the acquired image and using the improved MSRCR image enhancement algorithm for image enhancement processing, a higher quality skateboard image with stronger edge features is obtained, preparing for subsequent skateboard segmentation.

[0103] Step 2: Create a training sample set and a test sample set based on the dataset. The data in the training sample set is labeled, and the ratio of the number of samples in the training sample set to the number of samples in the test sample set is 8:2.

[0104] The process of creating the training sample set is as follows:

[0105] Step C1: Based on the obtained dataset, automatically cut and extract the primary sample set using the sliding window algorithm;

[0106] Step C2: Extract features from each image in the primary sample set and classify them using a clustering algorithm. Remove the samples without pantographs in the primary sample set to obtain the intermediate sample set.

[0107] Step C2.1: Extract the texture and color features of each image in the primary sample set;

[0108] Step C2.2: Merge texture features and color features by direct stitching to obtain the image's fused feature set;

[0109] Step C2.3: Perform clustering processing on the fusion feature set of the image using a clustering algorithm, analyze the categories generated by the clustering of the primary sample set, delete samples containing cloud and fog occlusion and shadow occlusion in the primary sample set, and retain samples containing pantographs.

[0110] Step C2.3.1: Set the neighborhood distance threshold between elements, and set the minimum number of contained points within the neighborhood radius with the neighborhood distance threshold as the threshold. If the number of elements contained in an element within the neighborhood radius is not less than the minimum number of contained points, then the element is a core object.

[0111] Step C2.3.2: Identify the core objects in the image fusion feature set and add them to the core object set;

[0112] Step C2.3.3: Randomly select an unvisited element from the core object set, first mark it as visited, then mark its category, and finally add the unvisited non-core objects within the neighborhood radius to the seed set;

[0113] Step C2.3.4: Determine if the seed set is empty. If it is, the cluster has been generated and proceed to step C2.3.6; otherwise, proceed to step C2.3.5.

[0114] Step C2.3.5: Randomly select a seed from the seed set and determine if the seed is a non-core object. If so, add the objects within the radius of the seed's domain to the seed set and jump to step C2.3.4.

[0115] Step C2.3.6: Determine whether all elements in the core object set have been accessed. If so, complete the clustering of the fusion feature set of the image; otherwise, proceed to step C2.3.3.

[0116] Step C3: Input the intermediate sample set into the semantic segmentation model in batches for iterative optimization training, and predict the samples after each iteration. Remove the erroneous samples from the intermediate sample set to obtain the training sample set. Specifically:

[0117] Step C3.1: Train the selected semantic segmentation model using an intermediate sample set;

[0118] Step C3.2: Use the trained semantic segmentation model to predict the samples in the intermediate sample set, and compare the prediction results with the true values ​​of the samples to obtain the prediction accuracy of each sample.

[0119] Step C3.3: Delete samples whose prediction accuracy is less than the preset overlap rate threshold;

[0120] Step C3.4: Determine whether the model accuracy is not less than the preset accuracy threshold. If yes, complete the creation of the target sample set; otherwise, return to step C3.1.

[0121] Step 3: Construct a pantograph slide segmentation model based on a convolutional neural network, and train the pantograph slide segmentation model using a labeled training sample set;

[0122] The pantograph slider segmentation model mentioned in step 3 includes:

[0123] The encoder module is used to perform multiple convolutional pooling downsampling operations on the image to extract multi-scale primary features;

[0124] The decoder module is used to perform deconvolution upsampling on the multi-scale primary features extracted by the encoder module to extract multi-scale object-level semantic features;

[0125] The multi-scale feature deconvolution module is used to perform deconvolution operations on multi-scale primary features and multi-scale object-level semantic features respectively, so as to unify the feature map size;

[0126] The feature fusion module superimposes and fuses the two feature maps output by the multi-scale feature deconvolution module;

[0127] The category determination module is used to perform category determination on the superimposed and fused feature maps to achieve image segmentation and obtain the pantograph slider region image.

[0128] That is, the pantograph slide segmentation model includes an encoder-decoder (without a class decision layer) and deconvolutional upsampling of two types of features (multi-scale feature deconvolution of the encoder and abstract high-level semantic feature deconvolution of the decoder). The two types of feature deconvolutional upsampling corresponding to the encoder and decoder modules include:

[0129] (1) Multi-scale feature deconvolution upsampling of the encoder module: Assuming the size of the input sample image is 64*64, the encoder module includes 3 convolutional layers with constant size and 3 pooling layers. Each of the first 3 convolutional layers is followed by a pooling layer (which reduces the feature size by half). The sizes of these 3 convolutional layers are 64*64, 32*32 and 16*16 respectively. The multi-level features such as the spectrum, texture and geometric shape of the object will be gradually extracted. The multi-scale feature deconvolution module then performs 2, 3 and 4 deconvolutions on these 3 convolutional layers respectively (each deconvolution doubles the feature size) to expand the feature size to 128*128.

[0130] (2) Deconvolution of abstract high-level semantic features in the decoder module: The decoder module includes 3 deconvolution layers and 3 convolution layers with constant size. Each deconvolution layer (which doubles the feature size) is followed by a convolution layer. These 3 convolution layers will extract abstract high-level semantic features at different scales, with sizes of 16*16, 32*32, and 64*64 respectively. The multi-scale feature deconvolution module then performs 4, 3, and 2 deconvolutions on these 3 convolution layers respectively (each deconvolution doubles the feature size), so that the feature size is increased to 128*128.

[0131] Next, the feature fusion module combines the features of these 6 convolutional layers with a feature size of 256*256, then passes them through two convolutional layers with the same size in the category determination module, and finally the Softmax layer performs the category determination.

[0132] It should be noted that in encoder-decoder semantic segmentation neural networks (such as U-Net and SegNet), the encoder mainly consists of convolutional layers and pooling layers to extract multi-level features of the object. The size of its convolutional layers gradually decreases, extracting multi-level features such as the object's spectrum, texture, and geometric shape. The decoder mainly consists of deconvolutional upsampling layers and class decision layers. Each deconvolution expands the feature size, gradually extracting abstract, high-level semantic features of the object. Therefore, combining the two types of multi-level feature deconvolution between the encoder and decoder will significantly improve the target extraction accuracy of this method, enabling precise identification of pantograph skateboards.

[0133] Accordingly, the image segmentation principle of the pantograph slider segmentation model is as follows: For an image of size n*n, the encoder module first performs convolutional pooling downsampling to extract multi-scale primary features. Then, the decoder module performs deconvolution upsampling on the features downsampled multiple times by the encoder module to extract multi-scale object-level semantic features. Next, the multi-scale feature deconvolution module performs deconvolution operations on the multi-scale primary feature map output by the encoder module and the multi-scale object-level semantic feature map output by the decoder module, expanding the size of all feature maps of different scales to (n*m)*(n*m). Then, the feature fusion module superimposes and fuses all feature maps of size (n*m)*(n*m). Finally, the category determination module performs category determination on the superimposed and fused feature maps to achieve image segmentation, obtaining a semantic segmentation result of size (n*m)*(n*m), which is the pantograph slider region image, where m = 2x, m is the multiple of the size of the output semantic segmentation result and the input image, and x is a positive integer.

[0134] Furthermore, the loss function in the pantograph sliding plate segmentation model training process described in step 3 is:

[0135] LOSS1=-[y*lnp+(1-y)*ln(1-p)],

[0136] Where y is the label of the training sample pixel, i.e., the pantograph skateboard pixel is 1 and the background pixel is 0, and p represents the probability that the training sample pixel is predicted to be the pantograph skateboard.

[0137] Step 4: Use the test sample set to test the accuracy of the pantograph sliding plate segmentation model and determine whether the accuracy exceeds the threshold. If it does, obtain the trained pantograph sliding plate segmentation model. Otherwise, increase the number of samples in the training sample set or modify the parameters of the pantograph sliding plate segmentation model and retrain the pantograph sliding plate segmentation model until the accuracy exceeds the threshold and then end the training.

[0138] Preprocessing the training samples can enhance the accuracy of target recognition. A large number of pantograph skateboard training samples and the required training labels from the image library are fed into a convolutional neural network (CNN) to obtain a CNN model for skateboard recognition. Then, the test samples from the image library are also preprocessed and fed into the CNN model to obtain the skateboard recognition prediction results. The given skateboard prediction results are compared with the labels in the test samples to obtain the final CNN test results.

[0139] After segmenting the skateboard using the convolutional neural network model, abnormal state detection of the skateboard can be performed. This project first establishes a pantograph skateboard dataset, and then applies model matching to the segmented skateboard images to achieve fault identification and location detection of carbon skateboards.

[0140] Step 5: Acquire images of the pantograph sliding plate in real time and perform image preprocessing. This step is the same as the image preprocessing method in Step 1.

[0141] Step 6: Input the preprocessed pantograph region image into the trained pantograph slide segmentation model to perform image segmentation and obtain the pantograph slide region image.

[0142] Step 7: Input the pantograph sliding plate area image into the trained sliding plate abnormality detection network to obtain the pantograph sliding plate abnormality detection results. The abnormality includes abnormal wear and crack faults.

[0143] The training of the skateboard abnormal state detection network is carried out using the collected pantograph skateboard full life cycle image data, so that the skateboard abnormal state detection network can perform deep learning on the normal wear and tear of each life node of the pantograph skateboard, thereby detecting abnormal states on the input real-time collected skateboard images.

[0144] The skateboard abnormal state detection network includes a feature extraction module, a dilated convolution module, an attention module, an upsampling module, and a convolution prediction module, wherein:

[0145] The feature extraction module is used to extract multi-scale features from the input pantograph area image to obtain multi-scale feature maps. The dilated convolution module is used to perform dilated convolution processing on the lowest-level feature map in the multi-scale feature map using a dilated spatial convolution pooling pyramid structure, and then input it into the attention module and the upsampling module. The attention module is used to perform iterative optimization processing on the feature maps of each scale except the lowest-level feature map in the multi-scale feature map and the corresponding scale feature maps output by the upsampling module using the erase attention method. The upsampling module is used to perform upsampling processing on the feature maps of different scales output by the dilated convolution module and the attention module. The convolution prediction module is used to perform convolution and prediction on the feature maps of different scales output by the dilated convolution module and the attention module to obtain pantograph sliding plate abnormality detection results based on different scales.

[0146] In this embodiment, the attention module uses the erase attention method to iteratively optimize the initial skateboard abnormal state detection results. The specific process of the erase attention method is as follows:

[0147] First, subtract the probability map of the multi-scale skateboard anomaly detection to be optimized from 0.5 to obtain an inverted weight matrix. Set the weights of the positions with values ​​less than 0 in the weight matrix to 0, erase the parts of the probability map of the multi-scale skateboard anomaly detection to be optimized with probabilities greater than 0.5, and then multiply the weight matrix by 2 to scale the weights to between 0 and 1 to obtain the output weight matrix.

[0148] The loss function for detecting abnormal states of the pantograph sliding plate is:

[0149]

[0150] Where Loss0 is the supervised loss function at the original resolution. To compute the supervised loss function at scale i, where i ranges from {2, 3, ..., n}, and n is the total number of selected original resolution out-of-scale values, G... j G is the true label for pixel j. j =1 indicates a pixel representing an abnormal state of the pantograph slider, P j Let j be the probability that pixel j is in an abnormal state of the pantograph slider, and N be the total number of pixels in an image.

[0151] This invention employs machine vision to capture pantograph images, combining deep learning technology with traditional image processing techniques to segment the acquired images, extract the pantograph slide plate, and use a slide plate state detection network to determine whether the carbon slide plate has cracks or abnormal wear. Compared with traditional pantograph slide plate detection methods, the detection method described in this invention has higher detection accuracy, stronger robustness, and can generally detect multiple anomaly types simultaneously. Furthermore, compared to traditional image segmentation methods that result in numerous missegments and omissions, this invention relies on semantic segmentation technology and utilizes a convolutional neural network to construct an image segmentation model. This model accurately segments the pantograph slide plate, thus avoiding the large number of missegments and omissions common in traditional image segmentation methods. It also avoids interference from shadows and other components that cause edge detection, preventing the detection of many irrelevant edge information, effectively improving the segmentation effect of pantograph slide plate images.

[0152] The technical solution provided by this invention has been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. It should be noted that those skilled in the art can make several improvements and modifications to this invention without departing from the principles of this invention, and these improvements and modifications also fall within the protection scope of the claims of this invention.

Claims

1. A method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks, characterized in that, Includes the following steps: Step 1: Obtain images of surface defects on the pantograph slider and perform image preprocessing to obtain a dataset; Step 2: Create a training sample set and a test sample set based on the dataset, where the data in the training sample set is labeled; Step 3: Construct a pantograph slide segmentation model based on a convolutional neural network, and train the pantograph slide segmentation model using a labeled training sample set; Step 4: Use the test sample set to test the accuracy of the pantograph sliding plate segmentation model and determine whether the accuracy exceeds the threshold. If it does, obtain the trained pantograph sliding plate segmentation model. Otherwise, increase the number of samples in the training sample set or modify the parameters of the pantograph sliding plate segmentation model and retrain the pantograph sliding plate segmentation model until the accuracy exceeds the threshold and then end the training. Step 5: Acquire images of the pantograph slider in real time and perform image preprocessing; Step 6: Input the preprocessed pantograph region image into the trained pantograph slide segmentation model to perform image segmentation and obtain the pantograph slide region image. Step 7: Input the pantograph skateboard area image into the trained skateboard abnormal state detection network to obtain the pantograph skateboard abnormal state detection results.

2. The method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks according to claim 1, characterized in that: The image preprocessing described in steps 1 and 5 includes image normalization, image filtering, and image enhancement.

3. The method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks according to claim 2, characterized in that: The specific steps of the image filtering process are as follows: Step A1: Perform grayscale mapping preprocessing on the original image after image normalization to obtain a grayscale image; Step A2: Perform median filtering on the grayscale image according to the median filtering algorithm to obtain a median-filtered image; Step A3: Apply a mean filter to the median-filtered image to obtain the filtered median-mean-filtered image.

4. The method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks according to claim 3, characterized in that: The specific steps of the median filtering algorithm are as follows: A k×k window is used to process the grayscale image; Calculate the average gray value of the pixels in the neighborhood of the center pixel within the filtering window; Sort all pixels within the filtering window in descending order of their grayscale values; Divide all pixels into several groups of pixels after sorting in descending order; The median gray value of each pixel group is the median of the desired neighborhood. Output the median-filtered image processed by the median filtering algorithm.

5. The method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks according to claim 2, characterized in that: The specific steps of the image enhancement process are as follows: Step B1: Input the image after the image filtering process; Step B2: Convert the input image from RGB space to HSV space to obtain the hue component, lightness component, and saturation component; Step B3: Perform adaptive weight calculation on the luminance component using the improved MSRCR algorithm to obtain the enhanced luminance component; Step B4: Inversely transform the hue component, the enhanced luminance component, and the saturation component to the RGB space to obtain the enhanced image.

6. The method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks according to claim 5, characterized in that: In step B3, the enhanced luminance components are obtained by adaptively calculating weights using the improved MSRCR algorithm. The specific process is as follows: Within the [0,255] region, the pixels of the brightness component are classified into dark pixels, medium-bright pixels, and bright pixels using a standard normal distribution model; Calculate the likelihood probabilities of the dark pixels, medium-bright pixels, and bright pixels respectively; The weights of the dark pixels, the medium-bright pixels, and the bright pixels are calculated based on the likelihood probabilities of the dark pixels, the medium-bright pixels, and the bright pixels. The luminance component is enhanced by amplifying each pixel based on the weights of the dark pixels, the medium-bright pixels, and the bright pixels, thus obtaining the enhanced luminance component.

7. The method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks according to claim 1, characterized in that: The ratio of the number of samples in the training sample set to the number of samples in the test sample set is 8:

2.

8. The method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks according to claim 1, characterized in that: The process of creating the training sample set in step 2 is as follows: Step C1: Based on the obtained dataset, automatically cut and extract the primary sample set using the sliding window algorithm; Step C2: Extract features from each image in the primary sample set and classify them using a clustering algorithm. Remove the samples without pantographs in the primary sample set to obtain the intermediate sample set. Step C3: Input the intermediate sample set into the semantic segmentation model in batches for iterative optimization training, and predict the samples after each iteration optimization, remove the erroneous samples in the intermediate sample set, and obtain the training sample set.

9. The method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks according to claim 1, characterized in that: The pantograph slider segmentation model mentioned in step 3 includes: The encoder module is used to perform multiple convolutional pooling downsampling operations on the image to extract multi-scale primary features; The decoder module is used to perform deconvolution upsampling on the multi-scale primary features extracted by the encoder module to extract multi-scale object-level semantic features; The multi-scale feature deconvolution module is used to perform deconvolution operations on multi-scale primary features and multi-scale object-level semantic features respectively, so as to unify the feature map size; The feature fusion module superimposes and fuses the two feature maps output by the multi-scale feature deconvolution module; The category determination module is used to perform category determination on the superimposed and fused feature maps to achieve image segmentation and obtain the pantograph slider region image.

10. The method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks according to claim 1, characterized in that: The skateboard abnormal state detection network includes a feature extraction module, a dilated convolution module, an attention module, an upsampling module, and a convolution prediction module, wherein: The feature extraction module is used to extract multi-scale features from the input pantograph area image to obtain multi-scale feature maps. The dilated convolution module is used to perform dilated convolution processing on the lowest-level feature map in the multi-scale feature map using a dilated spatial convolution pooling pyramid structure, and then input it into the attention module and the upsampling module. The attention module is used to perform iterative optimization processing on the feature maps of each scale except the lowest-level feature map in the multi-scale feature map and the corresponding scale feature maps output by the upsampling module using the erase attention method. The upsampling module is used to perform upsampling processing on the feature maps of different scales output by the dilated convolution module and the attention module. The convolution prediction module is used to perform convolution and prediction on the feature maps of different scales output by the dilated convolution module and the attention module to obtain pantograph sliding plate abnormality detection results based on different scales.

11. The method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks according to claim 1 or 10, characterized in that: The loss function for detecting abnormal states of the pantograph sliding plate is: Where Loss0 is the supervised loss function at the original resolution. To compute the supervised loss function at scale i, where i ranges from {2, 3, ..., n}, and n is the total number of selected original resolution out-of-scale values, G... j G is the true label for pixel j. j =1 indicates a pixel representing an abnormal state of the pantograph slider, P j Let j be the probability that pixel j is in an abnormal state of the pantograph slider, and N be the total number of pixels in an image.

12. The method for detecting abnormal states of pantograph sliding plates based on convolutional neural networks according to claim 10, characterized in that: The attention module uses an erasure attention method to iteratively optimize the initial skateboard abnormal state detection results. The specific process of the erasure attention method is as follows: First, subtract the probability map of the multi-scale skateboard anomaly detection to be optimized from 0.5 to obtain an inverted weight matrix. Set the weights of the positions with values ​​less than 0 in the weight matrix to 0, erase the parts of the probability map of the multi-scale skateboard anomaly detection to be optimized with probabilities greater than 0.5, and then multiply the weight matrix by 2 to scale the weights to between 0 and 1 to obtain the output weight matrix.

Images