An image-based method for detecting defects in the pantograph carbon strip

Abstract

The present invention discloses a method for detecting defects of pantograph carbon strips based on images, comprising the following steps: performing image enhancement on the collected carbon strip images to obtain enhanced images; performing image adaptive threshold segmentation on the enhanced images to obtain the image A after adaptive threshold segmentation; calculating the edges of the carbon strips of the image after adaptive threshold segmentation by using the Sobel operator; taking the enhanced image as input data and obtaining the probability map of the segmented carbon strips through the trained LiteSeg semantic segmentation network model; fusing the carbon strip probability map obtained by the LiteSeg network model with the carbon strip edge map obtained by the Sobel operator to obtain the outer contour of the carbon strip edge after fusion; obtaining the defective area of the carbon strip according to the outer contour of the carbon strip edge, calculating the area of the defective area of the carbon strip, and if the area is less than the set threshold, the area is discarded, and if the area is greater than the set threshold, it is determined that the carbon strip is damaged.

Description

Technical Field

[0001] The present invention relates to the field of rail transit, and specifically to a method for detecting defects in the carbon sliding strip of a pantograph based on images. Background Art

[0002] As an important electrical device for an electric locomotive to obtain power from the catenary, timely grasping the state of the pantograph plays an important role in ensuring the normal operation of the locomotive and the maintenance work of the locomotive. Among them, the carbon sliding strip is an important part of the pantograph, which provides the power required for the operation of the locomotive in the sliding state. However, when faults such as wear of the sliding plate occur in the carbon sliding strip of the pantograph, it may cause the pantograph to strike the bow or pull the net, resulting in the train losing power and stopping, and even causing a greater safety threat. Therefore, it is necessary to effectively monitor the pantograph and its sliding plate state.

[0003] In the initial stage of electrified railways, traditional detection of the carbon sliding strip of the pantograph mainly relied on manual observation and recording. However, this method is time-consuming, laborious and inefficient, and it is difficult to complete real-time monitoring. With the continuous development of the electrification cause, intelligent detection technologies have developed rapidly. For intelligent pantograph detection technologies, they are mainly divided into contact detection and non-contact detection. Contact detection mainly detects through various sensors installed on the pantograph of the inspection vehicle. Non-contact detection includes ranging technology and image processing technology detection. Ranging technology refers to the measurement of parameters and the detection of wear of the sliding plate through laser and ultrasonic waves. Image processing technology is to detect faults in the captured high-definition images through the method of image intelligent recognition. Relatively speaking, the accuracy of non-contact detection is higher and the robustness is stronger. Generally, multiple types of abnormalities can be detected simultaneously, becoming the mainstream technology for detecting the carbon sliding plate of the pantograph.

[0004] In the prior art, when the locomotive travels to complex backgrounds such as tunnels, the detection effect of the carbon sliding strip is seriously interfered, resulting in too many false alarms for the subsequent carbon sliding strip. The overall segmentation effect of the existing deep learning model detection method is better than that of traditional algorithms, but the segmentation of the edge area of the carbon sliding strip is not fine enough, and it is difficult to accurately locate the defects of the carbon sliding strip. Summary of the Invention

[0005] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a method for detecting defects in the carbon sliding strip of a pantograph based on images, including the following steps:

[0006] Step 1, collect carbon sliding strip images, and use the MSRCR algorithm to enhance the collected carbon sliding strip images to obtain enhanced images;

[0007] Step 2: Perform image adaptive threshold segmentation on the enhanced image. By calculating the mean value n of all pixel points in the enhanced image, modify the pixel point values greater than n in the image to 255 and those less than n to 0, obtaining the image A after adaptive threshold segmentation;

[0008] Step 3: Use the Sobel operator to calculate the carbon slider edges of the image after adaptive threshold segmentation, obtaining the upper and lower edge line segments of the carbon slider. Taking the upper and lower edge line segments of the carbon slider as boundaries, set all pixel values in the area between the upper and lower edge line segments to 0. Determine whether the image belongs to a complex background according to the proportion of image edge pixels in the whole image. If it belongs to a complex background, discard the image; if it does not belong to a complex background, retain the carbon slider image processed by the Sobel operator;

[0009] Step 4: Use the enhanced image as input data. After cleaning and data augmentation of the input data, train the LiteSeg semantic segmentation network model. Through the trained LiteSeg semantic segmentation network model, obtain the segmented carbon slider probability map;

[0010] Step 5: Set a threshold for the obtained carbon slider probability map as the ROI region. Perform ROI external region masking on the retained carbon slider image processed by the Sobel operator, retain the edge texture information inside the carbon slider, and fuse the carbon slider probability map obtained by the LiteSeg network model with the carbon slider edge map obtained by the Sobel operator to obtain the fused outer contour of the carbon slider edge;

[0011] Step 6: For the fused outer contour of the carbon slider edge, use the convex hull method to fill the fused outer contour of the carbon slider edge to obtain the convex hull contour of the carbon slider. By means of image difference, subtract the convex hull contour of the carbon slider from the original outer contour of the carbon slider to obtain the defective area of the carbon slider. Calculate the area of the defective area of the carbon slider. If the area is less than the set threshold, discard this area; if the area is greater than the set threshold, it is determined that the carbon slider is damaged.

[0012] Further, the using the Sobel operator to calculate the carbon slider edges of the image A after adaptive threshold segmentation, obtaining the upper and lower edge line segments of the carbon slider, includes:

[0013] Perform convolution operation on each pixel point in the image A with a 3×3 convolution kernel template to obtain the gray gradient map for vertical edge detection:

[0014]

[0015] Use the Hough transform to detect the upper and lower edges of the carbon slider. First, substitute all points (x, y) in the grayscale image G y into the polar coordinates

[0016] ρ = x Cosθ + y Sinθ

[0017] ρ represents the polar radius of the point (x, y), θ represents the polar angle of the point (x, y), and the ordered pair (ρ, θ) is the polar coordinate of the point (x, y);

[0018] Each point (x, y) obtains the corresponding curve in polar coordinates. When the number of curves intersecting at a point (ρ0, θ0) exceeds the set threshold, substitute the polar coordinates to obtain the straight line ρ0 = x Cosθ0 + y Sinθ0, which is the edge segment of the carbon slider, and obtain the upper and lower edge segments of the carbon slider.

[0019] Further, the probability map of the carbon slider obtained by the LiteSeg network model is fused with the edge map of the carbon slider obtained by the Sobel operator, and the following fusion formula is used:

[0020]

[0021] In the formula: represents the result after the fusion of the two algorithms, represents the classification probability of the i-th category, T sobel represents the value of this point, and c represents the total number of semantic segmentation categories.

[0022] Further, step six also includes: calculating the centroid (X G , Y G ) of the defective area:

[0023] X G = ∑ i w i x i / ∑ i w i

[0024] Y G = ∑ i w i y i / ∑ i w i

[0025] i is the discrete object, w i is the weight of the discrete object, and x i , y i are the coordinates of each discrete object;

[0026] Search upward from the position of the centroid until the upper edge of the carbon slider, select the center position of the defective area of the carbon slider to issue an alarm and mark the abnormal part.

[0027] The beneficial effects of the present invention are as follows: 1. The supervised classification learning method can detect abnormal defects in different states, but it is highly dependent on data and requires a balanced distribution of positive and negative samples. In the real railway background, abnormal data is often difficult to obtain, and the normal data collected is much larger than the abnormal data. The detection method of the present invention does not require abnormal samples and can successfully detect the defects of the carbon sliding strip by filling and differentiating the concave area.

[0028] 2. When the existing deep learning methods segment the edge information of an object, problems such as overfitting often occur, resulting in overly smooth image edges and unable to effectively segment the defective part of the carbon sliding strip. Traditional image processing techniques can accurately detect the edge information of an image, but they will also detect too much redundant edge information. The present invention refines the edge detection of the carbon sliding strip by combining the detection results of deep learning and the Sobel operator.

[0029] 3. Combine the Sobel operator with the Hough transform to eliminate the complex background, effectively suppress false alarms caused by the complex background in a single-frame image, and reduce the number of false alarms. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic flow diagram of a method for detecting defects in the carbon sliding strip of a pantograph based on an image;

[0031] Figure 2 It is a schematic flow diagram of the carbon sliding strip defect by the convex hull method

[0032] Figure 3 It is a schematic diagram for detecting the carbon sliding strip defect by the convex hull method. DETAILED DESCRIPTION OF THE INVENTION

[0033] The technical solution of the present invention will be further described in detail below with reference to the drawings, but the protection scope of the present invention is not limited to the following.

[0034] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail in combination with the drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention and are not used to limit the present invention, that is, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Usually, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0035] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of protection of the present invention. It should be noted that relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0036] Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0037] The features and performance of the present invention will be further described in detail below in conjunction with embodiments.

[0038] As Figure 1 shown, a method for detecting defects in the pantograph carbon strip based on images includes the following steps:

[0039] Step 1, collect the carbon strip image, and use the MSRCR algorithm to enhance the collected carbon strip image to obtain the enhanced image;

[0040] Step 2, perform image adaptive threshold segmentation on the enhanced image. By calculating the average value n of all pixel points of the enhanced image, modify the pixel point values greater than n in the image to 255, and modify the pixel point values less than n to 0 to obtain the image A after adaptive threshold segmentation;

[0041] Step 3, use the Sobel operator to calculate the edges of the carbon strip in the image after adaptive threshold segmentation to obtain the upper and lower edge line segments of the carbon strip. Taking the upper and lower edge line segments of the carbon strip as the boundaries, set all pixel values in the area between the upper and lower edge line segments to 0. Determine whether the image belongs to a complex background according to the proportion of image edge pixels in the whole image. If it belongs to a complex background, the image is excluded; if it does not belong to a complex background, the carbon strip image processed by the Sobel operator is retained;

[0042] Step 4: Using the enhanced image as input data, after cleaning and data augmentation of the input data, train the LiteSeg semantic segmentation network model. Through the trained LiteSeg semantic segmentation network model, obtain the segmented probability map of the carbon slider;

[0043] Step 5: Set a threshold for the obtained probability map of the carbon slider as the ROI region, mask the external region of the ROI for the remaining carbon slider image processed by the Sobel operator, retain the edge texture information inside the carbon slider, and fuse the probability map of the carbon slider obtained by the LiteSeg network model with the edge map of the carbon slider obtained by the Sobel operator to obtain the fused outer contour of the carbon slider edge;

[0044] Step 6: For the fused outer contour of the carbon slider edge, use the convex hull method to fill the fused outer contour of the carbon slider edge to obtain the convex hull contour of the carbon slider. Through the method of image difference, subtract the convex hull contour of the carbon slider from the original outer contour of the carbon slider to obtain the defective area of the carbon slider, calculate the area of the defective area of the carbon slider. If the area is less than the set threshold, discard this area. If the area is greater than the set threshold, it is determined that the carbon slider is damaged.

[0045] Calculating the edges of the carbon slider in the image A segmented by the adaptive threshold using the Sobel operator to obtain the upper and lower edge segments of the carbon slider, including:

[0046] Perform a convolution operation on each pixel point in the image A with a 3×3 convolution kernel template to obtain a gray gradient map for vertical edge detection:

[0047]

[0048] Use the Hough transform to detect the upper and lower edges of the carbon slider. First, for all points (x, y) in the grayscale image G y substitute them into the polar coordinates

[0049] ρ = xCosθ + ySinθ

[0050] ρ represents the polar radius of the point (x, y), θ represents the polar angle of the point (x, y), and the ordered pair (ρ, θ) is the polar coordinate of the point (x, y);

[0051] For each point (x, y), obtain the corresponding curve in polar coordinates. When the number of curves intersecting at a point (ρ0, θ0) exceeds the set threshold, substitute into the polar coordinates to obtain the straight line ρ0 = xCosθ0 + ySinθ0. This straight line is the edge segment of the carbon slider, and obtain the upper and lower edge segments of the carbon slider.

[0052] Fusing the probability map of the carbon slider obtained by the LiteSeg network model with the edge map of the carbon slider obtained by the Sobel operator, using the following fusion formula:

[0053]

[0054] Wherein: represents the result after the fusion of two algorithms, represents the classification probability for the i-th category, T sobel represents the value of this point, and c represents the total number of semantic segmentation categories.

[0055] Step six also includes: calculating the centroid (X G , Y G ) of the defective area:

[0056] X G = ∑ i w i x i / ∑ i w i

[0057] Y G = ∑ i w i y i / ∑ i w i

[0058] i is a discrete object, w i is the weight of the discrete object, and x i , y i are the coordinates of each discrete object;

[0059] Search upward according to the position of the centroid until the upper edge of the carbon slider, select the center position of the defective area of the carbon slider to send an alarm and mark the abnormal part.

[0060] Specifically, for the too dark or overexposed images existing in the original image, the MSRCR algorithm is used for image enhancement to reduce the adverse effects brought by the defects of the image itself data to the subsequent detection.

[0061] Step 2: Perform image adaptive threshold segmentation on the enhanced image. First, calculate the average value n of all pixel points after the enhanced image, modify the pixel point values greater than n in the image to 255, and modify the pixel point values less than n to 0. The modified binary image is the image after adaptive threshold segmentation.

[0062] Step 3: Use the Sobel operator to calculate the edge of the carbon slider, that is, perform a convolution operation with a 3×3 convolution kernel template on each pixel point in the binary image A to obtain the gray gradient map of vertical edge detection Then use the Hough transform to detect the upper and lower edges of the carbon slider, that is, first convert the grayscale image G yAll the points (x, y) are successively substituted into the polar coordinates ρ = xCosθ + ySinθ. Here, ρ represents the polar radius of the point (x, y), θ represents the polar angle of the point (x, y), and the ordered pair (ρ, θ) is the polar coordinate of the point (x, y). Each point (x, y) obtains the corresponding curve in polar coordinates. When the number of curves intersecting at a point (ρ0, θ0) exceeds the set threshold, substituting into the polar coordinates gives the straight line ρ0 = xCosθ0 + ySinθ0, which is recognized as the edge segment of the carbon slider, and the upper and lower edge segments of the carbon slider are obtained accordingly. Using the upper and lower edge segments of the carbon slider as boundaries, the pixel values in the area between the upper and lower edge segments are all set to 0, thereby only retaining the edge information outside the segments. Whether the image belongs to a complex background is judged according to the proportion of the edge pixels of the image in the whole image. If it belongs to a complex background, this image does not participate in the subsequent calculation to reduce the false alarm rate of the algorithm; if it does not belong to a complex background, the edge detection result is retained.

[0063] Step 4: Using the image after image enhancement in Step 1 as the input data, after cleaning and data augmentation of the data, train the LiteSeg semantic segmentation network model to obtain the probability map of the segmented carbon slider.

[0064] Step 5: Using the carbon slider probability map obtained in Step 4 for threshold setting as the ROI region, perform masking on the ROI external region of the carbon slider image processed by the Sobel operator in Step 3, only retaining the edge texture information inside the carbon slider, and fuse the carbon slider probability map obtained by the LiteSeg network model with the carbon slider edge map obtained by the Sobel operator. The fusion formula is:

[0065] In the formula: represents the result after the fusion of the two algorithms, represents the classification probability for the i-th category, T sobel represents the value of this point, and c represents the total number of categories of semantic segmentation.

[0066] Step 6: Obtain the outer contour of the edge of the fused carbon slider, use the convex hull method to fill it to obtain the convex hull contour of the carbon slider, and use the method of image difference to subtract it from the original outer contour of the carbon slider to obtain the defective area of the carbon slider. Calculate the area of each defective area of the carbon slider respectively. If the area is less than the set threshold, this area is discarded; if the area is greater than the set threshold, it is considered damaged, and calculate the centroid (X G , Y G ) of this defective area: X G =∑ i w i x i / ∑ i w i , Y G =∑ i wi y i / ∑ i w i , *Note: i is the discrete target, w i is the weight of the discrete target, x i , y i are the coordinates of each discrete target. Search upward from the position of the centroid until the upper edge of the carbon slider, select the center position of the defective area of the carbon slider to issue an alarm and mark the abnormal part

[0067] The above are only the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be changed within the scope of the concept described herein through the above teachings or the technology or knowledge in the relevant field. And the changes and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention shall fall within the protection scope of the appended claims of the present invention.

Claims

1. An image-based method for detecting defects in the carbon strip of a pantograph, characterized in that, It includes the following steps: Step 1: Collect the carbon brush images, and use the MSRCR algorithm to enhance the collected carbon brush images to obtain the enhanced images; Step 2: Perform image adaptive threshold segmentation on the enhanced images. By calculating the average value n of all pixel points of the enhanced images, modify the pixel point values greater than n in the images to 255, and modify the pixel point values less than n to 0 to obtain the image A after adaptive threshold segmentation; Step 3: Use the Sobel operator to calculate the edges of the carbon brush in the image after adaptive threshold segmentation to obtain the upper and lower edge line segments of the carbon brush. Taking the upper and lower edge line segments of the carbon brush as the boundaries, set all the pixel values in the area between the upper and lower edge line segments to 0. Judge whether the image belongs to a complex background according to the proportion of the image edge pixels in the whole image. If it belongs to a complex background, the image is excluded; if it does not belong to a complex background, the carbon brush image processed by the Sobel operator is retained; Step 4: Use the enhanced images as input data. After cleaning and data augmentation of the input data, train the LiteSeg semantic segmentation network model. Through the trained LiteSeg semantic segmentation network model, obtain the probability map of the segmented carbon brush; Step 5: Set the threshold for the obtained carbon brush probability map as the ROI area, mask the area outside the ROI of the retained carbon brush image processed by the Sobel operator, retain the edge texture information inside the carbon brush, and fuse the carbon brush probability map obtained by the LiteSeg network model with the carbon brush edge map obtained by the Sobel operator to obtain the fused outer contour of the carbon brush edge; Step 6: For the fused outer contour of the carbon brush edge, use the convex hull method to fill the fused outer contour of the carbon brush edge to obtain the convex hull contour of the carbon brush. By means of image difference, subtract the convex hull contour of the carbon brush from the original outer contour of the carbon brush to obtain the defective area of the carbon brush, and calculate the area of the defective area of the carbon brush. If the area is less than the set threshold, this area is discarded. If the area is greater than the set threshold, it is determined that the carbon brush is damaged; The calculation of the edges of the carbon brush in the image A after adaptive threshold segmentation by using the Sobel operator to obtain the upper and lower edge line segments of the carbon brush includes: Perform convolution operations on each pixel point in the image A through a 3×3 convolution kernel template to obtain the gray gradient map of longitudinal edge detection: Detect the upper and lower edges of the carbon slider using the Hough transform. First, substitute all the points (x, y) in the grayscale image G y into the polar coordinates in sequence ρ = xCosθ + ySinθ ρ represents the polar radius of the point (x, y), θ represents the polar angle of the point (x, y), and the ordered pair (ρ, θ) is the polar coordinate of the point (x, y); Each point (x, y) obtains the corresponding curve in polar coordinates. When the number of curves intersecting at a point (ρ0, θ0) exceeds the set threshold, substitute the polar coordinates to obtain the straight line ρ0 = xCosθ0 + ySinθ0. This straight line is the edge line segment of the carbon brush, and the upper and lower edge line segments of the carbon brush are obtained; The fusion of the carbon brush probability map obtained by the LiteSeg network model and the carbon brush edge map obtained by the Sobel operator adopts the following fusion formula: Where: represents the result after the fusion of two algorithms, represents the classification probability for the i-th category, T sobel represents the value of this point, and c represents the total number of semantic segmentation categories.

2. The method for detecting defects of the pantograph carbon strip based on images according to claim 1, wherein Step six further includes: calculating the centroid (X G , Y G ) of the defective area: X G = ∑ i w i x i / ∑ i w i Y G = ∑ i w i y i / ∑ i w i i is a discrete target, w i is the weight of the discrete target, x i , y i are the coordinates of each discrete target; Search upward according to the position of the centroid until the upper edge of the carbon slider, select the center position of the defective area of the carbon slider to issue an alarm and mark the defective area of the carbon slider.

Images