Rail surface defect identification method and device

Abstract

The present invention discloses a rail surface defect recognition method and device, wherein the method comprises: using a pre-trained continuous defect detection model to perform continuous defect recognition on a rail surface image, and recognize the rail position and continuous defect image; for each rail surface image in which no continuous defect is recognized, extracting light bands from the rail area according to the rail position, and generating a light band binary image; calculating the regional light band index of the rail area according to the light band binary image; inputting the regional light band index into a classifier to determine whether the rail has a suspected surface defect; inputting the rail surface image with suspected surface defects into a pre-trained local defect detection model for local defect recognition, and obtaining a local defect image, a local defect type, and a local defect position; and outputting a continuous defect image, a local defect image, a local defect type, and a corresponding local defect position. The present invention can comprehensively and accurately recognize rail surface defects in rail images.

Description

Technical Field

[0001] The present invention relates to the technical field of track detection and image recognition, and in particular to a method and device for identifying rail surface defects. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the invention recited in the claims. No admission is made that the description herein is prior art by inclusion in this section.

[0003] Rails are the main component of the entire railway track and one of the railway transportation infrastructures. Their function is to guide the direction of the train wheels, directly bear the force of the train wheels, and transmit it to the sleepers. The state of the rails directly affects the safety of train operation. Due to long-term exposure to the wild, the influence of various natural conditions such as wind, frost, rain, snow, high temperature and low temperature, as well as the pressure brought by train operation, the rails will inevitably suffer various damages, which will bring various safety hazards.

[0004] The main defects on the rail surface include: rail surface abrasion, peeling block, crack, rail head crushing, tread wave wear, etc. Different defects have different effects on train operation safety. In the inspection and maintenance of track safety facilities, rail surface defect detection is one of the most important links.

[0005] When performing rail image detection, a camera is usually mounted on the bottom of the inspection vehicle. When the inspection vehicle is running, the camera continuously shoots and images the rail surface to obtain the rail surface image. After that, the rail surface defects are identified through rail surface defect image recognition technology. Compared with general recognition tasks, rail surface defect detection and recognition has its own special difficulties and challenges. The environment where the railway passes is complex, and various natural conditions, such as light, wind, frost, rain and snow, make the quality of the collected on-site pictures different, which greatly increases the difficulty of defect recognition. In addition, the pattern of rail surface defects is also relatively complex. For example, some defects are relatively subtle and only occupy a small part compared to the rails, which also increases the difficulty of detection and recognition to a certain extent.

[0006] The prior art includes methods for identifying rail surface defects through deep learning, deep neural networks, deep forest models, etc.

[0007] Among them, the method of identifying rail surface defects through deep learning first requires manually collecting a large number of rail surface defect images and annotating the data, training the rail surface defect detection model based on the deep learning target positioning algorithm based on the annotated data, and then using the trained model to identify rail surface defects in the rail images. However, this technology requires a large number of rail defect image samples, and the detectable defects must be defects of the type that have appeared in the samples. For small samples or no samples, rail surface defects are prone to missed detection.

[0008] The deep neural network and deep forest model methods are used to model the rail surface scratches. After preprocessing the image, the rail surface defects in the image are detected through the trained model, including special processing of small sample problems. However, rail surface defects include continuous defects and local defects. This technology can detect local defects through target detection, but because the morphology and characteristics of continuous defects are different from those of local defects, this technology is prone to missed detection when detecting continuous defects due to the extreme aspect ratio of the defect target. In addition, it is also easy to miss defect types that have not appeared in the sample. Summary of the invention

[0009] An embodiment of the present invention provides a rail surface defect recognition method for comprehensively and accurately recognizing rail surface defects in a rail image by using an image recognition algorithm. The method comprises:

[0010] Acquire the captured rail surface image;

[0011] Utilize the pre-trained continuous defect detection model to perform continuous defect recognition on the rail surface image, identify the rail position and continuous defect image, where the rail position is the pixel row and column position of the rail in the image;

[0012] For each rail surface image in which no continuous defects are identified, light bands are extracted from the rail area according to the rail position to generate a light band binary image;

[0013] Calculate the regional light band index of the rail area according to the light band binary map;

[0014] Input the regional light band index into the classifier to determine whether the rail has suspected surface defects;

[0015] Inputting the rail surface image with suspected surface defects into the pre-trained local defect detection model to perform local defect recognition, and obtaining the local defect image, local defect type and local defect position;

[0016] Output continuous defect images, local defect images, local defect types and corresponding local defect locations.

[0017] The embodiment of the present invention further provides a rail surface defect recognition device, which is used to comprehensively and accurately recognize rail surface defects in rail images by using an image recognition algorithm. The device includes:

[0018] An acquisition module, used for acquiring the photographed rail surface image;

[0019] A continuous defect recognition module is used to perform continuous defect recognition on the rail surface image using a pre-trained continuous defect detection model, and identify the rail position and the continuous defect image, wherein the rail position is the pixel row and column position of the rail in the image;

[0020] A light band extraction module is used to extract light bands from the rail area according to the rail position for each rail surface image in which no continuous defects are identified, and generate a light band binary image;

[0021] A calculation module, used for calculating the regional light band index of the rail area according to the light band binary map;

[0022] A suspected defect identification module is used to input the regional light band index into the classifier to determine whether the rail has a suspected surface defect;

[0023] A local defect recognition module is used to input the rail surface image with suspected surface defects into a pre-trained local defect detection model for local defect recognition, and obtain a local defect image, local defect type and local defect position;

[0024] The recognition result output module is used to output continuous defect images, local defect images, local defect types and corresponding local defect positions.

[0025] An embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned rail surface defect identification method when executing the computer program.

[0026] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for executing the above-mentioned rail surface defect identification method.

[0027] In an embodiment of the present invention, a multi-level rail surface defect recognition method from continuous to local is proposed. First, continuous defect recognition is performed on the rail surface image, and then the rail surface state is automatically evaluated by a classifier to identify the rail surface image with possible surface defects. Then, the local defects are detected using a local defect detection model, thereby realizing the combination of deep learning and traditional classification methods, and further realizing comprehensive and automatic detection of global continuous defects and local defects on the rail surface, thereby improving the detection rate and accuracy of defects, and greatly improving the work efficiency of front-line inspection personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0029] Figure 1 This is a flow chart of a rail surface defect identification method according to an embodiment of the present invention;

[0030] FIG2( a ) is a schematic diagram of a rail having continuous corrugation defects according to an embodiment of the present invention;

[0031] FIG2( b ) is a schematic diagram of a rail having a local abrasion defect according to an embodiment of the present invention;

[0032] FIG3( a ) is a schematic diagram of an image annotation of a rail without continuous defects according to an embodiment of the present invention;

[0033] FIG3( b ) is a schematic diagram of annotating a rail image with continuous defects according to an embodiment of the present invention;

[0034] Figure 4 A schematic diagram of a process for generating a light band binary image in an embodiment of the present invention;

[0035] Figure 5 A schematic diagram of dividing a light band into sub-areas in an embodiment of the present invention;

[0036] Figure 6 A schematic diagram of a Faster R-CNN framework in the prior art;

[0037] Figure 7 A schematic diagram of a framework of a regional candidate network RPN in the prior art;

[0038] Figure 8 A schematic diagram of a process of using the Faster RCNN framework to identify the global state of a rail in an embodiment of the present invention;

[0039] Fig. 9 A schematic diagram of rail image annotation of a local defect in an embodiment of the present invention;

[0040] Fig.10 A schematic diagram of a process of identifying local defects of rails using a Faster RCNN framework in an embodiment of the present invention;

[0041] Fig.11 This is a schematic diagram of the structure of a rail surface defect identification device according to an embodiment of the present invention;

[0042] Fig.12 The figure is a schematic diagram of the structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0044] The embodiment of the present invention provides a method for identifying rail surface defects. Figure 1 As shown, the method includes steps 101 to 107:

[0045] Step 101: Acquire a captured rail surface image.

[0046] The rail surface image is obtained from the track inspection acquisition equipment. The track inspection acquisition equipment uses a linear array camera to continuously shoot the track. The rail surface image collected by the track inspection acquisition equipment is a grayscale image taken perpendicular to the rail surface.

[0047] According to the different defect forms, the defects on the rail surface can be divided into continuous defects and local defects. Continuous defects usually include corrugation, continuous scratches, continuous abrasions, etc.; local defects usually include local abrasions, local block loss, etc. For example, Figure 2(a) shows a rail defect image with continuous corrugation defects, and Figure 2(b) shows a rail defect image with local abrasions.

[0048] Step 102: Use the pre-trained continuous defect detection model to perform continuous defect recognition on the rail surface image to identify the rail position and the continuous defect image.

[0049] The rail position is the pixel row and column position of the rail in the image. The image is composed of pixels, and the rail position is the pixel row and column position of the rail, for example, from the 1st row of pixels to the 200th row of pixels in the vertical direction, and from the 80th column of pixels to the 120th column of pixels in the horizontal direction.

[0050] Continuous defects refer to continuous defects whose defect length in a single image exceeds 50% of the length of the rail in the image. Continuous defects occupy a large area, and a large part of continuous defects can be identified by performing comprehensive rail surface recognition on the rail surface image. Comprehensive rail surface state recognition takes the entire section of the rail in the image to be recognized as a whole to locate the rail and identify continuous defects. Global rail surface state recognition is achieved through a deep learning target detection model.

[0051] First, first sample images including rail images with continuous defects and rail images without continuous defects are obtained; the positions of complete rails are marked in all first sample images, and continuous defect identifiers are marked in rail images with continuous defects; then, a deep learning target detection model is trained using all marked first sample images to obtain a continuous defect detection model.

[0052] For example, the annotated rail images are shown in Figures 3(a) and 3(b), where Figure 3(a) is a schematic diagram of annotating a rail image without continuous defects, and Figure 3(b) is a schematic diagram of annotating a rail image with continuous defects. In the schematic diagram, the position of the complete rail is marked with a black thick box, and the text "continuous defect rail" is used as a continuous defect identifier. The text "no continuous defect rail" is also marked in the rail image without continuous defects, so that the deep learning target detection model can more clearly distinguish the two rail images during the training process.

[0053] The global rail surface state recognition result includes the position of the rail in the image and the category of the rail. The categories of rails are divided into rails without continuous defects and rails with continuous defects. The surface image of the rail with continuous defects is the continuous defect image.

[0054] Step 103: for each rail surface image in which no continuous defects are identified, light bands are extracted from the rail area according to the rail position to generate a light band binary image.

[0055] The global rail surface state recognition in step 102 can identify the continuous defect image and the rail position in all rail surface images. For the rail surface image without continuous defects, the light band is extracted from the rail area in the image according to the rail position. Specifically, the light band in the image can be extracted by an image segmentation algorithm to generate a light band binary image. In the image segmentation algorithm binary image, the result of the rail area is 255, and the result of other areas is 0. The image segmentation algorithm can use a semantic segmentation algorithm based on deep learning or use a static or dynamic threshold to directly binarize the image.

[0056] For example, Figure 4 The schematic diagram of the process of generating a binary image of light bands is shown in FIG. Figure 4 In the figure, region 1 and region 3 are marked with straight line boxes, and region 2 and region N are marked with dotted line boxes.

[0057] Step 104: Calculate the regional light band index of the rail area according to the light band binary image.

[0058] Taking into account that local defects are often small and occupy a small area in the rail surface image, the accuracy is low when using the complete rail surface image to identify local defects. In order to improve the accuracy of local defect identification, in an embodiment of the present invention, before calculating the regional light band index of the rail area based on the light band binary image, the light band in the light band binary image is divided into multiple sub-areas of equal length according to a first preset length. At the same time, considering that the local defect is small, if it is divided into two small parts by the boundary line of two sub-areas, each small part may be too small to be identified. Therefore, the two adjacent sub-areas are overlapped so that the two sub-areas have an overlapping area of ​​the second preset length, thereby avoiding the occurrence of the above-mentioned local defect being cut into two parts as much as possible. Exemplary, Figure 5 A schematic diagram showing a method of dividing a light band into regions is shown. Figure 5 As shown, there are certain overlapping areas between area 1 and area 2, area 2 and area 3, etc.

[0059] The first preset length and the second preset length can be set by the staff, and their specific length values ​​are not limited here.

[0060] After dividing the sub-regions, the regional light band index of the rail area is extracted based on the statistical combination of the light band area pixels. Specifically, the light band features are extracted from the light band binary image, and the light band features include the width of each row of light bands in the area, the total number of regional light band rows, the maximum value of the regional light band width, the minimum value of the regional light band width, the pixel value of each pixel in the area, and the total number of regional light band columns; the regional light band index of the rail area is calculated based on the light band features.

[0061] The regional light band indicators include regional average width, regional width standard deviation, regional width absolute extreme difference, regional width relative extreme difference, regional average gradient and regional light band area.

[0062] The calculation method of each area light band index is as follows:

[0063] ①. Average width of the area:

[0064]

[0065] In the above formula, μ represents the average width of the region; x i Indicates the width of the i-th row of light band; N indicates the total number of rows of regional light bands.

[0066] ②、Region width standard deviation:

[0067]

[0068] In the above formula, σ represents the standard deviation of the area width.

[0069] ③、Differentiation of absolute value of area width:

[0070] Δ abs =x max -x min

[0071] In the above formula, Δ abs represents the absolute extreme difference; x max Indicates the maximum width of the regional light band; x min Indicates the minimum width of the regional light band.

[0072] ④. Relative extreme value difference of area width:

[0073]

[0074] In the above formula, Δ rel Indicates the relative extreme value difference of the region width.

[0075] ⑤. Regional average gradient:

[0076]

[0077] In the above formula, represents the regional average gradient; p i,j It represents the pixel value of the pixel in the i-th row and j-th column, which can be 0 or 1; M represents the total number of columns in the regional light band.

[0078] ⑥. Area of ​​regional light band:

[0079]

[0080] In the above formula, S represents the area of ​​the regional light band.

[0081] It should be noted that in each row of pixels of the light band binary image, the pixel portion belonging to the light band is regarded as a row of light bands, and in each column of pixels of the light band binary image, the pixel portion belonging to the light band is regarded as a column of light bands.

[0082] Step 105: Input the regional light band index into the classifier to determine whether there are any suspected surface defects on the rail.

[0083] The classifier is pre-trained using multiple groups of regional light band indicators without local defects and multiple groups of regional light band indicators with local defects.

[0084] All regional light band indicators are combined into a one-dimensional feature vector, and then the regional light band indicators of each sub-area and the regional light band indicators of the complete rail area are input into the classifier respectively. Figure 5As shown, N groups of regional light bands calculated from regions 1, 2, 3 to N, and 1 group of regional light band indicators calculated from the complete rail region from region 1 to region N, a total of N+1 groups of regional light band indicators are input into the classifier to determine whether there are suspected defects on the rail surface.

[0085] In an embodiment of the present invention, the regional light band index calculated from the complete rail area is input into a classifier for surface suspected defect discrimination. The main purpose is to further screen whether there are continuous defects on the rail. In this way, continuous defects that are not identified by the continuous defect detection model can be checked for leakage, so as to obtain more accurate continuous defect identification results.

[0086] The classifier can be a naive Bayes classifier, a support vector machine (SVM) or a random forest classifier.

[0087] Step 106: Input the rail surface image with suspected surface defects into a pre-trained local defect detection model to perform local defect recognition, and obtain a local defect image, a local defect type, and a local defect position.

[0088] Local rail surface defect recognition requires the construction of local rail surface defect recognition features. First, a large number of second sample images need to be obtained, which include rail images with local defects; then, the local defect position and local defect type identification are marked in the second sample image, and the local defect type identification indicates the specific type of local defect, such as local abrasion, local block loss, etc.; then, the deep learning target detection model is trained using the marked second sample images to obtain a local defect detection model.

[0089] The rail surface image that is identified as having suspected surface defects is then input into the local defect detection model for local defect recognition, and the rail surface image with local defects, the local defect type and the corresponding local defect position are identified.

[0090] Step 107: output the continuous defect image, the local defect image, the local defect type and the corresponding local defect position.

[0091] In an embodiment of the present invention, a multi-level rail surface defect recognition method from continuous to local is proposed. First, continuous defect recognition is performed on the rail surface image, and then the rail surface state is automatically evaluated by a classifier to identify the rail surface image with possible surface defects. Then, the local defects are detected using a local defect detection model, thereby realizing the combination of deep learning and traditional classification methods, and further realizing comprehensive and automatic detection of global continuous defects and local defects on the rail surface, thereby improving the detection rate and accuracy of defects, and greatly improving the work efficiency of front-line inspection personnel.

[0092] In order to have a deeper understanding of the rail surface defect recognition method, the embodiments of the present invention will be specifically introduced below with reference to specific examples. In the following example, the rail surface defect recognition method is divided into four stages, namely, the global rail surface state recognition stage (i.e., the continuous defect recognition stage), the rail light band local feature discrimination stage (i.e., the surface suspected defect discrimination stage), the local rail surface defect recognition stage, and the result output stage.

[0093] 1. Global rail surface status identification

[0094] In the global rail surface state recognition stage, 2000 rail surface images are first prepared, of which 1000 are discontinuous defect rail surfaces and 1000 are continuous defect rail surfaces. The two types of rails are marked by frame according to their types. The marked images can be seen in Figure 3(a) and Figure 3(b).

[0095] The Faster R-CNN framework is then used to train the continuous defect detection model. Faster R-CNN is the first truly end-to-end deep learning detection algorithm. Its biggest innovation is that it adds a Region Proposal Network (RPN) to generate candidate boxes based on an anchor mechanism, and finally integrates feature extraction, candidate box selection, border regression and classification into one network, thereby effectively improving detection accuracy and efficiency.

[0096] The specific process is to scale the input image and then enter the convolution layer to extract features to obtain a feature map, and then send the feature map to the region candidate network to generate a series of possible candidate boxes for the target. Next, the original feature map and all the candidate boxes output by the region candidate network are input to the ROI pooling layer to extract and collect candidate regions, and calculate a fixed size 7×7 candidate region feature map, which is sent to the fully connected layer for target classification and coordinate regression. The framework of Faster R-CNN is as follows: Figure 6 As shown, the framework of the region candidate network RPN is as follows Figure 7 shown.

[0097] The core idea of ​​the candidate region network RPN is to use sliding windows and anchor mechanisms to generate candidate boxes. The specific method is to use the sliding window method to construct 9 candidate boxes of different aspect ratios and scales at the center of each sliding window on the 40×60 feature map obtained by the previous convolution layer, and map them to the image frame, and then sort them from large to small according to the score of each region, extract the first 2000 regions, perform non-maximum suppression on these 2000, and finally sort them again, and output 300 regions to Faster RCNN for prediction. In the ROI pooling stage, Faster RCNN maps about 300 candidate boxes obtained by the region candidate network to the convolution feature map. After ROI pooling, boxes of different sizes can obtain feature maps of fixed sizes.

[0098] The specific operations are as follows:

[0099] 1) According to the input image, map the ROI to the position corresponding to the feature map;

[0100] 2) Divide the mapped area into 7×7 areas;

[0101] 3) Perform maximum pooling operation on each region.

[0102] In this way, we can get corresponding images of fixed size from boxes of different sizes. Moreover, the size of the output feature map does not depend on the size of the ROI and the convolutional feature map. The biggest benefit of ROI pooling is that it greatly improves the training and testing speed. Faster R-CNN designed a network RPN to extract candidate regions, replacing the time-consuming selective search, which greatly improves the detection speed.

[0103] When using the Faster RCNN framework to identify the global state of rails, a recognition model is formed by training the labeled data. After the image to be identified is input into the model, the recognition is achieved through model reasoning. The specific recognition process is as follows: Figure 8 shown.

[0104] Global rail surface status recognition can first locate the position of the rail in the image, and secondly determine whether there are continuous defects in the rail.

[0105] 2. Identification of local features of rail light bands

[0106] For the image without continuous defects obtained in 1, the rail light band local feature discrimination module is used for identification. First, the rail sub-image is taken out from the image according to the rail position obtained in 1. Then the segmentation algorithm is used to binarize the rail area in the image. Here, the hard threshold method is used to segment the rail surface. The pixel values ​​with grayscale values ​​greater than or equal to 230 are binarized to 255, and the pixels with grayscale values ​​less than or equal to 230 are binarized to 0. After the binary image is generated, the binary image is divided into 8 equal parts vertically, and the average width of the region, the standard deviation of the region width, the absolute extreme difference of the region width, the relative extreme difference of the region width, the average gradient of the region, and the area of ​​the regional light band are extracted from the whole and each divided region to form a feature vector.

[0107] After extracting the features, the naive Bayes classifier is used to classify the features to determine whether they meet the defect standard. Naive Bayes classification is based on Bayes' theorem and assumes that the feature conditions are independent of each other. First, through a given training set, the independence of feature words is used as a premise to learn the joint probability distribution from input to output. Then, based on the learned model, input X to find the output Y that maximizes the posterior probability.

[0108] First, 500 feature vectors of local defects are extracted, and then 500 feature vectors without local defects are extracted, and the naive Bayes classifier is trained. Then, when identifying the surface state of the rail, the trained classifier is used to determine whether there are suspected defects in the corresponding area.

[0109] 3. Local rail surface defect identification

[0110] In the local rail surface defect recognition stage, we first prepare 1,000 images of local rail surface defects and mark the local defect locations on the rails. Then we use the Faster R-CNN framework to train the local rail surface defect recognition model. Fig. 9 shown.

[0111] When using the Faster R-CNN framework to identify local defects in rails, a local defect detection model is formed by training the labeled data. After the images in the 2 images that are suspected of surface defects are input into the model, recognition is achieved through model reasoning. For the specific recognition process, see Fig.10 shown.

[0112] Local defect recognition can first locate the location of the local defect in the image, and secondly identify the type of defect (rail surface scratches, rail surface block loss, etc.).

[0113] 4. Result output

[0114] Finally, the global defects identified in step 1 and the local defects identified in step 3 are output simultaneously, thus completing the rail surface defect recognition process.

[0115] The present invention also provides a rail surface defect identification device, as described in the following embodiments. Since the principle of the device to solve the problem is similar to that of the rail surface defect identification method, the implementation of the device can refer to the implementation of the rail surface defect identification method, and the repeated parts will not be repeated.

[0116] like Fig.11 As shown, the device 1100 includes an acquisition module 1101, a continuous defect recognition module 1102, a light band extraction module 1103, a calculation module 1104, a suspected defect judgment module 1105, a local defect recognition module 1106 and a recognition result output module 1107.

[0117] The acquisition module 1101 is used to acquire the captured rail surface image;

[0118] The continuous defect recognition module 1102 is used to perform continuous defect recognition on the rail surface image using a pre-trained continuous defect detection model, and recognize the continuous defect image and the rail position in the image, wherein the rail position is the pixel row and column position of the rail in the image;

[0119] The light band extraction module 1103 is used to extract the light bands of the rail area for each rail surface image in which no continuous defects are identified, and generate a light band binary image;

[0120] A calculation module 1104, used for calculating the regional light band index of the rail area according to the light band binary image;

[0121] The suspected defect determination module 1105 is used to input the regional light band index into the classifier to determine whether the rail has a suspected surface defect;

[0122] The local defect recognition module 1106 is used to input the rail surface image with suspected surface defects into a pre-trained local defect detection model to perform local defect recognition and obtain the local defect type and local defect location;

[0123] The recognition result output module 1107 is used to output the continuous defect image and the rail position in the image, as well as the local defect type and the corresponding local defect position.

[0124] In an implementation manner of the embodiment of the present invention, the device further includes a region division module, which is used to:

[0125] Dividing the light band in the light band binary image into a plurality of sub-regions of equal length according to a first preset length, wherein two adjacent sub-regions have an overlapping region of a second preset length;

[0126] Compute module for:

[0127] Calculate the regional light band index of each sub-area and the complete rail area according to the light band binary map;

[0128] Suspected defect identification module, used for:

[0129] The regional light band index of each sub-area and the regional light band index of the complete rail area are respectively divided into the classifier to determine whether there are suspected surface defects on the rail.

[0130] In an implementation of the embodiment of the present invention, the computing module is used to:

[0131] Extract light band features from the light band binary image, the light band features include the width of each row of light bands in the area, the total number of rows of regional light bands, the maximum width of the regional light bands, the minimum width of the regional light bands, the pixel value of each pixel in the area and the total number of columns of regional light bands;

[0132] The regional light band index of the rail area is calculated based on the light band characteristics.

[0133] In an implementation of the embodiment of the present invention, the computing module is used to:

[0134] Calculate the average width of the area based on the width of each row of light bands and the total number of rows of regional light bands;

[0135] The standard deviation of the zone width is calculated based on the average zone width, the width of each row of light bands, and the total number of rows of zone light bands;

[0136] Calculate the absolute extreme value difference of the regional width according to the maximum value of the regional light band width and the minimum value of the regional light band width;

[0137] According to the maximum value of the regional light band width, the minimum value of the regional light band width and the regional average width, the relative extreme value difference of the regional width is calculated;

[0138] Calculate the average gradient of the region according to the pixel value of each pixel in the region, the total number of rows of the regional light bands, and the total number of columns of the regional light bands;

[0139] Calculate the area of ​​the regional light band according to the pixel value of each pixel in the region, the total number of rows of the regional light band and the total number of columns of the regional light band;

[0140] The regional average width, regional width standard deviation, regional width absolute extreme value difference, regional width relative extreme value difference, regional average gradient and regional light band area are determined as the regional light band indicators of each region.

[0141] In an implementation of the embodiment of the present invention, the device further includes a module training module, which is used to:

[0142] Acquire a first sample image, wherein the first sample image includes a rail image with continuous defects and a rail image without continuous defects;

[0143] Mark the complete rail position and continuous defect identification in the rail image with continuous defects;

[0144] The deep learning target detection model is trained using labeled rail images with continuous defects and rail images without continuous defects to obtain a continuous defect detection model.

[0145] In one implementation of the embodiment of the present invention, the model training module is used to:

[0146] Acquire a second sample image, where the second sample image includes an image of a rail with local defects;

[0147] Marking the local defect position and the local defect type identifier in the second sample image;

[0148] The deep learning target detection model is trained using the annotated second sample image to obtain a local defect detection model.

[0149] In an embodiment of the present invention, a multi-level rail surface defect recognition method from continuous to local is proposed. First, continuous defect recognition is performed on the rail surface image, and then the rail surface state is automatically evaluated by a classifier to identify the rail surface image with possible surface defects. Then, the local defects are detected using a local defect detection model, thereby realizing the combination of deep learning and traditional classification methods, and further realizing comprehensive and automatic detection of global continuous defects and local defects on the rail surface, thereby improving the detection rate and accuracy of defects, and greatly improving the work efficiency of front-line inspection personnel.

[0150] An embodiment of the present invention further provides a computer device, Fig.12 Schematic diagram of a computer device in an embodiment of the present invention. The computer device can implement all steps in the rail surface defect identification method in the above embodiment. The computer device specifically includes the following contents:

[0151] Processor (processor) 1201, memory (memory) 1202, communication interface (CommunicationsInterface) 1203 and communication bus 1204;

[0152] The processor 1201, the memory 1202, and the communication interface 1203 communicate with each other through the communication bus 1204; the communication interface 1203 is used to realize information transmission between related devices;

[0153] The processor 1201 is used to call the computer program in the memory 1202, and the processor implements the rail surface defect identification method in the above embodiment when executing the computer program.

[0154] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for executing the above-mentioned rail surface defect identification method.

[0155] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0157] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0158] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0159] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying rail surface defects, characterized in that: The method comprises: Acquire the captured rail surface image; Utilize the pre-trained continuous defect detection model to perform continuous defect recognition on the rail surface image, identify the rail position and continuous defect image, where the rail position is the pixel row and column position of the rail in the image; For each rail surface image in which no continuous defects are identified, light bands are extracted from the rail area according to the rail position to generate a light band binary image; Calculate the regional light band index of the rail area according to the light band binary map; Input the regional light band index into the classifier to determine whether the rail has suspected surface defects; Inputting the rail surface image with suspected surface defects into the pre-trained local defect detection model to perform local defect recognition, and obtaining the local defect image, local defect type and local defect position; Output continuous defect images, local defect images, local defect types and corresponding local defect locations.

2. The method according to claim 1, characterized in that: Before calculating the regional light band index of the rail area according to the light band binary map, the method further includes: Dividing the light band in the light band binary image into a plurality of sub-regions of equal length according to a first preset length, wherein two adjacent sub-regions have an overlapping region of a second preset length; The regional light band index of the rail area is calculated based on the light band binary map, including: Calculate the regional light band index of each sub-area and the complete rail area according to the light band binary map; The regional light band index is input into the classifier to determine whether the rail has suspected surface defects, including: The regional light band index of each sub-area and the regional light band index of the complete rail area are respectively divided into the classifier to determine whether there are suspected surface defects on the rail.

3. The method according to claim 1 or 2, characterized in that: The regional light band index of the rail area is calculated based on the light band binary map, including: Extracting light band features from the light band binary image, wherein the light band features include the width of each row of light bands in the region, the total number of rows of regional light bands, the maximum value of the regional light band width, the minimum value of the regional light band width, the pixel value of each pixel in the region, and the total number of columns of regional light bands; The regional light band index of the rail area is calculated based on the light band characteristics.

4. The method according to claim 3, characterized in that The regional light band index of the rail area is calculated based on the light band characteristics, including: Calculate the average width of the area based on the width of each row of light bands and the total number of rows of regional light bands; The standard deviation of the zone width is calculated based on the average zone width, the width of each row of light bands, and the total number of rows of zone light bands; Calculate the absolute extreme value difference of the regional width according to the maximum value of the regional light band width and the minimum value of the regional light band width; According to the maximum value of the regional light band width, the minimum value of the regional light band width and the regional average width, the relative extreme value difference of the regional width is calculated; Calculate the average gradient of the region according to the pixel value of each pixel in the region, the total number of rows of the regional light bands, and the total number of columns of the regional light bands; Calculate the area of ​​the regional light band according to the pixel value of each pixel in the region, the total number of rows of the regional light band and the total number of columns of the regional light band; The regional average width, regional width standard deviation, regional width absolute extreme value difference, regional width relative extreme value difference, regional average gradient and regional light band area are determined as the regional light band indicators of each region.

5. The method according to claim 1, characterized in that Before using the pre-trained continuous defect detection model to continuously identify defects on the rail surface image, the method further includes: Acquire a first sample image, wherein the first sample image includes a rail image with continuous defects and a rail image without continuous defects; Marking the position of the complete rail in the first sample image, and marking the continuous defect mark in the rail image with continuous defects; The deep learning target detection model is trained using the annotated first sample image to obtain a continuous defect detection model.

6. The method according to claim 1, characterized in that Before inputting the rail surface image with suspected surface defects into the pre-trained local defect detection model for local defect recognition, the method further includes: Acquire a second sample image, wherein the second sample image includes an image of a rail having a local defect; Marking the local defect position and the local defect type identifier in the second sample image; The deep learning target detection model is trained using the annotated second sample image to obtain a local defect detection model.

7. A rail surface defect identification device, characterized in that: The device comprises: An acquisition module, used for acquiring the photographed rail surface image; A continuous defect recognition module is used to perform continuous defect recognition on the rail surface image using a pre-trained continuous defect detection model, and identify the rail position and the continuous defect image, wherein the rail position is the pixel row and column position of the rail in the image; A light band extraction module is used to extract light bands from the rail area according to the rail position for each rail surface image in which no continuous defects are identified, and generate a light band binary image; A calculation module, used for calculating the regional light band index of the rail area according to the light band binary map; A suspected defect identification module is used to input the regional light band index into the classifier to determine whether the rail has a suspected surface defect; A local defect recognition module is used to input the rail surface image with suspected surface defects into a pre-trained local defect detection model for local defect recognition, and obtain a local defect image, local defect type and local defect position; The recognition result output module is used to output continuous defect images, local defect images, local defect types and corresponding local defect positions.

8. The device according to claim 7, characterized in that The device also includes a region division module, which is used to: Dividing the light band in the light band binary image into a plurality of sub-regions of equal length according to a first preset length, wherein two adjacent sub-regions have an overlapping region of a second preset length; Compute module for: Calculate the regional light band index of each sub-area and the complete rail area according to the light band binary map; Suspected defect identification module, used for: The regional light band index of each sub-area and the regional light band index of the complete rail area are respectively divided into the classifier to determine whether there are suspected surface defects on the rail.

9. The device according to claim 7 or 8, characterized in that Compute module for: Extracting light band features from the light band binary image, wherein the light band features include the width of each row of light bands in the region, the total number of rows of regional light bands, the maximum value of the regional light band width, the minimum value of the regional light band width, the pixel value of each pixel in the region, and the total number of columns of regional light bands; The regional light band index of the rail area is calculated based on the light band characteristics.

10. The device according to claim 9, characterized in that Compute module for: Calculate the average width of the area based on the width of each row of light bands and the total number of rows of regional light bands; The standard deviation of the zone width is calculated based on the average zone width, the width of each row of light bands, and the total number of rows of zone light bands; Calculate the absolute extreme value difference of the regional width according to the maximum value of the regional light band width and the minimum value of the regional light band width; According to the maximum value of the regional light band width, the minimum value of the regional light band width and the regional average width, the relative extreme value difference of the regional width is calculated; Calculate the average gradient of the region according to the pixel value of each pixel in the region, the total number of rows of the regional light bands, and the total number of columns of the regional light bands; Calculate the area of ​​the regional light band according to the pixel value of each pixel in the region, the total number of rows of the regional light band and the total number of columns of the regional light band; The regional average width, regional width standard deviation, regional width absolute extreme value difference, regional width relative extreme value difference, regional average gradient and regional light band area are determined as the regional light band indicators of each region.

11. The device according to claim 7, characterized in that The device also includes a module training module, which is used to: Acquire a first sample image, wherein the first sample image includes a rail image with continuous defects and a rail image without continuous defects; Marking the position of the complete rail in the first sample image, and marking the continuous defect mark in the rail image with continuous defects; The deep learning target detection model is trained using the annotated first sample image to obtain a continuous defect detection model.

12. The device according to claim 7, characterized in that Model training module, used to: Acquire a second sample image, wherein the second sample image includes an image of a rail having a local defect; Marking the local defect position and the local defect type identifier in the second sample image; The deep learning target detection model is trained using the annotated second sample image to obtain a local defect detection model.

13. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the method according to any one of claims 1 to 6.

Images