A railway train wheelset surface defect detection method based on deep learning

Abstract

The present invention discloses a method for detecting surface defects of railway wheelsets based on deep learning, comprising the following steps: (1) collecting images of the surface of a train wheel set by a train wheel set surface defect detection system; (2) annotating the collected surface images, expanding the train wheel set images by data enhancement, and dividing the enhanced image data set into a training set, a validation set, and a test set; (3) establishing a train wheel set surface defect detection model based on deep learning, inputting the training set and the validation set into the train wheel set surface defect detection model in batches for training, and obtaining a trained train wheel set surface defect detection model; (4) inputting the test data set into the trained train wheel set surface defect detection model, and obtaining the shape and position of the defect in the test set image. The present invention extracts defect feature information in the sample by learning the sample image, avoids missing the detection of subtle defects in the defect image, and improves the detection accuracy and detection speed.

Description

Technical Field

[0001] The present invention relates to a method for detecting surface defects of train wheelsets, and in particular to a method for detecting surface defects of railway train wheelsets based on deep learning. Background Art

[0002] Railway transportation is the core link in the logistics industry operation chain. With the rapid development of railway transportation, more stringent requirements have been put forward for the management and maintenance of train wheels. The working environment of train wheels is harsh and accompanied by long-term operation. Cracks, scratches, dents and other defects often appear on the surface of train wheels. Therefore, the detection of surface defects of train wheels is an important guarantee for maintaining the safe operation of railway locomotives.

[0003] Traditional methods for detecting surface defects of train wheels mainly include manual visual inspection and traditional machine vision inspection. Manual visual inspection is affected by personal subjective factors, lacks uniformity in inspection standards, and with long-term high-intensity work, it will cause visual fatigue, which will reduce the reliability of the inspection results. In addition, this method is greatly affected by the working environment, making it difficult to ensure the quality of surface defect detection of train wheels. Compared with manual inspection, traditional machine vision inspection technology has the advantages of fast speed, good stability and high precision, but this method requires manual setting of the feature function of defects in the image. Therefore, when faced with extracting features of multiple types of targets, the feature extraction capability is not ideal, far from meeting the performance requirements of defect detection, and the on-site image acquisition process is easily disturbed by environmental noise, which also increases the difficulty of defect detection. Traditional machine vision defect detection technology is restricted by feature extraction capability and stability and can no longer meet the needs of practical applications.

[0004] With the introduction of AlexNet and the development of deep learning technology, although deep learning methods have begun to be applied in the field of machine vision, the existing deep learning methods have poor recognition accuracy for surface defects of railway train wheelsets, and in order to retain the deep learning features, a large number of parameters need to be saved, resulting in low detection efficiency. Therefore, it is an urgent technical problem to propose a railway train wheelset surface defect detection method based on deep learning that meets high accuracy and efficiency. Summary of the invention

[0005] Purpose of the invention: In view of the shortcomings of the deep learning defect detection model in the prior art that the recognition accuracy of small target defects is poor and the detection model needs to save a large number of parameters in order to learn deep features, the present invention provides a railway train wheelset surface defect detection method based on deep learning. The train wheelset surface defect detection model established by the detection method is based on the MobileNetV2 network and U-shaped architecture. The surface defect detection model of the surface defect detection model is enhanced by densely connecting multiple layers of features, the utilization of features at different levels by the model is enhanced, and the number of model parameters is reduced, thereby reducing the amount of calculation.

[0006] Technical solution: The railway train wheel surface defect detection method based on deep learning of the present invention comprises the following steps:

[0007] (1) Collecting images of the surface of railway train wheelsets through a train wheelset surface defect detection system;

[0008] (2) Annotate the collected surface images, expand the train wheel set images through data enhancement, and divide the enhanced image dataset into training set, validation set, and test set;

[0009] (3) Establish a train wheel surface defect detection model based on deep learning, input the training set and validation set into the train wheel surface defect detection model in batches for training, update the model parameters according to the set learning rate after each batch iteration, stop training when the model meets the conditions, save the model parameters, and obtain the trained train wheel surface defect detection model;

[0010] (4) The test data set is input into the trained train wheelset surface defect detection model to obtain the shape and position of the defects in the test set image.

[0011] In step (1), the motor in the train wheel surface defect detection system drives the camera to move horizontally on the guide rail, thereby realizing the image acquisition on the horizontal line of the railway train wheel; the rotating mechanism adjusts the angle of the railway train wheel. Whenever the industrial camera completes the image acquisition on a horizontal line, the rotating mechanism rotates at the angle set by the client, and the camera is driven by the motor on the guide rail to complete the image acquisition on the next horizontal line. The camera captures the image and transmits it to the railway train wheel surface defect detection client through the data acquisition card, and the detection results are recorded in the database. The client regularly generates a railway train wheel health report.

[0012] In step (2), the defective area in the surface image is labeled by Labelme to obtain labeled image data.

[0013] In step (2), the image data is cropped and enhanced step by step by a sliding window method, and the image data is expanded by noise, rotation and mirroring morphological operations.

[0014] In step (3), the deep learning-based train wheelset surface defect detection model includes an encoder and a decoder arranged in a U-shaped architecture, and the encoder is composed of MobileNetV2.

[0015] MoblieNetV2 reduces the number of training parameters in the train wheelset surface defect detection model through point-by-point convolution and channel-by-channel convolution.

[0016] In step (3), the encoder obtains multiple feature maps of different scales through multiple downsampling feature extractions, and the decoder restores the surface defect information of the railway train wheelset by connecting and fusing the feature maps of the same scale as the scale obtained by the encoder, thereby detecting the surface defects of the train wheelset.

[0017] In step (3), the activation function in the trained detection model is the GELU function, as shown in formula (1):

[0018]

[0019] Where x represents the input amount of the previous level network.

[0020] The training loss function in the trained detection model is the cross entropy loss, as shown in formula (2):

[0021]

[0022] Where N is the number of samples, y is the pixel matrix of the manually labeled defect image, and y' is the pixel matrix of the model prediction result image.

[0023] In step (3), during the training process of the deep learning-based train wheelset surface defect detection model, the Adam optimization algorithm and the back propagation algorithm are used to iteratively optimize the model parameters until the model loss value converges, the training is terminated, and the obtained model parameters are saved.

[0024] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0025] (1) The railway train wheelset surface defect detection model based on deep learning of the present invention is based on the lightweight model MobileNetV2 as the feature extraction network. The network parameters are relatively small, so that the detection speed is guaranteed. The detection FPS reaches 100, which improves the industrial-level defect detection speed.

[0026] (2) The encoder-decoder in the railway train wheelset surface defect detection model used in this paper adopts a U-shaped architecture design. The decoder output result is a fusion of features from different levels of the encoder, which avoids the model from missing minor defects in the defect image. The model detection accuracy reaches 98%, improving the industrial-level defect detection accuracy.

[0027] (3) Compared with the traditional defect detection method based on image processing, the train wheel surface defect detection method based on deep learning of the present invention has the characteristic of automatic extraction of target features by deep convolutional neural network. By learning the sample image, it can automatically extract the defect feature information in the sample, thereby avoiding the extraction of complex features in the image processing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a flow chart of a method for detecting surface defects of railway train wheelsets based on deep learning according to the present invention;

[0029] Figure 2 This is a schematic diagram of a railway train wheel set surface defect detection system of the present invention;

[0030] FIG3 is a schematic diagram of the structure of a railway train wheelset surface defect detection model based on deep learning according to the present invention;

[0031] Among them, Figure 3 (A) is the structure diagram of MobileNetV2 Block 1; Figure 3 (B) is the structure diagram of MobileNetV2 Block 2; Figure 3 (C) is the structural block diagram of the train wheel surface defect detection model;

[0032] Figure 4 It is a schematic diagram of the detection results of the railway train wheelset surface defect detection model of the present invention. DETAILED DESCRIPTION

[0033] like Figures 1 to 4 As shown, the railway train wheel surface defect detection method based on deep learning of the present invention includes the following process:

[0034] (1) The surface image of the railway train wheel set 2 is collected by the train wheel set surface defect detection system; the surface defect detection system consists of two parts: hardware and software. The hardware includes an industrial camera 3, a stepper motor 4, an image acquisition card, a rotating mechanism 1, a sliding guide rail 5 and an industrial computer 6. The stepper motor 4 drives the industrial camera 3 to move translationally on the guide rail mechanism 1, thereby realizing the image collection on the horizontal line of the railway train wheel set.

[0035] The angle adjustment of the railway train wheel set image acquisition is realized by the rotating mechanism 1. Whenever the industrial camera completes the image acquisition on a horizontal line, the rotating mechanism 1 rotates according to the angle set by the user on the client, and then the industrial camera is driven by the stepper motor on the guide rail to complete the image acquisition on the next horizontal line. The images collected by the industrial camera are transmitted to the railway train wheel set surface defect detection client through the data acquisition card. The client performs defect detection on the images collected by the industrial camera through the railway train wheel set surface defect detection system, and then records the detection results in the database. The client regularly generates a railway train wheel set health report.

[0036] (2) For the defective areas in the images collected by the system, the Labelme tool is used to manually annotate and generate a label image. The label image generated by manual annotation is processed by opening and union operations to reduce the pixel error caused by manual annotation. A sliding window is used to traverse and step-crop the data set processed by the above steps. The size of the sliding window cropping box is 512*512, and the step length is 256. Morphological processing methods such as random rotation, mirroring and noise are added to the data set after the sliding window cropping to enhance the data set. In this embodiment, the enhanced data set is finally divided into a training set, a validation set and a test set in a ratio of 3:1:1.

[0037] (3) A train wheel surface defect detection model based on deep learning is established, and the training set and validation set are input into the train wheel surface defect detection model for training to obtain the trained detection model.

[0038] In order to improve the portability of the train wheel surface defect detection model, the railway train wheel surface defect detection model based on deep learning developed in Python is converted into a deep learning model for C++ call through the Libtorch software package, so as to enhance the applicability of the deep learning model on multiple platforms. The client is developed on the Qt Creator platform. Specifically: Use Pytorch to build the required railway train wheel surface defect detection model based on deep learning, send the training set and validation set to the deep learning model training in batches, optimize the model parameters according to the set learning rate after each batch iteration, terminate the training when the model training effect reaches the optimal result, and save the deep learning model parameters with the best running results on the validation set.

[0039] In step (3), the structure of the railway train wheel surface defect detection model is shown in Figure 3(C). The model is built on the Pytorch framework and consists of two modules, an encoder and a decoder, which are combined according to a U-shaped architecture.

[0040] Among them, the encoder is composed of a lightweight feature extraction network MobileNetV2, which is composed of MobileNetBlock1 and MobileNetBlock2 stacked together. MobileNetBlock1 increases the dimension of the input data through 1*1 convolution, and then uses 3*3 convolution with a step size of 1 to extract features from the features, and finally uses 1*1 convolution as an activation function to reduce the dimension of the feature map, so that the multi-channel output results are fused with each other; the structure of MobileNetBlock2 is similar to MobileNetBlock1, except that in the second layer, the 3*3 convolution with a step size of 2 performs downsampling operations on the feature map.

[0041] The decoder module consists of 3*3 convolution and upsampling, and uses bilinear interpolation to expand the resolution of the input feature map to twice the original, thereby gradually restoring the spatial information of the detection image.

[0042] First, the input image is processed by the MobileNetV2 network in the encoder to generate a feature map. In the meantime, MobileNetBlock2 downsamples the input feature map to reduce the resolution of the feature map to 1 / 2 of the input image. The MobileNetV2 network performs 5 downsampling feature extractions to output feature maps that are 2, 4, 8, 16, and 32 times smaller in size. The decoder skips and fuses the feature maps corresponding to the scale of the encoder output feature map to generate a two-dimensional 512*512 image. The grayscale of the pixels in the output image corresponds to the probability value of the input image pixel being a defect or non-defective category.

[0043] Among them, the activation function in the train wheel surface defect detection model based on deep learning is the GELU function, which is expressed as formula (1), where x represents the input of the previous level network.

[0044]

[0045] The training set and validation set are sent to the built train wheelset surface defect detection model based on deep learning for training. During the model training process, the Adam optimization algorithm and back propagation algorithm are used to iteratively optimize the model parameters. After each iteration, the model parameters are updated according to the learning rate until the model loss value converges. The training is terminated and the obtained model parameters are saved.

[0046] The model training loss function is the cross entropy loss, which is expressed as formula (2), where N is the number of samples, y is the pixel matrix of the defect image manually annotated, and y' is the pixel matrix of the model prediction result image. Finally, the test set data is sent to the trained railway train wheel surface defect detection model to output the location information of the defects in the image.

[0047]

[0048] (4) Testing the surface defect data of train wheelsets: Input the surface image of train wheelsets into the trained surface defect detection model of train wheelsets, obtain the defective area in the detection image, and distinguish it from the non-defective area. The detection results are as follows: Figure 4 shown.

Claims

1. A railway train wheel set surface defect detection method based on deep learning, characterized in that: The following steps are involved: (1) Collecting images of the surface of railway train wheelsets through a train wheelset surface defect detection system; The motor in the train wheel surface defect detection system drives the camera to move and collect images on the horizontal line of the train wheel. After the rotating mechanism rotates the train wheel, the camera collects images on the next horizontal line. The image is detected by the detection client and the detection results are recorded in the database. (2) Annotate the collected surface images, expand the train wheel set images through data enhancement, and divide the enhanced image dataset into training set, validation set, and test set; (3) Establishing a train wheel surface defect detection model based on deep learning, inputting a training set and a validation set into the train wheel surface defect detection model for training, and obtaining a trained detection model; the train wheel surface defect detection model based on deep learning includes an encoder and a decoder arranged in a U-shaped architecture, and the encoder is composed of MobileNetV2; the MobileNetV2 reduces the number of training parameters in the train wheel surface defect detection model through point-by-point convolution and channel-by-channel convolution; The encoder obtains multiple feature maps of different scales through multiple downsampling feature extraction. The decoder restores the surface defect information of railway wheels by connecting and fusing the feature maps with the same scale as that obtained by the encoder, and then detects the surface defects of the wheels. The activation function in the trained detection model is the GELU function, as shown in formula (1): Where x represents the input amount of the previous level network; (4) The dataset is input into the trained train wheelset surface defect detection model to obtain the shape and position of the defects in the test set images.

2. The method for detecting surface defects of railway train wheelsets based on deep learning according to claim 1 is characterized in that: In step (2), the defective area in the surface image is labeled by Labelme to obtain labeled image data.

3. The method for detecting surface defects of railway train wheelsets based on deep learning according to claim 1 is characterized in that: In step (2), the image data is progressively cropped and enhanced by a sliding window method, and the image data is expanded by noise, rotation and mirror morphological operations.

4. The method for detecting surface defects of railway train wheelsets based on deep learning according to claim 1, characterized in that: The training loss function in the trained detection model is the cross entropy loss, as shown in formula (2): Where N is the number of samples, y is the pixel matrix of the manually labeled defect image, and y' is the pixel matrix of the model prediction result image.

5. The method for detecting surface defects of railway train wheelsets based on deep learning according to claim 1, characterized in that: In step (3), during the training process of the deep learning-based train wheelset surface defect detection model, the Adam optimization algorithm and the back propagation algorithm are used to iteratively optimize the model parameters until the training is terminated when the model loss value converges, and the obtained model parameters are saved.

Images