A convolutional neural network-based end-to-end turnout gap detection method

Abstract

The present application belongs to the technical field of deep learning and artificial intelligence, and particularly relates to a kind of convolutional neural network end-to-end switch gap detection method. Including collecting the switch gap image of the switch and switch driven switch conversion process of train passing through turnout;The collected gap image is screened and labeled to obtain a data set;The data set is divided into training set, validation set and test set according to the proportion 7:2:1;The training set is subjected to data enhancement processing;The neural network is trained using the data set to obtain a detection network;The collected gap image is input into the trained neural network for detection;The detection result is analyzed and the analysis result is given. Various types of switch gap detection systems can be integrated into one system, improving the universality and portability of the system.

Description

TECHNICAL FIELD

[0001] The present application relates to a switch gap detection method, in particular to a switch gap detection method based on a convolutional neural network end-to-end, and belongs to the field of detection technology and artificial intelligence. BACKGROUND

[0002] The turnout is a device in the railway that can make the train turn from one track to another. The switch machine drives the turnout to move to the positioning or reverse position to connect different tracks. The tightness of the turnout and the basic rail directly affects the safety of train operation. If the gap is too large, the "four open" phenomenon will occur, which may even lead to train derailment accidents. The gap is reflected by the gap in the switch machine. Real-time detection of the gap size is very important for the signal system and emergency response of the railway.

[0003] The traditional switch gap detection method is manual inspection or installation of photoelectric sensors, magnetic grid sensors, etc. in the switch machine. These methods are greatly affected by environmental factors and human factors. The current switch gap detection system used in the railway field is mainly based on video image detection method.

[0004] Chinese patent application (publication number CN110363742A) discloses a switch gap detection method based on CNN and image processing. The method first classifies the switch machine using a convolutional neural network to determine the type and positioning and reverse position information of the switch machine. Then, the traditional image processing algorithm is used to process the gap image to extract the gap feature straight lines L1 and L2, and thus calculate the switch gap value. This method is a two-step method, and the system is not as stable as the end-to-end detector. Moreover, the traditional image processing algorithm is greatly affected by the state of the switch gap image, such as the threshold value problem when binarizing the image, and the difficulty in extracting the gap feature straight line when the switch gap is contaminated by oil stains, etc. In addition, the gap feature straight line needs to be extracted in a fixed range of the image. These problems often greatly reduce the application effect of the traditional image processing algorithm on the switch gap. SUMMARY

[0005] To solve the problems in the above background art, the present application proposes a switch gap detection method based on a convolutional neural network end-to-end. This method can determine the type of switch gap, the positioning and reverse position state of switch gap, and the size of gap value through one algorithm, and the algorithm can get good results for the lightness of the image and the change of the gap position of different images. It has a fundamental improvement effect on the detection of contaminated switch gap value.

[0006] The technical solution of the present application to solve the above problems is: a switch gap detection method based on a convolutional neural network end-to-end, characterized by comprising the following steps:

[0007] Step one, collect the gap image of the switch machine driven by the train passing through the turnout and the conversion process of the turnout switch machine;

[0008] Step two, screen and label the collected gap image;

[0009] Step three, divide the data set of step two into training set, validation set and test set according to the proportion of 7:2:1;

[0010] Step four, data enhancement processing is performed on the training set divided in step three;

[0011] Step five, use the data set of step three to train the neural network and get the detection network;

[0012] Step six, input the collected gap image into the trained neural network for detection;

[0013] Step seven, analyze the detection result and give the analysis result.

[0014] Further, the gap image of the switch machine driven by the train passing through the turnout and the conversion process of the turnout switch machine in step one includes the positioning and reverse positioning images of ZD6, S700k, ZYJ7 and ZDJ9 switch machine gaps, and the data is balanced.

[0015] Further, the straight line of the switch machine gap image screened in step two is horizontal or vertical, otherwise the angle of the camera needs to be adjusted, and the annotation format in step two is a one-line seven-column vector (0, 0, 0, 1, 0, 0.21, 0.43), the first four bits represent the one-hot representation of the category of the gap of the switch machine, the fifth bit represents whether the gap image is positioning or reverse positioning, 0 represents positioning and 1 represents reverse positioning, the sixth bit represents the position of the first line of the gap relative to the image, and the seventh bit represents the position of the second line of the gap relative to the image.

[0016] Further, the data enhancement operation in step four includes random data erasure enhancement, fine tuning of image contrast, color saturation, hue, sharpness and data standardization processing, the purpose of data enhancement is to make the neural network more general, and the data enhancement operation can obtain a more optimal inference architecture.

[0017] Further, the neural network structure in step five is improved from the network structure of yolov3. The neural network selects darknet53 as the backbone feature extraction network. First, the input network picture is resized to (416, 416). In the convolution process, the width and height of the picture are continuously compressed, and the number of channels is continuously expanded, so that a series of feature layers can be obtained. The last three effective feature layers of darknet53, i.e. 13*13*1024, 26*26*512 and 52*52*216, are taken to further fuse the three effective feature layers. The neural network removes the three detection heads of yolov3 network, and performs GAP (Global Average Pooling) processing on the 52*52*128 feature layer. Then, two Fully_Connected layers are connected. The number of neurons in the first Fully_Connected layer is 50, and the number of neurons in the second Fully_Connected layer is 7.

[0018] Further, the loss function of the neural network in step five is divided into cross-entropy loss + L1 loss. The output result of the network is a tensor containing 7 values (T1, T2, T3, T4, T5, T6, T7). Before calculating the loss function, the first four values in the network output tensor are calculated by softmax, and the last three values are calculated by sigmoid to obtain a new tensor (t1, t2, t3, t4, t5, t6, t7). The label value in step two is (y1, y2, y3, y4, y5, y6, y7), and the loss function is:

[0019] ;

[0020] wherein , The third term of the loss function multiplied by the coefficient 2 is to make the detection of the neural network on the gap value of the switch more accurate. By expanding the coefficient, the network can be more sensitive to the error of the detected gap value.

[0021] Further, the detection result in step seven includes gap value, gap classification, gap positioning and reverse position classification information. If the absolute value of the gap value w is greater than the threshold value, it is considered that the gap is abnormal, and the alarm information is returned. If the gap classification score is less than the threshold value, it is considered that the gap is abnormal, and the alarm information is returned. If the gap positioning and reverse position classification score is less than the threshold value, it is considered that the gap is abnormal, and the alarm information is returned. If there is no abnormality, the gap value and the gap classification and positioning and reverse position classification results are returned.

[0022] Further, the detection network output tensor (T1, T2, T3, T4, T5, T6, T7) can calculate the size w of the gap value, the switch gap classification L 缺口 , switch gap classification score S 缺口 , positioning anti-position classification L 定位反位 , positioning anti-position classification score S 定位反位 , the formula is as follows:

[0023] ;

[0024] Wherein, if the image is the switch gap image of ZD6, ZYJ7 and ZDJ9, C takes the pixel height of the image, if the image is the switch gap image of S700k, C takes the pixel width of the image.

[0025] ;

[0026] ;

[0027] ;

[0028] .

[0029] Advantages of the present application:

[0030] Effectively solve the traditional computer vision algorithm is easily affected by light, oil and other environmental impact caused by misjudgment, effectively solve the traditional computer vision algorithm calculation switch gap value accuracy is not enough problem, the present application is end to end algorithm, an algorithm can solve the classification and gap value calculation problem of switch, avoid the instability of two-step method system. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description, obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creating labor.

[0032] Figure 1 is a flow chart of a switch gap detection method based on convolutional neural network end to end according to an embodiment of the present application;

[0033] Figure 2 is a flow chart of switch gap detection result analysis;

[0034] Figure 3 is a neural network structure diagram of switch gap detection;

[0035] Figure 4 is the result picture of ZDJ9 switch gap (with oil stain) detection. DETAILED DESCRIPTION

[0036] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the present application. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0037] The method for detecting switch gap based on convolutional neural network end-to-end provided by the present application will be described in more detail below with reference to the drawings and examples.

[0038] Figure 1 The method for detecting switch gap based on convolutional neural network end-to-end provided by the present application is a flow chart of a method for detecting switch gap based on convolutional neural network end-to-end, and the method for detecting switch gap based on convolutional neural network end-to-end comprises the following steps.

[0039] S1, collecting switch gap images in the process of train passing through turnout and switch driven turnout conversion;

[0040] S2, screening and labeling the collected gap images;

[0041] S3, dividing the data set in step S2 into a training set, a verification set, and a test set according to a proportion of 7:2:1;

[0042] S4, performing data enhancement processing on the training set divided in step S3;

[0043] S5, training a neural network using the data set in step S3 to obtain a detection network;

[0044] S6, inputting the collected gap images into the trained neural network for detection;

[0045] S7, analyzing the detection result and giving an analysis result.

[0046] REFERENCE Figure 1 AND Figure 2The switch gap images collected in step S1 include the positioning and reverse positioning images of ZD6, S700k, ZYJ7, and ZDJ9 switch gaps, and the data is balanced.

[0047] The image switch gap straight line screened in step S2 is in a horizontal or vertical state, otherwise the angle of the camera needs to be adjusted. The annotation format in step S2 is a one-line seven-column vector (0, 0, 0, 1, 0, 0.21, 0.43). The first four bits represent the one-hot representation of the type of switch gap, the fifth bit represents whether the gap image is positioning or reverse positioning, 0 represents positioning and 1 represents reverse positioning, the sixth bit represents the position of the first line of the gap gap relative to the image, and the seventh bit represents the position of the second line of the gap gap relative to the image.

[0048] The data enhancement operation in step S4 includes random data erasure enhancement, fine-tuning of image contrast, color saturation, hue, sharpness, and data standardization processing. The purpose of data enhancement is to make the neural network more generalizable, and the data enhancement operation can obtain a more accurate inference architecture.

[0049] The neural network structure in step S5 is improved from the network structure of yolov3, as shown in Figure 3 The neural network selects darknet53 as the backbone feature extraction network. First, the input network picture is resized to (416, 416). In the convolution process, the width and height of the picture are constantly compressed, and the number of channels is constantly expanded, so that a series of feature layers can be obtained. The last three effective feature layers of darknet53, i.e., 13*13*1024, 26*26*512, and 52*52*216, are taken to further feature fusion. The neural network removes the three detection heads of yolov3 network, and performs GAP (Global Average Pooling) processing on the 52*52*128 feature layer, and continues to connect two Fully_Connected layers. The number of neurons in the first Fully_Connected layer is 50, and the number of neurons in the second Fully_Connected layer is 7.

[0050] The loss function of the neural network in step S5 is cross-entropy loss+L1 loss, and the output of the network is a tensor containing 7 values (T1, T2, T3, T4, T5, T6, T7). Before calculating the loss function, the first four values in the network output tensor are calculated by softmax, and the last three values are calculated by sigmoid to obtain a new tensor (t1, t2, t3, t4, t5, t6, t7). The labeled values in step s2 are (y1, y2, y3, y4, y5, y6, y7), and the loss function is:

[0051] ;

[0052] wherein , The third term of the loss function multiplied by the coefficient 2 is to make the neural network more accurate in detecting the gap value of the switch machine. By expanding the coefficient, the network can be more sensitive to the error of the detected gap value.

[0053] The detection result in step S7 includes gap value, gap classification, gap positioning and reverse classification information. If the absolute value of the gap value w is greater than the threshold value, it is considered that the gap is abnormal, and the alarm information is returned. If the gap classification score is less than the threshold value, it is considered that the gap is abnormal, and the alarm information is returned. If the gap positioning and reverse classification score is less than the threshold value, it is considered that the gap is abnormal, and the alarm information is returned. If there is no abnormality, the gap value and the gap classification and positioning and reverse classification results are returned.

[0054] The tensor (T1, T2, T3, T4, T5, T6, T7) output by the detection network can calculate the size w of the gap value, the switch machine gap classification L 缺口 , the switch machine gap classification score S 缺口 , the positioning and reverse classification L 定位反位 , and the positioning and reverse classification score S 定位反位 , and the formula is as follows:

[0055] ;

[0056] If the image is a switch machine gap image of ZD6, ZYJ7, or ZDJ9, C takes the pixel height of the image, and if the image is a switch machine gap image of S700k, C takes the pixel width of the image.

[0057] ;

[0058] ;

[0059] ;

[0060] .

[0061] Figure 4 The results of detecting the ZDJ9 gap image by using the gap detection method based on the convolutional neural network end-to-end switching machine of the present application show that the detection method can not only correctly predict the type of the switching machine gap, but also accurately judge the size of the switching machine gap value.

[0062] The above is the preferred embodiment of the present application, so any equivalent changes or modifications made in accordance with the structure, features and principles described in the scope of the present application are included in the scope of the present application.

Claims

1.A method for detecting a gap of a switch machine based on a convolutional neural network end-to-end, characterized in that, The method comprises the following steps: Step one, collecting the switch machine gap images of the train passing through the turnout and the turnout switch machine driving the turnout conversion process; Step two, screening and labeling the collected gap images; Step three, dividing the step two dataset into training set, validation set, and test set according to the proportion of 7:2:1; Step four, performing data enhancement processing on the training set divided in step three; Step five, training the neural network using the dataset in step three to obtain a detection network; Step six, inputting the collected gap images into the trained neural network for detection; Step seven, analyzing the detection results and giving the analysis results; The neural network structure in step five is improved from the network structure of yolov3. The neural network selects darknet53 as the backbone feature extraction network. First, the input network picture is resized to (416, 416). In the convolution process, the width and height of the picture are continuously compressed, and the number of channels is continuously expanded, so that a series of feature layers can be obtained. The last three effective feature layers of darknet53, i.e. 13*13*1024, 26*26*512, and 52*52*216, are used for further feature fusion. The neural network removes the three detection heads of yolov3 network, and performs GAP (Global Average Pooling) processing on the 52*52*128 feature layer. Two Fully_Connected layers are further connected. The number of neurons in the first Fully_Connected layer is 50, and the number of neurons in the second Fully_Connected layer is 7. The loss function of the neural network in step five is divided into cross entropy loss + L1 loss. The output of the network is a tensor containing 7 values (T1, T2, T3, T4, T5, T6, T7). Before calculating the loss function, the first four values in the network output tensor are calculated by softmax, and the last three values are calculated by sigmoid to obtain a new tensor (t1, t2, t3, t4, t5, t6, t7). The annotation value in step two is (y1, y2, y3, y4, y5, y6, y7). The loss function is: loss = -∑ti*lnyi+∑|y5-t5|+2*∑|yj-tj|; where i∈{1, 2, 3, 4}, j∈{6, 7}, and the third term of the loss function is multiplied by a coefficient of 2 to make the neural network more accurate in detecting the switch machine gap value. By expanding the coefficient, the network can be more sensitive to the error of the detected gap value. 2.The switch gap detection method based on a convolutional neural network end-to-end according to claim 1, characterized in that, The collected images in step one include the positioning and reverse positioning images of ZD6, S700k, ZYJ7, and ZDJ9 switch machine gaps, and the data is balanced. 3.The switch gap detection method based on a convolutional neural network end-to-end according to claim 1, characterized in that, The straight line of the gap of the image switch machine in step two is horizontal or vertical, otherwise the angle of the camera needs to be adjusted. The annotation format in step two is a seven-column vector in one row, (0, 0, 0, 1, 0, 0.21, 0.43). The first four bits represent the one-hot representation of the category of the gap of the switch machine, the fifth bit represents whether the gap image is positioned or inverted, 0 represents positioning, and 1 represents inversion. The sixth bit represents the position of the first line of the gap relative to the image, and the seventh bit represents the position of the second line of the gap relative to the image. 4.The switch gap detection method based on a convolutional neural network end-to-end according to claim 1, characterized in that, The data enhancement operation in step four includes random data erasure enhancement, fine-tuning of image contrast, color saturation, hue, sharpness, and data standardization processing. The purpose of data enhancement is to make the neural network more generalizable. Data enhancement operations can obtain a more accurate inference architecture. 5.The switch gap detection method based on a convolutional neural network end-to-end according to claim 1, wherein, The detection result in step seven includes gap value, gap classification, gap positioning and inversion classification, etc. If the absolute value of the gap value w is greater than the threshold value, it is considered that the gap is abnormal, and the alarm information is returned. If the gap classification score is less than the threshold value, it is considered that the gap is abnormal, and the alarm information is returned. If the gap positioning and inversion classification score is less than the threshold value, it is considered that the gap is abnormal, and the alarm information is returned. If there is no abnormality, the gap value and the gap classification and positioning and inversion classification results are returned. 6.The switch gap detection method based on a convolutional neural network end-to-end according to claim 1, characterized in that, By detecting the tensors (T1, T2, T3, T4, T5, T6, T7) output by the network, the size w of the gap value, the switch gap classification L 缺口 , the switch gap classification score S 缺口 , the positioning anti-position classification L 定位反位 , the positioning anti-position classification score S 定位反位 , the formula is as follows: If the image is a switch machine gap image of ZD6, ZYJ7, or ZDJ9, C takes the pixel height of the image. If the image is a switch machine gap image of S700k, C takes the pixel width of the image. L 缺口 = argmax(T1, T2, T3, T4);

Images