Turnout gap detection method and system based on end-to-end depth coding

Abstract

The invention discloses a turnout gap detection method and system based on end-to-end depth coding, and belongs to the technical field of railway transportation safety monitoring and intelligent operation and maintaining.The method comprises the steps that an initial line anchor is set for a turnout gap image, a deep learning model is built by combining features extracted by the turnout gap image, and therefore two end side line segments of a turnout gap are predicted; the method comprises the following steps: performing positive and negative sample division on a predicted line segment through an intersection-parallel ratio of the predicted line segment and a real line segment, constructing line segment regression loss of a deep learning model through the intersection-parallel ratio so as to assist classification training of two end-side line segments, performing line segment regression training on only positive samples, and performing line segment prediction on a turnout gap image on the trained deep learning model. Obtaining a set of the two types of end-side line segments, and obtaining the length of the turnout gap through the shortest distance between the two types of end-side line segments in the set. According to the invention, high-precision rapid detection can be carried out on the turnout gap, and the application scenarios can be expanded to the fields of industrial production lines, constructional engineering, power equipment and the like.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of railway transportation safety monitoring and intelligent operation and maintenance, and particularly relates to a turnout gap detection method and system based on end-to-end deep coding. BACKGROUND

[0002] In the railway trunk line, intercity railway and urban rail transit system, the state of the turnout equipment needs to be monitored to improve the stable operation and safety of the rail transit, and the turnout gap detection is particularly important. The traditional turnout gap detection method includes two stages:

[0003] In the first stage, the photographed turnout gap image is obtained, and a detection and segmentation model is used to identify the representation rod position at the turnout gap in the form of a prediction box.

[0004] In the second stage, the gap size is calculated by the representation rod position and the distance of the manual reference point.

[0005] The above method needs a tedious manual configuration process due to the design of image detection and image segmentation, and needs to obtain labeled samples for training. The training and identification process requires a large model calculation, which increases the labor cost and equipment energy consumption, and is low in efficiency. At the same time, the traditional prediction box mode is not conducive to image fitting, which limits the accuracy of gap detection. SUMMARY

[0006] To solve the problems of the prior art and improve the efficiency of turnout gap detection and reduce the cost, the present application adopts the following technical solution:

[0007] The turnout gap detection method based on end-to-end deep coding includes the following steps:

[0008] An initial line anchor is set for the turnout gap image;

[0009] A deep learning model is constructed to obtain the turnout gap image and extract features therefrom. The offset features extracted and the corresponding initial line anchor are used to generate at least two prediction line segments on the end sides of the turnout gap;

[0010] The prediction line segments are divided into positive and negative samples according to the intersection-over-union of the prediction line segments and the real line segments;

[0011] The line segment regression loss of the deep learning model is constructed based on the intersection-over-union to assist the classification training of the line segments. Only the positive samples enter the line segment regression training;

[0012] The trained deep learning model is used to predict the line segments of the turnout gap image, and at least two types of prediction line segments on the end sides of the turnout gap are obtained;

[0013] The distance between the two types of predicted line segments is taken as the turnout gap length.

[0014] Further, the extracted offset features include end point offset, angle offset, length offset of the corresponding initial line anchor, and first line segment category and second line segment category of the line anchors at both ends of the turnout gap.

[0015] Further, the intersection of the IoU is based on the extracted offset features and the initial line anchor to obtain the end point coordinates, angle and length of the predicted line segment, the intermediate ray of the predicted line segment and the real line segment is calculated through the end point coordinates and the angle, and the intersection of the projections of the predicted line segment and the real line segment on the intermediate ray is obtained by combining the length.

[0016] Further, the union of the IoU is the minimum rectangular area enclosing the predicted line segment and the real line segment.

[0017] Further, the end point coordinates of the predicted line segment are the starting point of the predicted line segment, the ending point of the predicted line segment is obtained by combining the starting point coordinates with the product of the cosine value of the angle and the length, based on the horizontal axis and the vertical axis of the end point coordinates, the difference between the maximum and minimum values of the vertical axis and the difference between the maximum and minimum values of the horizontal axis of the starting point and the ending point of the predicted line segment and the real line segment are calculated, and the product of the two difference values is taken as the minimum rectangular area.

[0018] Further, the line segment regression loss is based on the cosine value of the angle difference between the predicted line segment and the real line segment and the IoU, and is constructed by balancing the parameters, and the predicted line segment with an IoU greater than a threshold value is taken as a positive sample for line segment regression training.

[0019] Further, the deep learning model also assists classification training by constructing a feature loss of contrast learning, and the feature loss of contrast learning is constructed by calculating the second norm mean of the feature difference between the positive sample and each negative sample, and then summing the result based on the one minus the mean of all positive samples.

[0020] Further, the setting of the initial line anchor is performed by multi-scale down-sampling the turnout gap image, and defining multiple initial line anchors including line anchor angle and line anchor length for each pixel of the down-sampled image; the feature extraction is performed on the obtained turnout gap image to obtain multiple high-dimensional feature maps, a high-dimensional feature map is segmented based on a certain dimension, and one segment of the feature map corresponds to one of the initial line anchors; the image line segment is predicted by the deep learning model, the pixel length is obtained through the image line segment, and the corresponding predicted line segment is obtained through mapping based on the real line segment.

[0021] Further, the intersection-over-union threshold and the confidence threshold are set, line segment prediction is performed on all high-dimensional feature maps of the turnout gap image based on the corresponding line anchors, the predicted line segments and the corresponding line segment class confidence thereof are obtained, the predicted line segments with high intersection-over-union and high confidence are obtained by screening the predicted line segments through the intersection-over-union threshold and the confidence threshold, and the shortest distance between two types of predicted line segments is obtained from the set as the turnout gap length.

[0022] The turnout gap detection system based on the end-to-end deep coding comprises a setting module, a deep learning module and an intersection-over-union calculation module, the turnout gap detection method based on the end-to-end deep coding is adopted, the initial line anchor is set for the turnout gap image to construct the deep learning model, the deep learning model is trained through the intersection-over-union of the predicted line segment and the real line segment, the line segments at both ends of the gap in the turnout gap image are predicted through the trained deep learning model, and the turnout gap length is obtained according to the shortest distance between the line segments.

[0023] The advantages and beneficial effects of the present application are as follows:

[0024] The present application realizes direct output of the gap size by constructing an end-to-end deep regression model, taking the gap image as the input, realizes faster recognition speed on the basis of maintaining reliable accuracy compared with the traditional method, and does not need manual labeling, and breaks away from the mode of predicting the frame in the traditional target detection method, detects in the mode of predicting the line segment, designs the anchor mode based on the line segment, the intersection-over-union algorithm based on the line segment and the non-maximum suppression algorithm, is more conducive to fitting the straight line on the image compared with the traditional mode of predicting the frame, and is convenient for predicting the distance between targets on the image. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a flowchart of the method in the embodiment of the present application.

[0026] Figure 2 is a schematic diagram of line segment intersection calculation in the embodiment of the present application.

[0027] Figure 3 is a schematic diagram of line segment union calculation in the embodiment of the present application.

[0028] Figure 4 is a structural schematic diagram of the system in the embodiment of the present application. DETAILED DESCRIPTION

[0029] The specific embodiments of the present application are described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0030] As Figure 1As shown, the turnout gap detection method based on end-to-end depth coding, by combining computer vision and deep learning, can be applied to the high-precision and rapid detection of turnout gaps, including the following steps:

[0031] Step S1: Set the size of the turnout gap image input to the model to [M,N], which is [640,640] in this embodiment. Pre-set three sets of line anchors: small, medium, and large. , , The turnout gap images were analyzed separately. ,Right now For each downsampling scale, the anchor set `anchor` defines three initial anchors for each pixel in the downsampling result of the turnout gap image. The angles of the three anchors corresponding to each pixel are initialized to [...]. ], The initial anchor length is 10. The initial anchor length is 5. The initial anchor length is 2.

[0032] Step S2: Input an RGB turnout gap image and resize it to [640, 640]. This image will be fed into a Feature Piramid Network to extract features, resulting in three high-dimensional feature maps with sizes of [missing values]. , , After dimensionality transformation and convolution, three high-dimensional feature maps are obtained respectively. That is, the sizes are respectively [ ], [ 114], [ The three tensors are obtained by dividing the third dimension 114 into three equal segments to correspond to the three initial line anchors. , , There are a total of 9 feature maps, namely arrive , arrive , arrive There are a total of 9 38-dimensional feature vectors, which correspond to arrive , arrive , arrive .

[0033] For each 38-dimensional feature vector, the first 4 dimensions are Represents prediction X-axis offset, Y-axis offset, angle offset, length offset. For example, for a size of , 4 dimensions are numbered from 0 to 3, The predicted line segment can be represented as:

[0034]

[0035] Where x, y represent the index in the feature map , i represents 4 different dimensions, represents the offset of the line segment in 4 different dimensions, for example: represents the 0th value of the vector in the xth row and yth column in the feature map , that is, the value of the predicted X-axis offset, represents the initial line anchor in 4 different dimensions, represents the predicted line segment attributes based on the initial line anchor after offset in different dimensions, which are X-axis coordinate, Y-axis coordinate, X-axis positive direction angle, and line segment length, respectively.

[0036] The present application defines two different categories of line segments, i.e. a type of line segment and b type of line segment in the turnout gap image, and the length of the final turnout gap is determined by the shortest distance between the two line segments. The 38-dimensional , i∈[0,3] is the estimated change amount of the line anchor, the next two dimensions, i∈[4,5] respectively represent the probability of being judged as a type of line segment and the probability of being judged as a type of line segment, and the next 32 dimensions are features for training, thus, the predicted category is , N=2 in the embodiment represents the number of line segment categories.

[0037] Since the initial line anchor is defined based on each pixel of the down-sampled turnout gap image, the deep learning model predicts the image line segment, and the pixel length is obtained through the image line segment, therefore, the pixel length needs to be mapped to the length of the real line segment, and an additional mapping relationship needs to be set, for example, 1px=0.1mm, and then the predicted line segment corresponding to the real line segment is obtained.

[0038] Step S3: In the model training stage, each predicted line segment is traversed, and the line segment intersection ratio of each real line segment is calculated, as shown in Figure 2 , Figure 3 , which specifically includes the following steps:

[0039] Step S31: input the predicted line segment and the real line segment , and set a screening threshold ;

[0040] ​Step S32: calculate the middle ray of the predicted line segment and the real line segment (the ray has no length);

[0041] Step S33: calculate the projection of the middle ray on the line segment , and save the distance between the projection start and end points , similarly, calculate the projection of the middle ray on the line segment , and save the distance between the projection start and end points , calculate the intersection of the line segment and the line segment , calculate the intersection of the line segment and the line segment ;

[0042] Step S34: calculate the end point coordinates of the predicted line segment , , , similarly, calculate the end point coordinates of the real line segment and , based on the start point coordinates and end point coordinates of the predicted line segment and the real line segment, calculate the area of the minimum wrapping rectangle of the red and blue strip line segment:

[0043]

[0044] Step S35: calculate the line segment intersection ratio , if , the predicted line segment is a positive sample, otherwise it is a negative sample;

[0045] Step S4: construct the loss function, , , represents the target confidence loss, , represents the line segment regression loss, , represents the classification loss, , represents the feature loss, in the embodiment of the present application, , , represents the balance parameter, the target confidence loss and the classification loss , with reference to the YOLO series, the present application proposes the line segment regression loss , only the predicted line segment determined as a positive sample by the line segment intersection ratio will enter the regression training of this loss, wherein is an adjustment parameter. In addition, the present application proposes a feature loss based on contrastive learning:

[0046]

[0047] , wherein, , represents the number of positive samples, , represents the number of negative samples,​ represents the feature extracted from the i-th positive sample, represents the feature extracted from the j-th negative sample, and the purpose is to improve the discrimination of positive samples and negative samples.

[0048] Step S5: In the inference stage, the output prediction line segment of step S2 is input to a line segment screening module, which is similar to the non-maximum suppression (NMS) in the detection frame mode. The specific steps are as follows:

[0049] Step S51: input the set IOU threshold and the confidence threshold , for all , based on the anchor to calculate the predicted line segment, and get the predicted line segment set , The confidence (i.e. the probability corresponding to the maximum probability class) is less than directly screened out;

[0050] Step S52: for the line segment in the predicted line segment set , if is not screened out, sort the index list according to the confidence from large to small;

[0051] Step S53: for each line segment corresponding to the index list , if the line intersection over union LineIOU is greater than the IOU threshold , then , is replaced by a new ;

[0052] Step S54: The remaining in is the final predicted line segment set.

[0053] Step S6: the shortest distance between the two types of line segments in the final line segment set is used as the predicted turnout gap length of the turnout gap image.

[0054] Figure 4 As shown in , the turnout gap detection system based on end-to-end deep coding includes a setting module, a deep learning module, and an intersection over union calculation module. The initial line anchor is set for the turnout gap image to construct the deep learning model. The deep learning model is trained by the intersection over union of the predicted line segment and the real line segment. The trained deep learning model is used to predict the line segments at both ends of the gap in the turnout gap image, and the shortest distance between the line segments is used to obtain the turnout gap length.

[0055]

[0055] The present application can detect the turnout gap with high precision and speed based on computer vision and deep learning, and its core application scene focuses on the state monitoring of turnout equipment in railway trunk lines, intercity railways and urban rail transit systems, and can be directly integrated into a turnout intelligent maintenance platform, a track safety early warning system or a railway infrastructure automatic detection terminal to obtain the size data of the turnout gap in real time and provide key technical support for turnout tightness and stability evaluation.

[0056] From the perspective of technology extension, the core innovations such as the end-to-end line segment prediction framework and the line segment-based intersection-over-union algorithm in the present application can also be extended to other scenarios that require high-precision size detection of linear targets, including but not limited to gap detection of mechanical parts in industrial production lines (such as precise guide rail butt joint gap), component splicing joint measurement in construction engineering, gap monitoring of power equipment (such as high-voltage switch contact), etc. The present application has cross-field technical adaptability and can widely serve the linear target size detection needs of the transportation, intelligent manufacturing, engineering detection and other industries.

[0057] The above examples are only used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting a turnout gap based on end-to-end deep coding, characterized in that: an initial line anchor is set for a turnout gap image; a deep learning model is constructed to obtain the turnout gap image and extract features therefrom, and a predicted line segment of at least two end sides of the turnout gap is generated from the extracted offset features and the corresponding initial line anchor; positive and negative samples of the predicted line segment are divided by an intersection-over-union of the predicted line segment and a real line segment; a line segment regression loss of the deep learning model is constructed based on the intersection-over-union to assist the classification training of the line segment, and only the positive sample enters the line segment regression training; the line segment prediction of the turnout gap image is performed by the trained deep learning model to obtain at least two types of predicted line segments of the end sides of the turnout gap; and the distance between the two types of predicted line segments is taken as the length of the turnout gap. The extracted offset features include the end point offset, angle offset and length offset of the corresponding initial line anchor, as well as the first line segment category and the second line segment category as the line anchors of the two ends of the turnout gap. The intersection of the intersection-over-union is based on the extracted offset features and the initial line anchor to obtain the end point coordinates, angle and length of the predicted line segment, the middle ray of the predicted line segment and the real line segment is calculated by the end point coordinates and the angle, and the intersection of the projections of the predicted line segment and the real line segment on the middle ray is obtained by combining the length. The union of the intersection-over-union is the minimum rectangular area wrapping the predicted line segment and the real line segment. The end point coordinates of the predicted line segment are the starting point of the predicted line segment, the ending point of the predicted line segment is obtained by the product of the cosine value of the angle and the length combined with the starting point coordinates, based on the horizontal axis and the vertical axis of the end point coordinates, the difference between the maximum and minimum values of the vertical axis and the difference between the maximum and minimum values of the horizontal axis of the starting point and the ending point of the predicted line segment and the real line segment are calculated, and the product of the two difference values is taken as the minimum rectangular area. The line segment regression loss is based on the cosine value of the angle difference between the predicted line segment and the real line segment and the intersection-over-union, and is constructed by balancing parameters, and the predicted line segment with the intersection-over-union greater than a threshold value is taken as a positive sample for line segment regression training. The deep learning model also assists the classification training by constructing a feature loss of contrast learning, and the feature loss of contrast learning is constructed by calculating the second norm mean of the feature difference between the positive sample and each negative sample, and then summing the results based on the one minus the mean of all positive samples.

2. The end-to-end deep learning based turnout gap detection method of claim 1, wherein: The setting of the initial line anchor is performed by multi-scale down-sampling the turnout gap image, and defining multiple initial line anchors for the pixels of each down-sampled image; the feature extraction is performed on the obtained turnout gap image to obtain multiple high-dimensional feature maps, the high-dimensional feature maps are segmented, and one segment of the feature map corresponds to one initial line anchor; 3. The end-to-end deep learning based turnout gap detection method of claim 2, wherein: The image line segment is predicted by the deep learning model, the pixel length is obtained from the image line segment, and the corresponding predicted line segment of the real line segment is obtained by mapping based on the real line segment.

4. The end-to-end deep learning based turnout gap detection method of claim 3, wherein: ​ 5. The end-to-end deep learning based turnout gap detection method of claim 4, wherein: ​ 6. The end-to-end deep learning based turnout gap detection method of claim 3, wherein: ​ 7. The end-to-end deep learning based turnout gap detection method of claim 1, wherein: ​ 8. The end-to-end deep learning based turnout gap detection method of claim 1, wherein: ​ ​ 9. The end-to-end deep learning based turnout gap detection method of claim 1, wherein: The intersection-over-union threshold and the confidence threshold are set, the turnout gap image is subjected to line segment prediction, the predicted line segment and the corresponding line segment category confidence are obtained, the predicted line segment is screened through the intersection-over-union threshold and the confidence threshold, the predicted line segment set with high intersection-over-union and high confidence is obtained, and the shortest distance between two types of predicted line segments is obtained from the set as the turnout gap length.

10. A system for detecting turnout gap based on end-to-end deep coding, comprising a setting module, a deep learning module, and an intersection and union ratio calculation module, characterized in that: The turnout gap detection method based on end-to-end depth coding in any one of claims 1 to 9 is used to set an initial line anchor for a turnout gap image to construct a deep learning model, the deep learning model is trained through the intersection-over-union of the predicted line segment and the real line segment, the line segments at both ends of the gap in the turnout gap image are predicted through the trained deep learning model, and the turnout gap length is obtained according to the shortest distance between the line segments.

Images