Railway turnout spacing detection method, device and system

Abstract

The present application discloses a method, device and system for detecting the spacing of railway turnouts, relating to the technical field of railways. The method includes: obtaining 3D image data of a railway track; identifying a turnout scene in the 3D image data based on a pre-trained turnout scene recognition network; extracting edge pixel points of turnout rails in the turnout scene; and detecting the turnout spacing according to the edge pixel points. The solution of the present application innovatively unifies turnout scene recognition and turnout spacing detection, improving the detection efficiency of turnout spacing.

Description

Technical Field

[0001] This application relates to the technical field of railway tracks, and in particular to a method, device, and system for detecting the spacing of railway turnouts. Background Art

[0002] A turnout is a necessary mechanical device for a train to turn into or cross another track and is an important part of railway operation. However, when a train passes through a turnout, it will generate a large impact force on the turnout, causing the position deviation of the stock rail and the switch rail of the turnout, changing the inner spacing between the stock rail and the switch rail of the turnout (i.e., the turnout spacing), reducing the service life and safety of the turnout, and even possibly causing traffic accidents. Therefore, measuring the turnout spacing is an important prerequisite for ensuring railway safety.

[0003] Currently, the detection of turnout spacing is mainly divided into the following two methods. One method is to monitor various signal parameters and environmental parameters during the operation of the turnout and its conversion equipment to provide a reference basis for the maintenance and overhaul of the turnout. This method mainly judges whether it works normally by collecting and analyzing the electrical signals of railway equipment. However, the electrical signals are easily restricted by the communication bandwidth and interfered by other cables during the collection and transmission process, and the accuracy and real-time performance of the data will be reduced. The other is to use the collected railway track data, through corresponding processing technologies, extract the required structural feature information, calculate the relevant parameters of the railway track, and judge whether they meet the safety standards. This method is greatly affected by light, and the number of ordinary tracks is much larger than the number of turnouts. The turnout images are mixed in the ordinary track images, resulting in low screening efficiency of the turnout images. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, and system for detecting the spacing of railway turnouts, so as to solve the problem of low detection efficiency of turnout spacing in the prior art.

[0005] To achieve the above purpose, this application provides a method for detecting the spacing of railway turnouts, including:

[0006] Obtaining 3D image data of a railway track;

[0007] Based on a pre-trained turnout scene recognition network, recognizing the turnout scene in the 3D image data;

[0008] Extracting the edge pixel points of the turnout rails in the turnout scene;

[0009] Detecting the turnout spacing according to the edge pixel points.

[0010] Optionally, after obtaining the 3D image data of the railway track, the method further includes:

[0011] Performing histogram equalization processing on the 3D image data.

[0012] Optionally, the training process of the turnout scene recognition network includes:

[0013] Establish the turnout scene recognition network based on a convolutional neural network;

[0014] Input a preset training sample into the turnout scene recognition network and output a prediction probability;

[0015] Input the prediction probability into a preset loss function and calculate the loss value of the turnout scene recognition network;

[0016] Optimize the parameters of the turnout scene recognition network according to the loss value by using a preset optimization algorithm.

[0017] Optionally, the 3D image data includes intensity image data;

[0018] Based on a pre-trained turnout scene recognition network, recognizing the turnout scene in the 3D image data includes:

[0019] Input the intensity image data into the turnout scene recognition network;

[0020] Extract the feature information in the intensity image data;

[0021] Calculate the probability representing the turnout scene according to the feature information to recognize the turnout scene in the 3D image data.

[0022] Optionally, the 3D image data includes depth image data;

[0023] Extracting the edge pixel points of the turnout rail of the turnout scene includes:

[0024] Perform rail surface area segmentation processing on the depth image data;

[0025] Obtain a turnout binary image according to the depth image data after the area segmentation processing;

[0026] Use an edge detection operator to detect the edge pixel points of the turnout binary image.

[0027] Optionally, detecting the turnout spacing according to the edge pixel points includes:

[0028] Determine multiple edge straight line segments of the turnout rail based on the Hough transform feature detection algorithm and the edge pixel points;

[0029] Perform fitting on multiple edge straight line segments respectively based on the least square method;

[0030] Detect the turnout spacing according to the edge straight line after fitting.

[0031] Optionally, detecting the turnout spacing according to the fitted edge straight line includes:

[0032] Selecting a first edge straight line and a second edge straight line for calculating the turnout spacing according to the distance between the fitted edge straight line and the origin of the parameter space;

[0033] Receiving a position parameter input by a user;

[0034] Determining the turnout spacing at a position corresponding to the position parameter input by the user according to the position parameter, the first edge straight line, and the second edge straight line.

[0035] An embodiment of the present application further provides a turnout spacing detection device for railways, including:

[0036] An acquisition module, configured to acquire 3D image data of a railway track;

[0037] An identification module, configured to identify a turnout scene in the 3D image data based on a pre-trained turnout scene recognition network;

[0038] An extraction module, configured to extract edge pixel points of turnout rails in the turnout scene;

[0039] A detection module, configured to detect the turnout spacing according to the edge pixel points.

[0040] Optionally, the device further includes:

[0041] A processing module, configured to perform histogram equalization processing on the 3D image data.

[0042] Optionally, the device further includes:

[0043] A training module, configured to train the turnout scene recognition network; wherein, the training module includes:

[0044] A building sub-module, configured to build the turnout scene recognition network based on a convolutional neural network;

[0045] An output sub-module, configured to input a preset training sample into the turnout scene recognition network and output a prediction probability;

[0046] A calculation sub-module, configured to input the prediction probability into a preset loss function and calculate a loss value of the turnout scene recognition network;

[0047] An optimization sub-module, configured to optimize parameters of the turnout scene recognition network according to the loss value by using a preset optimization algorithm.

[0048] Optionally, the 3D image data includes intensity image data;

[0049] The recognition module includes:

[0050] An input sub-module for inputting the intensity image data into the turnout scene recognition network;

[0051] An extraction sub-module for extracting feature information from the intensity image data;

[0052] A recognition sub-module for calculating the probability representing the turnout scene according to the feature information to recognize the turnout scene in the 3D image data.

[0053] Optionally, the 3D image data includes depth image data;

[0054] The extraction module includes:

[0055] A processing sub-module for performing rail surface area segmentation processing on the depth image data;

[0056] An acquisition sub-module for obtaining a turnout binary image according to the depth image data after area segmentation processing;

[0057] A first detection sub-module for detecting edge pixel points of the turnout binary image by using an edge detection operator.

[0058] Optionally, the detection module includes:

[0059] A determination sub-module for determining multiple edge straight line segments of the turnout rail based on the Hough transform feature detection algorithm and the edge pixel points;

[0060] A fitting sub-module for respectively fitting the multiple edge straight line segments based on the least squares method;

[0061] A second detection sub-module for detecting the turnout spacing according to the fitted edge straight line.

[0062] Optionally, the second detection sub-module includes:

[0063] A selection unit for selecting a first edge straight line and a second edge straight line for calculating the turnout spacing according to the distance between the fitted edge straight line and the origin of the parameter space;

[0064] A receiving unit for receiving the position parameters input by the user;

[0065] A determination unit for determining the turnout spacing at the position corresponding to the position parameter input by the user according to the position parameter, the first edge straight line and the second edge straight line.

[0066] An embodiment of the present application also provides a railway turnout spacing detection system, including: a processor, a memory, and a program stored on the memory and executable on the processor, and when the program is executed by the processor, the steps of the railway turnout spacing detection method described above are implemented.

[0067] An embodiment of the present application also provides a readable storage medium, on which a program is stored, and when the program is executed by a processor, the steps of the railway turnout spacing detection method described above are implemented.

[0068] The above technical solution of the present application has at least the following beneficial effects:

[0069] In the railway turnout spacing detection method according to the embodiment of the present application, first, 3D image data of a railway track is acquired; second, based on a pre-trained turnout scene recognition network, the turnout scene in the 3D image data is recognized; third, the edge pixel points of the turnout rails in the turnout scene are extracted; finally, the turnout spacing is detected according to the edge pixel points. In this way, by using the turnout scene recognition network to recognize the 3D image data, the recognition speed and accuracy of the turnout scene are improved, thereby improving the turnout spacing detection efficiency. Description of the Drawings

[0070] Figure 1 is a schematic flow chart of the railway turnout spacing detection method according to the embodiment of the present application;

[0071] Figure 2 is a schematic structural diagram of the turnout scene recognition network according to the embodiment of the present application;

[0072] Figure 3A is a schematic diagram of a straight line in a rectangular coordinate system according to the embodiment of the present application;

[0073] Figure 3B is a schematic diagram of a curve in a parameter annual inspection coordinate system according to the embodiment of the present application;

[0074] Figure 4 is a schematic flow chart of the training of the turnout scene recognition network according to the embodiment of the present application;

[0075] Figure 5 is a relationship diagram between the number of training iterations and the loss value according to the embodiment of the present application;

[0076] Figure 6 is a relationship diagram between the number of training iterations and the precision rate according to the embodiment of the present application;

[0077] Figure 7 is the ROC curve and AUC value of the training result according to the embodiment of the present application;

[0078] Figure 8Schematic structural diagram of the railway turnout spacing detection device according to the embodiment of the present application. Detailed implementation manners

[0079] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0080] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the associated objects before and after.

[0081] Next, in conjunction with the accompanying drawings, the railway turnout spacing detection method provided by the embodiments of the present application will be described in detail through specific embodiments and their application scenarios.

[0082] As Figure 1 shown, it is one of the flow schematic diagrams of the railway turnout spacing detection method according to the embodiment of the present application, and the method includes:

[0083] Step 101: Obtain 3D image data of a railway track;

[0084] Here, it should be noted that the 3D image data in this step can be data collected by a Ranger3 3D camera, which can capture the depth information and intensity information of the entire scene, including three parts: rails, fasteners, and sleepers. Among them, the depth information contains more track position information and can be used for rail surface segmentation and edge extraction. The intensity information contains more semantic information and can be used for neural network classification to identify the turnout scene.

[0085] Step 102: Based on a pre-trained turnout scene recognition network, identify the turnout scene in the 3D image data;

[0086] Here, it should be noted that the pre-trained turnout scene recognition network is a turnout scene recognition network that meets preset conditions, where the preset conditions can be the recognition error, recognition speed, etc. of the turnout scene recognition network.

[0087] In this step, the turnout scene recognition network is a turnout scene recognition network based on the AlexNet network. By applying deep learning to the recognition of railway turnouts and inputting the preprocessed railway data into the neural network for recognition and prediction, a large amount of railway data is effectively utilized, making up for the disadvantage of machine vision being inflexible in processing a large amount of data, and achieving accurate classification and recognition of ordinary tracks and turnouts.

[0088] Step 103: Extract the edge pixel points of the turnout rails in the turnout scene;

[0089] Since the turnout spacing is the distance between two rails, in this step, it is necessary to first obtain the pixel points that can represent the edges of the turnout rails.

[0090] Step 104: Detect the turnout spacing according to the edge pixel points.

[0091] Here, it should be noted that the three processing modules of railway data processing, recognition and classification, and feature detection are integrated into a whole. Inputting railway data and outputting detection results is simple in operation and improves the efficiency of turnout detection.

[0092] The railway turnout spacing detection method of the embodiment of the present application first obtains 3D image data of the railway track; secondly, based on the pre-trained turnout scene recognition network, recognizes the turnout scene in the 3D image data; thirdly, extracts the edge pixel points of the turnout rails in the turnout scene; finally, detects the turnout spacing according to the edge pixel points. In this way, by using the turnout scene recognition network to recognize the 3D image data, the recognition speed and accuracy of the turnout scene are improved, thereby enhancing the turnout spacing detection efficiency.

[0093] Here, it should be noted that there are boundaries between the rail surface and fasteners and sleepers of the track. The surface gray level of the rail surface in the collected 3D image data is relatively high, and the structural feature information of the track is obvious. However, since the railway track is an outdoor device, the data collection is greatly affected by environmental factors, and there are usually various random signal interferences, resulting in the interface between different structures in the collected 3D image data being not easy to distinguish, and the contrast between local images still needs to be improved. Therefore, as an optional implementation manner, after step 101 of obtaining the 3D image data of the railway track, the method further includes:

[0094] Perform histogram equalization processing on the 3D image data.

[0095] Here, it should be noted that histogram equalization, also known as histogram flattening, belongs to the point operation algorithm based on the spatial domain method in image enhancement algorithms. Its principle is to readjust the distribution of image gray levels, expand the gray levels with high frequencies, and merge the gray levels with low frequencies, that is, to change the pixel gray levels concentrated in the low gray level area or high gray level area into a uniform distribution within the entire gray level range, making the number of each gray level of the image roughly the same, thereby increasing the image contrast and making the visual effect of the image clearer. The function of histogram equalization is shown in the following formula:

[0096]

[0097] where n is the total number of pixels in the image, n i is the number of pixels at the i-th gray level, and L is the total number of image gray levels. After the transformation, the gray value is g(x, y), and the formula is as follows:

[0098] g(x,y) = P * S(k)

[0099] where P is the depth of the image gray level, generally 255.

[0100] Here, it should be noted that for images with overall contrast close and the distinguishability between local feature information not obvious, histogram equalization can be used to improve the visual effect of the image without affecting the overall image.

[0101] In this optional implementation, by applying gray level transformation, median filtering, and histogram equalization to 3D image data, the feature enhancement of 3D image data is carried out, the influence of interference signals on the image quality is weakened, the distinguishability between various features of the railway is increased, and the accuracy of computer feature extraction, scene recognition, and detection is improved.

[0102] Here, first of all, it should be noted that deep learning can effectively utilize a large amount of data, imitate the learning process of the human brain, and sequentially extract the structural features of the data through a multi-layer structure, and then analyze these data. Based on this, in order to solve the problem of low efficiency in screening turnout data, the embodiment of this application proposes to use a turnout scene recognition network based on the AlexNet network to learn the turnout structure features from a large amount of railway data and complete the rapid recognition of turnout data.

[0103] Therefore, as an optional implementation, the training process of the turnout scene recognition network includes:

[0104] (A) Establish a turnout scene recognition network based on a convolutional neural network;

[0105] Here, it should be noted that as Figure 2 shown, the turnout scene recognition network includes 8 network layers, and the first 5 layers ( Figure 2The five box structures starting from the left in the middle are convolutional layers, the latter three layers are fully connected layers, and the last fully connected layer outputs a two-dimensional vector representing the classification score result (predicted probability).

[0106] Among them, the convolutional layer is generally composed of convolution operations, activation operations, and pooling operations. Specifically:

[0107] The convolution operation connects neurons to partial regions of the input data through a convolution kernel to extract the features of the input data. In this way, the problems of a huge number of parameters and a huge amount of computation in traditional neural networks are solved. Here, it should be noted that in the embodiments of the present application, different convolution kernels extract different features. The low-level convolutional layers extract some low-level features such as corners and lines, while the high-level convolutional layers extract some more complex features.

[0108] The activation operation can increase the non-linear ability of the model, improve the expression ability and generalization ability, and follows the convolution operation in actual applications.

[0109] The pooling operation, that is, downsampling, aims to reduce the spatial size of the data in the neural network, reduce the number of parameters, save computing resources, speed up the computing speed, and can also effectively prevent the neural network from falling into overfitting. The pooling operation is independent for each depth slice. Generally, there are two pooling operation methods: max pooling and average pooling.

[0110] The convolution, activation, and pooling operations map the original data to a low-dimensional feature space. After multiple operations, the output data is abstract high-level features, and the fully connected layer can integrate these features, map the learned distributed feature representations to the sample label space, output a probability for various classification label situations, and provide a decision basis for the classification by the subsequent classifier.

[0111] (B) Input the preset training samples into the turnout scene recognition network and output the predicted probability;

[0112] In this step, the preset training samples are multiple 3D image data that have been pre-classified. Among them, "pre-classified" can be the image data obtained by pre-classifying multiple 3D image data according to whether the 3D image data is the image data of the turnout scene or the image data of the ordinary railway. That is to say, the training samples input into the turnout scene recognition network carry parameters representing their types; the predicted probability is the probability indicating whether it is a turnout.

[0113] Here, it should be noted that as Figure 2As shown, when using the turnout scene recognition network for turnout recognition, the railway intensity image is adjusted to a size of 227×227×3 as the input data. It is expanded to 27×27×96 using the first layer of convolution and pooling operations, and the ReLU activation function is used after each convolution. Then, through 4 such convolutional layers (as Figure 2 shown, there are only convolutions in the 3rd and 4th layers, without pooling), data of 6×6×256 is obtained. Subsequently, two fully connected layers are used to expand the data, and the scale of the output data is 4096×1. Finally, through the output layer, that is, a fully connected layer with two filters, a prediction result of 2×1 is output. In the output prediction result, the value of each row represents the evaluation score of this classification (the score estimate (probability) of the turnout scene and the score estimate (probability) of the ordinary railway scene). The category with the higher score is the classification result of the input data. The 3D image data is input into the turnout scene recognition network, and the label of each data is output, that is, whether it is a turnout.

[0114] (C) Input the predicted probability into a pre-set loss function to calculate the loss value of the turnout scene recognition network;

[0115] Here, it should be noted that the turnout recognition problem belongs to a binary classification problem. In a neural network, generally, cross-entropy is used as the loss function. Cross-entropy quantifies the gap between the distribution predicted by the model and the true sample distribution, guides the optimization direction of the neural network, and represents the ability of the network to distinguish samples. Among the large amount of railway data (3D data images) collected in the embodiments of this application, the data of ordinary tracks accounts for the vast majority, and the data of turnout tracks only accounts for a small part. To solve the problem of imbalance in the number of positive and negative samples, the embodiments of this application use focal loss (hereinafter referred to as FL) as the loss function.

[0116] FL is based on cross-entropy and adds a weight factor to control the contribution of positive and negative samples to the total loss. The cross-entropy loss function for binary classification is as follows:

[0117]

[0118] Among them, represents the predicted probability of the model, represents the label value. Then the FL loss function is as follows:

[0119]

[0120] α is the weight factor of positive and negative samples, which is calculated from the ratio of positive and negative samples, controls the contribution of positive and negative samples to the loss, that is, weakens the loss of the majority samples and increases the loss of the minority samples. γ is the weight factor for learning easy and difficult samples, that is, enhances the learning of difficult samples and weakens the learning of easy samples. In the embodiments of this application, γ is set to 2.

[0121] (D) According to the loss value, use a preset optimization algorithm to optimize the parameters of the turnout scene recognition network.

[0122] In this step, the Adam optimization algorithm is selected to optimize the turnout scene recognition network.

[0123] Here, it should be noted that neural networks often have two problems during the optimization process towards the optimal model: local minima and saddle points. Both of them can cause gradient dispersion and prevent the neural network from continuing to optimize towards the optimal solution. To solve these two problems, the Adam optimization algorithm combines the AdaGrad algorithm and the RM-Sprop algorithm, automatically adjusts the learning rate of each parameter according to the number of training times, and at the same time uses momentum to provide a basis for the forward trend of the parameter update direction, reducing the situation where the neural network falls into the local optimal solution.

[0124] The principle of the Adam optimization algorithm is to calculate the historical gradient decay by using the weighted mean and weighted biased variance of the gradient. The formula is as follows:

[0125]

[0126] m t = β1m t-1 + (1 - β1)g t

[0127] v t = β2v t-1 + (1 - β2)g t 2

[0128] Among them, g t refers to the gradient of the θ parameter at time t, m t refers to the weighted mean of the gradient, v t refers to the weighted biased variance, m t and v t The initial values of are 0. β1 and β2 are set decay factor coefficients and are constant values. Since during the calculation process, m t and v t tend to approach 0, so the formulas for m t and v t are corrected as follows:

[0129]

[0130] Therefore, the parameter update method of the Adam optimization algorithm is:

[0131]

[0132] Among them, η is the learning rate, and ε is a bias parameter set to prevent the denominator from being zero.

[0133] As an optional implementation, the 3D image data includes intensity image data;

[0134] Step 102: Based on a pre-trained turnout scene recognition network, recognize the turnout scene in the 3D image data, including:

[0135] Input the intensity image data into the turnout scene recognition network;

[0136] In this step, the intensity image data contains a lot of semantic information. By inputting the intensity image data into the turnout scene recognition network, the recognition of the turnout scene can be achieved according to these semantic information.

[0137] Extract the feature information in the intensity image data;

[0138] In this step, these feature information include low-level features such as angles and lines, as well as other more complex features. Specifically, these feature information can be extracted by multiple convolutional layers in the turnout scene recognition network, and multiple convolutional layers finally output abstract high-level features.

[0139] According to the feature information, calculate the probability representing the turnout scene to recognize the turnout scene in the 3D image data.

[0140] In this step, first, the fully connected layer in the turnout scene recognition network integrates these features, maps the learned distributed feature representation to the sample label space, and outputs a probability for various classification label situations; then, determine whether the 3D image data is a turnout scene according to the output probability. Specifically, when the output probability is greater than the preset probability, it is determined that the 3D image data is a turnout scene.

[0141] Here, it should be noted that the railway scene only includes the turnout scene and the ordinary track scene.

[0142] As an optional implementation, the 3D image data includes depth image data;

[0143] Here, it should be noted that the depth image data contains track position information and can be used for rail surface segmentation and edge extraction, etc.

[0144] Step 103: Extract the edge pixel points of the turnout rails in the 3D image data, including:

[0145] Perform rail surface area segmentation processing on the depth image data;

[0146] In this step, according to the acquisition characteristics of railway track data, the relative distance between the camera for acquiring 3D image data and the rail is approximately constant. Therefore, the depth of the rail surface is within a fixed range. In the embodiment of the present application, a corresponding depth threshold is set to segment the depth data of the turnout for the rail surface area to construct a point cloud map;

[0147] Obtain a binary image of the turnout according to the depth image data after region segmentation processing;

[0148] This step can specifically be: project the depth image data after region segmentation processing onto a two-dimensional plane, and then use a binary operation to obtain a binary image of the turnout.

[0149] Here, it should be noted that most of the point cloud data on the rail surface is continuously and uniformly distributed. However, due to factors such as the installation position of the line array industrial camera (the camera for acquiring 3D image data) and disturbances during acquisition, the edge data of the track is discontinuous and has noise. This phenomenon can be clearly seen in the binary image of the turnout. Therefore, the embodiment of the present application proposes that after obtaining the binary image of the turnout, median filtering is used for smoothing processing to improve the continuity of the edge, thereby facilitating subsequent edge detection.

[0150] Use an edge detection operator to detect the edge pixel points of the binary image of the turnout.

[0151] Here, it should be noted that the edge detection algorithm can be the Canny detection operator.

[0152] In this step, by setting certain conditions, such as pixel jumps, all edge pixel points in the image can be detected and merged into a set to distinguish different objects in the image. The Canny operator is one of the most commonly used detection operators in edge detection. It uses a non-maximum suppression algorithm and a double-threshold algorithm to improve the accuracy of edge detection and reduce the missed detection rate of edges.

[0153] Specifically, the detection process of the Canny detection operator is mainly carried out in four steps:

[0154] (1) Gaussian filtering to remove noise and improve the accuracy of the image detection result.

[0155] (2) Calculate the magnitude and direction of the image gray gradient through gradient operators in different directions.

[0156] (3) Apply non-maximum suppression to the gradient to determine the accurate position of the edge, exclude non-edge information, and retain the points with the maximum local gradient.

[0157] (4) Use the double-threshold method to determine the edge and obtain edge pixel points with high precision and high continuity.

[0158] Through the processing of the Canny operator, the edge points of the rail are linearly distributed, which is in line with the actual situation.

[0159] As an alternative implementation, in step 104, according to the edge pixel points, detecting the turnout spacing includes:

[0160] Step 1: Based on the Hough transform feature detection algorithm and the edge pixel points, determining multiple edge straight line segments of the turnout rail;

[0161] The Hough transform converts a curve in the Cartesian coordinate space into a point in the parameter space, transforming the problem of detecting the shape of the target into the problem of statistically detecting the peak of the aggregated points in the parameter space. In the embodiment of the present application, the edge image detected by the Canny operator is processed using the Hough transform to extract the edge straight line segments of the turnout rail.

[0162] The basic principle of the Hough transform is to exchange the parameters and variables of the straight line y = kx + b in the Cartesian coordinate space, that is, taking k and b as variables and x and y as known quantities. Therefore, the straight line y = kx + b in the Cartesian coordinate space represents a point (k, b) when transformed into the parameter space. Similarly, a point in the Cartesian coordinate system represents a curve in the parameter space, and its principle is as Figure 3A and Figure 3B .

[0163] The edge points of the turnout in the Cartesian coordinate system are transformed into curves in the parameter space. The edge points belonging to the same straight line will necessarily intersect at the same point in the parameter space. Therefore, the task of finding the edge straight line of the turnout rail is transformed into finding the intersection point where the most curves intersect in the parameter space.

[0164] Step 2: Based on the least squares method, respectively fitting the multiple edge straight line segments;

[0165] Here, it should be noted that the main purpose of detecting the railway turnout is to measure whether the turnout spacing meets the standards of railway safety specifications. As can be seen from the above analysis, the edge of the turnout is a straight line, and the analytical formula of the straight line is the basis for calculating the turnout spacing. Among them, the edge straight line obtained through the Canny operator and the Hough transform is a straight line segment, and there is still a certain gap between its integrity and the edge straight line of the turnout required in the embodiment of the present application. Therefore, on this basis, the embodiment of the present application uses the least squares method to fit the straight line segment to obtain a complete edge straight line of the turnout.

[0166] Assume that the distribution of the original data points converges near a straight line, and let the expression of this straight line be as follows:

[0167] y j = a0 + a1x

[0168] where a0 and a1 are arbitrary real numbers.

[0169] The basic principle of least squares is to take the sum of the squares of the differences between the measured value y i and the calculated value y j as the criterion for evaluating the straight line, and its expression is as follows:

[0170]

[0171] When reaches the minimum value, the obtained straight line is the straight line that minimizes the sum of the squares of the distances from the known data points to the straight line, and it is the best matching function searched by the algorithm.

[0172] Specifically, in this step, the straight line segment data obtained through the Canny operator and the Hough transform is used as the input, and the least squares method is used to calculate the straight line analytical formula.

[0173] Step 3: Detect the turnout spacing according to the fitted edge straight line.

[0174] As an optional implementation manner, detecting the turnout spacing according to the fitted edge straight line includes:

[0175] Selecting the first edge straight line and the second edge straight line for calculating the turnout spacing according to the distance between the fitted edge straight line and the origin of the parameter space;

[0176] Here, it should be noted that the fitting results of multiple edge straight lines based on the least squares method include four edge straight lines, and only two edge straight lines in the middle of the turnout are required to detect the turnout spacing. In the embodiments of the present application, the four straight lines are sorted from small to large according to the distances from the four straight lines to the origin in the parameter space, and it can be simply obtained that the distances from the edge straight lines in the middle of the turnout to the origin rank second and third among the four straight lines. Therefore, for the straight line fitting results, only the middle two straight lines need to be retained. That is to say, in this step, first, calculate the distances between each fitted straight line in the fitting results and the origin of the parameter space, and second, select the first edge straight line and the second edge straight line for calculating the turnout spacing according to the distances between the fitted straight lines and the origin, where the first edge straight line and the second edge straight line are the straight lines other than the edge straight lines with the closest and farthest distances among the four edge straight lines.

[0177] Receiving the position parameter input by the user;

[0178] In this step, the position parameter is the position of the turnout spacing that the user currently needs to detect. In the embodiments of the present application, the turnout spacing at the current position is measured according to different positions selected by the user, which increases the flexibility of turnout spacing detection and facilitates different detection requirements.

[0179] Determine the turnout spacing at the position corresponding to the position parameter input by the user according to the position parameter, the first edge line, and the second edge line.

[0180] Specifically, this step can be as follows: Obtain the manually selected position information and return the coordinate data. Substitute the ordinate value into the linear analytical formula of the stock rail (close to horizontal or vertical) to obtain the abscissa on the fitted line. Draw a perpendicular line from this point to the other fitted line (switch rail), and the distance between these two points is the turnout spacing information at the currently selected position that we need.

[0181] Here, it should be noted that the unit of the turnout spacing calculated directly from the image data is the number of pixels, while the data required for actual turnout safety detection is length, with the unit of mm. To meet the detection requirements, it is necessary to convert the measured data in pixels to the actual length.

[0182] In the embodiment of this application, the number of pixels is converted to the actual length by using the measurement ratio. The measurement ratio refers to the ratio of the actual size of the calibrated object in machine vision to its pixel size in the image. The distances from the calibrated object and the object to be measured to the camera are required to be the same.

[0183] Here, it should be noted that the rail surface width of the same railway is a definite value. At the same time, the plane formed by the rail surface and the turnout edge line is the same plane, and the distances from the rail surface and the turnout edge to the camera are the same, meeting the requirements of the calibrated object in the measurement ratio. Therefore, in the embodiment of this application, the rail surface of the rail is used as the calibrated object for this experiment to calculate the measurement ratio, and the actual distance of the turnout spacing is calculated by using this measurement ratio. In addition, the influence degrees of external factors such as the camera installation position on the rail surface and the turnout edge in the railway image are the same. Therefore, using other acquisition devices or obtaining railway data from other angles, the calculated turnout spacing results are the same, and this method has strong robustness to the input data.

[0184] The railway turnout spacing detection method in the embodiment of this application is as follows: First, obtain the 3D image data of the railway track; second, based on the pre-trained turnout scene recognition network, recognize the turnout scene in the 3D image data, thus improving the recognition efficiency and accuracy of the turnout scene; third, extract the edge pixel points of the turnout rails in the turnout scene based on the edge detection method of the Canny operator; then, based on the edge pixel points, extract the edge line segments based on the Hough transform and use the least squares method to fit the edge lines to obtain the fitted lines. Finally, detect the turnout spacing according to the fitted lines and the position parameters selected by the user, thus increasing the flexibility of the spacing detection and facilitating the satisfaction of different needs of users.

[0185] Next, the verification process of the results of the turnout spacing detection using the railway turnout spacing detection method in the embodiment of this application will be described.

[0186] First, the verification system is described as follows:

[0187] The turnout scene recognition and spacing detection system designed in the embodiments of this application is completed on a personal computer with the operating system Windows 10. Matlab and Python are used as programming languages, Matlab and PyCharm are used as development tools, and at the same time, the Matlab image processing toolbox, the PyTorch machine learning library, and the Visdom visualization tool are used as auxiliary tools to assist in the development of the detection system. The development environment for the entire experiment is shown in Table 1.

[0188] Table 1: Development environment for the recognition and edge detection system of turnout scenes

[0189]

[0190] The railway data used in the embodiments of this application comes from the China Academy of Railway Sciences (hereinafter referred to as the Academy of Railway Sciences). The Academy of Railway Sciences uses a professional inspection vehicle as the carrier platform, and uses the comprehensive inspection system for high-speed rail infrastructure to obtain railway information. At the same time, according to the spatial distribution relationship between the inspection vehicle and the railway track infrastructure, the front imaging components of the image acquisition system are scattered and arranged to achieve coverage acquisition of the line infrastructure. The main equipment of the image acquisition system includes various hardware devices such as line array industrial cameras, auxiliary light sources, power modules, and PLC controllers.

[0191] Secondly, the verification process is described as follows:

[0192] The turnout detection process based on deep learning proposed in the embodiments of this application mainly includes three parts: feature enhancement, classification recognition, and turnout spacing calculation, and each part contains several processing processes. The Ranger33D camera is used to collect railway samples, which contain depth information and intensity information. The intensity information is used for filtering, denoising, and histogram equalization to enhance the features between various structures in the railway image.

[0193] In order to quickly identify turnout data from a large number of railway samples, a turnout classification recognition network is proposed based on the AlexNet network in the embodiments of this application to predict class labels for railway samples. Among the railway samples collected in the embodiments of this application, there are 493 basic rails and 307 turnouts, for a total of 800. And data augmentation operations such as scaling, rotation, central cropping, and regularization are performed on them to increase the number of samples, and then the data set is divided into training samples, validation samples, and test samples according to the ratio of 60%, 20%, and 20% and input into the network. The training process is as Figure 4 shown. The Batch_size is set to 32, the epochs is 60, and the initial learning rate is 2×10 -4。During the testing phase, a railway image is input into the turnout scene recognition network. Finally, the turnout scene images are separated according to the labels, and the rail data is obtained from the images for straight line extraction of the rail edges and spacing calculation.

[0194] Figure 5 It is the relationship curve between the number of training iterations and the loss value. Figure 6 It is the relationship curve between the number of training iterations and the precision rate. According to the training results, as the number of training iterations (epoch) increases, the loss value continuously decreases, and the accuracy rate continuously increases. When the epoch is about 54, the loss reaches the minimum, which is 0.0073, and both the loss and the accuracy rate remain stable, indicating that the model has been trained successfully.

[0195] As shown in Table 2, the precision rate, recall rate, F1 score, and accuracy rate of the turnout recognition network are all above 0.91, and the accuracy rate reaches 93%. From Figure 7 the ROC curve and AUC value, it can be seen that the ROC curve is mainly concentrated in the upper left corner of the region, and the AUC is equal to 0.98, which is greater than 0.5 and close to 1, indicating that the classification performance of this model is very good and the prediction results are accurate.

[0196] Table 2

[0197]

[0198] Regarding the turnout detection accuracy, in the embodiments of the present application, images are collected in three characteristic regions of the turnout switch part, and the measurement results of manual detection and the detection of the embodiments of the present application are compared to verify the accuracy. Through experimental verification, the method of the embodiments of the present application can calculate the turnout spacing at different positions in the image, and the repeated measurement accuracy also meets the experimental requirements. In this article, three marking points are selected, and the measurements are carried out manually on site and using the turnout spacing detection system respectively. The comparison results are shown in Table 3.

[0199] Table 3

[0200]

[0201] It can be seen from the result comparison that the measurement error between the manual measurement and the method using the embodiments of the present application does not exceed 0.2 mm, which meets the range specified in the turnout detection regulations of the railway safety standard, proving that this method can meet the requirements of turnout detection.

[0202] As Figure 8 shown, the embodiments of the present application also provide a railway turnout spacing detection device, including:

[0203] An acquisition module 801, configured to acquire 3D image data of a railway track;

[0204] An identification module 802, configured to identify a turnout scene in 3D image data based on a pre-trained turnout scene identification network;

[0205] An extraction module 803, configured to extract edge pixel points of turnout rails in the turnout scene;

[0206] A detection module 804, configured to detect the turnout spacing according to the edge pixel points.

[0207] Optionally, the device further includes:

[0208] A processing module, configured to perform histogram equalization processing on the 3D image data.

[0209] Optionally, the device further includes:

[0210] A training module, configured to train the turnout scene identification network; wherein, the training module includes:

[0211] An establishment sub-module, configured to establish a turnout scene identification network based on a convolutional neural network;

[0212] An output sub-module, configured to input a preset training sample into the turnout scene identification network and output a prediction probability;

[0213] A calculation sub-module, configured to input the prediction probability into a preset loss function to calculate a loss value of the turnout scene identification network;

[0214] An optimization sub-module, configured to optimize parameters of the turnout scene identification network according to the loss value by using a preset optimization algorithm.

[0215] Optionally, the 3D image data includes intensity image data;

[0216] The identification module 802 includes:

[0217] An input sub-module, configured to input the intensity image data into the turnout scene identification network;

[0218] An extraction sub-module, configured to extract feature information in the intensity image data;

[0219] An identification sub-module, configured to calculate a probability representing the turnout scene according to the feature information to identify the turnout scene in the 3D image data.

[0220] Optionally, the 3D image data includes depth image data;

[0221] The extraction module 803 includes:

[0222] A processing sub-module, configured to perform rail surface area segmentation processing on the depth image data;

[0223] An acquisition sub-module, configured to obtain a turnout binary image according to the depth image data after region segmentation processing;

[0224] A first detection sub-module, configured to detect edge pixel points of the turnout binary image by using an edge detection operator.

[0225] Optionally, the detection module 804 includes:

[0226] A determination sub-module, configured to determine multiple edge straight line segments of the turnout rail based on the Hough transform feature detection algorithm and the edge pixel points;

[0227] A fitting sub-module, configured to respectively fit the multiple edge straight line segments based on the least squares method;

[0228] A second detection sub-module, configured to detect the turnout spacing according to the fitted edge straight line.

[0229] Optionally, the second detection sub-module includes:

[0230] A selection unit, configured to select a first edge straight line and a second edge straight line for calculating the turnout spacing according to the distance between the fitted edge straight line and the origin of the parameter space;

[0231] A receiving unit, configured to receive a position parameter input by a user;

[0232] A determination unit, configured to determine the turnout spacing at a position corresponding to the position parameter input by the user according to the position parameter, the first edge straight line, and the second edge straight line.

[0233] It should be noted that for the railway turnout spacing detection method provided in the embodiments of the present application, the execution subject may be a railway turnout spacing detection device, or a control module in the railway turnout spacing detection device for executing the loading method. In the embodiments of the present application, taking the railway turnout spacing detection device as an example to execute the railway turnout spacing detection method, the railway turnout spacing detection method provided in the embodiments of the present application is described.

[0234] The railway turnout spacing detection device provided in the embodiments of the present application can implement Figures 1 to 7 each process of the method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0235] The embodiments of the present application further provide a railway turnout spacing detection system, including: a processor, a memory, and a program stored on the memory and executable on the processor. When the program is executed by the processor, it implements each process of the railway turnout spacing detection method embodiment as described above, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0236] The embodiment of the present application also provides a readable storage medium, on which a program is stored. When the program is executed by a processor, it implements each process of the embodiment of the railway turnout spacing detection method and can achieve the same technical effect. To avoid repetition, it will not be elaborated here. Among them, the readable storage medium includes, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc.

[0237] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0238] The above is the preferred embodiment of the present application. It should be pointed out that for those of ordinary skill in the art, without departing from the principle described in the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A method for detecting the spacing of railway turnouts, characterized in that, Including: Obtaining 3D image data of a railway track; Identifying a turnout scene in the 3D image data based on a pre-trained turnout scene recognition network; Extracting edge pixel points of turnout rails in the turnout scene; Detecting the turnout spacing according to the edge pixel points; Wherein, the 3D image data includes intensity image data; Identifying a turnout scene in the 3D image data based on a pre-trained turnout scene recognition network includes: Inputting the intensity image data into the turnout scene recognition network; Extracting feature information in the intensity image data; Calculating a probability representing the turnout scene according to the feature information to identify the turnout scene in the 3D image data; Wherein, the 3D image data includes depth image data; Extracting edge pixel points of turnout rails in the turnout scene includes: Performing rail surface area segmentation processing on the depth image data; Obtaining a turnout binary image according to the depth image data after the area segmentation processing; Detecting edge pixel points of the turnout binary image by using an edge detection operator; Wherein, detecting the turnout spacing according to the edge pixel points includes: Determining multiple edge straight line segments of the turnout rails based on the Hough transform feature detection algorithm and the edge pixel points; Fitting each of the multiple edge straight line segments based on the least squares method; Detecting the turnout spacing according to the fitted edge straight lines.

2. The method according to claim 1, characterized in that, After obtaining the 3D image data of the railway track, the method further includes: Performing histogram equalization processing on the 3D image data.

3. The method according to claim 1, characterized in that, The training process of the turnout scene recognition network includes: Establishing the turnout scene recognition network based on a convolutional neural network; Inputting a preset training sample into the turnout scene recognition network to output a predicted probability; Inputting the predicted probability into a preset loss function to calculate a loss value of the turnout scene recognition network; Optimizing parameters of the turnout scene recognition network according to the loss value by using a preset optimization algorithm.

4. The method according to claim 1, characterized in that, Detecting the turnout spacing according to the fitted edge straight lines includes: Selecting a first edge straight line and a second edge straight line for calculating the turnout spacing according to the distance between the fitted edge straight line and the origin of the parameter space; Receiving a position parameter input by a user; Determining the turnout spacing at a position corresponding to the position parameter input by the user according to the position parameter, the first edge straight line, and the second edge straight line.

5. A device for detecting the spacing of railway turnouts, characterized in that, Including: An obtaining module for obtaining 3D image data of a railway track; An identifying module for identifying a turnout scene in the 3D image data based on a pre-trained turnout scene recognition network; An extracting module for extracting edge pixel points of turnout rails in the turnout scene; A detecting module for detecting the turnout spacing according to the edge pixel points; Wherein, the 3D image data includes intensity image data; The identifying module includes: An input sub-module for inputting the intensity image data into the turnout scene recognition network; An extracting sub-module for extracting feature information in the intensity image data; An identification sub-module, configured to calculate a probability representing the turnout scene based on the feature information, so as to identify the turnout scene in the 3D image data; Wherein, the 3D image data includes depth image data; The extraction module includes: A processing sub-module, configured to perform rail surface area segmentation processing on the depth image data; An acquisition sub-module, configured to obtain a turnout binary image according to the depth image data after the area segmentation processing; A first detection sub-module, configured to detect edge pixel points of the turnout binary image by using an edge detection operator; Wherein, the detection module includes: A determination sub-module, configured to determine multiple edge straight line segments of the turnout rail based on the Hough transform feature detection algorithm and the edge pixel points; A fitting sub-module, configured to respectively fit the multiple edge straight line segments based on the least square method; A second detection sub-module, configured to detect the turnout spacing according to the fitted edge straight lines; 6. The device according to claim 5, characterized in that, The device further includes: A processing module, configured to perform histogram equalization processing on the 3D image data; 7. The device according to claim 5, characterized in that, The device further includes: A training module, configured to train the turnout scene recognition network; wherein, the training module includes: A building sub-module, configured to build the turnout scene recognition network based on a convolutional neural network; An output sub-module, configured to input a preset training sample into the turnout scene recognition network and output a prediction probability; A calculation sub-module, configured to input the prediction probability into a preset loss function and calculate a loss value of the turnout scene recognition network; An optimization sub-module, configured to optimize parameters of the turnout scene recognition network according to the loss value by using a preset optimization algorithm; 8. The device according to claim 5, characterized in that, The second detection sub-module includes: A selection unit, configured to select a first edge straight line and a second edge straight line for calculating the turnout spacing according to the distance between the fitted edge straight line and the origin of the parameter space; A receiving unit, configured to receive a position parameter input by a user; A determination unit, configured to determine the turnout spacing at a position corresponding to the position parameter input by the user according to the position parameter, the first edge straight line and the second edge straight line; 9. A railway switch spacing detection system, characterized in that, Includes: A processor, a memory, and a program stored on the memory and executable on the processor, where the program, when executed by the processor, implements the steps of the railway turnout spacing detection method according to any one of claims 1 to 4; 10. A readable storage medium, characterized in that, A program is stored on the readable storage medium, and when the program is executed by a processor, it implements the steps of the railway turnout spacing detection method according to any one of claims 1 to 4.

Images