A method for detecting actual running area of a train in transit based on geometric feature mining

Abstract

The application discloses a kind of in-transit train actual running area detection method based on geometric feature mining, belongs to track area detection technical field, comprising: S1, the railway image when in-transit train runs is collected, using railway area segmentation model is handled, obtains rail segmentation result;S2, based on rail segmentation result, extract rail skeleton, form rail skeleton image;S3, in rail skeleton image, rail and intercept turnout are saved in the way of sliding search window, and the direction of turnout is identified;S4, based on the rail saved, the direction of the turnout and its direction, reconstructs in-transit train running railway area image, realizes in-transit train actual running area detection.The application reconstructs the area running in front of train, helps to judge whether the detected obstacle is on the track where train advances, to determine the dangerous degree of these obstacles to train, can improve the safety and efficiency of railway traffic, effectively reduce the incidence of traffic accidents.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of track area detection, and particularly relates to a method for detecting actual running area of an on-the-way train based on geometric feature mining. BACKGROUND

[0002] Railway is an important support for national development and an engine for modernization construction. With the progress of various industries in China, railway transportation demand increases, and higher standards for train safety are proposed. China's rail transportation is currently facing the pressure of large-scale flow of personnel and materials, including subways and light rails in the development of various cities. Rail transportation bears a large amount of transportation tasks and plays an important role in the field of transportation. This fully reflects the continuous increase in the workload of railway transportation, but the speedup of trains also brings some problems to be solved. At the same time, the number of trains running on the track is also increasing, especially during holidays, and the amount of railway transportation increases sharply. Therefore, in the face of the limited number of tracks and the increasing amount of railway transportation, how to ensure the safe running of a large number of running trains is a real problem.

[0003] Obstacle intrusion into the track line also brings a major hidden danger to railway safety. The International Railway Union claims that 90% of railway accidents are caused by third-party obstacles intruding into the railway lane. In order to ensure the safe running of trains, improving the active obstacle detection capability of trains has become an effective means to solve the problem. One of the basic requirements for active obstacle detection of trains is the detection of railway track area. Railway safety accidents not only bring social and economic losses and casualties, but also have a negative impact on the development of rail transportation. Therefore, it is necessary to pay high attention and take effective measures to solve the problem.

[0004] Accurate track area detection can provide an accurate train running area, so that an alarm can be issued when an obstacle is detected in the running area. In addition, accurate track area detection can locate railway switches to ensure that trains run in the predetermined direction. The actual railway is often more than one railway, and various types of switches appear on the railway, especially in the station, where switches are very important because they determine the running order of all trains entering and leaving the station. As a kind of line connection device between multiple tracks, the switch can control the train to change tracks between multiple tracks. If there is no arrangement of the control center and guidance of the switch, the trains will be in disorder or even collide. If the locomotive driver can timely learn the real-time information of the front track, he can judge in advance whether the route is safe and has time to take emergency measures to avoid or reduce accidents, so as to improve the safety of train operation.

[0005] Through the search of existing patents and related technologies, it is found that the methods for detecting railway area include:

[0006] (1) Chen Yurun and Zhang Xuanxiong proposed a method of track area detection, using deep learning and traditional algorithm, and proposed a high-performance multi-branch track area detection method. The method obtains track area information through semantic segmentation, instance segmentation, classification network and polygon fitting algorithm, and obtains excellent rail area contour.

[0007] (2) Wang Ziguang and Shu Guohua proposed a track area recognition method based on traditional image processing algorithm and deep learning. The traditional image processing algorithm is based on the edge features and straight line features of the track image, and comprehensively uses Sobel operator and Hough transform to identify the rails in the image, and then determines the track area. Deep learning technology develops rapidly and has great potential in image processing. Using the target detection algorithm of deep learning, based on MaskR-CNN model, the track area recognition system is trained, and the effectiveness of the system recognition is proved by verifying the effect on the actual picture.

[0008] The existing railway area segmentation method only solves the track detection method in simple cases, such as single track or parallel track. In fact, in the actual running environment of the train, various complex scenes in the track area alternate with each other, and railway turnout is often encountered. It is difficult to distinguish between different rails, and only the whole railway area is detected, which has the problem of difficult matching of left and right rails, causing same side repeated matching and opposite side repeated matching problem, and it is difficult to detect the actual running area of the train under the track turnout, and the danger level of the train running area cannot be divided. SUMMARY

[0009] In view of the above problems in the prior art, the in-train actual running area detection method based on geometric feature mining provided by the present application solves the problem of difficult matching of left and right rails in the existing detection method, causing same side repeated matching and opposite side repeated matching problem, and further affecting the precision and effect of detecting the actual running area of the train under the turnout.

[0010] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows: an in-train actual running area detection method based on geometric feature mining, comprising the following steps:

[0011] S1, collecting railway images during train running, using a railway area segmentation model based on information fusion to process the images, and obtaining rail segmentation results;

[0012] S2, based on the rail segmentation results, extracting the rail skeleton to form a rail skeleton image;

[0013] S3, in the rail skeleton image, save the rails and cut off the turnout in the form of a sliding search window, and identify the direction of the turnout;

[0014] S4. Based on the saved rails, the captured turnouts and their directions, reconstruct the railway area image of the train in transit to realize the detection of the actual operating area of ​​the train in transit.

[0015] Furthermore, in step S1, the railway region segmentation model based on information fusion includes a backbone network, a spatial void pooling pyramid, a pixel decoding module, a Transformer decoder module, and a classification predictor.

[0016] The backbone network is used to extract features from railway images to form a low-resolution feature map of the railway track.

[0017] The spatial void pooling pyramid is used to optimize the global and local features of the target region in the low-resolution feature map;

[0018] The pixel decoding module is used to gradually restore the size of the low-resolution feature map based on the optimized global and local features, and then gradually concatenate the restored feature map with the low-resolution feature map and perform feature upsampling to generate pixel embeddings.

[0019] The Transformer decoder module is used to generate the category prediction result and mask embedding vector of the target region based on the optimized global and local features;

[0020] The classification predictor is used to generate a mask prediction result for the target region based on the mask embedding vector and pixel embedding, and fuse the category prediction result of the target region to obtain the mask segmentation result of the railway track pixels in the target region, which is used as the rail segmentation result.

[0021] In the mask segmentation result of the railway track pixels in the target area, a pixel value of 0 represents the rail, a pixel value of 1 represents the railway area, and a pixel value of 2 represents the background.

[0022] Furthermore, the backbone network is an improved Xception_ECA network, which includes a Sep_ECA structure, which is a depthwise separable convolutional feature extraction structure with added SE attention mechanism.

[0023] In the pixel decoding module, the generated pixel embedding Among them, C ε H is the pixel embedding dimension, H is the height of the feature map, and W is the width of the feature map.

[0024] The Transformer decoder module uses an ensemble prediction mechanism to generate N pairs of class prediction results and mask embedding vectors.

[0025] In the Transformer decoder module, a corresponding linear classifier is set for each segment to predict the category of each track segment, and its expression is:

[0026]

[0027] In the formula, p i Let Δ be the class probability. K+1 The number of predicted categories plus the background categories, where K is the number of predicted categories, i is the ordinal number of predicted result pairs, and N is the number of generated predicted result pairs;

[0028] In the Transformer decoder module, a multilayer perceptron with two hidden layers converts the segmented embedding Q into N mask embedding vectors.

[0029] Furthermore, the method by which the classification predictor obtains the rail segmentation results is as follows:

[0030] The mask prediction result is obtained by multiplying the mask embedding vector by the pixel embedding dot product, and then activated by the Sigmoid function to obtain m. i [h,w]=sigmoid(ε mask [:,i]·ε pixel [:,h,w]);

[0031] pass Each pixel is assigned to one of N predicted probability-mask pairs, thereby segmenting the railway image into several segments. Segments with the same category label are merged to obtain the rail segmentation result.

[0032] Where, m i [h,w] represents the mask prediction result of pixel [h,w] in the target region, m i For the mask prediction result, m i ∈[0,1] H×W sigmoid(·) is the sigmoid activation function, ε mask Let ε be the mask embedding vector, [:,i] be the i-th mask embedding corresponding to the i-th pair of prediction results, and ε be the mask embedding vector. pixel For pixel embedding, [:,h,w] represents the coordinate information of the pixel embedding, H is the height of the feature map, and W is the width of the feature map;

[0033] argmax(·) assigns the pixel at position [h,w] to the probability mask pair corresponding to the i-th prediction result. This indicates that the "no object" category is not included, p i (c i ) represents the class probability, c i For category labels.

[0034] Furthermore, in the Transformer decoder module, the loss function DiceLoss for obtaining the category prediction result of the target region is:

[0035]

[0036] In the formula, y i The label value of pixel i is represented by y. j This represents the predicted value of pixel i, where N is the total number of pixels;

[0037] In the classification predictor, the loss function FocalLoss(p) is used to obtain the mask prediction result of the target region. t )for:

[0038] FocalLoss(p t )=-(λ-p t ) y log(p t )

[0039] In the formula, p t α is the probability predicted by the model. t y is the sample weight, and y is the adjustment factor;

[0040] The loss function L of the railway area segmentation model Total for:

[0041] L Total =λ focal FocalLoss+λ dice DiceLoss

[0042] In the formula, λ focal and λ dice These are the weights for the mask prediction loss and the class prediction loss, respectively.

[0043] Further, step S2 specifically includes:

[0044] S21. Based on the rail segmentation results, search for the track region in the railway image;

[0045] Among them, the area between the two rails is the track area;

[0046] S22. Based on the searched track region, extract the rail skeleton.

[0047] Further, step S3 specifically includes:

[0048] S31. In the rail skeleton image, determine whether the current search window contains two rails or three rails from left to right.

[0049] If there are two rails, proceed to step S32;

[0050] If there are three rails, proceed to step S33;

[0051] S32. Save the coordinate values ​​of the left and right rails, and adjust the size of the search window by extending 10 pixels along the rail direction. Return to step S31.

[0052] S33. Determine whether the current three rails are converging turnouts;

[0053] If so, proceed to step S34;

[0054] If not, proceed to step S35;

[0055] S34. Search from the top boundary to the bottom boundary and save the coordinates of the rails on the left and right sides;

[0056] S35. Extract the turnout image from the rail skeleton image within the current search window and identify the turnout direction;

[0057] When identifying the direction of the turnout, the current search window shows four rails from left to right;

[0058] If the turnout direction is left, proceed to step S36;

[0059] If the turnout direction is right, proceed to step S37;

[0060] S36. Save the first and third rails from left to right as the rails currently in operation of the train. At the same time, retain the fourth rail from left to right as the right boundary reference of the current search. Extend 10 pixels to the left and 10 pixels to the right along the left rail to adjust the size of the search window. Proceed to step S38.

[0061] S37. Save the second and fourth rails from left to right as the rails currently in operation of the train. At the same time, retain the first rail from left to right as the current search boundary reference. Extend 10 pixels to the right along the left rail and 10 pixels to the left boundary to adjust the size of the search window. Proceed to step S38.

[0062] S38. Slide the search window in the rail skeleton image and save the rail coordinates until there are only three rails from left to right in the current search window, then proceed to step S39.

[0063] S39. Extend outward by 10 pixels along the left and right rails to adjust the current search window size, and determine whether there are two or three rails in front.

[0064] If there are two rails, return to step S31;

[0065] If there are three rails from left to right, and it is not the first turnout, then return to step S33;

[0066] S310. Repeat steps S31 to S39 until all rails in the rail skeleton image are saved, turnouts are captured, and directions are identified.

[0067] Furthermore, in step S35, a railway turnout classification model based on grouped convolution is used to identify the turnout direction;

[0068] The railway turnout classification model includes, in sequence, an image preprocessing module, a feature extraction module, a ResNeXt residual structure block, a multi-scale attention module, an average pooling layer, a feature one-dimensionalization layer, a fully connected layer, and a classifier.

[0069] The ResNeXt residual structure block is equipped with channel attention mechanism and spatial attention mechanism; the multi-scale attention module includes multiple parallel dilated convolutional layers with different dilation rates.

[0070] Furthermore, the loss function of the railway turnout classification model is Loss r for:

[0071] Loss r =max(||f(A)-f(p)||) 2 -||f(A)-f(N)|| 2 +C,0)

[0072] In the formula, max(·) is the distance between category features, f(A) is the length of the target sample, f(p) is the length of the extracted image, f(N) is the length of the category different from A, A is the feature of the target sample, N is the category feature different from A, P represents the extracted image feature, and C is the upper boundary parameter.

[0073] The beneficial effects of this invention are as follows:

[0074] 1) The railway region segmentation model proposed in this invention can perform well in challenging scenarios such as shadows, reflections, tunnel environments, and track switching scenarios; it has achieved good results in turnout images with poor lighting and easy confusion with the background environment.

[0075] 2) The railway area image reconstruction method proposed in this invention can effectively solve the rail matching problem. At the same time, it can correctly reconstruct the running area in front of the train in challenging scenarios such as track switching, and plays a good role. The railway area reconstruction results have the potential to be applied to train active collision avoidance technology.

[0076] 3) This invention helps determine whether detected obstacles are on the train's track by reconstructing the area in front of the train, thereby determining the degree of danger these obstacles pose to the train. This method can improve the safety and efficiency of railway traffic and effectively reduce the incidence of traffic accidents. Attached Figure Description

[0077] Figure 1 The flowchart of the method for detecting the actual operating area of ​​trains on the road based on geometric feature mining provided by the present invention is shown. Detailed Implementation

[0078] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0079] This invention provides a method for detecting the actual operating area of ​​trains in transit based on geometric feature mining, such as... Figure 1 As shown, it includes the following steps:

[0080] S1. Collect railway images of trains in transit, process them using a railway region segmentation model based on information fusion, and obtain rail segmentation results.

[0081] S2. Based on the rail segmentation results, extract the rail skeleton to form a rail skeleton image;

[0082] S3. In the rail skeleton image, save the rails and extract the turnouts using a sliding search window, and identify the turnout direction;

[0083] S4. Based on the saved rails, the captured turnouts and their directions, reconstruct the railway area image of the train in transit to realize the detection of the actual operating area of ​​the train in transit.

[0084] In step S1 of this embodiment of the invention, the railway region segmentation model based on information fusion includes a backbone network, a spatial void pooling pyramid, a pixel decoding module, a Transformer decoder module, and a classification predictor.

[0085] The backbone network is used to extract features from railway images to form a low-resolution feature map of the railway track.

[0086] The spatial void pooling pyramid is used to optimize the global and local features of the target region in the low-resolution feature map;

[0087] The pixel decoding module is used to gradually restore the size of the low-resolution feature map based on the optimized global and local features, and then gradually concatenate the restored feature map with the low-resolution feature map and perform feature upsampling to generate pixel embeddings.

[0088] The Transformer decoder module is used to generate the category prediction result and mask embedding vector of the target region based on the optimized global and local features;

[0089] The classification predictor is used to generate a mask prediction result for the target region based on the mask embedding vector and pixel embedding, and fuse the category prediction result of the target region to obtain the mask segmentation result of the railway track pixels in the target region, which is used as the rail segmentation result.

[0090] In this embodiment of the invention, when training the railway region segmentation model, railway images with a resolution of 1920×1080 are acquired. The acquired railway images are manually screened to balance the amount of data from different scenes, ensuring the diversity and representativeness of the sample set. Only one keyframe is extracted from each second of railway image video, and image quality detection algorithms are used to delete samples with high similarity. The image data is labeled at the pixel level to train the model for segmentation tasks. Methods such as cross-validation are used to evaluate the accuracy of the labels. Inaccurate or inconsistent labels are corrected or deleted. The labeled labels are uniformly converted into PNG data format to facilitate model reading and training. Since the upper part of the image is the sky background, only the lower 60% of the image is input during actual training to reduce the impact of the background on training, i.e., the actual input size is 1920×648.

[0091] In step S1 of this embodiment, the backbone network of the railway region segmentation model is an improved Xception_ECA network. The improved Xception_ECA network includes a Sep_ECA structure, which is a depthwise separable convolutional feature extraction structure with added SE attention mechanism. Specifically, in the improved Xception_ECA network, max pooling operations are replaced by a step-by-step Sep_ECA structure. This allows the use of the Sep_ECA structure to extract feature factors, and additional batch normalization and ReLU activation are added after each 3×3 Sep_ECA operation. Simultaneously, the Sep_ECA structure employs a more efficient attention calculation method, improving model accuracy without significantly increasing computational burden. Therefore, the Sep_ECA structure in this embodiment can effectively reduce model parameters while ensuring efficient extraction of railway region features. The output of a single-layer Sep_ECA structure can be expressed as:

[0092] f(x)=σ(y Sep_ECA (x))

[0093] In the formula, y Sep_ECA Let f(x) represent the Sep_ECA structure, where f(x) is the output of the input feature map x after passing through the Sep_ECA structure, and σ is the activation function, which is the LeakyReLU function used for mapping.

[0094] Based on the above structural setup, the improved Xception_ECA network in this embodiment can better extract detailed features from the image, such as track details, curves, and straight lines, and can effectively distinguish between the background and foreground, thereby improving the segmentation accuracy of railway images.

[0095] In step S1 of this embodiment, the spatially hollow pooling pyramid in the railway region segmentation model mainly optimizes the global and local features of target regions (such as railway lines and rails) in railway images. Specifically, target regions such as railway lines and rails are usually long and thin in railway images, so a large receptive field and high resolution are needed to better identify and locate them. Meanwhile, since the background occupies a large proportion of railway images, using a spatially hollow pooling pyramid structure can avoid information redundancy and improve the accuracy of the model.

[0096] In step S1 of this embodiment, the low resolution of the brickwork in the pixel decoding module of the railway area segmentation model leads to information loss during downsampling. Therefore, the pixel decoding module is used to restore the size of the feature map to achieve high-precision detection of the track area and rail line. In this embodiment, the pixel embedding generated by the pixel decoding module... Among them, C ε H is the pixel embedding dimension, H is the height of the feature map, and W is the width of the feature map.

[0097] In step S1 of this embodiment of the invention, the Transformer decoder module in the railway region segmentation model uses an ensemble prediction mechanism to calculate its output from image features f and N learnable location embeddings, generating N pairs of category prediction results and mask embedding vectors, i.e., N C... Q Embedding of each dimension Global information about each segment of the prediction is encoded, and the decoder generates all predictions in parallel.

[0098] In this embodiment, the Transformer decoder module has a corresponding linear classifier for each segment to predict the category of each track segment. Its expression is:

[0099]

[0100] In the formula, pi Let Δ be the class probability. K+1 The number of predicted categories plus the background categories, where K is the number of predicted categories, i is the ordinal number of predicted result pairs, and N is the number of generated predicted result pairs;

[0101] If the embedding does not correspond to any region, an additional "no object" category will be predicted.

[0102] In this embodiment, in the Transformer decoder module, for mask prediction, a multilayer perceptron with two hidden layers converts the segmented embedding Q into N mask embedding vectors.

[0103] In step S1 of this embodiment of the invention, the method by which the classification predictor obtains the rail segmentation result is as follows:

[0104] The mask prediction result is obtained by multiplying the mask embedding vector by the pixel embedding dot product, and then activated by the Sigmoid function to obtain m. i [h,w]=sigmoid(ε mask [:,i]·ε pixel [:,h,w]);

[0105] pass Each pixel is assigned to one of N predicted probability-mask pairs, thereby segmenting the railway image into several segments. Segments with the same category label are merged to obtain the rail segmentation result.

[0106] Where, m i [h,w] represents the mask prediction result of pixel [h,w] in the target region, m i For the mask prediction result, m i ∈[0,1] H×W sigmoid(·) is the sigmoid activation function, ε mask Let ε be the mask embedding vector, [:,i] be the i-th mask embedding corresponding to the i-th pair of prediction results, and ε be the mask embedding vector. pixel For pixel embedding, [:,h,w] represents the coordinate information of the pixel embedding, H is the height of the feature map, and W is the width of the feature map;

[0107] argmax(·) assigns the pixel at position [h,w] to the probability mask pair corresponding to the i-th prediction result. This indicates that the "no object" category is not included, p i (c i ) represents the class probability, c i For category labels.

[0108] In this embodiment, based on the above model structure, in the mask segmentation result of the railway track pixels in the target area, a pixel value of 0 represents a rail, a pixel value of 1 represents a railway area, and a pixel value of 2 represents the background. Therefore, when the pixel value between two rails is 1, it indicates that the area between the two rails is a railway area, and at this time, the correct pair of tracks currently being traveled by the train can be matched.

[0109] In this embodiment, analysis of railway track data reveals that the background negative sample region constitutes the majority of the entire sample, and the ratio of railway area to rail line is approximately 83:17. To address the imbalanced nature of railway image data samples, the loss function DiceLoss for obtaining the category prediction result of the target region is set as follows in the Transformer decoder module:

[0110]

[0111] In the formula, y i The label value of pixel i is represented by y. j This represents the predicted value of pixel i, where N is the total number of pixels;

[0112] In this embodiment, analysis of railway track data reveals a strong correlation between railway areas and rails, necessitating the use of a region-dependent loss function. Therefore, this embodiment employs Dice loss as the category loss function because its design ensures that the loss at a given pixel depends not only on its own predicted value and label but also on the predicted values ​​and labels of other points in its related region. This allows Dice loss to effectively address the data imbalance problem, especially in image segmentation tasks. Therefore, in the classification predictor, the loss function FocalLoss(p) is used to obtain the mask prediction result of the target region. t )for:

[0113] FocalLoss(p t )=-(λ-p t ) y log(p t )

[0114] In the formula, p t α is the probability predicted by the model. t y is the sample weight, and y is the adjustment factor;

[0115] In this embodiment, based on the above-mentioned loss function settings, the overall loss function L of the railway area segmentation model is... Total for:

[0116] L Total =λ focal FocalLoss+λdice DiceLoss

[0117] In the formula, λ focal and λ dice These are the weights for the mask prediction loss and the class prediction loss, respectively.

[0118] In the embodiment of the present invention, step S2 specifically includes:

[0119] S21. Based on the rail segmentation results, search for the track region in the railway image;

[0120] Among them, the area between the two rails is the track area;

[0121] S22. Based on the searched track region, extract the rail skeleton.

[0122] In step S21 of this embodiment, based on the previously obtained rail segmentation results, the railway area where the train is currently running is analyzed from the railway image and is between 30% and 70% of the bottom of the image. Since the bottom of the rail segmentation results is often defective, the search starts from 8 pixels above the bottom. Starting from the image coordinates (640, 576), the search area for the train's running track is performed from left to right. Since the railway area and the rail are strongly correlated, the area of ​​the train's running track can be determined by the above method.

[0123] In step S22 of this embodiment, after determining the initial position of the bottom of the rail in the track area, the rail skeleton is extracted to facilitate subsequent turnout search. The skeleton extraction method based on the Guo-Hall algorithm used in this embodiment introduces two 4×4 templates to accommodate more complex line shapes. This algorithm extracts the skeleton by processing each pixel in a specific order, continuously applying templates to delete pixels until no further deletions are possible. This gradually removes details (such as branches and burrs) from the binary image, ultimately resulting in a single-pixel-wide line, called the rail skeleton.

[0124] In step S3 of this embodiment of the invention, after determining the initial position of the current track where the train is running and extracting the rail skeleton, a sliding search window is used to maintain the key points of the rail during actual train operation; that is, step S3 in this embodiment specifically involves:

[0125] S31. In the rail skeleton image, determine whether the current search window contains two rails or three rails from left to right.

[0126] If there are two rails, proceed to step S32;

[0127] If there are three rails, proceed to step S33;

[0128] S32. Save the coordinate values ​​of the left and right rails, and adjust the size of the search window by extending 10 pixels along the rail direction. Return to step S31.

[0129] S33. Determine whether the current three rails are converging turnouts;

[0130] If so, proceed to step S34;

[0131] If not, proceed to step S35;

[0132] S34. Search from the top boundary to the bottom boundary and save the coordinates of the rails on the left and right sides;

[0133] Among them, the saved rail coordinates, along with the subsequently identified rails, turnouts, and their directions, serve as the data basis for reconstructing the railway area image;

[0134] S35. Extract the turnout image from the rail skeleton image within the current search window and identify the turnout direction;

[0135] When identifying the direction of the turnout, the current search window shows four rails from left to right;

[0136] If the turnout direction is left, proceed to step S36;

[0137] If the turnout direction is right, proceed to step S37;

[0138] S36. Save the first and third rails from left to right as the rails currently in operation of the train. At the same time, retain the fourth rail from left to right as the right boundary reference of the current search. Extend 10 pixels to the left and 10 pixels to the right along the left rail to adjust the size of the search window. Proceed to step S38.

[0139] S37. Save the second and fourth rails from left to right as the rails currently in operation of the train. At the same time, retain the first rail from left to right as the current search boundary reference. Extend 10 pixels to the right along the left rail and 10 pixels to the left boundary to adjust the size of the search window. Proceed to step S38.

[0140] S38. Slide the search window in the rail skeleton image and save the rail coordinates until there are only three rails from left to right in the current search window, then proceed to step S39.

[0141] S39. Extend outward by 10 pixels along the left and right rails to adjust the current search window size, and determine whether there are two or three rails in front.

[0142] If there are two rails, return to step S31;

[0143] If there are three rails from left to right, and it is not the first turnout, then return to step S33;

[0144] S310. Repeat steps S31 to S39 until all rails in the rail skeleton image are saved, turnouts are captured, and directions are identified.

[0145] In step S35 of this embodiment, when identifying the turnout direction, since the turnouts to the left and right have a high degree of similarity in overall texture features, and the classification of turnout images is based on the opening and closing state of the turnout, it is necessary to distinguish the subtle feature differences in the opening and closing state of the turnout. In order to improve the efficiency and accuracy of turnout direction identification, in step S35 of this embodiment, a railway turnout classification model based on grouped convolution is used to identify the turnout direction.

[0146] The railway turnout classification model includes, in sequence, an image preprocessing module, a feature extraction module, a ResNeXt residual structure block, a multi-scale attention module, an average pooling layer, a feature one-dimensionalization layer, a fully connected layer, and a classifier.

[0147] The ResNeXt residual structure block is equipped with channel attention mechanism and spatial attention mechanism to further fuse and enhance the features of the extracted turnout image. Since different channels in the turnout image may contain different information, by introducing the channel attention mechanism, the model can pay more attention to the channels that are helpful to the classification task, thereby improving the classification accuracy.

[0148] In this embodiment, since turnout images are easily confused with the environment such as sleepers and have low contrast, and some turnout images have large areas of similar rail features, it is not easy to distinguish the type of turnout images by only a small range of turnout opening and closing states. Therefore, this embodiment adopts a multi-scale attention module, which can effectively extract the opening and closing state information of turnout images, reduce overfitting, and improve scale invariance.

[0149] Furthermore, the multi-scale attention module in this embodiment includes multiple parallel dilated convolutional layers with different dilation rates.

[0150] Specifically, the size and shape of turnout images can vary significantly. Different scales of convolutional kernels can be used to adapt to turnout images of varying sizes and shapes, thereby improving the network's robustness and generalization ability. Dilated convolutional layers are a type of convolutional operation that inserts holes into the convolutional kernel. This can expand the receptive field of the convolutional layer without increasing the number of parameters or computational cost, enabling better capture of global and local details in the turnout image. This invention utilizes dilated convolutional layers with different dilation rates to capture turnout pixel features at different proportions. While maintaining a constant output size, to fuse information from different scales, this invention uses multiple dilated convolutional layers in parallel, and then weights and sums their outputs to form a comprehensive feature map.

[0151] In the multi-scale attention module of this embodiment, the input feature map is represented as In, the output feature map is represented as Out, X1, X2, and X3 represent the outputs generated after convolution at different rates; {} represents the cascade algorithm; splicing; Represents convolution; Conv is used to represent a regular convolution kernel; Rate is used to represent dilated convolution kernels with different dilation rates; M c For channel attention maps; M s For a spatial attention graph, X1, X2, and X3 can be represented as:

[0152]

[0153]

[0154]

[0155] The output Out is represented as:

[0156]

[0157] In this embodiment, the differences between left-hand and right-hand traffic turnout data collected in actual operations are not always particularly obvious. For example, turnout sections are composed of stock rails and switch rails, and turnout images often have complex backgrounds such as sleepers. These images have highly similar backgrounds, making it difficult to determine left-hand or right-hand traffic based solely on local images. Therefore, the key to improving classification accuracy is to define the differences between these two categories of turnouts for left-hand and right-hand traffic. Considering that the turnout classification model in this embodiment needs to be deployed on actual running trains, and to improve inference speed, a loss function, Loss, is designed. r for:

[0158] Loss r =max(||f(A)-f(p)||) 2 -||f(A)-f(N)||2 +C,0)

[0159] In the formula, max(·) is the distance between category features, f(A) is the length of the target sample, f(p) is the length of the extracted image, f(N) is the length of the category different from A, A is the feature of the target sample, N is the category feature different from A, P represents the extracted image feature, and C is the upper boundary parameter.

[0160] In step S4 of this embodiment of the invention, the reconstructed railway area image is stitched together with the top 40% of the original image to obtain the final output, reconstructing the area in front of the train, realizing the detection of the actual operating area of ​​the train on the way, and thus helping to determine whether the detected obstacles are on the train's track, so as to determine the degree of danger of these obstacles to the train.

[0161] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

[0162] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A method for detecting the actual operating area of ​​trains in transit based on geometric feature mining, characterized in that, Includes the following steps: S1. Collect railway images of trains in transit, process them using a railway region segmentation model based on information fusion, and obtain rail segmentation results. In step S1, the railway region segmentation model based on information fusion includes a backbone network, a spatial void pooling pyramid, a pixel decoding module, a Transformer decoder module, and a classification predictor. The backbone network is used to extract features from railway images to form a low-resolution feature map of the railway track. The spatial void pooling pyramid is used to optimize the global and local features of the target region in the low-resolution feature map; The pixel decoding module is used to gradually restore the size of the low-resolution feature map based on the optimized global and local features, and then gradually concatenate the restored feature map with the low-resolution feature map and perform feature upsampling to generate pixel embeddings. The Transformer decoder module is used to generate the category prediction result and mask embedding vector of the target region based on the optimized global and local features; The classification predictor is used to generate a mask prediction result for the target region based on the mask embedding vector and pixel embedding, and fuse the category prediction result of the target region to obtain the mask segmentation result of the railway track pixels in the target region, which is used as the rail segmentation result. In the mask segmentation result of the railway track pixels in the target area, a pixel value of 0 represents the rail, a pixel value of 1 represents the railway area, and a pixel value of 2 represents the background. The loss function used in the Transformer decoder module to obtain the category prediction result of the target region. for: In the formula, Represents pixels The tag value, Represents pixels The predicted value, N This represents the total number of pixels. The loss function used in the classification predictor to obtain the mask prediction result of the target region. for: In the formula, The probability predicted by the model. It is the weight of the sample. As a regulating factor; The loss function of the railway area segmentation model for: In the formula, and These are the weights for mask prediction loss and class prediction loss, respectively. S2. Based on the rail segmentation results, extract the rail skeleton to form a rail skeleton image; S3. In the rail skeleton image, save the rails and extract the turnouts using a sliding search window, and identify the turnout direction; Step S3 specifically involves: S31. In the rail skeleton image, determine whether the current search window contains two rails or three rails from left to right; if there are two rails, proceed to step S32; if there are three rails, proceed to step S33. S32. Save the coordinate values ​​of the left and right rails, and adjust the size of the search window by extending 10 pixels along the rail direction. Return to step S31. S33. Determine whether the current three rails are merging turnouts; if yes, proceed to step S34; if no, proceed to step S35. S34. Search from the top boundary to the bottom boundary and save the coordinates of the rails on the left and right sides; S35. Extract the turnout image from the rail skeleton image within the current search window and identify the turnout direction; wherein, when identifying the turnout direction, the current search window contains four rails from left to right; if the turnout direction is left, proceed to step S36; if the turnout direction is right, proceed to step S37. S36. Save the first and third rails from left to right as the rails currently in operation of the train. At the same time, retain the fourth rail from left to right as the right boundary reference of the current search. Extend 10 pixels to the left and 10 pixels to the right along the left rail to adjust the size of the search window. Proceed to step S38. S37. Save the second and fourth rails from left to right as the rails currently in operation of the train. At the same time, retain the first rail from left to right as the current search boundary reference. Extend 10 pixels to the right along the left rail and 10 pixels to the left boundary to adjust the size of the search window. Proceed to step S38. S38. Slide the search window in the rail skeleton image and save the rail coordinates until there are only three rails from left to right in the current search window, then proceed to step S39. S39. Extend 10 pixels outward along the left and right rails to adjust the current search window size, and determine whether there are two or three rails ahead; if there are two rails, return to step S31; if there are three rails from left to right, and it is not the first turnout, return to step S33. S310. Repeat steps S31 to S39 until all rails in the rail skeleton image are saved, turnouts are captured, and directions are identified. S4. Based on the saved rails, the captured turnouts and their directions, reconstruct the railway area image of the train in transit to realize the detection of the actual operating area of ​​the train in transit.

2. The method for detecting the actual operating area of ​​trains in transit based on geometric feature mining according to claim 1, characterized in that, The backbone network is an improved version of the Xception_ECA network, which includes a Sep_ECA structure. The Sep_ECA structure is a depthwise separable convolutional feature extraction structure with added SE attention mechanism. In the pixel decoding module, the generated pixel embedding ;in, For pixel embedding dimension, The height of the feature map, The width of the feature map; The Transformer decoder module uses an ensemble prediction mechanism to generate... N The category prediction results and the mask embedding vector; In the Transformer decoder module, a corresponding linear classifier is set for each segment to predict the category of each track segment. Its expression is: In the formula, For class probability, To predict the number of categories plus the types of background, To predict the number of categories, i For the prediction result to the ordinal, N The logarithm of the generated prediction results; In the Transformer decoder module, a multilayer perceptron with two hidden layers embeds segments. Q Convert to N A mask embedding vector.

3. The method for detecting the actual operating area of ​​trains in transit based on geometric feature mining according to claim 2, characterized in that, The method by which the classification predictor obtains the rail segmentation results is as follows: Mask embedding vector and pixel embedding The dot product is used to obtain the mask prediction result, and then... Function activation, resulting in ; pass Assign each pixel to N One of the predicted probability-mask pairs is used to segment the railway image into several segments. Segments with the same category label are merged to obtain the rail segmentation result. in, For pixels in the target area The mask prediction results For the mask prediction result, , for Activation function For mask embedding vector, For the first i The first pair of prediction results corresponds to the first... i A mask embedding, For pixel embedding, The coordinate information embedded in the pixel. H The height of the feature map, W The width of the feature map; To position Pixels assigned to the first i For the probability mask pairs corresponding to the prediction results, This indicates that the "no object" category is not included. For class probability, For category labels.

4. The method for detecting the actual operating area of ​​trains in transit based on geometric feature mining according to claim 1, characterized in that, Step S2 specifically involves: S21. Based on the rail segmentation results, search for the track region in the railway image; Among them, the area between the two rails is the track area; S22. Based on the searched track region, extract the rail skeleton.

5. The method for detecting the actual operating area of ​​trains in transit based on geometric feature mining according to claim 4, characterized in that, In step S35, a railway turnout classification model based on grouped convolution is used to identify the turnout direction; The railway turnout classification model includes, in sequence, an image preprocessing module, a feature extraction module, a ResNeXt residual structure block, a multi-scale attention module, an average pooling layer, a feature one-dimensionalization layer, a fully connected layer, and a classifier. The ResNeXt residual structure block is equipped with channel attention mechanism and spatial attention mechanism; the multi-scale attention module includes multiple parallel dilated convolutional layers with different dilation rates.

6. The method for detecting the actual operating area of ​​trains in transit based on geometric feature mining according to claim 5, characterized in that, The loss function of the railway turnout classification model for: In the formula, The distance between categorical features. The length of the target sample. The length of the image extracted from the table. To and A The lengths of different categories, A Features of the target sample N To and A Different category features, P represents the extracted image features, and C is the upper boundary parameter.

Images