Urban rail train track line detection method and device

Abstract

The application provides a train track line detection method and device, and relates to the technical field of computers.The method comprises: acquiring a target image comprising a train track; inputting the target image into a target model to obtain a track line position and a category output by the target model; wherein the target model comprises: an instance detection sub-model, which is used for predicting a track line position and a category in the target image to obtain a track line position prediction result and a category prediction result; filtering spatial features and semantic features of the track line according to the track line position prediction result and the category prediction result to obtain effective spatial features and semantic features; and a shape prediction sub-model, which is used for obtaining target pixel points corresponding to the track line in the target image according to the effective spatial features and the semantic features, and determining the track line position and the category according to the target pixel points.The train track line detection method and device provided in the application can improve the efficiency and accuracy of track line detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a metro train track line detection method and device. BACKGROUND

[0002] The urban rail transit with large passenger capacity, high transportation efficiency and low energy consumption has become an inevitable choice to alleviate urban traffic congestion. With the increasing size of the metro train system, the safety hazards have also increased significantly. In order to reduce the safety hazards of train operation and improve the intelligent and digital level of metro train operation, an intelligent track perception system is urgently needed.

[0003] The track line, as a landmark target in the train operation environment, has important reference significance for train gauge detection, real-time positioning and front obstacle detection. Therefore, it is generally believed that the track line detection of the metro train is the basis of the intelligent track perception. At present, the track line detection generally uses a lane line segmentation method based on a convolutional neural network. However, due to the poor light environment in the subway tunnel and the alternating strong and weak light, the convolutional neural network cannot well extract the track feature information, which greatly reduces the track line detection accuracy and causes a large number of false detection and missed detection problems. SUMMARY

[0004] The present application provides a metro train track line detection method to solve the defects of track line false detection and missed detection in the prior art, which can improve the track line detection accuracy.

[0005] In a first aspect, the present application provides a train track line detection method, comprising:

[0006] obtaining a target image including a train track;

[0007] inputting the target image into a target model to obtain a track line position and a category output by the target model; the category includes a left track line and a right track line;

[0008] The target model comprises:

[0009] a feature extraction sub-model for extracting features of the target image to obtain spatial features and semantic features of the track line;

[0010] an instance detection sub-model for predicting the track line position and the category in the target image to obtain a track line position prediction result and a category prediction result;

[0011] screening the spatial features and the semantic features of the track line according to the track line position prediction result and the category prediction result to obtain effective spatial features and semantic features;

[0012] a shape prediction sub-model configured to obtain target pixel points corresponding to the track line in the target image according to the effective spatial features and semantic features, and determine the track line position and category according to the target pixel points.

[0013] In an embodiment, the target model further comprises a pairing sub-model.

[0014] The pairing sub-model is configured to predict an extension direction of each target pixel point in the case that the target image comprises multiple pairs of train tracks, and determine each pair of train tracks according to the prediction result.

[0015] In an embodiment, the instance detection sub-model is specifically configured to:

[0016] determine a target row number of a pixel point with a confidence greater than a preset threshold in the track line position prediction result and the category prediction result;

[0017] screen the spatial features and semantic features of the track line according to the target row number to obtain effective spatial features and semantic features;

[0018] Each spatial feature and semantic feature corresponds to a row number, and the row number is determined according to the pixel arrangement of the target image.

[0019] In an embodiment, the shape prediction sub-model is specifically configured to:

[0020] perform dimension reduction processing on the effective spatial features and semantic features corresponding to each row of pixels to obtain a track position vector, a track vertical vector, and a track offset vector;

[0021] cluster the track position vector, the track vertical vector, and the track offset vector according to prior data of the train track to obtain each target pixel point;

[0022] sort each target pixel point according to the row number corresponding to each effective spatial feature and semantic feature to obtain the track line position and category.

[0023] In an embodiment, the target model is determined in the following manner:

[0024] training an initial model according to a target image sample and a corresponding label to obtain the target model;

[0025] The label comprises a position of the train track and a category of the train track in the target image sample.

[0026] In an embodiment, the instance detection sub-model is determined in the following manner:

[0027] The initial detection sub-model is trained according to the target image sample and a corresponding label of the target image sample, to obtain the instance detection sub-model.

[0028] The label includes a position of the train track in the target image sample and a category of the train track.

[0029] In an embodiment, the feature extraction sub-model is a Transformer model, and the instance detection sub-model is a CNN model.

[0030] In a second aspect, an embodiment of the present application provides a train track line detection device, including:

[0031] An acquisition module is configured to acquire a target image including a train track.

[0032] A detection module is configured to input the target image into a target model, to obtain a track line position and a category output by the target model, wherein the category includes a left track line and a right track line.

[0033] The target model includes:

[0034] A feature extraction sub-model is configured to perform feature extraction on the target image, to obtain spatial features and semantic features of the track line.

[0035] An instance detection sub-model is configured to predict a track line position and a category in the target image, to obtain a track line position prediction result and a category prediction result.

[0036] The spatial features and the semantic features of the track line are screened according to the track line position prediction result and the category prediction result, to obtain effective spatial features and semantic features.

[0037] A shape prediction sub-model is configured to obtain target pixel points corresponding to the track line in the target image according to the effective spatial features and the semantic features, and to determine the track line position and the category according to the target pixel points.

[0038] In a third aspect, the present application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method of the first aspect.

[0039] In a fourth aspect, the present application further provides a non-transitory computer readable storage medium, having a computer program stored thereon, wherein the computer program is executable on a processor to implement the method of the first aspect.

[0040] The train track line detection method and device provided by the embodiment of the present application can automatically extract spatial features and semantic features from a train track image by constructing a target model, filter the extracted features according to the predicted track line position and category, and then obtain the track line position and category according to the effective features, so that the track line detection can be effectively performed under various light conditions, the track line mis-detection and missed detection problems can be avoided as much as possible, the efficient and real-time detection of the train track line under dense track lines and poor light conditions is realized, and thus the efficiency and accuracy of the track line detection are effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the present application or prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0042] Figure 1 FIG. 1 is a flowchart of a train track line detection method provided by the embodiment of the present application;

[0043] Figure 2 FIG. 2 is a structural diagram of a target model used in the train track line detection method provided by the embodiment of the present application;

[0044] Figure 3 FIG. 3 is a structural diagram of a Transformer feature encoding network provided by the embodiment of the present application;

[0045] Figure 4 FIG. 4 is a structure of a Transformer encoding layer provided by the embodiment of the present application;

[0046] Figure 5 FIG. 5 is a structure of a self-attention module provided by the embodiment of the present application;

[0047] Figure 6 FIG. 6 is a structural diagram of a train track line detection device provided by the embodiment of the present application;

[0048] Figure 7 FIG. 7 is a structural diagram of an electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions will be clearly and completely described below with reference to the drawings in the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0050] Figure 1 is a flowchart of a train track line detection method provided by an embodiment of the present application. Referring to Figure 1 , an embodiment of the present application provides a train track line detection method, which can include:

[0051] Step 110, obtaining a target image including a train track;

[0052] Step 120, inputting the target image into a target model to obtain a track line position and a category output by the target model; the category includes a left track line and a right track line;

[0053] The target model includes:

[0054] a feature extraction sub-model, configured to perform feature extraction on the target image to obtain spatial features and semantic features of the track line;

[0055] an instance detection sub-model, configured to predict the track line position and the category in the target image to obtain a track line position prediction result and a category prediction result;

[0056] screening the spatial features and the semantic features of the track line according to the track line position prediction result and the category prediction result to obtain effective spatial features and semantic features;

[0057] a shape prediction sub-model, configured to obtain target pixel points corresponding to the track line in the target image according to the effective spatial features and the semantic features, and determine the track line position and the category according to the target pixel points.

[0058] It should be noted that the execution subject of the train track line detection method provided by an embodiment of the present application can be a computer device, such as a mobile phone, a tablet computer, a notebook computer, a palm computer, etc. In the following, a detection system for executing the train track line detection method is taken as an example to illustrate the train track line detection method provided by an embodiment of the present application.

[0059] In step 110, a camera can be installed at a higher position in the train cab, facing the track line, so as to expand the sensing range of the camera while preventing obstruction by obstacles, and to achieve accurate collection of train track line data. The installed camera can collect track line video data of the front track area during train operation, and the train operation detection system can obtain relevant images of the train track from the track line video data as target images.

[0060] In step 120, the target model of the detection system can identify the target images, specifically, identify the train track line included in the target images, and obtain the position information and category information of the train track line. The position information of the train track line can be the position of each line of the train track line in the target image, specifically, the pixel of each line. The category information of the train track line can include left track line, right track line and background category. In the embodiments of the present application, the left track line and the right track line can be selected as the track line category information output by the target model. The detection system inputs the target images obtained in step 110 into the target model, and can obtain the track line position information and category information output by the target model, and can transmit the track line position and category as basic information into the train intelligent system.

[0061] Specifically, as shown in Figure 2 The target model can include:

[0062] The feature extraction sub-model can be used for feature extraction of the target image, specifically for extracting spatial features and semantic features of the train track line included in the target image. After the target model receives the target image, the feature extraction sub-model in the target model can perform feature encoding on the target image. The feature extraction sub-model can be a Transformer model, or a swin-Transformer model, a CaiT (Class Attention Image Transformer) model, a T2T-ViT (Tokens-to-Token-Vision Transformer) model, and a CPVT (Conditional Position encodings Visual Transformer) model, etc. that takes the Transformer model as the core structure. The swin-Transformer, CaiT, T2T-ViT, and CPVT models can all solve the problem of computational and memory overhead of the traditional Transformer model to a certain extent when processing large images, and achieve a balance between accuracy and efficiency. The swin-Transformer model and the Feature Pyramid Module (FPM) can also be selected for multi-scale feature extraction in the embodiments of the present application, which effectively extracts context information. The Transformer network can capture the relationship between different positions in the track line (spatial features) and context information (semantic features). The spatial features and semantic features can describe the shape, direction, continuity, and other features of the track line, providing information for subsequent extraction of track line prediction points and clustering.

[0063] The instance detection sub-model can be used to predict the position information and category information of the track line in the target image to obtain track line position prediction results and category prediction results, and then filter the spatial features and semantic features of the track line according to the track line position prediction results and the category prediction results, thereby obtaining effective spatial features and semantic features. Before the instance detection sub-model predicts the position and category information of the track line, the feature and position information of the track line can be learned first. When receiving the track line image, the instance detection sub-model can first preprocess the track line image, then predict the position and category of the preprocessed track line image, and analyze and post-process the prediction results of the position and category, filter the spatial features and semantic features with high credibility as effective spatial features and semantic features.

[0064] The shape prediction sub-model can be used to obtain target pixel points corresponding to the track lines in the target image according to the effective spatial features and the semantic features, and determine the positions and categories of the track lines according to the target pixel points. The shape prediction sub-model can obtain the effective spatial features and the semantic features output by the instance detection sub-model, and can obtain the most possible positions of the track lines in each row according to the effective spatial features and the semantic features, determine the positions as target pixel points of the row, and then cluster the target pixel points of each row to obtain the positions of the track lines, and obtain the category information of the track lines according to the spatial position relationship of the target pixel points, to determine the left track lines and the right track lines.

[0065] The train track line detection method provided by the embodiments of the present application can automatically extract spatial features and semantic features from a train track image by constructing a target model, filter the effective features according to the predicted positions and categories of the track lines, and obtain the positions and categories of the track lines according to the effective features, so that the track line detection can be effectively performed under various light conditions, the problems of track line false detection and missed detection can be avoided as much as possible, the efficient and real-time detection of the train track lines in dense track lines and poor light conditions is realized, and thus the efficiency and accuracy of the track line detection are effectively improved.

[0066] In one embodiment, the target model further includes a pairing sub-model.

[0067] The pairing sub-model is used to predict the extension directions of each target pixel point in the case where the target image includes multiple pairs of train tracks, and determine each pair of train tracks according to the prediction result.

[0068] In an ideal case, the track lines can be normally paired two by two. However, in practice, due to the excessive density of the track lines, or multiple track lines may share part of the track line target pixel points, for example, in the case of a turnout, the track lines two by two will be abnormally paired, and the detection of the track lines at the turnout will be abnormal. In addition, the poor light environment will also cause the prediction error of part of the track line target pixel points, resulting in errors in the subsequent track line clustering and shape prediction. Based on the above situation, the pairing sub-model is proposed in the embodiments.

[0069] As Figure 2As shown, the pair model can select a recurrent instance module (RIM), which can better overcome the problems of complex topological structures (such as dense lines and bifurcated lines) encountered in the track line detection process. The RIM structure can be constructed based on a long short-term memory (LSTM) model. In the case of including multiple pairs of train tracks in the target image, the RIM model can obtain a recursive prediction track line state vector si and a variable kernel parameter vector ki through learning. The vector si can represent two states of Boolean values: "continue" or "stop"; the vector ki contains kernel parameters for dynamic convolution in track line instance detection and shape prediction. The RIM module can predict the extension direction corresponding to each target pixel point based on the recursive prediction track line state vector si and the variable kernel parameter vector ki. In the prediction process, RIM cyclically predicts the track direction kernel parameters of the target pixels sharing the same track line until the state vector si is "stop", thereby realizing the direction prediction of the track line at the turnout, and determining the paired train tracks according to the prediction result.

[0070] The train track line detection method provided by the embodiments of the present application can realize accurate prediction of the direction of each track line at the turnout by adding RIM for each track target pixel point to perform direction prediction, greatly reduce the error matching rate of bifurcated lines, improve the detection accuracy in dense line environment, and solve the problem of track line pair prediction in complex topological structure. Moreover, the embodiments of the present application can achieve a processing frame rate of 30 FPS while ensuring the tracking accuracy of RIM, meeting the real-time deployment requirements.

[0071] In one embodiment, the instance detection sub-model is specifically used for:

[0072] Determining the target row number of the pixel point with a confidence greater than a preset threshold in the track line position prediction result and the category prediction result;

[0073] According to the target row number, the spatial features and semantic features of the track line are screened to obtain effective spatial features and semantic features;

[0074] Each spatial feature and semantic feature corresponds to a row number; the row number is determined according to the pixel arrangement of the target image.

[0075] The instance detection submodule can predict the track line position and category, and can screen the spatial features and semantic features of the track line according to the prediction result to obtain effective spatial features and semantic features. The embodiment of the application proposes a conditional track line detection strategy based on conditional convolution, which is a convolution operation with dynamic kernel parameters. The instance detection submodule can predict track line key pixels and regress a set of dynamic kernel parameters for each predicted track line pixel. In the prediction process, the instance detection submodule can first preprocess the image to be detected, such as adjusting the size, normalizing, removing noise, etc., so as to remove interference and improve the speed and accuracy of the prediction of the instance detection submodule. Then the instance detection submodule can perform object detection, specifically by inputting the preprocessed image into the trained instance detection model, using the model to perform object detection, and outputting the preliminary prediction result of the position and category of the track line. The instance detection submodule can continue to analyze and process the preliminary prediction result, specifically by screening the position and category of the detected track line according to the prediction result, such as screening the pixel points with a confidence greater than a preset threshold according to the confidence, Non-Maximum Suppression (NMS) processing, track line recognition, etc., determining the target row number of the pixel points, and screening the spatial features and semantic features of the track line according to the pixel points with a confidence greater than a preset threshold in the target row number, to obtain the effective spatial features and semantic features of the target row. It should be noted that each spatial feature and semantic feature corresponds to a row number, and the row number is determined according to the pixel arrangement of the target image. The complete track line image can be obtained by clustering the target pixels in each row.

[0076] The train track line detection method provided by the embodiment of the application designs a dedicated conditional convolution layer based on track line conditions, judges track line points based on pre-set conditions, and screens the extracted spatial features and semantic features through instance detection, which can avoid track line false detection problems, thereby improving the accuracy of track line detection.

[0077] In one embodiment, the shape prediction sub-model is specifically used for:

[0078] The effective spatial features and semantic features corresponding to each row of pixels are processed by dimension reduction to obtain a track position vector, a track vertical vector, and a track offset vector;

[0079] The track position vector, the track vertical vector, and the track offset vector are clustered according to the prior data of the train track to obtain each target pixel point;

[0080] The target pixel points are sorted according to the row numbers corresponding to each effective spatial feature and semantic feature to obtain the track line position and category.

[0081] The shape prediction sub-model can apply conditional convolution to specify track line shape, including track curvature, track vertical range and other important parameters in the shape prediction step. The conditional convolution layer mainly includes a convolution layer with a variable convolution kernel and a series of track line shape loss functions. Since each track line instance corresponds to a set of dynamic convolution kernels, the shape can be intelligently predicted at the pixel level.

[0082] After obtaining the effective spatial features and semantic features output by the instance detection sub-model, the shape prediction sub-model can perform dimension reduction processing on the effective features to obtain a track position vector, a track vertical vector and a track offset vector. The prediction of the train track line is based on the track position vector, the track vertical vector and the track offset vector. The track position vector represents the target pixel point position where the track line may appear in each row. The track vertical vector represents the pixel range occupied by the track line in the vertical direction of the image. The track offset vector predicts the offset value of the current row target pixel point relative to the actual position of the track line.

[0083] The shape prediction sub-model can cluster the track position vector, the track vertical vector and the track offset vector based on the prior data of the train track to obtain each target pixel point in each row, and sort each target pixel point according to the row number corresponding to each effective spatial feature and semantic feature output by the instance detection sub-model to obtain the track line position and category.

[0084] The shape prediction sub-model can predict the line shape based on common shape features of the track line. In the row-by-row prediction, the shape prediction sub-model can predict 3775 possible track line positions on each row, then cluster the track position vector, the track vertical vector and the track offset vector according to the prior data of the track shape stored by the detection system to obtain the target pixel point of each row, and can obtain the track line position and category according to the image from bottom to top according to each effective spatial feature and semantic feature.

[0085] The train track line detection method provided by the embodiment of the application designs a dedicated conditional convolution layer based on track line conditions, judges track line points based on pre-set conditions, and further filters the effective spatial features and semantic features obtained by instance detection through shape prediction, which can further avoid the problem of track line false detection and improve the accuracy of track line detection.

[0086] In one embodiment, the target model is determined in the following manner:

[0087] The initial model is trained according to the target image sample and the corresponding label to obtain the target model.

[0088] The label includes the position of the train track and the category of the train track in the target image sample.

[0089] The target model can be trained by taking multiple target images including track lines as samples, taking the track line detection result corresponding to the target image sample as a label, and training the initial model. When the accuracy of the initial model reaches the target requirement, the target model can be obtained. The track line detection result can include the position of the train track in the target image sample and the category of the train track. When the target model receives a train track line image, it can identify and output the corresponding train track line position and category.

[0090] In actual operation, track line images can be selected as the training set according to the track line video data collected by the camera. In order to achieve better training effect, track line images in poor exposure environments such as weak light and alternating light and dark can also be selected as the training set as much as possible. For example, 3000 track line data in high exposure environment in the tunnel, 2000 low light track line data and 3000 track line data in normal light can be selected to form the training set, so as to be more consistent with the actual situation.

[0091] Then a part of the training set needs to be selected as a test set to detect the effect of the target model trained. 500 pictures can be selected as the test set according to different light environments and different places. The test pictures can include various severe light environments, such as typical environments of high exposure and low light, so as to achieve the most actual detection effect.

[0092] The training set can be first instantiated and pixel-level labeled. Specifically, each pixel in each image is classified or segmented to identify different train parts or objects in the image. In this embodiment, only the track line and its category can be labeled. The track line category can be divided into three categories, the background category is represented by 0, the left track line is represented by 1, and the right track line is represented by 2. In order to reduce the detection difficulty and improve the detection accuracy of the track, only the track can be labeled in the labeling process, and other tracks do not need to be labeled, that is, the left track line of the track is 1, and the right track line of the track is 2.

[0093] The labeled track line image and the detection result are used to train the target model, and the accuracy of the target model can be detected by the images of the test set until the accuracy of the target model meets the demand.

[0094] The train track line detection method provided by the embodiment of the application can further filter the effective spatial features and semantic features obtained by shape detection and instance detection through training the target model to predict the position and category of the track line, can further avoid the problem of track line false detection, and improve the automation rate and accuracy of track line detection.

[0095] In one embodiment, the instance detection sub-model is determined by the following method:

[0096] The initial detection sub-model is trained according to the target image sample and a corresponding label of the target image sample, to obtain the instance detection sub-model.

[0097] The label includes a position of the train track in the target image sample and a category of the train track.

[0098] The instance detection sub-model can take multiple target images including track lines as samples, take a track line detection result corresponding to the target image sample as a label, train the initial detection sub-model, and obtain the instance detection sub-model when the accuracy of the initial detection sub-model reaches a target requirement. The track line detection result can include a position of the train track in the target image sample and a category of the train track. When the instance detection sub-model receives a train track image, the corresponding train track position and category can be recognized and output.

[0099] The train track detection method provided in the embodiments of the present application can realize automatic prediction of instance detection by training an initial detection sub-model to predict the position and category of a track line image, and can help improve the efficiency and accuracy of track line detection.

[0100] In one embodiment, the feature extraction sub-model is a Transformer model, and the instance detection sub-model is a CNN model.

[0101] As shown in Figure 3 , the feature extraction sub-model can select a Transformer model. Each small module of the feature extraction sub-model adopts a Transformer self-attention mechanism. The Transformer self-attention mechanism first preliminarily extracts image features through a convolution operation with a kernel of 3, then learns deep semantic information of the track line through a self-attention module, and finally adjusts the extracted spatial features and deep semantic features through a convolution operation with a kernel of 3. The structure of each encoding layer is as shown in Figure 4 .

[0102] The self-attention module extracts different feature weights through three linear layers, integrates the three feature weights, and assigns the integrated feature weights to the encoded features, to realize self-learning of the track line features. A schematic diagram of the self-attention module is as shown in Figure 5 .

[0103] In the example detection process, an image data set containing track lines can be collected and labeled first, and the position and category information of the track lines are labeled, and then model selection and training are performed. An instance detection submodel suitable for the instance detection task can be selected, such as a FasterR-CNN (Faster Region based Convolutional Neural Network), a YOLO (You Only Look Once), a MaskR-CNN (Mask Region based Convolutional Neural Network) model, and the like. In this embodiment of the application, a convolutional neural network (CNN) model is selected as the instance detection submodel. Then, the collected data set is used as a sample, and the data labeled with the position and category information of the track lines is used as a label to train the initial detection submodel, so that the model can learn the features and position information on the track lines. Through continuous training and iteration, the initial detection submodel gradually meets the accuracy requirement and becomes an instance detection submodel.

[0104] The train track line detection method provided in this embodiment of the application selects a Transformer model as a feature extraction submodel and selects a CNN model as an instance detection submodel, greatly improves the feature extraction efficiency of a small track line in the distance and the feature extraction capability in a poor light fiber condition, guarantees the detection and clustering accuracy of subsequent track line points, and can ensure the automation of feature extraction and instance detection. Through continuous iteration and update, the accuracy of feature extraction and instance detection is further improved, and the efficiency and accuracy of the track line detection method are effectively improved.

[0105] The train track line detection device provided in this application is described below. The train track line detection device described below can be referred to in correspondence with the train track line detection method described above.

[0106] Figure 6 FIG. 1 is a structural schematic diagram of a train track line detection device provided in this embodiment of the application.

[0107] Referring to Figure 6 The train track line detection device provided in this embodiment of the application can include:

[0108] The acquisition module 610 is configured to acquire a target image including a train track.

[0109] The detection module 620 is configured to input the target image into a target model to obtain track line positions and categories output by the target model. The categories include left track lines and right track lines.

[0110] The target model includes:

[0111] a feature extraction sub-model configured to perform feature extraction on the target image to obtain spatial features and semantic features of the track line;

[0112] an instance detection sub-model configured to predict a track line position and a track line category in the target image to obtain a track line position prediction result and a track line category prediction result;

[0113] screen the spatial features and the semantic features of the track line according to the track line position prediction result and the track line category prediction result to obtain effective spatial features and semantic features;

[0114] a shape prediction sub-model configured to obtain target pixel points corresponding to the track line in the target image according to the effective spatial features and the semantic features, and determine the track line position and the track line category according to the target pixel points.

[0115] The train track line detection device provided by the embodiments of the present application can automatically perform feature extraction on a train track image to obtain spatial features and semantic features, screen the extracted features according to a predicted track line position and a predicted track line category to obtain effective features, and then obtain a track line position and a track line category according to the effective features. The device can effectively detect track lines under various light conditions, avoid track line mis-detection and missed detection as much as possible, and realize efficient and real-time detection of train track lines in dense track lines and poor light conditions, thereby effectively improving the efficiency and accuracy of track line detection.

[0116] In one embodiment, the target model further includes a pairing sub-model, and the pairing sub-model is specifically configured to:

[0117] when the target image includes multiple pairs of train tracks, predict an extension direction corresponding to each target pixel point, and determine each pair of train tracks according to a prediction result.

[0118] In one embodiment, the instance detection sub-model is specifically configured to:

[0119] determine a target row number of a pixel point having a confidence greater than a preset threshold in the track line position prediction result and the track line category prediction result;

[0120] screen the spatial features and the semantic features of the track line according to the target row number to obtain effective spatial features and semantic features;

[0121] Each spatial feature and semantic feature corresponds to a row number, and the row number is determined according to a pixel arrangement of the target image.

[0122] In one embodiment, the shape prediction sub-model is specifically configured to:

[0123] dimensionally reduce the effective spatial features and semantic features corresponding to each row of pixels to obtain a track position vector, a track vertical vector and a track offset vector;

[0124] cluster the track position vector, the track vertical vector and the track offset vector according to prior data of the train track to obtain each target pixel point;

[0125] sort the target pixel points according to the number of rows corresponding to each effective spatial feature and semantic feature to obtain the track line position and category.

[0126] In one embodiment, the target model is determined in the following manner:

[0127] training an initial model according to target image samples and corresponding labels to obtain the target model;

[0128] The labels include the position of the train track and the category of the train track in the target image samples.

[0129] In one embodiment, the instance detection sub-model is determined in the following manner:

[0130] training an initial detection sub-model according to target image samples and corresponding labels to obtain the instance detection sub-model;

[0131] The labels include the position of the train track and the category of the train track in the target image samples.

[0132] In one embodiment, the feature extraction sub-model is a Transformer model, and the instance detection sub-model is a CNN model.

[0133] It should be noted that the train track line detection device provided by the embodiments of the present application can achieve the same technical effects based on the above train track line detection method, and the same parts and beneficial effects in the method embodiments will not be described in detail.

[0134] Figure 7 An example of an electronic device is shown in the entity structure diagram as shown in Figure 7As shown, the electronic device can include a processor 710, a communications interface 720, a memory 730, and a communications bus 740, wherein the processor 710, the communications interface 720, and the memory 730 complete mutual communication through the communications bus 740. The processor 710 can invoke a logical instruction in the memory 730 to execute a train track line detection method, for example, including:

[0135] Obtaining a target image including a train track;

[0136] Inputting the target image into a target model to obtain a track line position and a category output by the target model; the category includes a left track line and a right track line;

[0137] The target model includes:

[0138] A feature extraction sub-model for performing feature extraction on the target image to obtain spatial features and semantic features of a track line;

[0139] An instance detection sub-model for predicting a track line position and a category in the target image to obtain a track line position prediction result and a category prediction result;

[0140] Screening the spatial features and the semantic features of the track line according to the track line position prediction result and the category prediction result to obtain effective spatial features and semantic features;

[0141] A shape prediction sub-model for obtaining target pixel points corresponding to the track line in the target image according to the effective spatial features and the semantic features, and determining the track line position and the category according to the target pixel points.

[0142] In addition, the logical instruction in the memory 730 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0143] In another aspect, the present application also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the train track line detection method provided by the above method, for example comprising:

[0144] obtaining a target image including a train track;

[0145] inputting the target image into a target model to obtain a track line position and a category output by the target model; the category includes a left track line and a right track line;

[0146] wherein the target model comprises:

[0147] a feature extraction sub-model configured to perform feature extraction on the target image to obtain spatial features and semantic features of the track line;

[0148] an instance detection sub-model configured to predict the track line position and the category in the target image to obtain a track line position prediction result and a category prediction result;

[0149] screening the spatial features and the semantic features of the track line according to the track line position prediction result and the category prediction result to obtain effective spatial features and semantic features;

[0150] a shape prediction sub-model configured to obtain target pixel points corresponding to the track line in the target image according to the effective spatial features and the semantic features, and determine the track line position and the category according to the target pixel points.

[0151] The device embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme. Those skilled in the art can understand and implement without creative labor.

[0152] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0153] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of detecting a train track line, characterized by, The method comprises the following steps: acquiring a target image comprising a train track; inputting the target image into a target model to obtain a track line position and a category output by the target model; the category comprises a left track line and a right track line; wherein the target model comprises: a feature extraction sub-model, configured to perform feature extraction on the target image to obtain spatial features and semantic features of a track line; an instance detection sub-model, configured to predict a track line position and a category in the target image to obtain a track line position prediction result and a category prediction result; screening the spatial features and the semantic features of the track line according to the track line position prediction result and the category prediction result to obtain effective spatial features and semantic features; a shape prediction sub-model, configured to obtain target pixel points corresponding to the track line in the target image according to the effective spatial features and the semantic features, and determine the track line position and the category according to the target pixel points; the instance detection sub-model is specifically configured to: determine a target row number of a pixel point with a confidence greater than a preset threshold in the track line position prediction result and the category prediction result; screen the spatial features and the semantic features of the track line according to the target row number to obtain effective spatial features and semantic features; wherein each spatial feature and semantic feature corresponds to a row number; the row number is determined according to a pixel arrangement of the target image; the shape prediction sub-model is specifically configured to: perform dimension reduction processing on the effective spatial features and the semantic features corresponding to each row of pixels to obtain a track position vector, a track vertical vector, and a track offset vector; cluster the track position vector, the track vertical vector, and the track offset vector according to prior data of a train track to obtain target pixel points; sort the target pixel points according to the row numbers corresponding to each effective spatial feature and semantic feature to obtain the track line position and the category.

2. The train track line detection method according to claim 1, characterized in that, The target model further comprises a pairing sub-model; the pairing sub-model is configured to predict an extension direction corresponding to each target pixel point in the case that the target image comprises multiple pairs of train tracks, and determine each pair of train tracks according to the prediction result.

3. The train track line detection method according to claim 1 or 2, characterized by, The target model is determined in the following manner: training an initial model according to target image samples and corresponding labels to obtain the target model; wherein the labels comprise a position of the train track and a category of the train track in the target image samples.

4. The train track line detection method according to claim 1 or 2, characterized by, The instance detection sub-model is determined in the following manner: training an initial detection sub-model according to target image samples and corresponding labels to obtain the instance detection sub-model; wherein the labels comprise a position of the train track and a category of the train track in the target image samples.

5. The train track line detection method according to claim 1 or 2, characterized by, The feature extraction sub-model is a Transformer model, and the instance detection sub-model is a CNN model.

6. A train track line detection device characterized by comprising: The method comprises the following steps: an acquisition module, configured to acquire a target image comprising a train track; The detection module is configured to input the target image into a target model to obtain a track line position and a category output by the target model, wherein the category includes a left track line and a right track line. The target model includes: A feature extraction sub-model configured to perform feature extraction on the target image to obtain spatial features and semantic features of the track line; An instance detection sub-model configured to predict the track line position and the category in the target image to obtain a track line position prediction result and a category prediction result; The spatial features and the semantic features of the track line are screened according to the track line position prediction result and the category prediction result to obtain effective spatial features and semantic features; A shape prediction sub-model configured to obtain target pixel points corresponding to the track line in the target image according to the effective spatial features and the semantic features, and determine the track line position and the category according to the target pixel points; The instance detection sub-model is specifically configured to: determine a target row number of a pixel point with a confidence greater than a preset threshold in the track line position prediction result and the category prediction result; screen the spatial features and the semantic features of the track line according to the target row number to obtain the effective spatial features and the semantic features; Each spatial feature and semantic feature corresponds to a row number, and the row number is determined according to a pixel arrangement of the target image. The shape prediction sub-model is specifically configured to: perform dimension reduction processing on the effective spatial features and the semantic features corresponding to each row of pixels to obtain a track position vector, a track vertical vector, and a track offset vector; cluster the track position vector, the track vertical vector, and the track offset vector according to prior data of a train track to obtain each target pixel point; sort the target pixel points according to the row numbers corresponding to each effective spatial feature and semantic feature to obtain the track line position and the category.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the train track line detection method of any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the train track line detection method of any one of claims 1 to 5.

Images