System and Method for Predicting Train Travelable Area Based on Rail Instance Segmentation

Abstract

System and method for predicting train drivable area based on rail instance segmentation. The fields involved include: a data acquisition module and a rail instance segmentation model. Based on deep learning, the present invention collects and annotates image data of the track through the data acquisition module, annotates the current track as the same instance, and the turnout information is implicitly included in the annotation stage. By using a large number of annotated pictures and inputting the obtained instances into the rail instance segmentation model, the implicit turnout information is learned in an end-to-end manner, which not only reduces the computational amount but also improves the performance. It releases the limitation that the prediction result depends on the rail segmentation result, especially the result of the turnout section, and directly predicts the drivable area based on the end-to-end method, thereby improving the accuracy through a large number of annotated pictures.

Description

Technical Field

[0001] The present invention relates to the technical field of track prediction, and specifically relates to a system and method for predicting the train drivable area based on rail instance segmentation. Background Art

[0002] Rail transit refers to a type of transportation vehicle or transportation system in which the operating vehicle needs to travel on a specific track. The most typical rail transit is the railway system composed of traditional trains and standard railways. With the diversified development of train and railway technologies, rail transit presents more and more types, which are not only spread over long-distance land transportation, but also widely used in medium and short-distance urban public transportation. In rail transit, the distribution of rails is complex and changeable. When a train travels to a turnout section, it is necessary to accurately predict which rail the train will turn into, that is, to predict the drivable area. The existing technical solutions are divided into two stages. First, the track is segmented, and on this basis, the turnout is detected. According to the turnout information, the train drivable area is predicted. The turnout detection highly depends on the result of track segmentation. Since it is difficult to accurately segment the track, it is difficult to solve the problem of rail segmentation adhesion at the turnout, which brings certain difficulties to the analysis of turnout information. Therefore, it brings certain difficulties to predicting the drivable area. Summary of the Invention

[0003] The embodiments of the present invention provide a system and method for predicting the train drivable area based on rail instance segmentation. Based on deep learning, the data acquisition module labels the current track as the same instance, and the turnout information is implicitly included in the labeling stage. Using a large number of labeled pictures, the rail instance segmentation model can learn the implicit turnout information in an end-to-end manner, which not only reduces the calculation amount but also improves the performance, and solves the problems existing in the current train operation track prediction technology that the turnout detection highly depends on the result of track segmentation, it is difficult to solve the problem of rail segmentation adhesion at the turnout, which brings certain difficulties to the analysis of turnout information, and the low accuracy of predicting the drivable area.

[0004] A system for predicting the train drivable area based on rail instance segmentation includes: a data acquisition module and a rail instance segmentation model;

[0005] The data acquisition module is used to acquire track image information, label the image, and transmit the data to the rail instance segmentation model;

[0006] The rail instance segmentation model is used to receive the data transmitted by the data acquisition module and analyze the data to predict the drivable area of the track;

[0007] Further, the data acquisition module includes a collection unit and a labeling unit. The collection unit is used to collect images of the track, and the labeling unit is used to label the key points on the edge of the track in the images of the track collected by the collection unit and connect these key points to form a complete contour instance of the track, forming a closed polygon, and different instances are assigned different IDs;

[0008] Further, obtaining the image information of the track includes the image of the track and the turnout information of the track.

[0009] Further, the railway track instance segmentation model includes an image binary segmentation module, a track pixel embedding module, a clustering module, and a detection and prediction module. The image binary segmentation module is used to separate the background from the lane lines to obtain lane line pixel points. The track pixel embedding module is used to obtain the distance vectors between the image pixel points. The clustering module is used to match all the obtained image pixel points with the instance IDs. The detection and prediction module is used to detect the lane line instances and predict the drivable area.

[0010] Further, the track pixel embedding module is used to train different pixel vectors for each pixel point based on the method of distance metric learning;

[0011]

[0012] Where, in the formula L var is the variance loss, and L dist is the distance loss; C is the cluster, that is, the number of tracks; c is the number of cluster centers; cA, cB represent different tracks; Nc is the number of track pixels; x i is the pixel coordinate of the track in the original image; μ c is the average value of the cluster center coordinates, and μ cA , μ cB correspond to the pixel point coordinates of the cluster centers of different tracks; δ v , δ d are the set variance and distance thresholds; the variance loss makes x i approach the mean value μ c of the track, and the distance loss will make the cluster centers of each track move away from each other.

[0013] In a second aspect, an embodiment of the present invention provides a method for predicting the drivable area of a train based on railway track instance segmentation, including the following steps:

[0014] S1, acquisition and annotation. The acquisition unit acquires an image of the track, and the annotation unit labels the key points on the edge of the track in the image of the track acquired by the acquisition unit and connects these key points to form a complete contour instance of the track, forming a closed polygon, assigns different IDs to different instances, and inputs the instance data into the railway track instance segmentation model;

[0015] S2, Analyze and predict. Input the track image into the binary segmentation module to separate the background from the lane lines and obtain the binary image of the lane line pixel points. Input the obtained binary image into the pixel embedding module to get the distance vector between the image pixel points. The clustering module uses the DBSCAN algorithm to cluster the obtained distance vector between the pixel points, and the output result is the instance ID corresponding to each pixel in the image, forming an instance segmentation map. After the detection and prediction module detects different lane line instances, it uses polynomial fitting for the pixel points to parameterize each track. Since the acquired image of the railway track is relatively special and the current tracks are basically in the same position at the bottom of the image, the current left and right tracks can be determined, and the drivable area can be determined according to the instance segmentation result.

[0016] The beneficial effects of the above technical solutions provided by the embodiments of the present invention at least include: Based on deep learning, the present invention collects and labels the image data of the track through the data acquisition module, labels the current track as the same instance, and the turnout information is implicitly included in the labeling stage. Using a large number of labeled pictures, the obtained instances are input into the railway track instance segmentation model to learn the implicitly included turnout information in an end-to-end manner, which not only reduces the calculation amount but also improves the performance, and eliminates the dependence of the prediction result on the railway track segmentation result, especially the limitation of the turnout section result. Based on the end-to-end method, the drivable area is directly predicted, and the accuracy is improved through a large number of labeled pictures.

[0017] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in the written specification, claims, and drawings.

[0018] The technical solutions of the present invention will be further described in detail below through the drawings and embodiments. Description of the Drawings

[0019] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0020] Figure 1 It is a schematic diagram of the track structure disclosed in the embodiment of the present invention;

[0021] Figure 2 It is a schematic diagram of the instance segmentation structure disclosed in the embodiment of the present invention;

[0022] Figure 3 It is the mask image corresponding to different tracks disclosed in the embodiment of the present invention;

[0023] Figure 4 The effect diagram of instance segmentation disclosed in the embodiment of the present invention;

[0024] Figure 5 The schematic structural diagram of the system for predicting the train drivable area based on rail instance segmentation disclosed in the embodiment of the present invention;

[0025] Figure 6 The schematic flow diagram of the method for predicting the train drivable area based on rail instance segmentation disclosed in the embodiment of the present invention.

[0026] Reference numerals:

[0027] 1 - Data acquisition module; 101 - Acquisition unit; 102 - Annotation unit; 2 - Rail instance segmentation model; 201 - Image binary segmentation module; 202 - Track pixel embedding module; 203 - Clustering module; 204 - Detection and prediction module. Specific embodiments

[0028] Hereinafter, exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0029] Embodiment 1

[0030] As Figures 1-5 shown, the embodiment of the present invention provides a system for predicting the train drivable area based on rail instance segmentation, including: a data acquisition module 1 and a rail instance segmentation model 2;

[0031] The data acquisition module 1 is used to acquire track image information, annotate the image, and transmit the data to the rail instance segmentation model 2. The data acquisition module 1 includes an acquisition unit 101 and an annotation unit 102. The acquisition unit 101 is used to acquire the image of the track, and the acquired track image information includes the image of the track and the turnout information of the track. The annotation unit 102 is used to annotate the key points on the edge of the track in the image acquired by the acquisition unit 101, connect these key points, form the contour instance of the complete track, form a closed polygon, and assign different IDs to different instances;

[0032] As Figures 1-4 shown, specifically:

[0033] (1) Select key points along the edge of the track and connect these points until the contour of the track is outlined, forming a closed polygon, and different tracks, that is, different instances, are given different IDs. As Figure 1As shown, the tracks from left to right are the first instance, the second instance, the third instance, and the fourth instance respectively;

[0034] (2)Save the coordinates of the key points in a json file by ID, classify them, and export;

[0035] (3)Generate a mask image for each track instance according to the point coordinates in the json file by ID. For example, Figure 1 There are four tracks in it, that is, four instances, then the image needs to be input into the model together with the Figure 2 four mask images in it for training. In the mask image, the pixel values of the track area are set to 255, and the pixel values of other background areas are all 0. Name the mask images according to the ID of the instance.

[0036] The railway track instance segmentation model 2 is used to receive the data transmitted by the data acquisition module 1 and analyze and predict the drivable area of the track. The railway track instance segmentation model 2 includes an image binary segmentation module 201, a track pixel embedding module 202, a clustering module 203, and a detection and prediction module 204. The image binary segmentation module 201 is used to separate the background from the lane lines to obtain lane line pixel points. The track pixel embedding module 202 is used to obtain the distance vector between image pixel points. The clustering module 203 is used to match all the obtained image pixel points with the instance ID. The detection and prediction module 204 is used to detect lane line instances and predict the drivable area;

[0037] Specifically:

[0038] (1)Load the original image after scaling into the trained binary segmentation module. The output result is a binary image in which the track pixels are separated from the background pixels. In the model training stage, because the pixel data distribution of the tracks and the background in the image is uneven, the standard cross-entropy loss function is used to train the binary segmentation module, which can update the network parameter values more quickly;

[0039] (2)Input the binary image obtained in step (1) into the pixel embedding module. The output is the distance vector between each pixel point in the picture. In the model training stage, in order to distinguish which track the obtained pixel points belong to, a method based on distance metric learning is used to train different pixel vectors for each pixel point;

[0040]

[0041] Among them, in the formula, L var is the variance loss, and L dist is the distance loss; C is the cluster, that is, the number of tracks; c is the number of cluster centers; cA and cB represent different tracks; Nc is the number of track pixels; x iare the pixel coordinates of the tracks in the original image; μ c is the average value of the cluster center coordinates, μ cA , μ cB are the pixel coordinates of the cluster centers corresponding to different tracks; δ v , δ d is the set variance and distance threshold; the variance loss causes x i to approach the mean μ c of the track, and the distance loss causes the cluster centers of each track to move away from each other;

[0042] The variance loss causes x i to approach the mean μc of the track, and the distance loss causes the cluster centers of each track to move away from each other;

[0043] (3) Based on the loss function obtained in step (2), cluster the obtained distance vectors between pixel points using the DBSCAN algorithm. The output result is the instance id corresponding to each pixel in the image, that is, the instance segmentation map:

[0044] a. Given a distance threshold and a minimum number of samples;

[0045] b. If the number of pixel points around a certain pixel point that satisfy the distance threshold is greater than the minimum number of samples, then this point is called a core object. Traverse all pixel points to find the set of core objects;

[0046] c. Select a core object as the cluster center, find all density-reachable pixel points and generate a clustering cluster;

[0047] d. Remove the found density-reachable samples from the remaining core objects;

[0048] e. Repeat steps c and d from the updated core objects until all core objects have been traversed; that is, all pixels are assigned to the corresponding tracks.

[0049] (4) When the detection and prediction module 204 detects different lane line instances, use polynomial fitting for pixel points to parameterize each track. Because the collected images of the railway tracks are relatively special, the current tracks are basically in the same position at the bottom of the image, so the current left and right tracks can be determined, and the drivable area can be determined according to the instance segmentation result.

[0050] The present invention overcomes the problem that the prior art highly depends on the track segmentation result, resulting in the difficulty in solving the problem of rail segmentation adhesion at turnouts due to the difficulty in accurately segmenting the tracks. Based on deep learning, the data acquisition module 1 collects and annotates the image data of the tracks, annotates the current track as the same instance, and the turnout information is implicitly included in the annotation stage. By using a large number of annotated pictures, the obtained instances are input into the rail instance segmentation model 2 to learn the implicitly included turnout information in an end-to-end manner, which not only reduces the computational amount but also improves the performance, and eliminates the limitation that the prediction result depends on the rail segmentation result, especially the result of the turnout section. Based on the end-to-end method, the drivable area is directly predicted, and the accuracy is improved through a large number of annotated pictures.

[0051] Embodiment 2

[0052] The embodiment of the present invention also discloses a method for predicting the drivable area of a train based on rail instance segmentation, as Figures 1-6 , including the following steps:

[0053] S1, acquisition and annotation: The acquisition unit 101 acquires the image of the track, and the annotation unit 102 annotates the key points on the edge of the track in the image acquired by the acquisition unit 101 and connects these key points to form the contour instance of the complete track, forming a closed polygon. Different instances are given different IDs and the instance data is input into the rail instance segmentation model 2;

[0054] Specifically, the acquisition unit 101 acquires the image data of the track, the annotation unit 102 annotates the acquired image, selects key points along the edge of the track, and connects these points until the contour of the track is outlined, forming a closed polygon. Different tracks, that is, different instances, are given different IDs, and a mask image is generated for each track instance according to the ID;

[0055] S2, analysis and prediction: The track image is input into the binary segmentation module to separate the background from the lane lines, and a binary image of the lane line pixel points is obtained; the obtained binary image is input into the pixel embedding module to obtain the distance vector between the image pixel points. The clustering module 203 clusters the obtained distance vector between the pixel points by using the DBSCAN algorithm, and the output result is the instance ID corresponding to each pixel in the image, forming an instance segmentation map; after the detection prediction module 204 detects different lane line instances, it uses polynomial fitting for the pixel points to parameterize each track. Because the acquired image of the rail is relatively special, the current track is basically at the same position at the bottom of the image, and the current left and right tracks can be determined. The drivable area is determined according to the instance segmentation result;

[0056] Specifically, in the model training stage, the binary segmentation module uses the standard cross entropy loss function to train the binary segmentation module because the pixel data of the track and background in the image are unevenly distributed. The original image is scaled and loaded into the trained binary segmentation module. The output result is a binary image that separates the track pixels from the background pixels; the binary image is input into the pixel embedding module, and the output is the distance vector between each pixel in the image; the clustering module 203 clusters the obtained distance vector between pixels using the DBSCAN algorithm. The output result is the instance ID corresponding to each pixel in the image, that is, the instance segmentation map. When the detection prediction module 204 detects different lane line instances, it uses polynomial fitting pixels to parameterize each track. Because the collected image of the railroad track is relatively special, the current track is basically in the same position at the bottom of the image, so the current left and right tracks can be determined, and then the drivable area can be determined based on the instance segmentation result.

[0057] The method for predicting the train drivable area based on track instance segmentation disclosed in the present embodiment is based on deep learning. The image data of the track is collected and labeled by the data acquisition module 1, and the current track is labeled as the same instance. The labeling stage includes turnout information. A large number of labeled pictures are used, and the obtained instances are input into the track instance segmentation model 2 to learn the implicit turnout information in an end-to-end manner, which not only reduces the amount of calculation but also improves the performance. It eliminates the limitation that the prediction result depends on the track segmentation result, especially the turnout section result, and directly predicts the drivable area based on an end-to-end manner, thereby improving the accuracy through a large number of labeled pictures.

[0058] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0059] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0060] Those skilled in the art should also understand that all the illustrative logical blocks, modules, circuits, and algorithmic steps described in conjunction with the embodiments herein can be implemented as electronic hardware, computer software, or a combination thereof. To clearly illustrate the interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps have been generally described in terms of their functions above. Whether such a function is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Skilled technicians can implement the described functions in a flexible manner for each specific application. However, such implementation decisions should not be construed as departing from the scope of protection of this disclosure.

[0061] The steps of the methods or algorithms described in conjunction with the embodiments herein can be directly embodied as hardware, software modules executed by a processor, or a combination thereof. The software modules can be located in a RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the art. An exemplary storage medium is connected to the processor so that the processor can read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. The ASIC can be located in a user terminal. Of course, the processor and the storage medium can also exist as discrete components in the user terminal.

[0062] For software implementation, the technologies described in this application can be implemented using modules (e.g., procedures, functions, etc.) that execute the functions described in this application. These software codes can be stored in a memory unit and executed by a processor. The memory unit can be implemented inside the processor or outside the processor. In the latter case, it is communicatively coupled to the processor by various means, which are well-known in the art.

[0063] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments. However, those of ordinary skill in the art should recognize that the various embodiments can be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, this term is covered in a manner similar to the term "including" as interpreted when "including" is used as a transitional word in the claims. In addition, any term "or" used in the claims or the specification is intended to mean "non-exclusive or".

Claims

1. A system for predicting the train's drivable area based on rail instance segmentation, characterized in that, Including: A data acquisition module and a railway track instance segmentation model; The data acquisition module is used to acquire track image information, annotate the image, and transmit the data to the railway track instance segmentation model; The data acquisition module includes a collection unit and an annotation unit. The collection unit is used to collect images of the track, and the annotation unit is used to annotate the key points on the edge of the track in the image collected by the collection unit and connect these key points to form a complete contour instance of the track, forming a closed polygon, and assigning different IDs to different instances; Among them, acquiring the track image information includes the image of the track and the turnout information of the track; The railway track instance segmentation model is used to receive the data transmitted by the data acquisition module and analyze and predict the drivable area of the track; The railway track instance segmentation model includes an image binary segmentation module, a track pixel embedding module, a clustering module, and a detection and prediction module. The image binary segmentation module is used to separate the background from the lane lines to obtain lane line pixel points. The track pixel embedding module is used to obtain the distance vector between image pixel points. The clustering module is used to match all the obtained image pixel points with the instance IDs. The detection and prediction module is used to detect lane line instances and predict the drivable area; Among them, the track pixel embedding module is used to train different pixel vectors for each pixel point based on the method of distance metric learning; ； Among them, in the formula, L var is the variance loss, and L dist is the distance loss; C is the cluster, that is, the number of orbits; c is the number of cluster centers; cA and cB represent different orbits; Nc is the number of orbit pixels; x i is the pixel coordinate of the orbit in the original image; μ c is the average value of the cluster center coordinates, μ cA , μ cB corresponds to the pixel point coordinates of the cluster centers of different orbits; δ v , δ d are the set variance and distance thresholds; the variance loss makes x i approach the mean value μ c of the orbit, and the distance loss will make the cluster centers of each orbit move away from each other.

2. A prediction method applying the system for predicting the train's drivable area based on rail instance segmentation as claimed in claim 1, characterized in that: Including the following steps: S1. Acquisition and annotation: The collection unit collects images of the track, and the annotation unit annotates the key points on the edge of the track in the image collected by the collection unit and connects these key points to form a complete contour instance of the track, forming a closed polygon, assigning different IDs to different instances, and inputting the instance data into the railway track instance segmentation model; S2. Analysis and prediction: Input the track image into the binary segmentation module to separate the background from the lane lines to obtain a binary image of lane line pixel points; input the obtained binary image into the pixel embedding module to obtain the distance vector between image pixel points. The clustering module uses the DBSCAN algorithm to cluster the obtained distance vectors between pixel points, and the output result is the instance ID corresponding to each pixel in the image, forming an instance segmentation map; after the detection and prediction module detects different lane line instances, it uses polynomial fitting to fit the pixel points, parameterizes each track, and determines the drivable area according to the instance segmentation result.

Images