Track Line Detection Method and System Based on Dynamic Anchor

Abstract

This invention discloses a track detection method based on dynamic line anchors, comprising: collecting images containing track targets and track instance annotations, and preprocessing them to obtain a dataset composed of original image tensors and mask image tensors; training a track detection model using an end-to-end deep neural network structure based on dynamic line anchors and track contrastive learning based on the obtained dataset; inputting the image to be detected into the trained track detection model, and outputting the track detection result. This invention effectively extracts track features by combining dynamic line anchors and track contrastive learning, improving the accuracy of track detection and significantly enhancing the accuracy of detecting complex tracks in real-world environments, which is more beneficial for the future development of the autonomous driving field.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method and system for track line detection based on dynamic line anchors. Background Technology

[0002] In recent years, intelligent driving technology has continued to develop, greatly improving convenience and safety. Among these advancements, intelligent trains employ advanced sensors and computer vision algorithms to assist or replace train drivers in perceiving and analyzing the train's operating environment, effectively reducing the risk of accidents caused by human error. Track detection, as a component of intelligent driving, is a necessary and sufficient condition for many subsequent tasks and possesses significant research value.

[0003] Traditional image-processing-based track detection methods typically rely on the edge and linear features of the track. While these methods achieve good results under simple conditions, they lack robustness in practical applications. With the emergence and development of deep neural networks, their ability to learn both low-level and high-level semantic information, coupled with their fast inference speed and robustness, has made them a popular choice for railway detection.

[0004] In recent years, many innovative methods have emerged in lane detection research, similar to track detection, based on deep neural networks. Broadly speaking, these methods can be categorized into segmentation-based methods, anchor-based methods, keypoint-based methods, and parameter prediction methods. Among these methods, due to the good compatibility between anchors and track lines, the focus of track detection research can be placed on anchors. Specifically, an anchor is a thin ray determined by its starting point and slope. By placing a large number of predefined anchors as references at the image boundaries, feature pooling is used to obtain the region of interest features corresponding to the anchors, predicting the offset between the trajectory and the anchor line. Finally, the trajectory instance is determined based on whether the anchor is selected and the offset.

[0005] However, this also brings some problems. Too many anchors mean only a few are actually active during inference, leading to a waste of computational and memory resources. Conversely, too few anchors may fail to cover potential locations on the track, significantly reducing detection results. Summary of the Invention

[0006] The present invention aims to at least partially solve one of the technical problems existing in the related art.

[0007] One objective of this invention is to provide a track line detection method based on dynamic line anchors. By combining dynamic line anchors and track comparative learning, track line features are effectively extracted, improving the accuracy of network model detection and providing an effective track line detection solution for safe and efficient automatic train operation.

[0008] Another object of the present invention is to provide a track line detection system based on dynamic line anchors.

[0009] To achieve the above objectives, the present invention provides a track line detection method based on dynamic anchors, comprising:

[0010] Collect images containing orbital targets and track line instance annotations, and preprocess them to obtain a dataset consisting of the original image tensor and the mask image tensor;

[0011] Based on the acquired dataset, an end-to-end deep neural network structure based on dynamic anchor and track contrast learning was used to train and obtain a track line detection model.

[0012] The image to be detected is input into the trained track line detection model, and the track line detection result is output.

[0013] A further preferred technical solution of the present invention is that the set includes images of orbital targets and annotations of orbital line instances, and is preprocessed to obtain a dataset composed of original image tensors and mask image tensors; including:

[0014] Track images and track line instance annotations were selected from a multi-scene track dataset that includes tunnels, cities, strong light, and severe weather.

[0015] Perform scaling, normalization, image enhancement, and masking operations on track images and track line instance annotations;

[0016] The dataset consists of the original image tensor and the mask image tensor, and is divided into training set, test set and validation set.

[0017] Preferably, the step of training a track line detection model using an end-to-end deep neural network structure based on dynamic line anchors and track comparison learning, according to the acquired dataset, includes:

[0018] Construct an end-to-end deep neural network structure based on dynamic line anchor and track contrast learning;

[0019] The original image tensor and the mask image tensor are fed into the deep neural network for training. The weights of the deep neural network are adjusted to obtain the trained trajectory line detection model.

[0020] The trained trajectory detection model is tested and validated, and its hyperparameters are adjusted.

[0021] Preferably, the constructed end-to-end deep neural network structure based on dynamic line anchor and track comparison learning includes a track comparison learning module, a layer aggregation module, and a line anchor self-assignment module.

[0022] The orbital comparison learning module consists of a main encoder and a momentum encoder. The main encoder is used for high-level semantic feature extraction, which includes the shape, texture and structure of the object. The momentum encoder is used to generate stable and consistent feature representations and slowly update the encoder parameters through a momentum update mechanism.

[0023] The layer aggregation module consists of multiple sub-aggregation modules. Each sub-aggregation module performs pooling, segmentation, and concatenation operations on the features extracted from each layer of the backbone network in the main encoder, performs proportional segmentation of the features, and then fuses these segmented features along the channel dimension.

[0024] The line anchor self-assignment module is used to sample and pool the fused features to generate sampled feature maps and line anchor feature maps.

[0025] Preferably, the training method for the end-to-end deep neural network structure based on dynamic line anchor and track contrast learning includes:

[0026] Features of the original image tensor and the mask image tensor are obtained through forward inference, and the error between the two features is obtained.

[0027] The obtained error is used for backpropagation to update the weights of the deep neural network.

[0028] Preferably, the method involves obtaining features of the original image tensor and the mask image tensor through forward inference, and then obtaining the error between the two features; specifically, the method is as follows:

[0029] The original image tensor and the mask image tensor are respectively input into the main encoder and momentum encoder of the track contrast learning module to extract features and obtain the basic features of the two images.

[0030] The basic features of the two obtained images are compared and learned. Feature representations are learned by maximizing the similarity between similar samples and minimizing the similarity between different samples, while the comparison error value is calculated.

[0031] The features extracted from each layer by the main encoder are input to the aggregation module for feature fusion. By weighting or concatenating the features from different layers, a more representative feature representation is generated.

[0032] The fused features are sampled and pooled into feature maps and line anchor feature maps. The feature maps predict the approximate location of track line instances in the image and calculate the classification error value. The line anchor feature maps calculate the bias between the track line instances and the predicted track lines and calculate the regression error value.

[0033] Preferably, the total error during training is a weighted sum of the comparison error, classification error, and regression error. This total error is used for backpropagation to update the weights of the deep neural network.

[0034] Another aspect of the present invention provides a track line detection system based on dynamic line anchors, comprising:

[0035] The data collection module is used to collect images containing orbital targets and orbital line instance annotations, and to preprocess them to obtain a dataset consisting of the original image tensor and the mask image tensor.

[0036] The model building module is used to train an end-to-end deep neural network structure based on dynamic line anchors and track comparison learning to obtain a track line detection model based on the acquired dataset.

[0037] The track line detection module is used to input the image to be detected into the trained track line detection model and output the track line detection results.

[0038] In another aspect, the present invention provides a non-transitory computer-readable storage medium storing computer instructions thereon, which instruct a computer to execute the above-described track line detection method based on dynamic anchors to perform track line detection.

[0039] In another aspect, the present invention provides an electronic device, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus, and the processor calls logical instructions in the memory to execute the above-described track line detection method based on dynamic line anchors to perform track line detection.

[0040] In another aspect, the present invention provides a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer executes the above-described track line detection method based on dynamic line anchors to perform track line detection.

[0041] Beneficial Effects: This invention constructs a track detection model based on dynamic anchors for track detection. This model achieves self-assignment of anchors by adaptively adjusting anchor parameters multiple times, thereby improving detection performance. Specifically, this invention designs a layer aggregation module to align and aggregate multi-scale semantic features extracted from the backbone network, establishing global long-range dependencies. Then, the self-assignment module adaptively adjusts anchor parameters multiple times based on the aggregated features to assign anchors. These assigned anchors are used to accurately match track instances.

[0042] Furthermore, to handle low visibility and occlusion scenarios, this invention proposes a track comparison learning method. The track comparison learning module combines locally extracted features with globally captured cross-similar features to achieve accurate railway detection.

[0043] In summary, this invention effectively extracts track features by combining dynamic line anchors and track comparison learning, thereby improving the accuracy of track detection. This significantly enhances the accuracy of detecting complex tracks in real-world environments and is more conducive to the future development of autonomous driving. Attached Figure Description

[0044] Figure 1 This is an overall flowchart of the track line detection model during the training phase in an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, embodiments of this invention, and should not be construed as limiting the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. In the description of this invention, it should be understood that the terminology used is for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0046] The following is combined with Figure 1 This invention describes the track line detection method and system based on dynamic line anchors provided by the present invention.

[0047] Example 1: This example provides a track line detection method based on dynamic line anchors, including preprocessing an image containing track targets and track line instance annotations to obtain an original image tensor and a mask image tensor; constructing an end-to-end deep neural network model based on dynamic line anchors and track comparison learning; feeding the original image tensor and the mask image tensor into the deep neural network model for training to obtain the trained model weights; and inputting a new image into the trained network model to obtain the track line detection result.

[0048] The method provided in this embodiment effectively extracts track line features by combining dynamic line anchors and track contrast learning, thereby improving the accuracy of network model detection and providing an effective track line detection solution for safe and efficient automatic train operation.

[0049] Specifically, the implementation steps of this embodiment are as follows:

[0050] S1. Collect images containing orbital targets and orbital line instance annotations, and preprocess them to obtain a dataset consisting of the original image tensor and the mask image tensor; including:

[0051] S11. Select and annotate track images and track line instances from a multi-scene track dataset that includes tunnels, cities, strong light, and severe weather.

[0052] S12. Perform scaling, normalization, image enhancement, and masking operations on the track image and track line instance annotations.

[0053] S13. Obtain the dataset consisting of the original image tensor and the mask image tensor, and divide the dataset into training set, test set and validation set.

[0054] S2. Construct an end-to-end deep neural network structure based on dynamic line anchors and track contrast learning. This deep neural network structure includes: a track contrast learning module, a layer aggregation module, and a line anchor self-assignment module;

[0055] The orbital contrast learning module consists of a main encoder and a momentum encoder. The main encoder is used to extract high-level semantic features, including the shape, texture, and structure of the object. The momentum encoder is used to generate more stable and consistent feature representations. It uses a momentum update mechanism to slowly update the encoder parameters, thereby avoiding drastic changes in feature representations.

[0056] The layer aggregation module consists of multiple sub-aggregation modules. Each sub-aggregation module performs pooling, segmentation, and concatenation operations on the features extracted from each layer of the backbone network in the main encoder. Semantic information is extracted from the feature maps, the features are proportionally segmented, and then these segmented features are concatenated along the channel dimension. To improve the interactivity between different segments, segment shuffling and grouped convolution are employed. Finally, the feature maps output from the two processes are fused together through addition, effectively reducing the sampling of the feature maps while preserving scale information.

[0057] The line anchor self-assignment module samples and pools the fused features to generate a sampled feature map and a line anchor feature map. After establishing a long-term horizontal dependency between the line anchor feature map and the feature map, this information is forwarded through a fully connected layer to predict the adjustment offset of the line anchor. Utilizing the positional information of the line anchor, each detection instance can retrieve line anchor features from the feature map through region-of-interest alignment. Through multiple adjustments, the fit between the line anchor and the track instance is improved, thereby enhancing the accuracy of track detection.

[0058] S3. The original image tensor and the mask image tensor from step S1 are fed into the deep neural network constructed in step S2 for training to obtain the trained model weights. This step includes forward inference, calculation of error values, and backpropagation to update the weights. Specifically:

[0059] S31. Input the original image tensor and mask image tensor from step S1 into the feature extraction network to obtain basic feature information containing orbital line features and positional information;

[0060] S32. Based on the two features extracted in step S31, comparative learning is performed to generate a second view of the comparative learning method by masking the input image. To reveal the local features of the masked patches, a cross-similarity operation is performed, utilizing the similarity between surrounding features and original image features. The cross-similarity between each patch and feature map is stored in a separate tensor, which is then concatenated in their respective order. This not only identifies the similarity between each patch on one feature map and all patches on another feature map but also preserves their positional information. Therefore, the comparative loss can reflect the relationship between the positional values ​​and similarity values ​​of the neural network.

[0061] S33. The input layer aggregation module performs feature fusion on the features extracted by the encoder in step S31. Starting with pooling, semantic information is extracted from the feature maps, and then the features are proportionally segmented. These segmented features are then concatenated along the channel dimension. To improve the interactivity between different segments, segment shuffling and group convolution are used. Finally, the feature maps output from the two processes are fused together through addition, effectively reducing the sampling of feature maps while preserving scale information.

[0062] S34. The features fused in step S33 are sampled and pooled to generate sampled feature maps and line anchor feature maps. Following this, a fully connected layer processes the sampled feature maps to extract initial line anchor attributes. The sampled features of the line anchor serve as a similarity evaluation matrix, and the sampled feature maps are projected as values ​​into the respective feature spaces. After establishing long-term horizontal dependencies between the line anchor feature maps and the feature maps, this information is forwarded through the fully connected layer to predict the adjusted offset of the line anchor. Utilizing the positional information of the line anchors, each detection instance can retrieve line anchor features from the feature map through region-of-interest alignment. Through multiple adjustments, the fit between the line anchors and track instances is improved, thereby enhancing the accuracy of track detection.

[0063] S35. Calculate the total error. The total error value is obtained by weighted summation of the comparison error value, classification error value, and regression error value. Then, backpropagate based on the obtained error to update the weights of the deep neural network.

[0064] S36. Test and validate the trained trajectory detection model using the test set and validation set, and adjust the model's hyperparameters.

[0065] S4. Input the image to be detected into the trained track line detection model and output the track line detection result.

[0066] This embodiment's method designs a layer aggregation module to align and aggregate multi-scale semantic features extracted from the backbone network, establishing global long-range dependencies. Then, a self-assignment module adaptively adjusts the anchor parameters multiple times based on the aggregated features to assign anchors. These assigned anchors are used to accurately match track instances. Furthermore, to handle low-visibility and occluded scenarios, this embodiment proposes a track contrastive learning method, combining locally extracted features with globally captured cross-similar features to achieve accurate railway detection.

[0067] Example 2: This example provides a track line detection system based on dynamic anchors, including:

[0068] The data collection module is used to collect images containing orbital targets and orbital line instance annotations, and to preprocess them to obtain a dataset consisting of the original image tensor and the mask image tensor; corresponding to step S1 in embodiment 1.

[0069] The model building module is used to train and obtain a track line detection model based on the acquired dataset using an end-to-end deep neural network structure based on dynamic line anchors and track comparison learning; corresponding to steps S2-S3 in Example 1.

[0070] The track line detection module is used to input the image to be detected into the trained track line detection model and output the track line detection result; corresponding to step S4 in embodiment 1.

[0071] Example 3: This example provides a non-transitory computer-readable storage medium storing computer instructions that cause a computer to execute the track line detection method based on dynamic line anchors in Example 1 to perform track line detection. The method includes:

[0072] Collect images containing orbital targets and track line instance annotations, and preprocess them to obtain a dataset consisting of the original image tensor and the mask image tensor;

[0073] Based on the acquired dataset, an end-to-end deep neural network structure based on dynamic anchor and track contrast learning was used to train and obtain a track line detection model.

[0074] The image to be detected is input into the trained track line detection model, and the track line detection result is output.

[0075] Example 4: This example provides an electronic device that may include a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The processor can call logical instructions from the memory to execute the track line detection method based on dynamic anchors described in Example 1, performing track line detection. This method includes:

[0076] Collect images containing orbital targets and track line instance annotations, and preprocess them to obtain a dataset consisting of the original image tensor and the mask image tensor;

[0077] Based on the acquired dataset, an end-to-end deep neural network structure based on dynamic anchor and track contrast learning was used to train and obtain a track line detection model.

[0078] The image to be detected is input into the trained track line detection model, and the track line detection result is output.

[0079] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0080] Example 5: This example provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the track line detection method based on dynamic line anchors in Example 1 to perform track line detection. The method includes:

[0081] Collect images containing orbital targets and track line instance annotations, and preprocess them to obtain a dataset consisting of the original image tensor and the mask image tensor;

[0082] Based on the acquired dataset, an end-to-end deep neural network structure based on dynamic anchor and track contrast learning was used to train and obtain a track line detection model.

[0083] The image to be detected is input into the trained track line detection model, and the track line detection result is output.

[0084] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0085] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A track line detection method based on dynamic anchors, characterized in that, include: Collect images containing orbital targets and track line instance annotations, and preprocess them to obtain a dataset consisting of the original image tensor and the mask image tensor; Based on the acquired dataset, an end-to-end deep neural network structure based on dynamic line anchors and track contrast learning is used to train and obtain a track line detection model; the end-to-end deep neural network structure based on dynamic line anchors and track contrast learning includes a track contrast learning module, a layer aggregation module, and a line anchor self-assignment module. The orbital comparison learning module consists of a main encoder and a momentum encoder. The main encoder is used for high-level semantic feature extraction, which includes the shape, texture and structure of the object. The momentum encoder is used to generate stable and consistent feature representations and slowly update the encoder parameters through a momentum update mechanism. The layer aggregation module consists of multiple sub-aggregation modules. Each sub-aggregation module performs pooling, segmentation, and concatenation operations on the features extracted from each layer of the backbone network in the main encoder, extracts semantic information from the feature map, performs proportional segmentation of the features, and then fuses these segmented features along the channel dimension. The anchor self-allocation module adaptively adjusts the anchor parameters multiple times based on the aggregated features to allocate anchors; the anchor self-allocation module is used to sample and pool the fused features to generate sampled feature maps and anchor feature maps. The image to be detected is input into the trained track line detection model, and the track line detection result is output.

2. The track line detection method based on dynamic anchors according to claim 1, characterized in that, The set contains images of orbital targets and annotations of orbital line instances, which are preprocessed to obtain a dataset consisting of the original image tensor and the mask image tensor; including: Track images and track line instance annotations were selected from a multi-scene track dataset that includes tunnels, cities, strong light, and severe weather. Perform scaling, normalization, image enhancement, and masking operations on track images and track line instance annotations; The dataset consists of the original image tensor and the mask image tensor, and is divided into training set, test set and validation set.

3. The track line detection method based on dynamic anchors according to claim 1, characterized in that, Based on the acquired dataset, an end-to-end deep neural network structure based on dynamic anchor and track comparison learning is used to train and obtain a track line detection model. include: Construct an end-to-end deep neural network structure based on dynamic line anchor and track contrast learning; The original image tensor and the mask image tensor are fed into the deep neural network for training. The weights of the deep neural network are adjusted to obtain the trained trajectory line detection model. The trained trajectory detection model is tested and validated, and its hyperparameters are adjusted.

4. The track line detection method based on dynamic anchors according to claim 3, characterized in that, The training method for the end-to-end deep neural network structure based on dynamic anchor and track contrast learning includes: Features of the original image tensor and the mask image tensor are obtained through forward inference, and the error between the two features is obtained. The obtained error is used for backpropagation to update the weights of the deep neural network.

5. The track line detection method based on dynamic anchors according to claim 4, characterized in that, The method involves obtaining features of the original image tensor and the mask image tensor through forward inference, and then obtaining the error between the two features; the specific method is as follows: The original image tensor and the mask image tensor are respectively input into the main encoder and momentum encoder of the track contrast learning module to extract features and obtain the basic features of the two images. The basic features of the two obtained images are compared and learned. Feature representations are learned by maximizing the similarity between similar samples and minimizing the similarity between different samples, while the comparison error value is calculated. The features extracted from each layer by the main encoder are input to the aggregation module for feature fusion. By weighting or concatenating the features from different layers, a more representative feature representation is generated. The fused features are sampled and pooled into feature maps and line anchor feature maps. The feature maps predict the approximate location of track line instances in the image and calculate the classification error value. The line anchor feature maps calculate the bias between the track line instances and the predicted track lines and calculate the regression error value.

6. The track line detection method based on dynamic anchors according to claim 5, characterized in that, The total error during training is a weighted sum of the contrast error, classification error, and regression error. This total error is used for backpropagation to update the weights of the deep neural network.

7. A track line detection system based on dynamic line anchors, characterized in that, include: The data collection module is used to collect images containing orbital targets and orbital line instance annotations, and to preprocess them to obtain a dataset consisting of the original image tensor and the mask image tensor. The model building module is used to train a track detection model based on the acquired dataset using an end-to-end deep neural network structure based on dynamic line anchors and track comparison learning; the end-to-end deep neural network structure based on dynamic line anchors and track comparison learning includes a track comparison learning module, a layer aggregation module, and a line anchor self-assignment module. The orbital comparison learning module consists of a main encoder and a momentum encoder. The main encoder is used for high-level semantic feature extraction, which includes the shape, texture and structure of the object. The momentum encoder is used to generate stable and consistent feature representations and slowly update the encoder parameters through a momentum update mechanism. The layer aggregation module consists of multiple sub-aggregation modules. Each sub-aggregation module performs pooling, segmentation, and concatenation operations on the features extracted from each layer of the backbone network in the main encoder, extracts semantic information from the feature map, performs proportional segmentation of the features, and then fuses these segmented features along the channel dimension. The anchor self-allocation module adaptively adjusts the anchor parameters multiple times based on the aggregated features to allocate anchors; the anchor self-allocation module is used to sample and pool the fused features to generate sampled feature maps and anchor feature maps. The track line detection module is used to input the image to be detected into the trained track line detection model and output the track line detection results.

8. A non-transitory computer-readable storage medium storing computer instructions thereon, the computer instructions causing a computer to execute the track line detection method based on dynamic line anchors as described in any one of claims 1-6, and to perform track line detection.

9. An electronic device, comprising: The system includes a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The processor calls logical instructions from the memory to execute the track line detection method based on dynamic line anchors as described in any one of claims 1-6, and performs track line detection.

Images