Track line detection method and system based on dynamic line anchor

Abstract

The invention discloses a track line detection method based on a dynamic line anchor, and the method comprises the steps: collecting an image containing a track target and a track line instance label, carrying out the preprocessing, and obtaining a data set composed of an original image tensor and a mask image tensor; according to the obtained data set, an end-to-end deep neural network structure based on dynamic line anchor and track contrast learning is adopted, and a track line detection model is obtained through training; and inputting a to-be-detected image into the trained track line detection model, and outputting a track line detection result. According to the method, through combination of dynamic line anchoring and track comparative learning, track line features are effectively extracted, the accuracy of track line detection is improved, the accuracy of complex track line detection in a real environment at present is effectively improved, and development of the automatic driving field in the future is facilitated.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a track line detection method and system based on dynamic line anchors. Background Art

[0002] In recent years, intelligent driving technology has been continuously developing, greatly improving convenience and safety. Among them, intelligent trains use advanced sensors and computer vision algorithms to assist or replace train drivers in perceiving and analyzing the train operating environment, effectively reducing the risk of accidents caused by human factors. Track line detection, as part of intelligent driving, is a necessary and sufficient condition for many subsequent tasks and has important research value.

[0003] Traditional image processing-based track line detection methods usually rely on the edge and linear features of the track. This method achieves good results under simple conditions, but lacks robustness in practical applications. With the emergence and development of deep neural networks, it has become a popular choice for railway detection due to its ability to learn low-level and high-level semantic information, coupled with its fast inference speed and robustness.

[0004] Based on deep neural networks, many innovative methods have emerged in recent years in the study of lane detection, which is similar to track detection. Broadly speaking, these methods can be divided into segmentation-based methods, anchor-based methods, keypoint-based methods, and parameter prediction methods. Among these methods, since line anchors have good compatibility with track lines, the research focus can be placed on line anchors for track detection. Specifically, a line anchor is an elongated ray determined by a starting point and a slope. By placing a large number of predefined line anchors at the image boundary as a reference, and then using a feature pool to obtain the region of interest features corresponding to the line anchors, the offset between the track and the anchor line is predicted. Finally, the track instance is determined based on whether the line anchor is selected and the offset.

[0005] However, this also brings some problems. The number of anchor lines is often too large, and only a few of them are actually active during inference, resulting in a waste of computational and memory resources. On the contrary, too few anchor lines may cause them to fail to cover the potential location of the track, greatly degrading the detection results. Summary of the invention

[0006] The present invention aims to solve one of the technical problems existing in the related art at least to a certain extent.

[0007] One object of the present invention is to provide a track line detection method based on dynamic line anchors. By combining dynamic line anchors with track comparative learning, track line features can be effectively extracted, the accuracy of network model detection can be improved, and an effective track line detection solution can be provided for safe and efficient automatic train operation.

[0008] Another object of the present invention is to provide a track line detection system based on dynamic line anchors.

[0009] In order to achieve the above-mentioned object, the present invention provides a track line detection method based on a dynamic line anchor, comprising:

[0010] Collect images containing track targets and track line instance annotations, and perform preprocessing to obtain a dataset consisting of original image tensors and mask image tensors;

[0011] According to the acquired data set, an end-to-end deep neural network structure based on dynamic line anchors and track contrast learning is used to train a track line detection model.

[0012] The image to be detected is input into the trained track line detection model, and the track line detection result is output.

[0013] A further preferred technical solution of the present invention is that the set includes images of track targets and track line instance annotations, and is preprocessed to obtain a data set consisting of an original image tensor and a mask image tensor; including:

[0014] Select track images and track line instance annotations from a multi-scenario track dataset including tunnels, cities, strong light, and bad weather;

[0015] Scaling, normalization, image enhancement, and masking of track images and track instance annotations;

[0016] A dataset consisting of an original image tensor and a mask image tensor is obtained, and the dataset is divided into a training set, a test set, and a validation set.

[0017] Preferably, the method comprises: training a track line detection model based on the acquired data set by adopting an end-to-end deep neural network structure based on dynamic line anchors and track contrast learning; comprising:

[0018] Construct an end-to-end deep neural network structure based on dynamic line anchors and track contrast learning;

[0019] The original image tensor and the mask image tensor are sent to the deep neural network for training, and the weight of the deep neural network is adjusted to obtain a trained track line detection model;

[0020] Test and verify the trained track line detection model and adjust the model's hyperparameters.

[0021] Preferably, the end-to-end deep neural network structure based on dynamic line anchor and track contrast learning is constructed, and the deep neural network structure includes a track contrast learning module, a layer aggregation module and a line anchor self-assignment module;

[0022] The track contrastive learning module is composed of a main encoder and a momentum encoder. The main encoder is used to extract high-level semantic features, including the shape, texture and structure of the object. The momentum encoder is used to generate stable and consistent feature representations and slowly update the encoder parameters through a momentum update mechanism.

[0023] The layer aggregation module consists of multiple sub-aggregation modules. Each sub-aggregation module performs pooling, segmentation, and concatenation operations on the features of each layer extracted by the backbone network in the main encoder, proportionally segments the features, and then fuses these segmented features along the channel dimension.

[0024] The line anchor self-assignment module is used to sample and pool the fused features to generate sampled feature maps and line anchor feature maps.

[0025] Preferably, the end-to-end deep neural network structure training method based on dynamic line anchor and track contrast learning includes:

[0026] Obtain the features of the original image tensor and the mask image tensor through forward reasoning, and obtain the error between the two features;

[0027] Back propagation is performed based on the obtained error to update the weights of the deep neural network.

[0028] Preferably, the features of the original image tensor and the mask image tensor are obtained by forward reasoning, and the error between the two features is obtained; the specific method is:

[0029] The original image tensor and the mask image tensor are respectively input into the main encoder and momentum encoder of the track contrast learning module to extract features and obtain the basic features of the two images;

[0030] The basic features of the two images are used for comparative learning. The feature representation is learned by maximizing the similarity between similar samples and minimizing the similarity between different samples, and the comparative error value is calculated at the same time.

[0031] The features of each layer extracted by the main encoder are input into the layer aggregation module for feature fusion. By weighted averaging or concatenating the features at different levels, a more representative feature representation is generated.

[0032] The fused features are sampled and pooled into feature maps and line anchor feature maps. The feature map predicts the approximate position of the track line instance in the image and calculates the classification error value. The line anchor feature map calculates the offset between the track line instance and the predicted track line, and calculates the regression error value.

[0033] Preferably, the total error during the training process is a weighted sum of the comparison error value, the classification error value, and the regression error value, and the total error is used for back propagation to update the weights of the deep neural network.

[0034] Another aspect of the present invention provides a track line detection system based on a dynamic line anchor, comprising:

[0035] The data collection module is used to collect images containing track targets and track line instance annotations, and perform preprocessing to obtain a data set consisting of an original image tensor and a mask image tensor;

[0036] A model building module is used to train a track line detection model based on the acquired data set using an end-to-end deep neural network structure based on dynamic line anchors and track contrast learning;

[0037] The track line detection module is used to input the image to be detected into the trained track line detection model and output the track line detection result.

[0038] Another aspect of the present invention provides a non-transitory computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions enable a computer to execute the above-mentioned track line detection method based on dynamic line anchors to perform track line detection.

[0039] Yet another aspect of the present invention provides an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus, and the processor calls the logic instructions in the memory to execute the above-mentioned track line detection method based on dynamic line anchors to perform track line detection.

[0040] On the other hand, the present invention provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer executes the above-mentioned track line detection method based on dynamic line anchors to perform track line detection.

[0041] Beneficial effects: The present invention constructs a track line detection model based on dynamic line anchors for track detection. The model realizes self-allocation of line anchors by adaptively adjusting line anchor parameters multiple times, thereby improving detection performance. Specifically, the present invention designs a layer aggregation module to align and aggregate multi-scale semantic features extracted from the backbone network to establish global long-range dependencies. Then, the self-allocation module adaptively adjusts the line anchor parameters multiple times according to the aggregated features to allocate line anchors. These allocated line anchors are used to accurately match track instances.

[0042] In addition, in order to deal with low visibility and occlusion scenes, the present invention proposes a track contrast learning method. The track contrast learning module combines the locally extracted features with the globally captured cross-similarity features to achieve accurate railway detection.

[0043] In summary, the present invention effectively extracts track line features and improves the accuracy of track line detection through the combination of dynamic line anchors and track comparative learning. It effectively improves the accuracy of current complex track line detection in real environments, which is more conducive to the future development of the field of autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is an overall flow chart of the track line detection model in the training phase in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments, and they should not be understood as limitations on the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0046] Combine the following Figure 1 The present invention describes a track line detection method and system based on dynamic line anchors.

[0047] Embodiment 1: This embodiment provides a track line detection method based on dynamic line anchors, including preprocessing an image containing track targets and track line instance annotations to obtain an original image tensor and a mask image tensor; constructing an end-to-end deep neural network model based on dynamic line anchors and track contrast learning; sending the original image tensor and the mask image tensor into the deep neural network model for training respectively to obtain the trained model weights; inputting a new image into the trained network model to obtain a track line detection result.

[0048] The method provided in this embodiment effectively extracts track line features through the combination of dynamic line anchors and track comparative learning, improves the accuracy of network model detection, and provides an effective track line detection solution for safe and efficient automatic train operation.

[0049] Specifically, the implementation steps of this embodiment are:

[0050] S1. Collect images containing track targets and track line instance annotations, and perform preprocessing to obtain a data set consisting of original image tensors and mask image tensors; including:

[0051] S11, selecting track images and track line instance annotations from a multi-scenario track dataset including tunnels, cities, strong light and bad weather;

[0052] S12, performing scaling, normalization, image enhancement and masking operations on the track image and the track line instance annotations;

[0053] S13. Obtain a data set consisting of an original image tensor and a mask image tensor, and divide the data set into a training set, a test set, and a validation set.

[0054] S2. Construct an end-to-end deep neural network structure based on dynamic line anchor and track contrast learning. The deep neural network structure includes: track contrast learning module, layer aggregation module and line anchor self-assignment module;

[0055] Among them, the track contrast learning module is composed of a main encoder and a momentum encoder. The main encoder is used to extract high-level semantic features, which include the shape, texture, and structure of the object. The momentum encoder is used to generate a more stable and consistent feature representation, and the parameters of the encoder are slowly updated through the momentum update mechanism to avoid drastic changes in the feature representation.

[0056] The layer aggregation module consists of multiple sub-aggregation modules, each of which performs pooling, segmentation, and concatenation operations on the features of each layer extracted by the backbone network in the main encoder. Semantic information is extracted from the feature map, the features are proportionally segmented, and then these segmented features are connected along the channel dimension. At the same time, in order to improve the interactivity between different segments, paragraph shuffling and grouped convolution are used. Finally, the feature maps output by the two processes are fused together through addition operations, which effectively reduces the sampling of feature maps while retaining the scale information.

[0057] The line anchor self-assignment module samples and pools the fused features to generate sampled feature maps and line anchor feature maps. After creating long-term horizontal dependencies between the line anchor feature map and the feature map, this information is forwarded through the fully connected layer to predict the adjustment offset of the line anchor. Using the position information of the line anchor, each detection instance can retrieve the line anchor feature from the feature map through region of interest alignment. Through multiple adjustments, the fit between the line anchor and the track instance is improved, thereby improving the accuracy of track detection.

[0058] S3, the original image tensor and the mask image tensor in step S1 are respectively sent to the deep neural network constructed in step S2 for training to obtain the trained model weights. This step includes the process of forward reasoning, calculating the error value and back propagating to update the weights. Specifically:

[0059] S31, inputting the original image tensor and the mask image tensor in step S1 into a feature extraction network to obtain basic feature information including track line features and position information;

[0060] S32, performing contrastive learning based on the two features extracted in step S31, and generating a second view of the contrastive learning method by masking the input image. In order to reveal the local features of the masked patch, a cross-similarity operation is performed, using the similarity between the surrounding features and the original image features. The cross-similarity between each block and feature map is stored in a separate tensor, and then they are connected in their respective order. In this way, not only can the similarity between each patch on one feature map and all patches on another feature map be identified, but also their position information can be retained. Therefore, the contrast loss can reflect the relationship between the position value and the similarity value of the neural network.

[0061] S33, the features of each layer extracted by the encoder in step S31 are input to the layer aggregation module for feature fusion, starting from the pooling operation, extracting semantic information from the feature map, and then proportionally segmenting the features. Then these segmented features are connected along the channel dimension. In order to improve the interactivity between different segments, segment shuffling and group convolution are used. Finally, the feature maps output by the two processes are fused together through addition operations, which effectively reduces the sampling of the feature maps while retaining the scale information.

[0062] S34, sampling and pooling operations are performed on the features fused in step S33 to generate sampled feature maps and line anchor feature maps. After this, a fully connected layer processes the sampled feature maps to extract the initial line anchor properties. The sampled features of the line anchors are used as similarity evaluation matrices, and the sampled feature maps are projected into each feature space as values. After creating a long-term horizontal dependency between the line anchor feature map and the feature map, the information is forwarded through the fully connected layer to predict the adjusted offset of the line anchor. Using the position information of the line anchor, each detection instance can retrieve the line anchor features from the feature map through region of interest alignment. Through multiple adjustments, the fit between the line anchor and the track instance is improved, thereby improving the accuracy of track detection.

[0063] S35. Calculate the total error. The total error value is obtained by weighted summation of the comparison error value, the classification error value and the regression error value. Perform back propagation based on the obtained error to update the weights of the deep neural network.

[0064] S36. Use the test set and validation set to test and validate the trained track line detection model, and adjust the hyperparameters of the model.

[0065] S4. Input the image to be detected into the trained track line detection model, and output the track line detection result.

[0066] The method of this embodiment designs a layer aggregation module to align and aggregate the multi-scale semantic features extracted from the backbone network and establish global long-range dependencies. Then, the self-assignment module adaptively adjusts the anchor parameters multiple times to assign anchors based on the aggregated features. These assigned anchors are used to accurately match track instances. In addition, in order to handle low visibility and occluded scenes, this embodiment proposes a track contrast learning method that combines locally extracted features with globally captured cross-similar features to achieve accurate railway detection.

[0067] Embodiment 2: This embodiment provides a track line detection system based on a dynamic line anchor, comprising:

[0068] The data collection module is used to collect images containing track targets and track line instance annotations, and perform preprocessing to obtain a data set consisting of an original image tensor and a mask image tensor; corresponding to step S1 in Example 1.

[0069] The model building module is used to train a track line detection model based on the acquired data set using an end-to-end deep neural network structure based on dynamic line anchors and track contrast learning; corresponding to steps S2-S3 in Example 1.

[0070] The track line detection module is used to input the image to be detected into the trained track line detection model and output the track line detection result; corresponding to step S4 in Example 1.

[0071] Embodiment 3: This embodiment provides a non-transitory computer-readable storage medium, on which computer instructions are stored, and the computer instructions enable the computer to execute the track line detection method based on dynamic line anchors in Embodiment 1 to perform track line detection. The method includes:

[0072] Collect images containing track targets and track line instance annotations, and perform preprocessing to obtain a dataset consisting of original image tensors and mask image tensors;

[0073] According to the acquired data set, an end-to-end deep neural network structure based on dynamic line anchors and track contrast learning is used to train a track line detection model.

[0074] The image to be detected is input into the trained track line detection model, and the track line detection result is output.

[0075] Embodiment 4: This embodiment provides an electronic device, which may include: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus. The processor may call the logic instructions in the memory to execute the track line detection method based on dynamic line anchors in Embodiment 1 to perform track line detection. The method includes:

[0076] Collect images containing track targets and track line instance annotations, and perform preprocessing to obtain a dataset consisting of original image tensors and mask image tensors;

[0077] According to the acquired data set, an end-to-end deep neural network structure based on dynamic line anchors and track contrast learning is used to train a track line detection model.

[0078] The image to be detected is input into the trained track line detection model, and the track line detection result is output.

[0079] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0080] Embodiment 5: This embodiment provides a computer program product, which includes a computer program. The computer program can be stored in a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the track line detection method based on dynamic line anchors in Embodiment 1 to perform track line detection. The method includes:

[0081] Collect images containing track targets and track line instance annotations, and perform preprocessing to obtain a dataset consisting of original image tensors and mask image tensors;

[0082] According to the acquired data set, an end-to-end deep neural network structure based on dynamic line anchors and track contrast learning is used to train a track line detection model.

[0083] The image to be detected is input into the trained track line detection model, and the track line detection result is output.

[0084] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0085] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A track line detection method based on dynamic line anchors, characterized in that: include: Collect images containing track targets and track line instance annotations, and perform preprocessing to obtain a dataset consisting of original image tensors and mask image tensors; According to the acquired data set, an end-to-end deep neural network structure based on dynamic line anchors and track contrast learning is used to train a track line detection model. The image to be detected is input into the trained track line detection model, and the track line detection result is output.

2. The track line detection method based on dynamic line anchor according to claim 1, characterized in that: The set includes images of track targets and track line instance annotations, and is preprocessed to obtain a data set consisting of an original image tensor and a mask image tensor; including: Select track images and track line instance annotations from a multi-scenario track dataset including tunnels, cities, strong light, and bad weather; Scaling, normalization, image enhancement, and masking of track images and track instance annotations; A dataset consisting of an original image tensor and a mask image tensor is obtained, and the dataset is divided into a training set, a test set, and a validation set.

3. The track line detection method based on dynamic line anchor according to claim 1, characterized in that: According to the acquired data set, an end-to-end deep neural network structure based on dynamic line anchors and track contrast learning is adopted to train a track line detection model; include: Construct an end-to-end deep neural network structure based on dynamic line anchors and track contrast learning; The original image tensor and the mask image tensor are sent to the deep neural network for training, and the weight of the deep neural network is adjusted to obtain a trained track line detection model; Test and verify the trained track line detection model and adjust the model's hyperparameters.

4. The track line detection method based on dynamic line anchor according to claim 3, characterized in that: The end-to-end deep neural network structure based on dynamic line anchor and track contrast learning is constructed, and the deep neural network structure includes a track contrast learning module, a layer aggregation module and a line anchor self-assignment module; The track contrast learning module is composed of a main encoder and a momentum encoder. The main encoder is used to extract high-level semantic features, including the shape, texture and structure of the object. The momentum encoder is used to generate stable and consistent feature representations and slowly update the encoder parameters through a momentum update mechanism. The layer aggregation module consists of multiple sub-aggregation modules. Each sub-aggregation module performs pooling, segmentation, and concatenation operations on the features of each layer extracted by the backbone network in the main encoder, extracts semantic information from the feature map, performs proportional segmentation on the features, and then fuses these segmented features along the channel dimension. The line anchor self-assignment module is used to sample and pool the fused features to generate sampled feature maps and line anchor feature maps.

5. The track line detection method based on dynamic line anchor according to claim 4, characterized in that: The end-to-end deep neural network structure training method based on dynamic line anchor and track contrast learning includes: Obtain the features of the original image tensor and the mask image tensor through forward reasoning, and obtain the error between the two features; Back propagation is performed based on the obtained error to update the weights of the deep neural network.

6. The track line detection method based on dynamic line anchor according to claim 5, characterized in that: The features of the original image tensor and the mask image tensor are obtained by forward reasoning, and the error between the two features is obtained; the specific method is: The original image tensor and the mask image tensor are respectively input into the main encoder and momentum encoder of the track contrast learning module to extract features and obtain the basic features of the two images; The basic features of the two images are used for comparative learning. The feature representation is learned by maximizing the similarity between similar samples and minimizing the similarity between different samples, and the comparative error value is calculated at the same time. The features of each layer extracted by the main encoder are input into the layer aggregation module for feature fusion. By weighted averaging or concatenating the features at different levels, a more representative feature representation is generated. The fused features are sampled and pooled into feature maps and line anchor feature maps. The feature map predicts the approximate position of the track line instance in the image and calculates the classification error value. The line anchor feature map calculates the offset between the track line instance and the predicted track line, and calculates the regression error value.

7. The track line detection method based on dynamic line anchor according to claim 6, characterized in that: The total error during the training process is the weighted sum of the comparison error value, classification error value, and regression error value. The total error is used for back propagation to update the weights of the deep neural network.

8. A track line detection system based on dynamic line anchors, characterized in that: include: The data collection module is used to collect images containing track targets and track line instance annotations, and perform preprocessing to obtain a data set consisting of an original image tensor and a mask image tensor; A model building module is used to train a track line detection model based on the acquired data set using an end-to-end deep neural network structure based on dynamic line anchors and track contrast learning; The track line detection module is used to input the image to be detected into the trained track line detection model and output the track line detection result.

9. A non-transitory computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions enable a computer to execute the track line detection method based on dynamic line anchors as described in any one of claims 1 to 7 to perform track line detection.

10. An electronic device comprising: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus, and the processor calls the logic instructions in the memory to execute the track line detection method based on dynamic line anchors as described in any one of claims 1-7 to perform track line detection.

Images