Track line detection method and system based on hybrid attention mechanism and associated pixels

Abstract

The application provides a track line detection method and system based on a mixed attention mechanism and associated pixels, and the method comprises the following steps: selecting track images and track line labels of a mixed scene from a track line database, and converting the track images and the track line labels into binary masks; pairing the track images with the binary masks; and dividing the track images and the binary masks into a training set and a test set according to a corresponding proportion; constructing a track line detection network model based on a mixed attention mechanism and associated pixels; inputting the training set divided according to the proportion into a deep neural network for training; inputting the paired color images and binary masks into the track line detection network model to obtain feature information of surrounding pixel points of each pixel point, and realizing prediction on a horizontal associated pixel domain and a vertical associated pixel domain. The application effectively extracts unique features of track lines by combining a mixed attention mechanism and associated pixels, improves the accuracy of network model detection, and guarantees the accuracy of application.

Description

Technical Field

[0001] This invention relates to a method and system for detecting orbital lines based on a hybrid attention mechanism and associated pixels, belonging to the field of image processing technology. Background Technology

[0002] Currently, track line detection technology has become an important branch of research in the fields of computer vision and artificial intelligence, and is mainly used in track-related tasks such as track defect detection, track deformation analysis, and foreign object intrusion detection.

[0003] In recent years, deep neural networks have achieved remarkable results in image segmentation and object detection due to their high inference speed, robustness, and ability to learn low-level and high-level semantic information. However, their practical application is still constrained by several factors: interference conditions such as strong light, darkness, and external object occlusion; the relatively small size of current track line-related datasets limits the feature extraction capabilities of simple deep neural networks; more importantly, current track line detection segmentation methods still perform poorly on images with complex textures and cluttered backgrounds. Furthermore, current deep learning-based track line detection methods do not consider the overall characteristics of track lines in the image, resulting in poor detection performance.

[0004] In recent years, hybrid attention mechanisms have been increasingly applied to detection tasks in image processing. Compared to traditional convolutional neural networks, they perform adaptive feature weighting at both the spatial and channel levels, thus better capturing feature information at different scales.

[0005] In few-sample image detection tasks, optimizing the performance of deep neural networks by combining the geometric structural features of the target is an effective technical approach. By combining a hybrid attention mechanism with associated pixels, the structural and positional features of the target can be effectively utilized, while also enhancing the feature extraction capability of deep neural networks for small datasets such as orbital lines.

[0006] In view of this, it is indeed necessary to propose a trajectory line detection method and system based on a hybrid attention mechanism and associated pixels to solve the above problems. Summary of the Invention

[0007] The purpose of this invention is to provide a trajectory line detection method based on a hybrid attention mechanism and associated pixels. By combining the hybrid attention mechanism and associated pixels, key features of the trajectory line can be effectively extracted, improving the accuracy and running speed of the network model detection and ensuring the real-time performance of the application.

[0008] To achieve the above objectives, this invention provides a trajectory line detection method based on a hybrid attention mechanism and associated pixels, comprising the following steps:

[0009] Step 1: Select color track images and track line annotations for various scenes from the track line database. Convert the track line annotations into binary masks using an image processing function. Pair the binary masks with the color track images. Then divide the paired images into training and testing sets according to the corresponding proportions.

[0010] Step 2: Construct an end-to-end trainable orbital detection network model based on a hybrid attention mechanism and associated pixels;

[0011] Step 3: Input the training set divided in Step 1 into the orbital line detection network model for training;

[0012] Step 4: Input the paired color track image and binary mask obtained in Step 1 into the track line detection network model in Step 2 to obtain the feature information of the surrounding pixels of each pixel, and realize the prediction of the horizontal and vertical associated pixel domains.

[0013] As a further improvement of the present invention, the end-to-end trainable track detection network model in step 2 includes a basic feature extraction network, a hybrid attention mechanism encoding network, and a post-processing clustering network.

[0014] As a further improvement of the present invention, step 2 specifically includes the following steps:

[0015] Step 21: Input the paired color image and binary mask from Step 1 into the feature extraction network to obtain basic feature information containing orbital line features and positional information;

[0016] Step 22: Input the basic feature information obtained in step 21 into the hybrid attention mechanism encoding network to obtain horizontal and vertical associated pixels that include orbital space and channel level;

[0017] Step 23: Input the horizontal and vertical associated pixels and binary mask obtained in step 22 into the post-processing clustering network in sequence to cluster the lane pixels and obtain the corresponding lane instances;

[0018] Step 24: Based on the imbalance between the track and the background in the track image, train the track line detection network model using the cross-entropy loss function.

[0019] As a further improvement of the present invention, the feature extraction network in step 21 is composed of an encoder and a decoder. The encoder is used to extract high-level semantic features, which include the shape, texture and structure of the object. The decoder is used to perform image segmentation and pixel-level semantic analysis and capture details and edge information in the image.

[0020] As a further improvement of the present invention, in step 22, basic feature information of the trajectory line is extracted from the feature extraction network, and then a hybrid attention network module is used to realize deeper feature extraction, including spatial and channel; the hybrid attention network module includes a channel attention module and a spatial attention module, the channel attention module is used to highlight important features, and the spatial hybrid attention module is used to focus on the feature information around the pixel.

[0021] As a further improvement of the present invention, in step 23, a horizontal correlation vector and a vertical correlation vector are assigned to all pixels in the track image. The horizontal correlation vector points to the center of each track line instance in each row. The track line pixels are aggregated into different track line parts in the horizontal direction using the binary mask obtained in step 1. The vertical correlation vector points to the center of the track line in the previous row, thereby connecting the track line parts in different rows into a single track line instance. The track line instance is then stored in a matrix, and the matrix is ​​returned as output. As a further improvement of the present invention, the loss function in step 24, cross-entropy loss, is used to address the imbalance between track line and background categories. By adjusting the weights of the loss function, it focuses more on samples that are difficult to classify and less on samples that are easy to classify.

[0022] Another objective of this invention is to provide a trajectory line detection system based on a hybrid attention mechanism and associated pixels that applies the above-described method.

[0023] To achieve the above objectives, the present invention provides a track line detection system based on a hybrid attention mechanism and associated pixels, and applies the above-described method to perform track line detection.

[0024] The beneficial effects of this invention include: It effectively utilizes the complementary features extracted by convolutional neural networks and hybrid attention mechanisms. A basic feature extraction network is used to extract primary features, which are then combined with a hybrid attention encoding network to obtain horizontal and vertically correlated pixels encompassing the track space and channel levels. Finally, binary encoding is used to cluster track line instances. A cross-entropy loss function is employed to address the issue of unevenness between track lines and the background in the track line dataset. In summary, the hybrid attention mechanism and correlated pixel network provided by this invention improve the accuracy of track line detection, effectively enhancing the accuracy of complex track line detection in real-world environments, and is more conducive to the future development of the autonomous driving field. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the trajectory line detection method based on a hybrid attention mechanism and associated pixels according to the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] It should be emphasized that, in describing this invention, various formulas and constraints are distinguished by consistent reference numerals, but it is not excluded that different reference numerals may be used to identify the same formulas and / or constraints. The purpose of this arrangement is to more clearly illustrate the features of this invention.

[0028] This invention is a trajectory line detection method based on a hybrid attention mechanism and associated pixels, such as... Figure 1 As shown, the track line detection method includes the following steps:

[0029] Step 1: Select color images and annotations of various track lines from the track line database. Use image processing functions to convert the track line annotations into binary masks. Pair the binary masks with the color images. Then divide the paired images into training and testing sets according to the corresponding ratios.

[0030] Step 2: Construct an end-to-end trainable orbital detection network model based on a hybrid attention mechanism and associated pixels;

[0031] Step 3: Input the training set divided in Step 1 into the deep neural network for training;

[0032] Step 4: Input the paired color image and binary mask obtained in Step 1 into the orbital detection network model in Step 2 to obtain the feature information of the surrounding pixels of each pixel, and realize the prediction of the horizontal and vertical associated pixel domains.

[0033] The following describes in detail the trajectory line detection method based on hybrid attention mechanism and associated pixels of the present invention with reference to the accompanying drawings.

[0034] Specifically, in step 2, the end-to-end trainable track detection network model includes: a feature extraction network, a global attention encoding network, a track line classification and prediction network, and a structural representation network. The construction process of the end-to-end trainable track detection network model is as follows:

[0035] Step 21: Input the paired color image and binary mask from Step 1 into the feature extraction network to obtain basic feature information containing orbital line features and positional information;

[0036] Step 22: Input the basic feature information obtained in Step 21 into the hybrid attention mechanism encoding network to obtain horizontal and vertical associated pixels that include orbital space and channel level;

[0037] Step 23: Input the horizontal and vertical associated pixels and binary mask obtained in step 22 into the post-processing clustering network in sequence to cluster the lane pixels and obtain the corresponding lane instances.

[0038] Step 24: Based on the imbalance between the track and the background in the track image, train the track line detection network model using the cross-entropy loss function.

[0039] The feature extraction network in step 21 combines an encoder and a decoder, fully utilizing the deep feature representation capabilities of the encoder and the segmentation and semantic analysis capabilities of the decoder. Step 22 extracts basic feature information of the track lines from the feature extraction network, followed by a hybrid attention network module to achieve deeper feature extraction, including spatial and channel features. Step 23 assigns a horizontal and vertical association domain to all pixels in the track image; these are essentially vectors. The horizontal association vector points to the center of each track line instance in each row. Using the binary mask obtained in step 1, track line pixels are aggregated into different track line segments in the horizontal direction. The vertical association vector points to the center of the track line in the previous row, thus connecting track line segments from different rows into a single track line instance. The track line instances are then stored in a matrix, which is returned as output.

[0040] The present invention also provides a system that can perform the aforementioned steps.

[0041] This invention effectively utilizes convolutional neural networks and hybrid attention mechanisms to extract comprehensive features of channels and space, performs detection in the horizontal and vertical pixel domains, and combines binary masks for clustering, successfully detecting track line instances, greatly improving the accuracy of network model detection in complex and multi-track line scenarios.

[0042] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for detecting orbital lines based on a hybrid attention mechanism and associated pixels, characterized in that, Includes the following steps: Step 1: Select color track images and track line annotations for various scenes from the track line database. Convert the track line annotations into binary masks using an image processing function. Pair the binary masks with the color track images. Then divide the paired images into training and testing sets according to the corresponding proportions. Step 2: Construct an end-to-end trainable orbital line detection network model based on a hybrid attention mechanism and associated pixels; the end-to-end trainable orbital line detection network model includes a basic feature extraction network, a hybrid attention mechanism encoding network, and a post-processing clustering network, specifically including the following steps: Step 21: Input the paired color image and binary mask from Step 1 into the feature extraction network to obtain basic feature information containing orbital line features and positional information; Step 22: Input the basic feature information obtained in step 21 into the hybrid attention mechanism encoding network to obtain horizontal and vertical associated pixels that include orbital space and channel level; Step 23: Input the horizontal and vertical associated pixels and binary mask obtained in Step 22 into the post-processing clustering network to cluster the lane pixels and obtain the corresponding lane instances; specifically: assign a horizontal association vector and a vertical association vector to all pixels in the track image. The horizontal association vector points to the center of each track line instance in each row. Using the binary mask obtained in Step 1, the track line pixels are aggregated into different track line parts in the horizontal direction. The vertical association vector points to the center of the track line in the previous row, thereby connecting the track line parts in different rows into a track line instance. Then, the lane line instances are stored in a matrix, and the matrix is ​​returned as output. Step 24: Based on the imbalance between the track and the background in the track image, train the track line detection network model using the cross-entropy loss function; Step 3: Input the training set divided in Step 1 into the orbital line detection network model for training; Step 4: Input the paired color track image and binary mask obtained in Step 1 into the track detection network model in Step 2 to obtain the feature information of the surrounding pixels of each pixel, and realize the prediction of the horizontal and vertical associated pixel domains.

2. The orbital line detection method based on hybrid attention mechanism and associated pixels according to claim 1, characterized in that: The feature extraction network in step 21 consists of an encoder and a decoder. The encoder is used to extract high-level semantic features, including the shape, texture and structure of the object. The decoder is used to segment the image and analyze the semantics at the pixel level and capture details and edge information in the image.

3. The orbital line detection method based on hybrid attention mechanism and associated pixels according to claim 1, characterized in that: In step 22, basic feature information of the trajectory line is extracted from the feature extraction network. Subsequently, the hybrid attention network module realizes deeper feature extraction, including spatial and channel attention. The hybrid attention network module includes a channel attention module and a spatial attention module. The channel attention module is used to highlight important features, and the spatial hybrid attention module is used to focus on the feature information around the pixel.

4. The orbital line detection method based on hybrid attention mechanism and associated pixels according to claim 3, characterized in that: The loss function in step 24, cross-entropy loss, is used to address the imbalance between the orbital lines and the background categories. By adjusting the weights of the loss function, it focuses more on samples that are difficult to classify and less on samples that are easy to classify.

5. A trajectory line detection system based on a hybrid attention mechanism and associated pixels, characterized in that, Track line detection is performed using the method described in any one of claims 1 to 4.

Images