A road traffic marking detection method based on efficient visual transformation

Abstract

The present invention relates to the technical field of road traffic marking detection, and provides a road traffic marking detection method based on efficient visual transformation. The present invention merges, screens, expands and annotates a road traffic marking data set; constructs a road traffic marking detection model consisting of a feature extraction network, a multi-scale feature map enhancement network and a detection head; inputs the feature extraction network for feature extraction; inputs the multi-scale feature map enhancement network for feature fusion; inputs the detection head for detection and classification to obtain the final detection result. According to the scheme of the present invention, the feature extraction network uses an efficient attention mechanism to enhance the ability to capture global context information and improve the detection accuracy of complex road traffic markings; the multi-scale feature map enhancement network uses a bottom-up path structure to enhance the ability to capture small target features, thereby improving the recognition accuracy and robustness under complex backgrounds.

Description

Technical Field

[0001] The invention belongs to the technical field of road traffic marking detection, and in particular relates to a road traffic marking detection method based on efficient visual transformation. Background Art

[0002] With the rapid development of artificial intelligence technology, intelligent transportation systems have become an important direction for the development of modern transportation. As a key research field in intelligent transportation, autonomous driving technology has received widespread attention and has made significant progress. The detection and recognition of road traffic markings are crucial in autonomous driving systems and directly affect the decision-making process of vehicles. Accurate road marking detection can provide real-time road information for intelligent vehicles, thereby improving road traffic safety. At the same time, efficient road traffic marking detection technology is not only a basic requirement of intelligent transportation systems, but also an important support for the development of autonomous driving technology.

[0003] The existing algorithms for road traffic marking detection and recognition are mainly divided into traditional algorithms and deep learning algorithms. In particular, the deep learning algorithms have powerful feature extraction and expression capabilities that significantly improve the detection accuracy. However, there are still some problems with the existing deep learning algorithms: First, the increase in the number of model parameters leads to a decrease in detection speed, especially on resource-constrained edge devices, the real-time and deployment costs of the model have become urgent issues to be solved; second, complex road environments such as low contrast, occlusion, and diverse backgrounds further aggravate the instability of marking detection, making it difficult for the model to achieve robust performance. In addition, when dealing with small target markings, due to the uneven distribution of data, missed detection and false detection problems are often prone to occur, which seriously affects the model's ability to detect small target markings. At the same time, the lack of effective fusion capabilities of multi-scale feature maps results in information loss in feature expressions at different resolutions, further limiting the detection accuracy of the model in complex scenes. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a road traffic marking detection method based on efficient visual transformation, which aims to solve the problems mentioned in the background technology.

[0005] The present invention provides a road traffic marking detection method based on efficient visual transformation, comprising the following steps:

[0006] Step S1: merging a road traffic marking dataset, the road traffic marking dataset including a plurality of road traffic marking images, expanding and annotating the merged road traffic marking dataset to obtain an expanded and annotated road traffic marking dataset;

[0007] Step S2: construct a road traffic marking detection model, which consists of a feature extraction network, a multi-scale feature map enhancement network and a detection head;

[0008] The feature extraction network consists of an efficient visual transformer network, a fast spatial pyramid pooling block, and a cross-scale pixel-level spatial attention block, wherein the efficient visual transformer network consists of a first two-dimensional convolution block, an efficient visual transformer building block, and an efficient visual transformer downsampling block, and the efficient visual transformer building block consists of a first depthwise separable convolution block, a first residual block, a first feedforward neural network block, and a local window attention mechanism block;

[0009] The multi-scale feature map enhancement network consists of a bidirectional attention fusion block, a second 2D convolution block, a transposed convolution block, a dot product block, an enhanced cross-stage local network feature extraction block, and an addition block;

[0010] Step S3: the expanded and annotated road traffic marking data set is input into a feature extraction network for feature extraction to obtain a plurality of multi-scale feature maps; wherein the plurality of road traffic marking images in the expanded and annotated road traffic marking data set are subjected to local feature correlation calculation and feature fusion in an efficient visual transformer building block;

[0011] Step S4: several multi-scale feature maps are input into a multi-scale feature map enhancement network for feature fusion to obtain several enhanced multi-scale feature maps; wherein several multi-scale feature maps are feature selectively enhanced in a bidirectional attention fusion block;

[0012] Step S5: Several enhanced multi-scale feature maps are input into the detection head for detection and classification to obtain the final detection result.

[0013] Furthermore, step S1 includes the following steps:

[0014] Step S101: merging the CeyMo dataset and the VPGNet dataset, screening road traffic marking images that meet the road traffic marking standards, and classifying the road traffic marking images that meet the road traffic marking standards to obtain the merged and screened CeyMo dataset and VPGNet dataset, wherein the CeyMo dataset and the VPGNet dataset are both road traffic marking datasets;

[0015] Step S102: Counting the number of road traffic marking images in each category of road traffic marking images in the merged and filtered CeyMo dataset and VPGNet dataset, and identifying categories with insufficient number of road traffic marking images;

[0016] Step S103: Collect a number of road traffic marking images of the same category as the road traffic marking images whose number is insufficient, manually annotate the collected road traffic marking images, and obtain the expanded and manually annotated CeyMo dataset and VPGNet dataset.

[0017] Further, in step S3, the expanded and annotated road traffic marking data set is input into a feature extraction network for feature extraction to obtain a plurality of multi-scale feature maps; the following steps are included:

[0018] Step S301: the expanded and annotated road traffic marking data set is input into the efficient visual transformer network in the feature extraction network, and the efficient visual transformer network includes four first two-dimensional convolution blocks, three efficient visual transformer building blocks and two efficient visual transformer downsampling blocks; in the efficient visual transformer network, a plurality of road traffic marking images are input into the four first two-dimensional convolution blocks for downsampling and feature extraction to obtain a plurality of first multi-scale feature maps; then they are input into the first efficient visual transformer building block for local feature correlation calculation and feature fusion to obtain a plurality of second multi-scale feature maps; then they are input into the first efficient visual transformer downsampling block for downsampling and feature fusion to obtain a plurality of third multi-scale feature maps; then they are input into the second efficient visual transformer building block for local feature correlation calculation and feature fusion to obtain a plurality of fourth multi-scale feature maps; then they are input into the second efficient visual transformer downsampling block for downsampling and feature fusion to obtain a plurality of fifth multi-scale feature maps; then they are input into the third efficient visual transformer building block for local feature correlation calculation and feature fusion to obtain a plurality of sixth multi-scale feature maps;

[0019] Step S302: inputting a plurality of sixth multi-scale feature maps into a fast spatial pyramid pooling block in a feature extraction network to obtain a plurality of seventh multi-scale feature maps;

[0020] Step S303: Several seventh multi-scale feature maps are input into the cross-scale pixel-level spatial attention block in the feature extraction network to obtain several eighth multi-scale feature maps, where the eighth multi-scale feature map is a 32-fold down-sampled multi-scale feature map.

[0021] Further, in step S301, a plurality of first multi-scale feature maps are input into a first efficient visual transformer building block for local feature correlation calculation and feature fusion to obtain a plurality of second multi-scale feature maps, including the following steps:

[0022] The efficient visual transformer building block consists of two first depthwise separable convolution blocks, five first residual blocks, two first feedforward neural network blocks and a local window attention mechanism block; a number of first multi-scale feature maps are input into the first first depthwise separable convolution block in the efficient visual transformer building block to capture local features, and then input into the first first residual block to fuse input features and output features; then input into the first first feedforward neural network block to enhance feature expression, and then input into the second first residual block to fuse input features and output features, and then input into the local window attention mechanism block to calculate local feature correlation, and then input into the third first residual block to fuse input features and output features, and finally input into the second first depthwise separable convolution block, the fourth first residual block, the second first feedforward neural network block and the fifth first residual block in sequence to further extract and integrate features to obtain a number of second multi-scale feature maps.

[0023] Further, in step S301, the plurality of second multi-scale feature maps are input into the first efficient visual transformer downsampling block for downsampling and feature fusion to obtain a plurality of third multi-scale feature maps. The specific steps are as follows:

[0024] The efficient visual transformer downsampling block consists of three second depth separable convolution blocks, four second residual blocks, two second feedforward neural network blocks and a compression and excitation attention mechanism block; several second multi-scale feature maps are input into the first second depth separable convolution block in the efficient visual transformer downsampling block to capture local features, and then input into the first second residual block to fuse input features and output features, and then input into the first second feedforward neural network block to enhance feature expression, and then input into the second second residual block to fuse input features and output features, and then input into the second second depth separable convolution block to capture local features, and then input into the compression and excitation attention mechanism block for global pooling processing and calculation of channel attention weights, and then input into the third second depth separable convolution block to capture local features, and then input into the third second residual block to fuse input features and output features, and then input into the second second feedforward neural network block to enhance feature expression, and then input into the fourth second residual block to fuse input features and output features to obtain several third multi-scale feature maps.

[0025] Further, step S4 includes the following steps:

[0026] Step S401: the multi-scale feature map enhancement network consists of five bidirectional attention fusion blocks, three second two-dimensional convolution blocks, two transposed convolution blocks, two dot product blocks, two enhanced cross-stage local network feature extraction blocks and two addition blocks; a plurality of eighth multi-scale feature maps are input into the first bidirectional attention fusion block in the multi-scale feature map enhancement network for feature selective enhancement, and then input into the first second two-dimensional convolution block for convolution operation, and then input into the first transposed convolution block for upsampling, and then input into the third bidirectional attention fusion block for feature selective enhancement;

[0027] After the plurality of fourth multi-scale feature maps are sequentially input into the second bidirectional attention fusion block and the second second two-dimensional convolution block for processing, shallow features of the plurality of fourth multi-scale feature maps are obtained;

[0028] The selectively enhanced eighth multi-scale feature maps are then sequentially input into the first dot product block and the first addition block, and feature fused with the shallow features of the fourth multi-scale feature maps to obtain the fused 16-fold downsampled multi-scale feature maps;

[0029] The feature selective enhancement in the bidirectional attention fusion block is to process long-distance dependent information through global pooling in the horizontal and vertical directions, and generate directional weights in combination with channel information, which can enhance the feature expression of small targets in road traffic marking images and key areas in complex backgrounds;

[0030] Step S402: the fused multiple 16-fold downsampled multi-scale feature maps are input into the first enhanced cross-stage local network feature extraction block to extract high-order semantic information and integrate multi-path information, thereby obtaining multiple first enhanced multi-scale feature maps;

[0031] Or the fused 16-fold downsampled multi-scale feature maps are input into the second transposed convolution block for upsampling to obtain a number of 8-fold downsampled multi-scale feature maps, which are then input into the fifth bidirectional attention fusion block for feature selective enhancement, and then sequentially input into the second dot product block and the second addition block for feature fusion with the shallow features of the second multi-scale feature maps to obtain a number of fused 8-fold downsampled multi-scale feature maps; wherein the several second multi-scale feature maps are sequentially input into the fourth bidirectional attention fusion block and the third second two-dimensional convolution block for processing to obtain the shallow features of the several second multi-scale feature maps;

[0032] Step S403: Several fused 8-fold downsampled multi-scale feature maps are input into a second enhanced cross-stage local network feature extraction block to extract high-order semantic information and integrate multi-path information to obtain several second enhanced multi-scale feature maps.

[0033] Furthermore, in step S401, feature selective enhancement is performed in the bidirectional attention fusion block, and the specific processing flow is as follows:

[0034] Step S4011: Input features of the bidirectional attention fusion block First, through the adaptive average pooling operation, aggregation is performed in the width and height directions respectively to obtain the horizontal output features of the bidirectional attention fusion block And the vertical output features of the bidirectional attention fusion block , the formulas are expressed as:

[0035] ;

[0036] ;

[0037] Where: It is the input feature of the bidirectional attention fusion block; It is the horizontal output feature of the bidirectional attention fusion block; Represents the vertical output features of the bidirectional attention fusion block; It is an adaptive average pooling operation; is the height of the multi-scale feature map of the processed road traffic marking image; is the width of the multi-scale feature map of the processed road traffic marking image;

[0038] Step S4012: Vertical output features of the bidirectional attention fusion block The height and width are transposed, and the formula is expressed as:

[0039] ;

[0040] Where: is the vertical output feature of the transposed bidirectional attention fusion block; RearrangeDims is the operation of exchanging the order of dimensions; To keep the order of the 0th dimension, the batch remains unchanged; to keep the order of the 1st dimension, the number of channels remains unchanged; to move the order of the 3rd dimension, the width moves to the 2nd position; to move the order of the 2nd dimension, the height moves to the 3rd position;

[0041] Step S4013: Fusion of horizontal output features of the bidirectional attention block And the vertical output features of the transposed bidirectional attention fusion block The splicing is performed along the height dimension, and the formula is expressed as:

[0042] ;

[0043] Where: It is the concatenated output feature of the bidirectional attention fusion block; For splicing; The second dimension of the multi-scale feature map of the processed road traffic marking image is spliced;

[0044] Step S4014: Use point-by-point convolution to concatenate the output features of the bidirectional attention fusion block Perform dimensionality reduction mapping, the formula is expressed as:

[0045] ;in ;

[0046] Where: It is point-wise convolution; The output features after dimensionality reduction mapping of the concatenated output features of the bidirectional attention fusion block, The number of output channels of the multi-scale feature map of the processed road traffic marking image; is the number of channels after the dimensionality reduction of the multi-scale feature map of the processed road traffic marking image; max is the maximum value function; is the dimensionality reduction factor;

[0047] Step S4015: Output features after dimensionality reduction mapping of the concatenated output features of the bidirectional attention fusion block Then perform batch normalization and then pass the activation function The fusion features are obtained, and the formulas are expressed as follows:

[0048] ;

[0049] ;

[0050] Where: The output features after batch normalization of the dimension reduction mapping output features of the bidirectional attention fusion block; Represents batch normalization operation; The batch-normalized output features of the bidirectional attention fusion block are then passed through the activation function The obtained fusion features; It is a variant of the rectified linear unit activation function;

[0051] Step S4016: The batch-normalized output features of the bidirectional attention fusion block are then passed through the activation function The obtained fusion features Split by height and width, the formula is:

[0052] ;

[0053] Where: For splitting; Output features for horizontal splitting of bidirectional attention fusion blocks; The vertical split output features of the bidirectional attention fusion block;

[0054] Step S4017: Vertically split output features of the bidirectional attention fusion block Transposed back to the original shape again, the formula is expressed as:

[0055] ;

[0056] Where: The vertical split output features of the transposed bidirectional attention fusion block;

[0057] Step S4018: Use point-by-point convolution to output features of the horizontal split of the bidirectional attention fusion block And the vertical split output features of the transposed bidirectional attention fusion block Processing to generate horizontal output attention weights of bidirectional attention fusion block And the vertical output attention weights of the bidirectional attention fusion block , the formulas are expressed as:

[0058] ;

[0059] ;

[0060] Where: for Activation function; Output attention weights for the horizontal bidirectional attention fusion block; is the vertical output attention weight of the bidirectional attention fusion block;

[0061] Step S4019: Horizontal output attention weights of the bidirectional attention fusion block And the vertical output attention weights of the bidirectional attention fusion block Multiply them together to get the bidirectional fusion weight of the bidirectional attention fusion block , the formula is:

[0062] ;

[0063] Where: is the bidirectional fusion weight of the bidirectional attention fusion block;

[0064] Bidirectional fusion weights of the bidirectional attention fusion block And the input features of the bidirectional attention fusion block Multiply them together to get the final output features of the bidirectional attention fusion block , the formula is:

[0065] ;

[0066] In the formula, is the final output feature of the bidirectional attention fusion block.

[0067] Further, the detection route of the detection head in step S5 is divided into a first branch and a second branch, and the plurality of enhanced multi-scale feature maps are processed by the first branch to obtain the bounding box positions of the plurality of road traffic marking images, and the plurality of enhanced multi-scale feature maps are processed by the second branch to obtain the category probabilities of the plurality of road traffic marking images;

[0068] There are three detection heads; several eighth multi-scale feature maps are sequentially input into the first bidirectional attention fusion block and the first second two-dimensional convolution block for processing, and then input into the first detection head; several first enhanced multi-scale feature maps are input into the second detection head; and several second enhanced multi-scale feature maps are input into the third detection head.

[0069] Further, several enhanced multi-scale feature maps are processed by the first branch to obtain the bounding box positions of several road traffic marking images, and several enhanced multi-scale feature maps are processed by the second branch to obtain the category probabilities of several road traffic marking images. The specific steps are:

[0070] Step S501: the first branch includes a first convolution layer, a second convolution layer and a first two-dimensional convolution layer; a plurality of enhanced multi-scale feature maps are input into the first convolution layer and the second convolution layer in the first branch for feature extraction, and finally the first two-dimensional convolution layer outputs the bounding box positions of a plurality of road traffic marking images;

[0071] Step S502: The second branch includes a first depth-wise separable convolution layer, a third convolution layer, a second depth-wise separable convolution layer, a fourth convolution layer, and a second two-dimensional convolution layer. Several enhanced multi-scale feature maps are sequentially input into the first depth-wise separable convolution layer, the third convolution layer, the second depth-wise separable convolution layer, the fourth convolution layer, and the second two-dimensional convolution layer in the second branch for feature extraction. Finally, the category probabilities of several road traffic marking images are output through the second two-dimensional convolution layer.

[0072] Further, in step S5, several enhanced multi-scale feature maps are input into the detection head for detection and classification, using The loss function is used as the classification loss function, The loss function is the binary cross entropy function Introduce sliding average, adaptive weighting and dynamic loss weight adjustment mechanism.

[0073] The present invention has the following technical effects:

[0074] (1) By merging, filtering, expanding and annotating the road traffic marking dataset, the label distribution of the road traffic marking dataset is made more balanced, and the detection and recognition capabilities of the road traffic marking detection model for categories with insufficient number of road traffic marking images in the road traffic marking dataset are improved.

[0075] (2) The Efficient Visual Transformer Network can significantly improve the efficiency of feature extraction by combining an efficient attention mechanism with convolution operations, and enhance the information interaction between feature channels, thereby improving the ability to capture global context information. In addition, the Efficient Visual Transformer building block in the Efficient Visual Transformer Network can improve the accuracy and speed of road traffic marking detection while ensuring efficient memory usage, especially improving the accuracy of complex road traffic marking detection.

[0076] (3) The multi-scale feature map enhancement network achieves efficient integration of features of different scales by adopting a bottom-up path structure. The multi-scale feature map enhancement network gradually upsamples low-resolution features and then fuses them with high-resolution shallow features in multiple stages, which can supplement shallow detail information and significantly optimize the feature expression of small target areas. The bidirectional attention fusion block in the multi-scale feature map enhancement network can enhance the ability to capture small target features and improve the recognition accuracy of road traffic marking images in complex backgrounds. In addition, by reducing computational complexity, the multi-scale feature map enhancement network can ensure high detection accuracy while having good real-time performance, significantly improving the robustness and practical application performance of the road traffic marking detection model.

[0077] (4) The EMASlideloss loss function replaces the binary cross entropy loss function in the original YOLOv11 network. The EMASlideloss loss function combines the sliding window mechanism with the exponential moving average strategy to dynamically adjust the weight of the loss function, effectively reducing the negative impact of large targets on small targets. In addition, the EMASlideloss loss function is more flexible in handling categories with insufficient number of road traffic marking images in the road traffic marking dataset, so that the road traffic marking detection model can improve the reasoning performance when detecting categories with insufficient number of road traffic marking images and small targets in the road traffic marking dataset, further improving the stability and generalization ability of the road traffic marking detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] A more complete understanding of exemplary embodiments of the present invention may be obtained by referring to the following drawings:

[0079] Figure 1 A schematic diagram of the structure of a road traffic marking detection model provided by an embodiment of the present invention;

[0080] Figure 2 A schematic diagram of the structure of a feature extraction network provided by an embodiment of the present invention;

[0081] Figure 3 A schematic diagram of the structure of a multi-scale feature map enhancement network provided by an embodiment of the present invention;

[0082] Figure 4 A schematic diagram of the structure of a building block of an efficient visual transformer provided by an embodiment of the present invention;

[0083] Figure 5 A schematic diagram of the structure of a detection head provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0084] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0085] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs; the terms used herein are only for the purpose of describing specific embodiments rather than limiting the present invention.

[0086] The embodiment of the present invention provides a road traffic marking detection method based on efficient visual transformation, comprising the following steps:

[0087] Step S1: merging a road traffic marking dataset, the road traffic marking dataset including a plurality of road traffic marking images, expanding and annotating the merged road traffic marking dataset to obtain an expanded and annotated road traffic marking dataset;

[0088] Step S2: construct a road traffic marking detection model, which consists of a feature extraction network, a multi-scale feature map enhancement network and a detection head;

[0089] The feature extraction network consists of an efficient visual transformer network, a fast spatial pyramid pooling block, and a cross-scale pixel-level spatial attention block, wherein the efficient visual transformer network consists of a first two-dimensional convolution block, an efficient visual transformer building block, and an efficient visual transformer downsampling block, and the efficient visual transformer building block consists of a first depthwise separable convolution block, a first residual block, a first feedforward neural network block, and a local window attention mechanism block;

[0090] The multi-scale feature map enhancement network consists of a bidirectional attention fusion block, a second 2D convolution block, a transposed convolution block, a dot product block, an enhanced cross-stage local network feature extraction block, and an addition block;

[0091] Step S3: the expanded and annotated road traffic marking data set is input into a feature extraction network for feature extraction to obtain a plurality of multi-scale feature maps; wherein the plurality of road traffic marking images in the expanded and annotated road traffic marking data set are subjected to local feature correlation calculation and feature fusion in an efficient visual transformer building block;

[0092] Step S4: several multi-scale feature maps are input into a multi-scale feature map enhancement network for feature fusion to obtain several enhanced multi-scale feature maps; wherein several multi-scale feature maps are feature selectively enhanced in a bidirectional attention fusion block;

[0093] Step S5: Several enhanced multi-scale feature maps are input into the detection head for detection and classification to obtain the final detection result.

[0094] It can be understood that the structural diagram of the road traffic marking detection model is as follows Figure 1 As shown; the structural diagram of the feature extraction network is as follows Figure 2 As shown; the structure of the multi-scale feature map enhancement network is shown as Figure 3 shown.

[0095] Specifically, step S1 includes the following steps:

[0096] Step S101: merging the CeyMo dataset and the VPGNet dataset, screening road traffic marking images that meet the road traffic marking standards, and classifying the road traffic marking images that meet the road traffic marking standards to obtain the merged and screened CeyMo dataset and VPGNet dataset, wherein the CeyMo dataset and the VPGNet dataset are both road traffic marking datasets;

[0097] Step S102: Counting the number of road traffic marking images in each category of road traffic marking images in the merged and filtered CeyMo dataset and VPGNet dataset, and identifying categories with insufficient number of road traffic marking images;

[0098] Step S103: Collect a number of road traffic marking images of the same category as the road traffic marking images whose number is insufficient, manually annotate the collected road traffic marking images, and obtain the expanded and manually annotated CeyMo dataset and VPGNet dataset.

[0099] It can be understood that the CeyMo dataset is a benchmark dataset for road marking detection with high resolution, diversity and accuracy, and the VPGNet dataset is a lane line and road sign detection and recognition dataset with rich annotation categories and scene changes. The merged CeyMo dataset and VPGNet dataset are expanded and manually annotated to construct a road traffic marking dataset containing 5958 road traffic marking images, covering 11 common traffic marking types.

[0100] Specifically, in step S3, the expanded and annotated road traffic marking data set is input into a feature extraction network for feature extraction to obtain a plurality of multi-scale feature maps; the steps include:

[0101] Step S301: the expanded and annotated road traffic marking data set is input into the efficient visual transformer network in the feature extraction network, and the efficient visual transformer network includes four first two-dimensional convolution blocks, three efficient visual transformer building blocks and two efficient visual transformer downsampling blocks; in the efficient visual transformer network, a plurality of road traffic marking images are input into the four first two-dimensional convolution blocks for downsampling and feature extraction to obtain a plurality of first multi-scale feature maps; then they are input into the first efficient visual transformer building block for local feature correlation calculation and feature fusion to obtain a plurality of second multi-scale feature maps; then they are input into the first efficient visual transformer downsampling block for downsampling and feature fusion to obtain a plurality of third multi-scale feature maps; then they are input into the second efficient visual transformer building block for local feature correlation calculation and feature fusion to obtain a plurality of fourth multi-scale feature maps; then they are input into the second efficient visual transformer downsampling block for downsampling and feature fusion to obtain a plurality of fifth multi-scale feature maps; then they are input into the third efficient visual transformer building block for local feature correlation calculation and feature fusion to obtain a plurality of sixth multi-scale feature maps;

[0102] Step S302: inputting a plurality of sixth multi-scale feature maps into a fast spatial pyramid pooling block in a feature extraction network to obtain a plurality of seventh multi-scale feature maps;

[0103] It can be understood that the fast spatial pyramid pooling block consists of a first input 1×1 convolution layer, a maximum pooling layer, and a first output 1×1 convolution layer; a plurality of sixth multi-scale feature maps are input into the first input 1×1 convolution layer in the fast spatial pyramid pooling block, the number of channels of the plurality of sixth multi-scale feature maps is halved, and then input into the maximum pooling layer for three maximum pooling processes of the same scale, and finally input into the first output 1×1 convolution layer for feature fusion, to obtain a plurality of seventh multi-scale feature maps;

[0104] Step S303: inputting a plurality of seventh multi-scale feature maps into a cross-scale pixel-level spatial attention block in a feature extraction network to obtain a plurality of eighth multi-scale feature maps, where the eighth multi-scale feature map is a 32-fold down-sampled multi-scale feature map;

[0105] It can be understood that the cross-scale pixel-level spatial attention block is composed of a second input 1×1 convolutional layer, several self-attention mechanism blocks and a second output 1×1 convolutional layer; several seventh multi-scale feature maps are input into the second 1×1 convolutional layer in the cross-scale pixel-level spatial attention block, and the number of input channels of several seventh multi-scale feature maps is expanded to twice the number of intermediate layer channels, and then a part of the seventh multi-scale feature maps retains some features through channel segmentation, and another part of the seventh multi-scale feature maps is input into several self-attention mechanism blocks, and the self-attention mechanism is used to capture cross-channel dependencies and extract global features, and then the partial features and global features are spliced, and finally input into the second output 1×1 convolutional layer for feature fusion to obtain several eighth multi-scale feature maps.

[0106] Specifically, in step S301, a plurality of first multi-scale feature maps are input into a first efficient visual transformer building block for local feature correlation calculation and feature fusion to obtain a plurality of second multi-scale feature maps, including the following steps:

[0107] like Figure 4 As shown, the efficient visual transformer building block consists of two first depthwise separable convolution blocks, five first residual blocks, two first feedforward neural network blocks, and a local window attention mechanism block;

[0108] Several first multi-scale feature maps are input into the first first depthwise separable convolution block in the efficient visual transformer building block to capture local features, and then input into the first first residual block to fuse input features and output features; then input into the first first feedforward neural network block to enhance feature expression, and then input into the second first residual block to fuse input features and output features, and then input into the local window attention mechanism block to calculate local feature correlation, and then input into the third first residual block to fuse input features and output features, and finally input into the second first depthwise separable convolution block, the fourth first residual block, the second first feedforward neural network block and the fifth first residual block in sequence to further extract and integrate features to obtain several second multi-scale feature maps.

[0109] It can be understood that the specific process of inputting the first multi-scale feature map into the efficient visual transformer building block for processing is as follows:

[0110] Step S30121: The first multi-scale feature map is input into the first first depth-separable convolutional block to capture local features by expanding the local receptive field of view, and then input into the first first residual block to fuse the input features and output features through a jump connection. The formula is expressed as:

[0111] ;

[0112] Where: The output feature of the first multi-scale feature map is processed by the first first depth-separable convolution block and the first first residual block in sequence; is the input feature of the first multi-scale feature map; is the depth-wise separable convolution function; is the residual;

[0113] Step S30122: input the first first feedforward neural network block to perform channel dimension increase and channel dimension reduction processing to enhance feature expression capability, and then input the second first residual block to fuse input features and output features through jump connection, which is expressed as:

[0114] ;

[0115] Where: The output feature of the first multi-scale feature map is processed sequentially by the first first depth-separable convolution block, the first first residual block, the first first feedforward neural network block and the second first residual block; is the number of channels of the multi-scale feature map of the processed road traffic marking image; is the rectified linear unit activation function;

[0116] Step S30123: Input the local window attention mechanism block to calculate the local feature correlation, and then input the third first residual block to fuse the input features and output features through jump connection. The formula is expressed as:

[0117] ;

[0118] Where: The output features of the first multi-scale feature map processed sequentially by the first first depth separable convolution block, the first first residual block, the first first feedforward neural network block, the second first residual block, the local window attention mechanism block and the third first residual block; is the local window attention mechanism function;

[0119] Step S30124: input the second first depth separable convolution block and the fourth first residual block, the formula is expressed as:

[0120] ;

[0121] Where: The output features of the first multi-scale feature map processed sequentially by the first first depth separable convolution block, the first first residual block, the first first feedforward neural network block, the second first residual block, the local window attention mechanism block, the third first residual block, the second first depth separable convolution block and the fourth first residual block;

[0122] Step S30125: input into the second first feedforward neural network block and the fifth first residual block for further feature extraction and integration, the formula is expressed as:

[0123] ;

[0124] Where: are the output features of several second multi-scale feature maps.

[0125] Among them, the local window attention mechanism block consists of local window division, cascade group attention mechanism block, interaction and fusion, and window restoration. The specific processing flow of the local window attention mechanism block is:

[0126] When the length and width of the resolution of the first multi-scale feature map in the input local window attention mechanism block are larger than the window size, the first multi-scale feature map in the input local window attention mechanism block is divided into multiple non-overlapping local windows, each with a size of , if the length and width cannot be If it is divisible, the feature map is zero-filled, and the formula is expressed as:

[0127] ;

[0128] Where: The output features of the first multi-scale feature map in the input local window attention mechanism block; Partition operation for local windows;

[0129] Then each window applies the cascade group attention mechanism block independently: first, each local window feature is divided into M sub-features, each sub-feature corresponds to an attention head, and each sub-feature is mapped to the query, key, and value. Then, the attention distribution is calculated for each attention head, and the output of each attention head is used as the input of the next attention head. Finally, the outputs of all attention heads are concatenated and projected back to the original dimension through a linear layer. The formula is expressed as:

[0130] ;

[0131] ;

[0132] ;

[0133] ;

[0134] ;

[0135] Where: is the output feature of the cascade group attention mechanism block; are the output features of the first multi-scale feature map in the input local window attention mechanism block. The first, second and Mth sub-features of ; For the The query vector of the attention head; For the The key vector of the attention head; For the The value vector of the attention head; are the jth linear layers for projection into query, key, and value subspaces, respectively; is the attention score of the jth attention head; is the normalization function; T is the transposition operation; is the dimension of the key vector K; is the attention bias of the jth attention head; is the output feature of the jth attention head; is the output feature of the j+1th attention head; is the projection layer; For splicing; are the attention scores of the first, second, and Zth attention heads respectively;

[0136] Output features of cascade group attention Further processing is performed through depthwise separable convolution, and finally the features calculated in each window are restored to the global feature map to obtain the output features of the local window attention mechanism block. , the formula is:

[0137] ;

[0138] Where: is the output feature of the local window attention mechanism block; Restores the operation for the window.

[0139] Specifically, in step S301, a plurality of second multi-scale feature maps are input into the first efficient visual transformer downsampling block for downsampling and feature fusion to obtain a plurality of third multi-scale feature maps. The specific steps are as follows:

[0140] The efficient visual transformer downsampling block consists of three second depth separable convolution blocks, four second residual blocks, two second feedforward neural network blocks and a compression and excitation attention mechanism block; several second multi-scale feature maps are input into the first second depth separable convolution block in the efficient visual transformer downsampling block to capture local features, and then input into the first second residual block to fuse input features and output features, and then input into the first second feedforward neural network block to enhance feature expression, and then input into the second second residual block to fuse input features and output features, and then input into the second second depth separable convolution block to capture local features, and then input into the compression and excitation attention mechanism block for global pooling processing and calculation of channel attention weights, and then input into the third second depth separable convolution block to capture local features, and then input into the third second residual block to fuse input features and output features, and then input into the second second feedforward neural network block to enhance feature expression, and then input into the fourth second residual block to fuse input features and output features to obtain several third multi-scale feature maps.

[0141] Specifically, step S4 includes the following steps:

[0142] Step S401: the multi-scale feature map enhancement network consists of five bidirectional attention fusion blocks, three second two-dimensional convolution blocks, two transposed convolution blocks, two dot product blocks, two enhanced cross-stage local network feature extraction blocks and two addition blocks; a plurality of eighth multi-scale feature maps are input into the first bidirectional attention fusion block in the multi-scale feature map enhancement network for feature selective enhancement, and then input into the first second two-dimensional convolution block for convolution operation, and then input into the first transposed convolution block for upsampling, and then input into the third bidirectional attention fusion block for feature selective enhancement;

[0143] After the plurality of fourth multi-scale feature maps are sequentially input into the second bidirectional attention fusion block and the second second two-dimensional convolution block for processing, shallow features of the plurality of fourth multi-scale feature maps are obtained;

[0144] The selectively enhanced eighth multi-scale feature maps are then sequentially input into the first dot product block and the first addition block, and feature fused with the shallow features of the fourth multi-scale feature maps to obtain the fused 16-fold downsampled multi-scale feature maps;

[0145] The bidirectional attention fusion block performs feature selective enhancement by processing long-distance dependency information through global pooling in the horizontal and vertical directions, and generates directional weights based on channel information to enhance the feature expression of small targets in road traffic marking images and key areas in complex backgrounds.

[0146] Step S402: the fused multiple 16-fold downsampled multi-scale feature maps are input into the first enhanced cross-stage local network feature extraction block to extract high-order semantic information and integrate multi-path information, thereby obtaining multiple first enhanced multi-scale feature maps;

[0147] Or the fused 16-fold downsampled multi-scale feature maps are input into the second transposed convolution block for upsampling to obtain a number of 8-fold downsampled multi-scale feature maps, which are then input into the fifth bidirectional attention fusion block for feature selective enhancement, and then sequentially input into the second dot product block and the second addition block for feature fusion with the shallow features of the second multi-scale feature maps to obtain a number of fused 8-fold downsampled multi-scale feature maps; wherein the several second multi-scale feature maps are sequentially input into the fourth bidirectional attention fusion block and the third second two-dimensional convolution block for processing to obtain the shallow features of the several second multi-scale feature maps;

[0148] Step S403: Several fused 8-fold downsampled multi-scale feature maps are input into a second enhanced cross-stage local network feature extraction block to extract high-order semantic information and integrate multi-path information to obtain several second enhanced multi-scale feature maps.

[0149] Specifically, in step S401, feature selective enhancement is performed in the bidirectional attention fusion block, and the specific processing flow is as follows:

[0150] Step S4011: Input features of the bidirectional attention fusion block First, through the adaptive average pooling operation, aggregation is performed in the width and height directions respectively to obtain the horizontal output features of the bidirectional attention fusion block And the vertical output features of the bidirectional attention fusion block , the formulas are expressed as:

[0151] ;

[0152] ;

[0153] Where: It is the input feature of the bidirectional attention fusion block; It is the horizontal output feature of the bidirectional attention fusion block; Represents the vertical output features of the bidirectional attention fusion block; It is an adaptive average pooling operation; is the height of the multi-scale feature map of the processed road traffic marking image; is the width of the multi-scale feature map of the processed road traffic marking image;

[0154] Step S4012: Vertical output features of the bidirectional attention fusion block The height and width are transposed, and the formula is expressed as:

[0155] ;

[0156] Where: is the vertical output feature of the transposed bidirectional attention fusion block; RearrangeDims is the operation of exchanging the order of dimensions; To keep the order of the 0th dimension, the batch remains unchanged; to keep the order of the 1st dimension, the number of channels remains unchanged; to move the order of the 3rd dimension, the width moves to the 2nd position; to move the order of the 2nd dimension, the height moves to the 3rd position;

[0157] Step S4013: Fusion of horizontal output features of the bidirectional attention block And the vertical output features of the transposed bidirectional attention fusion block The splicing is performed along the height dimension, and the formula is expressed as:

[0158] ;

[0159] Where: It is the concatenated output feature of the bidirectional attention fusion block; For splicing; The second dimension of the multi-scale feature map of the processed road traffic marking image is spliced;

[0160] Step S4014: Use point-by-point convolution to concatenate the output features of the bidirectional attention fusion block Perform dimensionality reduction mapping, the formula is expressed as:

[0161] ;in ;

[0162] Where: It is point-wise convolution; The output features after dimensionality reduction mapping of the concatenated output features of the bidirectional attention fusion block, The number of output channels of the multi-scale feature map of the processed road traffic marking image; is the number of channels after the dimensionality reduction of the multi-scale feature map of the processed road traffic marking image; max is the maximum value function; is the dimensionality reduction factor;

[0163] Step S4015: Output features after dimensionality reduction mapping of the concatenated output features of the bidirectional attention fusion block Then perform batch normalization and then pass the activation function The fusion features are obtained, and the formulas are expressed as follows:

[0164] ;

[0165] ;

[0166] Where: The output features after batch normalization of the dimension reduction mapping output features of the bidirectional attention fusion block; Represents batch normalization operation; The batch-normalized output features of the bidirectional attention fusion block are then passed through the activation function The obtained fusion features; It is a variant of the rectified linear unit activation function;

[0167] Step S4016: The batch-normalized output features of the bidirectional attention fusion block are then passed through the activation function The obtained fusion features Split by height and width, the formula is:

[0168] ;

[0169] Where: For splitting; Output features for horizontal splitting of bidirectional attention fusion blocks; The vertical split output features of the bidirectional attention fusion block;

[0170] Step S4017: Vertically split output features of the bidirectional attention fusion block Transposed back to the original shape again, the formula is expressed as:

[0171] ;

[0172] Where: The vertical split output features of the transposed bidirectional attention fusion block;

[0173] Step S4018: Use point-by-point convolution to output features of the horizontal split of the bidirectional attention fusion block And the vertical split output features of the transposed bidirectional attention fusion block Processing to generate horizontal output attention weights of bidirectional attention fusion block And the vertical output attention weights of the bidirectional attention fusion block , the formulas are expressed as:

[0174] ;

[0175] ;

[0176] Where: for Activation function; Output attention weights for the horizontal bidirectional attention fusion block; is the vertical output attention weight of the bidirectional attention fusion block;

[0177] Step S4019: Horizontal output attention weights of the bidirectional attention fusion block And the vertical output attention weights of the bidirectional attention fusion block Multiply them together to get the bidirectional fusion weight of the bidirectional attention fusion block , the formula is:

[0178] ;

[0179] Where: is the bidirectional fusion weight of the bidirectional attention fusion block;

[0180] Bidirectional fusion weights of the bidirectional attention fusion block And the input features of the bidirectional attention fusion block Multiply them together to get the final output features of the bidirectional attention fusion block , the formula is:

[0181] ;

[0182] In the formula, is the final output feature of the bidirectional attention fusion block.

[0183] Specifically, the detection route of the detection head in step S5 is divided into a first branch and a second branch, a plurality of enhanced multi-scale feature maps are processed by the first branch to obtain the bounding box positions of a plurality of road traffic marking images, and a plurality of enhanced multi-scale feature maps are processed by the second branch to obtain the category probabilities of a plurality of road traffic marking images;

[0184] There are three detection heads; several eighth multi-scale feature maps are sequentially input into the first bidirectional attention fusion block and the first second two-dimensional convolution block for processing, and then input into the first detection head; several first enhanced multi-scale feature maps are input into the second detection head; and several second enhanced multi-scale feature maps are input into the third detection head.

[0185] The structural diagram of the detection head is as follows: Figure 5 shown.

[0186] Specifically, several enhanced multi-scale feature maps are processed by the first branch to obtain the bounding box positions of several road traffic marking images, and several enhanced multi-scale feature maps are processed by the second branch to obtain the category probabilities of several road traffic marking images. The specific steps are:

[0187] Step S501: the first branch includes a first convolution layer, a second convolution layer and a first two-dimensional convolution layer; a plurality of enhanced multi-scale feature maps are input into the first convolution layer and the second convolution layer in the first branch for feature extraction, and finally the first two-dimensional convolution layer outputs the bounding box positions of a plurality of road traffic marking images;

[0188] Step S502: The second branch includes a first depth-wise separable convolution layer, a third convolution layer, a second depth-wise separable convolution layer, a fourth convolution layer, and a second two-dimensional convolution layer. Several enhanced multi-scale feature maps are sequentially input into the first depth-wise separable convolution layer, the third convolution layer, the second depth-wise separable convolution layer, the fourth convolution layer, and the second two-dimensional convolution layer in the second branch for feature extraction. Finally, the category probabilities of several road traffic marking images are output through the second two-dimensional convolution layer.

[0189] Specifically, in step S5, several enhanced multi-scale feature maps are input into the detection head for detection and classification. The loss function is used as the classification loss function, The loss function is the binary cross entropy function Introduce sliding average, adaptive weighting and dynamic loss weight adjustment mechanism.

[0190] Step S5011: The binary cross entropy function The formula is:

[0191] ;

[0192] Where: is the binary cross entropy function ; N is the total number of road traffic marking images in the road traffic marking dataset; is the category label of the i-th road traffic marking image; represents the probability that the i-th road traffic marking image is predicted to be each road traffic marking image category;

[0193] Step S5012: Dynamically track the mean of the intersection-over-combination ratio during the training process through a sliding average, which is used as a parameter for weight adjustment. The sliding average formula is expressed as:

[0194] ;

[0195] in ;

[0196] Where: IOU is the intersection-over-union ratio between the predicted box and the real box of the road traffic marking image; is the moving average intersection-combination ratio; is the dynamically adjusted attenuation coefficient; is the intersection-over-union ratio of the predicted box and the real box of the current road traffic marking image; is the initial fixed attenuation factor; is the current training iteration number; A hyperparameter to control the smooth update speed of IOU;

[0197] Step S5013: Compare the intersection of the predicted frame and the real frame based on the current road traffic marking image IoU The road traffic marking image is divided into simple road traffic marking image, medium road traffic marking image and difficult road traffic marking image according to the relationship between them, and weighted processing is performed respectively. The adaptive weighted formula is expressed as:

[0198] ;

[0199] in ;

[0200] ;

[0201] ;

[0202] ;

[0203] ;

[0204] ;

[0205] Where: is the weight after adaptive weighting of the road traffic marking image; are the weights of simple, medium and difficult road traffic marking images respectively; It is a simple road traffic marking image; It is a medium road traffic marking image; Difficult road traffic marking images; Indicates The IOU value of the road traffic marking image prediction;

[0206] Step S5014: Combining the weights after adaptive weighting of the road traffic marking image ,right After correction, the final loss function The calculation formula is expressed as:

[0207] ;

[0208] Where: is the final loss function.

Claims

1. A road traffic marking detection method based on efficient visual transformation, characterized in that: The following steps are involved: Step S1: merging a road traffic marking dataset, the road traffic marking dataset including a plurality of road traffic marking images, expanding and annotating the merged road traffic marking dataset to obtain an expanded and annotated road traffic marking dataset; Step S2: construct a road traffic marking detection model, which consists of a feature extraction network, a multi-scale feature map enhancement network and a detection head; The feature extraction network consists of an efficient visual transformer network, a fast spatial pyramid pooling block, and a cross-scale pixel-level spatial attention block, wherein the efficient visual transformer network consists of a first two-dimensional convolution block, an efficient visual transformer building block, and an efficient visual transformer downsampling block, and the efficient visual transformer building block consists of a first depthwise separable convolution block, a first residual block, a first feedforward neural network block, and a local window attention mechanism block; The multi-scale feature map enhancement network consists of a bidirectional attention fusion block, a second 2D convolution block, a transposed convolution block, a dot product block, an enhanced cross-stage local network feature extraction block, and an addition block; Step S3: the expanded and annotated road traffic marking data set is input into a feature extraction network for feature extraction to obtain a plurality of multi-scale feature maps; wherein the plurality of road traffic marking images in the expanded and annotated road traffic marking data set are subjected to local feature correlation calculation and feature fusion in an efficient visual transformer building block; Step S4: several multi-scale feature maps are input into a multi-scale feature map enhancement network for feature fusion to obtain several enhanced multi-scale feature maps; wherein several multi-scale feature maps are feature selectively enhanced in a bidirectional attention fusion block; Step S5: Several enhanced multi-scale feature maps are input into the detection head for detection and classification to obtain the final detection result.

2. The road traffic marking detection method based on efficient visual transformation according to claim 1, characterized in that: Step S1 includes the following steps: Step S101: merging the CeyMo dataset and the VPGNet dataset, screening road traffic marking images that meet the road traffic marking standards, and classifying the road traffic marking images that meet the road traffic marking standards to obtain the merged and screened CeyMo dataset and VPGNet dataset, wherein the CeyMo dataset and the VPGNet dataset are both road traffic marking datasets; Step S102: Counting the number of road traffic marking images in each category of road traffic marking images in the merged and filtered CeyMo dataset and VPGNet dataset, and identifying categories with insufficient number of road traffic marking images; Step S103: Collect a number of road traffic marking images of the same category as the road traffic marking images whose number is insufficient, manually annotate the collected road traffic marking images, and obtain the expanded and manually annotated CeyMo dataset and VPGNet dataset.

3. The road traffic marking detection method based on efficient visual transformation according to claim 2, characterized in that: In step S3, the expanded and annotated road traffic marking data set is input into a feature extraction network for feature extraction to obtain a plurality of multi-scale feature maps; the steps include: Step S301: the expanded and annotated road traffic marking data set is input into the efficient visual transformer network in the feature extraction network, and the efficient visual transformer network includes four first two-dimensional convolution blocks, three efficient visual transformer building blocks and two efficient visual transformer downsampling blocks; in the efficient visual transformer network, a plurality of road traffic marking images are input into the four first two-dimensional convolution blocks for downsampling and feature extraction to obtain a plurality of first multi-scale feature maps; then they are input into the first efficient visual transformer building block for local feature correlation calculation and feature fusion to obtain a plurality of second multi-scale feature maps; then they are input into the first efficient visual transformer downsampling block for downsampling and feature fusion to obtain a plurality of third multi-scale feature maps; then they are input into the second efficient visual transformer building block for local feature correlation calculation and feature fusion to obtain a plurality of fourth multi-scale feature maps; then they are input into the second efficient visual transformer downsampling block for downsampling and feature fusion to obtain a plurality of fifth multi-scale feature maps; then they are input into the third efficient visual transformer building block for local feature correlation calculation and feature fusion to obtain a plurality of sixth multi-scale feature maps; Step S302: inputting a plurality of sixth multi-scale feature maps into a fast spatial pyramid pooling block in a feature extraction network to obtain a plurality of seventh multi-scale feature maps; Step S303: Several seventh multi-scale feature maps are input into the cross-scale pixel-level spatial attention block in the feature extraction network to obtain several eighth multi-scale feature maps, where the eighth multi-scale feature map is a 32-fold down-sampled multi-scale feature map.

4. The road traffic marking detection method based on efficient visual transformation according to claim 3, characterized in that: In step S301, a plurality of first multi-scale feature maps are input into a first efficient visual transformer building block for local feature correlation calculation and feature fusion to obtain a plurality of second multi-scale feature maps, including the following steps: The efficient visual transformer building block consists of two first depthwise separable convolution blocks, five first residual blocks, two first feedforward neural network blocks and a local window attention mechanism block; a number of first multi-scale feature maps are input into the first first depthwise separable convolution block in the efficient visual transformer building block to capture local features, and then input into the first first residual block to fuse input features and output features; then input into the first first feedforward neural network block to enhance feature expression, and then input into the second first residual block to fuse input features and output features, and then input into the local window attention mechanism block to calculate local feature correlation, and then input into the third first residual block to fuse input features and output features, and finally input into the second first depthwise separable convolution block, the fourth first residual block, the second first feedforward neural network block and the fifth first residual block in sequence to further extract and integrate features to obtain a number of second multi-scale feature maps.

5. The road traffic marking detection method based on efficient visual transformation according to claim 4, characterized in that: In step S301, the plurality of second multi-scale feature maps are input into the first efficient visual transformer downsampling block for downsampling and feature fusion to obtain a plurality of third multi-scale feature maps. The specific steps are as follows: The efficient visual transformer downsampling block consists of three second depth separable convolution blocks, four second residual blocks, two second feedforward neural network blocks and a compression and excitation attention mechanism block; several second multi-scale feature maps are input into the first second depth separable convolution block in the efficient visual transformer downsampling block to capture local features, and then input into the first second residual block to fuse input features and output features, and then input into the first second feedforward neural network block to enhance feature expression, and then input into the second second residual block to fuse input features and output features, and then input into the second second depth separable convolution block to capture local features, and then input into the compression and excitation attention mechanism block for global pooling processing and calculation of channel attention weights, and then input into the third second depth separable convolution block to capture local features, and then input into the third second residual block to fuse input features and output features, and then input into the second second feedforward neural network block to enhance feature expression, and then input into the fourth second residual block to fuse input features and output features to obtain several third multi-scale feature maps.

6. The road traffic marking detection method based on efficient visual transformation according to claim 5, characterized in that: Step S4 includes the following steps: Step S401: the multi-scale feature map enhancement network consists of five bidirectional attention fusion blocks, three second two-dimensional convolution blocks, two transposed convolution blocks, two dot product blocks, two enhanced cross-stage local network feature extraction blocks and two addition blocks; a plurality of eighth multi-scale feature maps are input into the first bidirectional attention fusion block in the multi-scale feature map enhancement network for feature selective enhancement, and then input into the first second two-dimensional convolution block for convolution operation, and then input into the first transposed convolution block for upsampling, and then input into the third bidirectional attention fusion block for feature selective enhancement; After the plurality of fourth multi-scale feature maps are sequentially input into the second bidirectional attention fusion block and the second second two-dimensional convolution block for processing, shallow features of the plurality of fourth multi-scale feature maps are obtained; The selectively enhanced eighth multi-scale feature maps are then sequentially input into the first dot product block and the first addition block, and feature fused with the shallow features of the fourth multi-scale feature maps to obtain the fused 16-fold downsampled multi-scale feature maps; The feature selective enhancement in the bidirectional attention fusion block is to process long-distance dependency information through global pooling in the horizontal and vertical directions, and generate directional weights in combination with channel information; Step S402: the fused multiple 16-fold downsampled multi-scale feature maps are input into the first enhanced cross-stage local network feature extraction block to extract high-order semantic information and integrate multi-path information, thereby obtaining multiple first enhanced multi-scale feature maps; Or the fused 16-fold downsampled multi-scale feature maps are input into the second transposed convolution block for upsampling to obtain a number of 8-fold downsampled multi-scale feature maps, which are then input into the fifth bidirectional attention fusion block for feature selective enhancement, and then sequentially input into the second dot product block and the second addition block for feature fusion with the shallow features of the second multi-scale feature maps to obtain a number of fused 8-fold downsampled multi-scale feature maps; wherein the several second multi-scale feature maps are sequentially input into the fourth bidirectional attention fusion block and the third second two-dimensional convolution block for processing to obtain the shallow features of the several second multi-scale feature maps; Step S403: Several fused 8-fold downsampled multi-scale feature maps are input into a second enhanced cross-stage local network feature extraction block to extract high-order semantic information and integrate multi-path information to obtain several second enhanced multi-scale feature maps.

7. The road traffic marking detection method based on efficient visual transformation according to claim 6, characterized in that: In step S401, feature selective enhancement is performed in the bidirectional attention fusion block. The specific processing flow is as follows: Step S4011: Input features of the bidirectional attention fusion block First, through the adaptive average pooling operation, aggregation is performed in the width and height directions respectively to obtain the horizontal output features of the bidirectional attention fusion block And the vertical output features of the bidirectional attention fusion block , the formulas are expressed as: ； ； Where: It is the input feature of the bidirectional attention fusion block; It is the horizontal output feature of the bidirectional attention fusion block; Represents the vertical output features of the bidirectional attention fusion block; It is an adaptive average pooling operation; is the height of the multi-scale feature map of the processed road traffic marking image; is the width of the multi-scale feature map of the processed road traffic marking image; Step S4012: Vertical output features of the bidirectional attention fusion block The height and width are transposed, and the formula is expressed as: ； Where: is the vertical output feature of the transposed bidirectional attention fusion block; RearrangeDims is an operation to exchange the order of dimensions; To preserve the order of the 0th dimension, the batch is kept unchanged; Keep the order of the first dimension, and the number of channels will remain unchanged; If the third dimension is moved, the width will be moved to the second position; if the second dimension is moved, the height will be moved to the third position. Step S4013: Fusion of horizontal output features of the bidirectional attention block And the vertical output features of the transposed bidirectional attention fusion block The splicing is performed along the height dimension, and the formula is expressed as: ； Where: It is the concatenated output feature of the bidirectional attention fusion block; For splicing; The second dimension of the multi-scale feature map of the processed road traffic marking image is spliced; Step S4014: Use point-by-point convolution to concatenate the output features of the bidirectional attention fusion block Perform dimensionality reduction mapping, the formula is expressed as: ;in ; Where: It is point-wise convolution; The output features after dimensionality reduction mapping of the concatenated output features of the bidirectional attention fusion block, The number of output channels of the multi-scale feature map of the processed road traffic marking image; is the number of channels after the dimensionality reduction of the multi-scale feature map of the processed road traffic marking image; max is the maximum value function; is the dimensionality reduction factor; Step S4015: Output features after dimensionality reduction mapping of the concatenated output features of the bidirectional attention fusion block Then perform batch normalization and then pass the activation function The fusion features are obtained, and the formulas are expressed as follows: ； ； Where: The output features after batch normalization of the dimension reduction mapping output features of the bidirectional attention fusion block; Represents batch normalization operation; The batch-normalized output features of the bidirectional attention fusion block are then passed through the activation function The obtained fusion features; It is a variant of the rectified linear unit activation function; Step S4016: The batch-normalized output features of the bidirectional attention fusion block are then passed through the activation function The obtained fusion features Split by height and width, the formula is: ； Where: For splitting; Output features for horizontal splitting of bidirectional attention fusion blocks; The vertical split output features of the bidirectional attention fusion block; Step S4017: Vertically split output features of the bidirectional attention fusion block Transposed back to the original shape again, the formula is expressed as: ； Where: The vertical split output features of the transposed bidirectional attention fusion block; Step S4018: Use point-by-point convolution to output features of the horizontal split of the bidirectional attention fusion block And the vertical split output features of the transposed bidirectional attention fusion block Processing to generate horizontal output attention weights of bidirectional attention fusion block And the vertical output attention weights of the bidirectional attention fusion block , the formulas are expressed as: ； ； Where: for Activation function; Output attention weights for the horizontal bidirectional attention fusion block; is the vertical output attention weight of the bidirectional attention fusion block; Step S4019: Horizontal output attention weights of the bidirectional attention fusion block And the vertical output attention weights of the bidirectional attention fusion block Multiply them together to get the bidirectional fusion weight of the bidirectional attention fusion block , the formula is: ； Where: is the bidirectional fusion weight of the bidirectional attention fusion block; Bidirectional fusion weights of the bidirectional attention fusion block And the input features of the bidirectional attention fusion block Multiply them together to get the final output features of the bidirectional attention fusion block , the formula is: ； In the formula, is the final output feature of the bidirectional attention fusion block.

8. The method for detecting road traffic markings based on efficient visual transformation according to claim 7, characterized in that: The detection route of the detection head in step S5 is divided into a first branch and a second branch, and the plurality of enhanced multi-scale feature maps are processed by the first branch to obtain the bounding box positions of the plurality of road traffic marking images, and the plurality of enhanced multi-scale feature maps are processed by the second branch to obtain the category probabilities of the plurality of road traffic marking images; There are three detection heads; several eighth multi-scale feature maps are sequentially input into the first bidirectional attention fusion block and the first second two-dimensional convolution block for processing, and then input into the first detection head; several first enhanced multi-scale feature maps are input into the second detection head; and several second enhanced multi-scale feature maps are input into the third detection head.

9. The road traffic marking detection method based on efficient visual transformation according to claim 8, characterized in that: Several enhanced multi-scale feature maps are processed by the first branch to obtain the bounding box positions of several road traffic marking images, and several enhanced multi-scale feature maps are processed by the second branch to obtain the category probabilities of several road traffic marking images. The specific steps are: Step S501: the first branch includes a first convolution layer, a second convolution layer and a first two-dimensional convolution layer; a plurality of enhanced multi-scale feature maps are input into the first convolution layer and the second convolution layer in the first branch for feature extraction, and finally the first two-dimensional convolution layer outputs the bounding box positions of a plurality of road traffic marking images; Step S502: The second branch includes a first depth-wise separable convolution layer, a third convolution layer, a second depth-wise separable convolution layer, a fourth convolution layer, and a second two-dimensional convolution layer. Several enhanced multi-scale feature maps are sequentially input into the first depth-wise separable convolution layer, the third convolution layer, the second depth-wise separable convolution layer, the fourth convolution layer, and the second two-dimensional convolution layer in the second branch for feature extraction. Finally, the category probabilities of several road traffic marking images are output through the second two-dimensional convolution layer.

10. The road traffic marking detection method based on efficient visual transformation according to claim 9, characterized in that: In step S5, several enhanced multi-scale feature maps are input into the detection head for detection and classification. The loss function is used as the classification loss function, The loss function is the binary cross entropy function Introduce sliding average, adaptive weighting and dynamic loss weight adjustment mechanism.

Images