A road traffic marking detection method based on aggregated prompt enhancement

Abstract

The invention discloses a road traffic marking detection method based on aggregated prompt enhancement, comprising the following steps: collecting road traffic marking images; constructing a traffic marking detection model, wherein the traffic marking detection model consists of an aggregated prompt enhancer, a feature extraction backbone network and a cross-stage feature fusion module; using the aggregated prompt enhancer to perform data enhancement on the traffic marking images in a road traffic marking data set, and inputting the enhanced images into the feature extraction backbone network for feature extraction, and fusing the extracted features through the cross-stage feature fusion module to obtain a final detection result; the invention introduces an aggregated prompt enhancer to perform adaptive enhancement on the road traffic marking images, thereby improving the detection accuracy and robustness of the traffic marking detection model; through upper and lower anchor feature extraction blocks, remote context information can be introduced and captured in parallel, thereby effectively solving the problems of large target scale changes and diverse ranging environments in road traffic marking detection tasks.

Description

Technical Field

[0001] The invention relates to the technical field of road traffic marking detection, and in particular to a road traffic marking detection method based on aggregated prompt enhancement. Background Art

[0002] As an important part of road traffic, road traffic markings play a key role in guiding traffic and indicating driving directions. Accurate identification and detection of road traffic markings are of great significance to traffic management and the realization of autonomous driving technology. First of all, accurate traffic marking detection can provide drivers with timely and accurate lane information. Secondly, the autonomous driving system needs to understand the road conditions and plan the driving path by detecting road markings. High-precision marking detection technology is an important basis for the safe and reliable operation of autonomous vehicles. Finally, road marking detection technology also plays an important role in intelligent transportation systems. It can provide real-time road marking status information to traffic management departments, and can promptly detect and report damage and wear of road markings, provide data support for road maintenance and marking redrawing, and ensure that the markings are clear and visible.

[0003] The existing road traffic marking detection models generally adopt traditional deep learning methods, which are insufficient in detection accuracy and difficult to deal with the problem of image quality degradation under complex weather conditions. For example, in low-light environments such as night and cloudy days, the edge clarity and contrast of road markings are reduced, affecting the detection accuracy. In addition, in rainy and snowy weather, water droplets or snowflakes will adhere to the road markings, causing the edges of the markings to be blurred or completely blocked, increasing the difficulty of detection. In addition, the urban road traffic environment is complex and changeable, and traffic markings may be blocked by vehicles or other obstacles, resulting in the model being unable to accurately identify and distinguish different traffic markings. In addition, during the driving process of the vehicle, the scale of the traffic markings captured by the on-board camera often varies greatly, which requires not only the ability to detect large targets at normal distances, but also the ability to detect small targets at long distances. Traditional image processing methods often cannot provide sufficiently accurate traffic marking detection results, and are prone to false detection or missed detection. Further research and improvement are needed to improve the model performance and generalization ability to cope with complex application environments. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a road traffic marking detection method based on aggregated prompt enhancement, which aims to solve the problems mentioned in the background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a road traffic marking detection method based on aggregated prompt enhancement, comprising the following steps:

[0006] Step S1: construct a road traffic marking dataset, which includes a number of road traffic marking images;

[0007] Step S2: constructing a traffic marking detection model, wherein the traffic marking detection model is composed of an aggregated prompt enhancer, a feature extraction backbone network, and a cross-stage feature fusion module; the aggregated prompt enhancer is set at the input end of the feature extraction backbone network;

[0008] The feature extraction backbone network consists of a first convolution block, a second convolution block, a first context anchor feature extraction block, a third convolution block, a second context anchor feature extraction block, a fourth convolution block, a third context anchor feature extraction block, a fifth convolution block, a fourth context anchor feature extraction block and a fast spatial pyramid pooling block, which are connected in sequence;

[0009] The cross-stage feature fusion module includes a first feature fusion block, a second feature fusion block, a third feature fusion block, a fourth feature fusion block, a sixth convolution block, a seventh convolution block, a first detection block, a second detection block and a third detection block;

[0010] Step S3: using an aggregated prompt enhancer to perform data enhancement on the traffic marking images in the road traffic marking dataset;

[0011] Step S4: inputting the traffic line image after data enhancement into the feature extraction backbone network to extract features and obtain a feature map;

[0012] Step S5: Input the feature map extracted in S4 into the cross-stage feature fusion module for feature fusion and obtain the final detection result.

[0013] Furthermore, the first context anchor feature extraction block, the second context anchor feature extraction block, the third context anchor feature extraction block and the fourth context anchor feature extraction block have the same structure, all of which are composed of replacing the bottleneck module in the original c2f module with the context anchor attention block; the first context anchor feature extraction block, the second context anchor feature extraction block, the third context anchor feature extraction block and the fourth context anchor feature extraction block are all three-layer structures, the first layer and the third layer are convolutional layers with a size of 1×1 and a stride of 1, and the second layer is three context anchor attention blocks that interact with each other.

[0014] Furthermore, the processing flow of the first context anchor feature extraction block, the second context anchor feature extraction block, the third context anchor feature extraction block and the fourth context anchor feature extraction block is as follows:

[0015] The input of the first context anchor feature extraction block, the second context anchor feature extraction block, the third context anchor feature extraction block and the fourth context anchor feature extraction block first generates an intermediate feature map through the convolution layer of the first layer. After the segmentation operation, the generated intermediate feature map is split into two parts. One part is input into the three context anchor attention blocks that interact with each other in the second layer, and the other part is fused in series with the outputs of the first two context anchor attention blocks in the second layer and then input into the convolution layer of the third layer to obtain the output of the context anchor feature extraction block.

[0016] Furthermore, the processing flow of the traffic marking detection model is as follows:

[0017] The output of the aggregated prompt enhancer, i.e., the traffic marking image after data enhancement, is sequentially passed through the first convolution block, the second convolution block, the first context anchor feature extraction block, the third convolution block, the second context anchor feature extraction block, the fourth convolution block, the third context anchor feature extraction block, the fifth convolution block, the fourth context anchor feature extraction block and the fast spatial pyramid pooling block; the output of the fast spatial pyramid pooling block is upsampled and fused in series with the output of the third context anchor feature extraction block, and the fusion is input into the first feature fusion block after the fusion is in series; the output of the first feature fusion block is upsampled and fused in series with the output of the second context anchor feature extraction block, and the fusion is input into the second feature fusion block after the fusion is in series; the output of the second feature fusion block is respectively input into the first detection block and the sixth convolution block; the output of the sixth convolution block is fused in series with the output of the first feature fusion module, and the fusion is input into the third feature fusion block after the fusion is in series; the output of the third feature fusion block is respectively input into the second detection block and the seventh convolution block; the output of the seventh convolution block is fused in series with the output of the fast spatial pyramid pooling block, and the output of the fourth feature fusion block is input into the third detection block;

[0018] The outputs of the first detection block, the second detection block and the third detection block are the final detection results output by the traffic marking detection model.

[0019] Furthermore, the aggregated prompt enhancer is composed of an aggregated lifting generation block, a first content-driven prompt block, a second content-driven prompt block, a third content-driven prompt block, a first receptive field attention convolution, a second receptive field attention convolution, a third receptive field attention convolution, a fourth receptive field attention convolution and a fifth receptive field attention convolution; the specific process of the aggregated prompt enhancer performing data enhancement on the traffic marking image in the road traffic marking dataset is as follows:

[0020] First, the first receptive field attention convolution is used to obtain the high-resolution features of the input image, i.e., the road traffic marking image, and the high-resolution features are input into the second receptive field attention convolution. The output of the second receptive field attention convolution is downsampled to the third receptive field attention convolution. The output of the third receptive field attention convolution is downsampled to the fourth receptive field attention convolution. The output of the fourth receptive field attention convolution is downsampled to the fifth receptive field attention convolution.

[0021] Let the initial prompt of the aggregated lifting generation block be expressed as ∈ ,in, , , Respectively represent the initial prompt height, width and channels, represents the set of real numbers; first, the initial prompt After the Hardsish activation function, the first multi-scale prompt sequence is generated, and the first multi-scale prompt sequence is input into the first transposed convolutional layer with a step size of 2 to generate prompts , will prompt After the Hardsish activation function, a second multi-scale prompt sequence is generated, which is input into the second transposed convolutional layer with a step size of 2 to generate prompts. ;

[0022] The output of the fifth receptive field attention convolution and the initial prompt Input the first content-driven prompt block for interaction. After the interaction is completed, perform upsampling operation and fused in series with the output of the fourth receptive field attention convolution. After the series fusion, input the sixth receptive field attention convolution for processing. The output of the sixth receptive field attention convolution and the prompt are combined. The second content-driven prompt block is input for interaction. After the interaction is completed, the upsampling operation is performed and the output of the third receptive field attention convolution is fused in series. After the series fusion, the seventh receptive field attention convolution is input for processing. The output of the seventh receptive field attention convolution and the prompt are combined. The third content-driven prompt block is input for interaction. After the interaction is completed, an upsampling operation is performed and the output of the second receptive field attention convolution is fused in series. After the series fusion, the eighth receptive field attention convolution is input for processing. Finally, the output of the eighth receptive field attention convolution and the road traffic marking image are added by element-wise addition to obtain the output of the aggregated prompt enhancer, that is, the traffic marking image after data enhancement.

[0023] Furthermore, the second receptive field attention convolution, the third receptive field attention convolution, the fourth receptive field attention convolution and the fifth receptive field attention convolution all have a step size of 1 and a size of 3×3.

[0024] Furthermore, the contextual anchor attention block consists of an average pooling layer, a 1×1 convolutional layer, a depthwise convolutional layer, a depthwise convolutional layer, and a convolutional layer connected sequentially;

[0025] The processing flow of the contextual anchor attention block is:

[0026] The input of the contextual anchor attention block is first average pooled through the average pooling layer, and then convolved through a 1×1 convolutional layer to obtain the output :

[0027] ;

[0028] In the formula, represents the average pooling operation; It represents the output of the contextual anchor attention block after the input passes through the average pooling layer and the 1×1 convolution layer in sequence; represents a 1×1 convolutional layer; represents the input of the contextual anchor attention block; Represents a specific channel in the input of the contextual anchor attention block, ranging from 0 to , Indicates the maximum index value in a specific channel number;

[0029] Will After two deep convolutional layers in sequence, it is expressed as:

[0030] ;

[0031] ;

[0032] In the formula, express Output after the first deep convolutional layer; express Output after the second deep convolutional layer; represents a deep convolutional layer;

[0033] The output of two deep convolutional layers After another 1×1 convolutional layer and Sigmoid activation function, the output of the contextual anchor attention block is obtained. , Indicates the number of channels , the height is , the width is A three-dimensional tensor of ;

[0034] It is expressed as:

[0035] .

[0036] Furthermore, the road traffic marking dataset is used to train the traffic marking detection model in a supervised learning manner, using the binary cross entropy loss function As the classification loss function, CIOU loss function and DFL loss function are used as regression loss functions; training is performed for 300 iterations with a learning rate of 0.01, and Lion optimizer is used for optimization. After all training is completed, the parameters of the optimal traffic marking detection model are saved, and finally tested and evaluated on the test set, which is obtained by dividing the road traffic marking data set;

[0037] Binary Cross Entropy Function It is expressed as:

[0038] ;

[0039] In the formula, Represents the total number of samples, i.e., road traffic marking images in the road traffic marking dataset; Indicates The label of the category to which the sample belongs; Indicates The probability that a sample is predicted to be each category;

[0040] CIOU loss function It is expressed as:

[0041] ;

[0042] In the formula, It is the intersection of the predicted box and the true box; It is the Euclidean distance between the center point of the predicted box and the true box; It is the diagonal length of the minimum closure area of ​​the predicted box and the true box; is the difference in aspect ratio between the predicted box and the real box; is the equilibrium parameter;

[0043] The DFL loss function is expressed as:

[0044] ;

[0045] In the formula, They are the predicted value and near-prediction value output by the traffic line detection model respectively; Respectively The actual value of each label, the label integral value, and the adjacent label integral value; Represents the DFL loss function.

[0046] Compared with the existing technology, the present invention has the following beneficial effects:

[0047] (1) The present invention adaptively enhances the road traffic marking image by introducing an aggregated prompt enhancer. The image enhancement strategy can be dynamically adjusted according to the real-time environmental conditions, which can effectively improve the recognizability of traffic markings under different lighting and climatic conditions, thereby improving the detection accuracy and robustness of the traffic marking detection model. It can be used not only under conventional conditions, but also to cope with road traffic marking detection under adverse weather conditions such as low illumination, rain and snow.

[0048] (2) The present invention introduces upper and lower anchor feature extraction blocks into the traffic marking detection model, so that the model can more flexibly capture contextual information when processing traffic markings of different scales and complex backgrounds. This not only enhances the model's ability to extract local features of traffic markings, but also improves its understanding of the contextual environment, thereby effectively reducing false detections and missed detections caused by background interference.

[0049] (3) By replacing the bottleneck module in the original c2f module in the traffic marking detection model with a contextual anchor attention block, the present invention can introduce and capture remote context information in parallel, effectively solving the problems of large target scale variation and diverse ranging environments in the road traffic marking detection task, and improving the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a structural schematic diagram of the traffic marking detection model of the present invention.

[0051] Figure 2 It is a schematic diagram of the structure of the aggregated prompt enhancer of the present invention.

[0052] Figure 3 It is a structural schematic diagram of the upper and lower anchor feature extraction blocks of the present invention.

[0053] Figure 4 Schematic diagram of the structure of the contextual anchor attention block. DETAILED DESCRIPTION

[0054] The present invention provides a technical solution: a road traffic marking detection method based on aggregated prompt enhancement, comprising the following steps:

[0055] Step S1: construct a road traffic marking dataset, which includes a number of road traffic marking images.

[0056] Step S2: Construct a traffic line detection model, such as Figure 1 As shown, Figure 1 middle, represents upsampling, represents serial fusion; the traffic marking detection model consists of an aggregated prompt enhancer, a feature extraction backbone network and a cross-stage feature fusion module; the aggregated prompt enhancer is set at the input end of the feature extraction backbone network.

[0057] Step S3: using an aggregated prompt enhancer to perform data enhancement on the traffic marking images in the road traffic marking dataset.

[0058] Step S4: Input the data-enhanced traffic marking image into the feature extraction backbone network to extract features and obtain a feature map.

[0059] Step S5: Input the feature map extracted in step S4 into the cross-stage feature fusion module for feature fusion and obtain the final detection result.

[0060] The feature extraction backbone network is composed of a first convolution block, a second convolution block, a first upper and lower anchor feature extraction block, a third convolution block, a second upper and lower anchor feature extraction block, a fourth convolution block, a third upper and lower anchor feature extraction block, a fifth convolution block, a fourth upper and lower anchor feature extraction block and a fast spatial pyramid pooling block (Spatial Pyramid Pooling Fast, referred to as SPPF) connected in sequence.

[0061] The cross-stage feature fusion module includes a first feature fusion block, a second feature fusion block, a third feature fusion block, a fourth feature fusion block, a sixth convolution block, a seventh convolution block, a first detection block, a second detection block and a third detection block;

[0062] The output of the aggregated prompt enhancer (the traffic marking image after data enhancement) passes through the first convolution block, the second convolution block, the first upper and lower anchor feature extraction block, the third convolution block, the second upper and lower anchor feature extraction block, the fourth convolution block, the third upper and lower anchor feature extraction block, the fifth convolution block, the fourth upper and lower anchor feature extraction block and the fast spatial pyramid pooling block in sequence; the output of the fast spatial pyramid pooling block is upsampled and fused in series with the output of the third upper and lower anchor feature extraction block, and the series fusion is input into the first feature fusion block after the series fusion, the output of the first feature fusion block is upsampled and fused in series with the output of the second upper and lower anchor feature extraction block after the series fusion, and the output of the second feature fusion block is respectively input into the first detection block and the sixth convolution block, the output of the sixth convolution block is fused in series with the output of the first feature fusion module, and the series fusion is input into the third feature fusion block, the output of the third feature fusion block is respectively input into the second detection block and the seventh convolution block, the output of the seventh convolution block and the output of the fast spatial pyramid pooling block are fused in series and input into the fourth feature fusion block, and the output of the fourth feature fusion block is input into the third detection block.

[0063] Among them, the outputs of the first detection block, the second detection block and the third detection block are the final detection results.

[0064] Among them, the first detection block, the second detection block and the third detection block have the same structure and are composed of two branches. Both branches are composed of two consecutive convolution blocks with a size of 3×3, a step size of 1, and a padding number of 1. The two branches are used for bounding box prediction and category prediction, respectively; in addition, the output ends of the two branches are connected to a separate two-dimensional convolutional layer (Conv2d) with a size of 1×1 and a step size of 1, which are used to obtain the prediction results of the bounding box and category, respectively.

[0065] like Figure 2 As shown, Figure 2 middle, represents the Hardsish activation function, represents element-wise addition, represents upsampling, represents downsampling, represents serial fusion; the aggregate prompt enhancer consists of an aggregated lifting generation block, multiple content-driven prompt blocks (CPB) and multiple receptive field attention convolutions (RFAConv); the specific process of the aggregate prompt enhancer to perform data enhancement on the traffic marking images in the road traffic marking dataset is as follows:

[0066] First, the first receptive field attention convolution (RFAConv) is used to obtain the high-resolution features of the input image, i.e., the road traffic marking image, and the high-resolution features are input into the second receptive field attention convolution. The output of the second receptive field attention convolution is downsampled to the third receptive field attention convolution, the output of the third receptive field attention convolution is downsampled to the fourth receptive field attention convolution, and the output of the fourth receptive field attention convolution is downsampled to the fifth receptive field attention convolution.

[0067] Among them, the second receptive field attention convolution, the third receptive field attention convolution, the fourth receptive field attention convolution and the fifth receptive field attention convolution all have a step size of 1 and a size of 3×3.

[0068] In the aggregated lifting generation block, the hint is a learnable parameter used to encode the degradation-specific contextual information. Let the initial hint of the aggregated lifting generation block be represented as ∈ ,in, , , Respectively represent the initial prompt height, width and channels, represents the set of real numbers; first, the initial prompt After the Hardsish activation function, the first multi-scale prompt sequence is generated, and the first multi-scale prompt sequence is input into the first transposed convolutional layer (TConv) with a step size of 2 to generate prompts , will prompt After the Hardsish activation function, a second multi-scale prompt sequence is generated, which is input into the second transposed convolutional layer (TConv) with a step size of 2 to generate prompts .

[0069] The output of the fifth receptive field attention convolution and the initial prompt Input the first content-driven prompt block for interaction. After the interaction is completed, perform upsampling operation and fused in series with the output of the fourth receptive field attention convolution. After the series fusion, input the sixth receptive field attention convolution for processing. The output of the sixth receptive field attention convolution and the prompt are combined. The second content-driven prompt block is input for interaction. After the interaction is completed, the upsampling operation is performed and the output of the third receptive field attention convolution is fused in series. After the series fusion, the seventh receptive field attention convolution is input for processing. The output of the seventh receptive field attention convolution and the prompt are combined. The third content-driven prompt block is input for interaction. After the interaction is completed, an upsampling operation is performed and the output of the second receptive field attention convolution is fused in series. After the series fusion, the eighth receptive field attention convolution is input for processing. Finally, the output of the eighth receptive field attention convolution and the road traffic marking image are added by element-wise addition to obtain the output of the aggregated prompt enhancer, that is, the traffic marking image after data enhancement.

[0070] like Figure 3 As shown, Figure 3 middle, represents the Sigmoid activation function, Represents a split operation, Represents serial fusion; the structures of the first upper and lower anchor feature extraction block (c2f_CAA), the second upper and lower anchor feature extraction block, the third upper and lower anchor feature extraction block, and the fourth upper and lower anchor feature extraction block are the same, and they are all composed of replacing the bottleneck module in the original c2f module with the context anchor attention block (Context Anchor Attention).

[0071] The upper and lower anchor feature extraction blocks process features of different scales in parallel, so that the traffic marking detection model can obtain the morphological information of traffic markings more comprehensively. This fusion of multi-scale features enables the traffic marking detection model to maintain high accuracy when detecting small or blurred markings.

[0072] The upper and lower anchor feature extraction blocks can effectively eliminate feature redundancy and extract more discriminative feature information; this makes subsequent feature processing and classification more efficient, and improves the response speed and real-time detection capability of the entire traffic marking detection model.

[0073] The upper and lower anchor feature extraction blocks suppress non-marking interference by focusing on the feature information related to traffic markings, improve the recognition ability of the traffic marking detection model, and thus reduce the false detection rate.

[0074] Among them, the first upper and lower anchor feature extraction block (c2f_CAA), the second upper and lower anchor feature extraction block, the third upper and lower anchor feature extraction block and the fourth upper and lower anchor feature extraction block are all three-layer structures. The first and third layers are convolutional layers with a size of 1×1 and a stride of 1. The second layer is three contextual anchor attention blocks that interact with each other.

[0075] The inputs of the first upper and lower anchor feature extraction block (c2f_CAA), the second upper and lower anchor feature extraction block, the third upper and lower anchor feature extraction block, and the fourth upper and lower anchor feature extraction block are first passed through the convolutional layer of the first layer to generate an intermediate feature map. After the segmentation operation, the generated intermediate feature map is split into two parts. One part is input into the three interactive contextual anchor attention blocks of the second layer, and the other part is fused in series with the outputs of the first two contextual anchor attention blocks of the second layer and then input into the convolutional layer of the third layer to obtain the output of the upper and lower anchor feature extraction block (c2f_CAA).

[0076] like Figure 4 As shown, Figure 4 middle, represents the Sigmoid activation function; the context anchor attention block (Context AnchorAttention) consists of an average pooling layer, a convolutional layer (1×1), a depthwise convolutional layer, a depthwise convolutional layer, and a convolutional layer (1×1) connected sequentially.

[0077] The processing flow of the contextual anchor attention block is:

[0078] The input of the contextual anchor attention block is first average pooled through the average pooling layer, and then convolved through a convolutional layer (1×1) to obtain the output :

[0079] ;

[0080] In the formula, represents the average pooling operation; Represents the output of the contextual anchor attention block after the input passes through the average pooling layer and the convolutional layer (1×1) in sequence; represents a convolutional layer (1×1); represents the input of the contextual anchor attention block; Represents a specific channel in the input of the contextual anchor attention block, ranging from 0 to , Indicates the maximum index value in a specific channel number.

[0081] Will After two deep convolutional layers in sequence, it is expressed as:

[0082] ;

[0083] ;

[0084] In the formula, express Output after the first deep convolutional layer; express Output after the second deep convolutional layer; represents a depthwise convolutional layer.

[0085] The output of two deep convolutional layers After another convolutional layer (1×1) and Sigmoid activation function, the output of the context anchor attention block is obtained. , Indicates the number of channels , the height is , the width is A three-dimensional tensor of .

[0086] It is expressed as:

[0087] .

[0088] Among them, the road traffic marking dataset is used to train the traffic marking detection model in a supervised learning manner, and the binary cross entropy loss function is used. As the classification loss function, CIOU loss function (Complete Intersection over Union) and DFL loss function (Distribution Focal Loss) are used as regression loss functions; training is performed for 300 epochs with a learning rate of 0.01. The Lion optimizer is used for optimization, and the parameters of the optimal traffic marking detection model are saved after all training is completed. Finally, the model is tested and evaluated on the test set, which is obtained by dividing the road traffic marking dataset.

[0089] Binary Cross Entropy Function It is expressed as:

[0090] ;

[0091] In the formula, Represents the total number of samples (road traffic marking images) in the road traffic marking dataset; Indicates The label of the category to which the sample belongs; Indicates The probability that a sample is predicted to be each category.

[0092] The CIOU loss function introduces more complex geometric constraints on top of the standard IOU (Intersection over Union) loss function, aiming to better capture the overlapping area between the prediction box and the target box, and also considers the aspect ratio and center point distance of the box, thus improving the accuracy of target positioning. It is expressed as:

[0093] ;

[0094] In the formula, It is the intersection of the predicted box and the true box; It is the Euclidean distance between the center point of the predicted box and the true box; It is the diagonal length of the minimum closure area of ​​the predicted box and the true box; is the difference in aspect ratio between the predicted box and the real box; is a balance parameter.

[0095] The DFL loss function is expressed as:

[0096] ;

[0097] In the formula, They are the predicted value and near-prediction value output by the traffic line detection model respectively; Respectively The actual value of each label, the label integral value, and the adjacent label integral value; Represents the DFL loss function.

[0098] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A road traffic marking detection method based on aggregated prompt enhancement, characterized in that: The steps include: Step S1: construct a road traffic marking dataset, which includes a number of road traffic marking images; Step S2: constructing a traffic marking detection model, wherein the traffic marking detection model is composed of an aggregated prompt enhancer, a feature extraction backbone network, and a cross-stage feature fusion module; the aggregated prompt enhancer is set at the input end of the feature extraction backbone network; The feature extraction backbone network consists of a first convolution block, a second convolution block, a first context anchor feature extraction block, a third convolution block, a second context anchor feature extraction block, a fourth convolution block, a third context anchor feature extraction block, a fifth convolution block, a fourth context anchor feature extraction block and a fast spatial pyramid pooling block, which are connected in sequence; The cross-stage feature fusion module includes a first feature fusion block, a second feature fusion block, a third feature fusion block, a fourth feature fusion block, a sixth convolution block, a seventh convolution block, a first detection block, a second detection block and a third detection block; Step S3: using an aggregated prompt enhancer to perform data enhancement on the traffic marking images in the road traffic marking dataset; Step S4: inputting the traffic line image after data enhancement into the feature extraction backbone network to extract features and obtain a feature map; Step S5: input the feature map extracted in S4 into the cross-stage feature fusion module to perform feature fusion and obtain the final detection result; The first context anchor feature extraction block, the second context anchor feature extraction block, the third context anchor feature extraction block and the fourth context anchor feature extraction block have the same structure, which is to replace the bottleneck module in the original c2f module with the context anchor attention block; the first context anchor feature extraction block, the second context anchor feature extraction block, the third context anchor feature extraction block and the fourth context anchor feature extraction block are all three-layer structures, the first and third layers are convolutional layers with a size of 1×1 and a stride of 1, and the second layer is three context anchor attention blocks that interact with each other; The processing flow of the first context anchor feature extraction block, the second context anchor feature extraction block, the third context anchor feature extraction block and the fourth context anchor feature extraction block is: The inputs of the first context anchor feature extraction block, the second context anchor feature extraction block, the third context anchor feature extraction block, and the fourth context anchor feature extraction block are firstly passed through the convolution layer of the first layer to generate an intermediate feature map. The generated intermediate feature map is split into two parts after the segmentation operation. One part is input into the three interactive context anchor attention blocks of the second layer, and the other part is fused in series with the outputs of the first two context anchor attention blocks of the second layer and then input into the convolution layer of the third layer to obtain the output of the context anchor feature extraction block. The aggregated prompt enhancer consists of an aggregated lifting generation block, a first content-driven prompt block, a second content-driven prompt block, a third content-driven prompt block, a first receptive field attention convolution, a second receptive field attention convolution, a third receptive field attention convolution, a fourth receptive field attention convolution and a fifth receptive field attention convolution; the specific process of the aggregated prompt enhancer for data enhancement of traffic marking images in the road traffic marking dataset is as follows: First, the first receptive field attention convolution is used to obtain the high-resolution features of the input image, i.e., the road traffic marking image, and the high-resolution features are input into the second receptive field attention convolution. The output of the second receptive field attention convolution is downsampled to the third receptive field attention convolution. The output of the third receptive field attention convolution is downsampled to the fourth receptive field attention convolution. The output of the fourth receptive field attention convolution is downsampled to the fifth receptive field attention convolution. Let the initial prompt of the aggregated lifting generation block be expressed as ∈ ,in, , , Respectively represent the initial prompt height, width and channels, represents the set of real numbers; first, the initial prompt After the Hardsish activation function, the first multi-scale prompt sequence is generated, and the first multi-scale prompt sequence is input into the first transposed convolutional layer with a step size of 2 to generate prompts , will prompt After the Hardsish activation function, a second multi-scale prompt sequence is generated, which is input into the second transposed convolutional layer with a step size of 2 to generate prompts. ; The output of the fifth receptive field attention convolution and the initial prompt The first content is input to drive the prompt block for interaction. After the interaction is completed, the upsampling operation is performed and the output of the fourth receptive field attention convolution is fused in series. After the series fusion, the sixth receptive field attention convolution is input for processing. The output of the sixth receptive field attention convolution and the prompt are combined. Input the second content-driven prompt block for interaction. After the interaction is completed, perform upsampling operation and fused in series with the output of the third receptive field attention convolution. After the series fusion, input the seventh receptive field attention convolution for processing. The output of the seventh receptive field attention convolution and the prompt are combined. The third content-driven prompt block is input for interaction. After the interaction is completed, an upsampling operation is performed and the output of the second receptive field attention convolution is fused in series. After the series fusion, the eighth receptive field attention convolution is input for processing. Finally, the output of the eighth receptive field attention convolution and the road traffic marking image are added by element-wise addition to obtain the output of the aggregated prompt enhancer, that is, the traffic marking image after data enhancement.

2. The method for detecting road traffic markings based on aggregated prompt enhancement according to claim 1, characterized in that: The processing flow of the traffic marking detection model is as follows: The output of the aggregated prompt enhancer, i.e., the traffic marking image after data enhancement, is sequentially passed through the first convolution block, the second convolution block, the first context anchor feature extraction block, the third convolution block, the second context anchor feature extraction block, the fourth convolution block, the third context anchor feature extraction block, the fifth convolution block, the fourth context anchor feature extraction block and the fast spatial pyramid pooling block; the output of the fast spatial pyramid pooling block is upsampled and fused in series with the output of the third context anchor feature extraction block, and the fusion is input into the first feature fusion block after the fusion is in series; the output of the first feature fusion block is upsampled and fused in series with the output of the second context anchor feature extraction block, and the fusion is input into the second feature fusion block after the fusion is in series; the output of the second feature fusion block is respectively input into the first detection block and the sixth convolution block; the output of the sixth convolution block is fused in series with the output of the first feature fusion module, and the fusion is input into the third feature fusion block after the fusion is in series; the output of the third feature fusion block is respectively input into the second detection block and the seventh convolution block; the output of the seventh convolution block is fused in series with the output of the fast spatial pyramid pooling block, and the output of the fourth feature fusion block is input into the third detection block; The outputs of the first detection block, the second detection block and the third detection block are the final detection results output by the traffic marking detection model.

3. The method for detecting road traffic markings based on aggregated prompt enhancement according to claim 2, characterized in that: The second receptive field attention convolution, the third receptive field attention convolution, the fourth receptive field attention convolution and the fifth receptive field attention convolution all have a stride of 1 and a size of 3×3.

4. The method for detecting road traffic markings based on aggregated prompt enhancement according to claim 3, characterized in that: The contextual anchor attention block consists of an average pooling layer, a 1×1 convolutional layer, a depthwise convolutional layer, a depthwise convolutional layer, and a convolutional layer connected sequentially; The processing flow of the contextual anchor attention block is: The input of the contextual anchor attention block is first average pooled through the average pooling layer, and then convolved through a 1×1 convolutional layer to obtain the output : ； In the formula, represents the average pooling operation; It represents the output of the contextual anchor attention block after the input passes through the average pooling layer and the 1×1 convolution layer in sequence; represents a 1×1 convolutional layer; represents the input of the contextual anchor attention block; Represents a specific channel in the input of the contextual anchor attention block, ranging from 0 to , Indicates the maximum index value in a specific channel number; Will After two deep convolutional layers in sequence, it is expressed as: ； ； In the formula, express Output after the first deep convolutional layer; express Output after the second deep convolutional layer; represents a deep convolutional layer; The output of two deep convolutional layers After another 1×1 convolutional layer and Sigmoid activation function, the output of the contextual anchor attention block is obtained. , Indicates the number of channels , the height is , the width is A three-dimensional tensor of ; It is expressed as: 。 5. The method for detecting road traffic markings based on aggregated prompt enhancement according to claim 4, characterized in that: The road traffic marking dataset is used to train the traffic marking detection model in a supervised learning manner, using the binary cross entropy loss function. As the classification loss function, CIOU loss function and DFL loss function are used as regression loss functions; training is performed for 300 iterations with a learning rate of 0.01, and Lion optimizer is used for optimization. After all training is completed, the parameters of the optimal traffic marking detection model are saved, and finally tested and evaluated on the test set, which is obtained by dividing the road traffic marking data set; Binary Cross Entropy Function It is expressed as: ； In the formula, Represents the total number of samples, i.e., road traffic marking images in the road traffic marking dataset; Indicates The label of the category to which the sample belongs; Indicates The probability that a sample is predicted to be each category; CIOU loss function It is expressed as: ； In the formula, It is the intersection of the predicted box and the true box; It is the Euclidean distance between the center point of the predicted box and the true box; It is the minimum diagonal length of the enclosed area between the predicted box and the true box; is the difference in aspect ratio between the predicted box and the real box; is the equilibrium parameter; The DFL loss function is expressed as: ； In the formula, They are the predicted value and near-prediction value output by the traffic line detection model respectively; Respectively The actual value of each label, the label integral value, and the adjacent label integral value; Represents the DFL loss function.

Images