A high-resolution traffic sign rapid detection method based on adaptive region screening

Abstract

The application provides a high-resolution traffic sign rapid detection method based on adaptive region screening, proposes an adaptive region screening model, extracts image features and uses image blocking technology to realize the purpose of judging whether each image block contains a traffic sign, the model comprises four parts of image adjustment, multi-scale feature fusion, region feature extraction and category prediction, proposes a traffic sign detection model based on image segmentation technology, a multi-scale semantic segmentation module CAM, extracts multi-scale features, realizes simple fusion, further extracts the fused features to obtain multi-scale segmentation graphs corresponding to different resolutions, introduces the above segmentation graphs into a multi-scale feature fusion network of the traffic sign detection model, guides and improves the information extraction of the foreground features, and Tex explores the background area of the features again after the fused feature output, and refines the input features again. The application can help the traffic sign detection to realize the balance between precision and real-time performance.

Description

Technical Field

[0001] This invention belongs to the field of computer vision traffic sign detection technology, specifically relating to a fast detection method for high-resolution traffic signs based on adaptive region filtering. Background Technology

[0002] To capture the traffic scene within the driver's field of vision as much as possible, vehicle-mounted cameras acquire images with relatively high resolution. This results in public transportation sign detection datasets having higher image resolutions than typical images. For example, commonly used traffic sign datasets such as TT100K, GTSDB, CCTSDB2021, and STSD have image resolutions of 2048×2048, 1360×800, 1280×720 (partially), and 1280×960, respectively. High-resolution images impose a computational burden on the detector for feature extraction and also increase the difficulty of training. During training, an 8GB GPU needs to be batch-controlled to train on images with a resolution of 2048×2048. To improve model performance without changing the image resolution, only higher-performance devices, larger memory spaces, or multi-GPU servers can be selected, leading to a heavy training burden and expensive equipment requirements.

[0003] To reduce the requirements for equipment and training, existing techniques preprocess images in the dataset by adjusting image size and dividing the image into blocks.

[0004] Resizing the image involves directly reducing the image resolution using the bilinear interpolation function. The advantage of this preprocessing method is that the reduced resolution also decreases the computational burden. However, it also results in the loss of a significant amount of target information, as the bilinear interpolation function, being a fixed function, cannot distinguish between the target and the background.

[0005] Image segmentation uses processing methods such as sliding windows to decompose a high-resolution image into many smaller-resolution images according to set rules. In order to ensure that the target at the edge of the image block is not lost, a certain degree of overlap is maintained between the image blocks.

[0006] Traffic sign detection not only requires high accuracy, but also real-time performance and lightweight design for practical use. However, the preprocessing methods commonly used in existing approaches, such as adjusting image size or dividing the image into blocks, struggle to balance the demands for real-time performance and high accuracy.

[0007] In addition, in real-world traffic scenarios, traffic signs face challenges such as scale variations, lighting changes, occlusion, damage, and similar interfering objects, which reduce the model's detection performance and lead to false positives and false negatives. Using image segmentation for preprocessing results in incomplete traffic signs at the segment edges. Considering the need for lightweight traffic sign detection, lightweight techniques often reduce the model's learning ability, thus lowering accuracy and leading to more false negatives. Existing methods mainly improve model performance by incorporating attention, optimizing multi-scale feature fusion, and replacing the loss function with the latest version; however, these methods struggle to address the imbalance between accuracy and inference speed caused by image preprocessing methods.

[0008] How to improve the real-time performance and lightweight design of traffic sign detection models while ensuring their accuracy has become the main challenge facing current traffic sign detection research. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the prior art and provide a rapid detection method for high-resolution traffic signs based on adaptive region filtering.

[0010] To address the challenge of balancing detection accuracy and inference speed in existing image preprocessing methods, such as image adjustment and image segmentation, which suffer from high resolution in traffic sign images, this invention proposes an adaptive region filtering model that integrates image adjustment and image segmentation preprocessing methods with deep learning technology. This model utilizes a backbone network to quickly extract multi-scale features from the scaled-down high-resolution image. It then uses a feature pyramid to fuse multi-scale features from the bottom up to initially optimize shallow features. Next, an improved ASFF is used to further optimize shallow features by calculating the spatial and channel information of the multi-scale features. Then, feature stacking is used to obtain the residuals of the shallow features. Furthermore, the C2f module proposed by YOLOv8 is used to refine the residual feature information. Finally, image segmentation technology is used to divide the shallow features, and a three-branch feature extraction module is used to obtain comprehensive features from different regions. These comprehensive features are then input into a classification head to determine whether different regions contain traffic signs. This adaptive region filtering model quickly filters out a large number of image regions in high-resolution images that do not contain traffic signs, thereby reducing the time spent on subsequent detection and improving the model's real-time performance at a comprehensive level.

[0011] To address the issues of false positives and false negatives in lightweight traffic sign detection models when facing complex and ever-changing traffic environments, this invention utilizes image segmentation technology to assist traffic sign detection, thereby improving the performance of lightweight traffic sign detection. The model extracts multi-scale feature information using a backbone network. First, a multi-scale weak semantic segmentation module is proposed, inputting multi-scale features to obtain semantic segmentation maps corresponding to different feature scales. Next, the multi-scale semantic segmentation maps are used to enhance features at corresponding scales. Then, the aforementioned semantic segmentation maps are used to optimize the FPN+PAN feature fusion network, using the segmentation maps to guide the model in focusing on and aligning features of interest. Finally, a feature refinement module is added after the output features to re-explore and optimize the results for background region features.

[0012] To achieve the above objectives, the present invention provides the following technical solution: a rapid detection method for high-resolution traffic signs based on adaptive region filtering, comprising the following steps:

[0013] S1: The image size is adjusted and input into the backbone network of the adaptive region filtering model. For example, the resolution of a 2048×2048 image in the TT100K dataset is adjusted to 1024×1024 or 800×800, or the resolution of a 1360×800 image in the GTSDB dataset is adjusted to 640×640. Adjusting the resolution to a smaller value helps speed up the inference of the adaptive filtering model. The backbone network is then used to extract relevant information about traffic signs from the adjusted image. Image scaling reduces the resolution of the target, making detailed classification difficult. However, the adaptive region filtering model classifies objects based on whether they are targets, transforming the multi-class problem of detailed traffic sign classification into a binary classification problem. It does not consider the similarity between traffic signs, greatly reducing the classification difficulty and the impact of image resolution on classification.

[0014] S2: The multi-scale features (C3, C4, C5) extracted from the backbone network are fed into the feature pyramid FPN and the multi-scale features are initially fused through the bottom-up subnetwork to obtain (P3, P4, P5). The fused multi-scale features are then input into the improved ASFF module ASFF-CAM. By redistributing weights using spatial attention and feature attention, the multi-scale features are fused again to obtain the optimized shallow features.

[0015] S3: The fused features and P3 are stacked at the channel level (Concat) to achieve residual connection, and then the residual features are input into the C2f module proposed by the YOLOv8 detector to optimize the feature information.

[0016] S4: The features obtained from process S3 are input into the region feature extraction module. The region feature extraction module consists of three steps: first, all regions are calculated and candidate boxes are obtained; second, region features are extracted from image features using the candidate boxes; and finally, comprehensive features are extracted based on the features of each region. This invention combines the sliding window algorithm in image segmentation to map the candidate boxes of the original image segments to the feature level, and uses the ROI-Align algorithm to extract features (B*N, C, h, w) of different candidate regions, where B is the image batch, C is the channel feature, N is the corresponding candidate region, and h and w represent the feature scale of the candidate region. After obtaining the features, it is necessary to extract the comprehensive features of each candidate region. For this purpose, we propose a three-branch feature pooling module, which includes three feature extraction methods: max pooling, average pooling, and adaptive pooling. Finally, the three pooling features are aggregated (B, 3*C, N, 1).

[0017] S5: Input the aggregated features into the classification head to obtain the judgment result for each region, and set the corresponding threshold according to the loss function. For example, the threshold of the BCE loss function is 0.001, and the threshold of the Focal loss function is 0.05.

[0018] S6: Based on the prediction results of the adaptive region filtering model, filter the original image and retain only the image patches corresponding to the prediction results that are higher than the threshold.

[0019] S7: Retain the information of the selected image blocks (position relative to the original image) and input the image blocks into the backbone network for traffic sign detection based on image segmentation technology to extract effective information related to traffic signs.

[0020] S8: Input the multi-scale features (C3, C4, C5, note the distinction from the above) extracted by the backbone network into the CAM module to obtain multi-scale segmentation maps that can handle features with different resolutions, and use them as spatial attention (1+M)*F to enhance the features (C3, C4, C5).

[0021] S9: The enhanced features are input into the improved FPN+PAN feature fusion module to achieve multi-scale feature fusion. The improved FPN+PAN uses the multi-scale segmentation map generated by CAM to align the features between feature fusions, reducing the impact of redundant information in high-level features.

[0022] S10: Input the fused multi-scale features into the corresponding TeX modules to refine the features, and then input them into the detector to obtain the detection results.

[0023] S11: Collect the detection results for each image patch, use the retained image patch information to map the detection results to the original image, and use the non-maximum suppression algorithm (NMS) to eliminate redundant detection results.

[0024] S1-S6 represent the adaptive region filtering model proposed in this invention. This model extracts image features and utilizes image segmentation technology to determine whether each image block contains a traffic sign. The model includes four parts: image adjustment, multi-scale feature fusion, region feature extraction, and category prediction. In S2, multi-scale feature fusion (ASFF-CAM) is achieved by combining channel attention and spatial attention, optimizing shallow feature information and preparing for subsequent region feature extraction. In S4, image segmentation technology is introduced into the feature segmentation level to achieve adaptive region filtering, and a three-branch feature pooling module is proposed, focusing on the highest point information in the foreground, global information, and adaptively obtained feature synthesis information, integrating the features corresponding to the image blocks. S7-S11 represent the traffic sign detection model based on image segmentation technology proposed in this invention. In S8, a multi-scale semantic segmentation module (CAM) is proposed, incorporating image segmentation technology. Multi-scale features are extracted, simple fusion is achieved, and then the fused features are further extracted to obtain multi-scale segmentation maps corresponding to different resolutions. In S9, the above segmentation maps are introduced into the multi-scale feature fusion network of the traffic sign detection model to guide and improve the extraction of foreground feature information. In S10, a feature refinement module Tex is proposed. Multi-scale feature fusion pays more attention to foreground features. After the fused feature output, Tex explores the background region of the feature again and refines the input feature again. This is beneficial as a supplement when the former segmentation is not fine enough and as a way to suppress feature background noise.

[0025] Furthermore, the ASFF-CAM module adds channel attention fusion to the existing ASFF module, which is based on spatial attention fusion of multi-scale features. Unlike traditional ASFF, the module's ultimate goal is to obtain optimized shallow features rather than new multi-scale features, so additional branches are removed. Spatial attention fusion first uses a 1×1 convolution and nearest-neighbor upsampling to align the resolution and number of channels of features between 4 and 5 to 3. Then, it uses another 1×1 convolution to compress the number of channels between the three features to 8. The compressed features are then stacked at the channel level. The stacked features are then input into a 1×1 convolution to map to 3 channels, and the softmax function is used to obtain the proportion of the three features. Finally, the multi-scale features are fused using the formula F1*W1 + F2*W2 + F3*W3. F1, F2, and F3 correspond to the three feature layers, and W1, W2, and W3 correspond to the three spatial attention weights. Channel attention fusion first involves element-wise summing of the aligned three-layer features. Then, the max-pooling and average-pooling features of each feature are extracted. Borrowing from the SENet compression and dilation principle, corresponding weighted features (B, 3*C, 1, 1) are obtained, and the features are adjusted to the form (B, C, 3, 1). The softmax function is applied to the third channel. Features are then fused using F1*CW1 + F2*CW2 + F3*CW3, where CW1, CW2, and CW3 correspond to spatial channel attention weights. After obtaining the spatial and channel fusion features, the fusion of the two types of features is achieved using the C2f approach. First, a 1×1 convolution is used to initially fuse the features and divide them into two parts. One part remains unchanged, while the other part is input into a convolution with four consecutive 3×3 channels compressed to 1 / 4 to obtain rich receptive field information. Features are retained after each convolution, and then stacked in the channel layers to form complete features. Finally, the retained features and the rich receptive field features are fused by element-wise summing.

[0026] Furthermore, the three-branch feature pooling module: max pooling and average pooling are similar to the general case, while adaptive pooling utilizes the principle of spatial attention to obtain the attention level of each pixel. First, it obtains the maximum and average features at the channel level and then stacks these two types of features. Then, a 7×7 large kernel convolution is used to map the channels from 2 to 1. Next, softmax activation is applied to the spatial layer to obtain a probability map representing the probability of each pixel. Finally, the feature information is aggregated at the spatial level using the probability map. In the three-branch feature pooling module, max pooling obtains the most prominent representation of the region, average pooling obtains the background information of the region, and adaptive pooling obtains the comprehensive feature information after learning. This helps the model pay attention to both global and local information, improving the attention given to small target features that occupy a smaller proportion.

[0027] Furthermore, the multi-scale semantic segmentation module (CAM) first inputs multi-scale feature information into the module, uses FPN to aggregate multi-scale feature information from bottom to top to obtain optimized shallow features, then borrows the ELAN idea (cascaded 3×3 convolutions and retain each result) to extract feature information with rich receptive fields, then inputs the features into 3×3 convolutions to reduce the resolution and maps 3×3 convolutions to channel 1 to obtain the segmentation map, repeating the above process to obtain multi-scale segmentation maps corresponding to features at different resolutions.

[0028] Furthermore, the FPN+PAN feature fusion module is improved by incorporating the influence of the segmentation map into the multi-scale feature fusion part of the original FPN+PAN network. Different scale features require different regions of interest. For example, shallow features focus on small target information, while deep features focus on large target information. The segmentation map is used to adjust the bottom-up and top-down fusion, aligning the regions of interest of the fused features and improving model performance.

[0029] Furthermore, the feature refinement module Tex module: While the CAM module is responsible for focusing on the region of interest, the Tex module, in contrast to CAM, mainly targets the non-region of interest in different output results. It uses cascaded 3×3 dilated convolutions (1, 2, 3) to extract feature information of the non-region of interest and refine the input features.

[0030] Compared with the prior art, the beneficial effects of the present invention are:

[0031] 1) This invention proposes an adaptive region filtering model that combines the advantages of image adjustment and image segmentation. This model enables rapid filtering of image blocks that do not contain traffic signs, reducing the computational overhead of inferring image features during subsequent detection. Overall, it accelerates traffic sign detection in high-resolution images. Because the filtered image blocks retain the target's resolution without reducing target information, it alleviates the problem of reduced model accuracy due to image size adjustment. In summary, the adaptive region filtering model helps traffic sign detection achieve a balance between accuracy and real-time performance.

[0032] 2) This invention proposes a traffic sign detection model assisted by image segmentation technology. A multi-scale semantic segmentation module generates probability maps corresponding to different resolutions. These probability maps are used to enhance multi-scale feature information and optimize the multi-scale feature fusion network. Similarly, the probability maps are used to refine the output features and optimize feature information at different resolutions. This helps improve the model's learning of target features, especially for lightweight models, reduces missed detections of traffic signs, and improves the overall performance of the model. Attached Figure Description

[0033] Figure 1 Displaying the location and image blocks of traffic signs in traffic images;

[0034] Figure 2 This provides an overall framework for rapid traffic sign detection in high-resolution images based on adaptive region filtering.

[0035] Figure 3 For ASFF-CAM module;

[0036] Figure 4 It is a three-branch feature pooling module;

[0037] Figure 5 CAM is a multi-scale semantic segmentation module.

[0038] Figure 6 The Mask-att module is used to enhance features using semantic segmentation graphs;

[0039] Figure 7 The Mask-align module optimizes feature fusion using semantic segmentation graphs;

[0040] Figure 8 A TeX module for feature refinement;

[0041] Figure 9 This is the C2f module proposed by YOLOv8. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. The specific embodiments described herein are only for explaining the technical solutions of this invention and are not intended to limit the invention.

[0043] Example

[0044] Please see Figure 2 As shown, this embodiment provides a method for rapid detection of high-resolution traffic signs based on adaptive region filtering, including:

[0045] In terms of the implementation environment, the operating system selected in this embodiment is Windows 10, but it can also run in a Linux environment. The deep learning framework for the fast detection model of traffic signs in high-resolution images based on adaptive region filtering is PyTorch 1.7 with CUDA 10.1. The hardware environment is an Intel(R) Core(TM) i9-10900K CPU @ 3.70GHz processor and an NVIDIA Quadro RTX 4000 graphics card. During the training phase, the model's weight decay is 0.0005 and the batch size is 32.

[0046] 1. Adaptive Region Filtering Model

[0047] like Figure 2The adaptive region selection model shown in the upper part consists of four parts: image adjustment, multi-scale feature fusion, region feature extraction, and category prediction.

[0048] Image adjustment, as described above, utilizes bilinear interpolation or nearest neighbor algorithms to reduce image resolution and accelerate the inference time for adaptive filtering.

[0049] Multi-scale feature fusion is an improvement based on the ASFF module. After using the Feature Pyramid Network (FPN) to achieve initial bottom-up fusion and obtain new multi-scale feature information, ASFF-CAM is used to fuse the aforementioned features again to optimize shallow features. The process involves taking the multi-scale features (C3, C4, C5) extracted by the YOLOv8n backbone network and inputting them into the Feature Pyramid Network (FPN). First, feature C5 is upsampled using the nearest neighbor algorithm and stacked with C4 at the channel level to form a single feature. Then, a 1×1 convolution is used to fuse the two feature channels and map them to the number of channels in C4 to achieve multi-scale feature fusion and obtain P4. The above operation is repeated to obtain (C5, P4, P3).

[0050] ASFF-CAM module: Inputs the initially fused features into ASFF-CAM, such as... Figure 3 The method first aligns the features between channels 4 and 5 using a 1×1 convolution and nearest-neighbor upsampling, achieving spatial attention fusion. Next, a 1×1 convolution is used again to compress the number of channels between the three features to 8. The compressed features are then stacked at the channel level. The stacked features are then input into a 1×1 convolution to map to 3 channels, and the softmax function is used to obtain the proportion of each feature. Finally, the multi-scale features are fused using the formula F1*W1+F2*W2+F3*W3. F1, F2, and F3 correspond to the three layers of features, and W1, W2, and W3 correspond to the three spatial attention weights. For channel attention fusion, the aligned three layers of features are first element-wise summed. Then, the max-pooling and average-pooling features of each feature are extracted. Borrowing from the SENet compression and dilation principle, weighted features corresponding to (B, 3*C, 1, 1) are obtained. The features are then adjusted to the form (B, C, 3, 1), and the softmax function is applied to the third channel. The F1*CW1+F2*CW2+F3*CW3 feature fusion method is used, where CW1, CW2, and CW3 correspond to spatial channel attention weights. After obtaining spatial and channel fusion features, the fusion of the two types of features is achieved by borrowing the idea of ​​C2f. First, a 1×1 convolution is used to initially fuse the features and divide them into two parts. One part is retained unchanged, while the other part is input into a convolution with four consecutive 3×3 channels compressed to 1 / 4 to obtain rich receptive field information. Features are retained after each convolution, and then stacked in the channel layer to form complete features. Finally, the retained features and rich receptive field features are fused by element-wise addition.

[0051] The fused features are first combined with the initial shallow fusion features P3 using channel stacking to achieve residual connections, and then input into the C2f module to learn the features. The specific process of the C2f module is as follows: Figure 9 As shown, the input features are first divided into two features of the same scale using a 1×1 convolution. The second feature is then input into a feature extraction module composed of 3×3 convolutions to obtain high receptive field features. Finally, the obtained features are stacked and fused using a 1×1 convolution. Image segmentation combined with ROIAlign technology is used to segment the features, and the segmented features are input into a three-branch feature pooling module to learn comprehensive features.

[0052] Three-branch feature pooling module: such as Figure 4 As shown, max pooling and average pooling are the same as in the general case. Adaptive pooling uses the principle of spatial attention to obtain the attention level of each pixel. First, the maximum feature and average feature are obtained at the channel level and then the two features are stacked. Then, a 7×7 large kernel convolution is used to map the channel from 2 to 1. Then, softmax activation is applied to the spatial layer to obtain a probability map representing the probability of each pixel. Finally, the feature information is aggregated at the spatial level using the probability map and three-branch features are stacked at the channel level.

[0053] 2. Traffic sign detection based on image segmentation technology

[0054] Multi-scale segmentation map extraction module CAM module: such as Figure 5 As shown, firstly, multi-scale feature information is input into the module, and multi-scale feature information is aggregated from bottom to top using FPN to obtain optimized shallow features. Then, the ELAN idea (cascaded 3×3 convolutions and retain each result) is used to extract feature information with rich receptive fields. Next, the feature input is reduced in resolution by 3×3 convolution and mapped to channel 1 by 3×3 convolution to obtain the segmentation map. The above process is repeated to obtain multi-scale segmentation maps corresponding to features of different resolutions.

[0055] Improved FPN+PAN feature fusion module: such as Figure 6 As shown, the effect of adding the above segmentation map to the multi-scale feature fusion part of the original FPN+PAN network is that different scale features need to focus on different regions. For example, shallow features focus on small target information, while deep features focus on large target information. The segmentation map is used to adjust the bottom-up and top-down fusion, align the regions of interest of the fused features, and improve the model performance.

[0056] like Figure 7 As shown, this is the Mask-align module that optimizes feature fusion using semantic segmentation graphs. Figure 7 (a) in the text refers to the Mask-align module for upsampling processing. Figure 7(b) in the diagram refers to the Mask-align module for downsampling processing. align This refers to the method of adjusting the segmentation map to align multi-scale feature attention. For example, when fusing features from layer 5 and layer 4 in FPN, it affects features from layer 4 and layer 3. Therefore, Mask... align It's a fusion of Mask3 and Mask4, and the principle is similar.

[0057] Feature refinement module TeX module: such as Figure 8 As shown, while the CAM module is responsible for focusing on the region of interest, the Tex module, in contrast to CAM, mainly targets the non-interest regions of interest in different output results. It uses cascaded 3×3 dilated convolutions (1, 2, 3) to extract feature information of the non-interest regions and refine the input features.

[0058] This invention designs a traffic sign detection system based on image segmentation technology, using the YOLOv5 object detection network as a baseline. The network results are as follows: Figure 2 As shown in the lower part, this invention makes targeted improvements to the problem of small target detection in traffic signs in the following two aspects: 1) The network adds a multi-scale segmentation module (CAM) to YOLOv5 and improves the FPN+PAN feature fusion network used in YOLOv5 by using multi-scale segmentation maps. Before feature fusion, the segmentation map is used to enhance the target features, and in the multi-scale fusion part, the segmentation map is used to align the features. 2) A feature refinement TeX module is proposed. Before the multi-scale fusion features are input into the detection head, the multi-scale segmentation map (1-mask) is used in reverse to explore the background region features, thereby refining the features of the background region and compensating for the possibility of insufficient segmentation in CAM.

[0059] 3. The detection method combining an adaptive region selection model and an image segmentation-based traffic sign detection model includes the following steps:

[0060] S1: The image size is adjusted and input into the backbone network of the adaptive region filtering model. For example, the resolution of a 2048×2048 image in the TT100K dataset is adjusted to 1024×1024 or 800×800, or the resolution of a 1360×800 image in the GTSDB dataset is adjusted to 640×640. Adjusting the resolution to a smaller value helps speed up the inference of the adaptive filtering model. The backbone network is then used to extract relevant information about traffic signs from the adjusted image. Image scaling reduces the resolution of the target, making detailed classification difficult. However, the adaptive region filtering model classifies objects based on whether they are targets, transforming the multi-class problem of detailed traffic sign classification into a binary classification problem. It does not consider the similarity between traffic signs, greatly reducing the classification difficulty and the impact of image resolution on classification.

[0061] S2: The multi-scale features (C3, C4, C5) extracted from the backbone network are fed into the feature pyramid FPN and the multi-scale features are initially fused through the bottom-up subnetwork to obtain (P3, P4, P5). The fused multi-scale features are then input into the improved ASFF module ASFF-CAM. By redistributing weights using spatial attention and feature attention, the multi-scale features are fused again to obtain the optimized shallow features.

[0062] S3: The fused features and P3 are stacked at the channel level (Concat) to achieve residual connection, and then the residual features are input into the C2f module proposed by the YOLOv8 detector to optimize the feature information.

[0063] S4: The features obtained from process S3 are input into the region feature extraction module. The region feature extraction module consists of three steps: first, all regions are calculated and candidate boxes are obtained; second, region features are extracted from image features using the candidate boxes; and finally, comprehensive features are extracted based on the features of each region. This invention combines the sliding window algorithm in image segmentation to map the candidate boxes of the original image segments to the feature level, and uses the ROI-Align algorithm to extract features (B*N, C, h, w) of different candidate regions, where B is the image batch, C is the channel feature, N is the corresponding candidate region, and h and w represent the feature scale of the candidate region. After obtaining the features, it is necessary to extract the comprehensive features of each candidate region. For this purpose, we propose a three-branch feature pooling module, which includes three feature extraction methods: max pooling, average pooling, and adaptive pooling. Finally, the three pooling features are aggregated (B, 3*C, N, 1).

[0064] S5: Input the aggregated features into the classification head to obtain the judgment result for each region, and set the corresponding threshold according to the loss function. For example, the threshold of the BCE loss function is 0.001, and the threshold of the Focal loss function is 0.05.

[0065] S6: Based on the prediction results of the adaptive region filtering model, filter the original image and retain only the image patches corresponding to the prediction results that are higher than the threshold.

[0066] S7: Retain the information of the selected image blocks (position relative to the original image) and input the image blocks into the backbone network for traffic sign detection based on image segmentation technology to extract effective information related to traffic signs.

[0067] S8: Input the multi-scale features (C3, C4, C5, note the distinction from the above) extracted by the backbone network into the CAM module to obtain multi-scale segmentation maps that can handle features with different resolutions, and use them as spatial attention (1+M)*F to enhance the features (C3, C4, C5).

[0068] S9: The enhanced features are input into the improved FPN+PAN feature fusion module to achieve multi-scale feature fusion. The improved FPN+PAN uses the multi-scale segmentation map generated by CAM to align the features between feature fusions, reducing the impact of redundant information in high-level features.

[0069] S10: Input the fused multi-scale features into the corresponding TeX modules to refine the features, and then input them into the detector to obtain the detection results.

[0070] S11: Collect the detection results for each image patch, use the retained image patch information to map the detection results to the original image, and use the non-maximum suppression algorithm (NMS) to eliminate redundant detection results.

[0071] The above description merely illustrates preferred embodiments of the present invention, and while the description is relatively specific and detailed, it should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications, improvements, and substitutions without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A method for rapid detection of high-resolution traffic signs based on adaptive region filtering, characterized in that: Includes the following steps: S1: The image is resized and input into the backbone network of the adaptive region filtering model. The backbone network is used to extract effective information related to traffic signs in the resized image. S2: The multi-scale features (C3, C4, C5) extracted from the backbone network are fed into the Feature Pyramid Network (FPN) to achieve initial multi-scale feature fusion through bottom-up subnetworks, resulting in (P3, P4, P5). The fused multi-scale features are then input into the Multi-Scale Spatial-Channel Feature Fusion (ASFF-CAM) module. Using spatial attention and feature attention to redistribute weights, the multi-scale features are fused again to obtain optimized shallow features. The ASFF-CAM module adds channel attention to the ASFF's multi-scale feature fusion based on spatial attention. Force fusion of multi-scale features differs from traditional ASFF. The module's ultimate goal is to obtain optimized shallow features rather than new multi-scale features, so additional branches are removed. Spatial attention fusion first uses 1×1 convolution and nearest neighbor upsampling to align the resolution and number of channels of features between 4 and 5 using 3. Then, it uses another 1×1 convolution to compress the number of channels between the three features to 8. The compressed features are stacked at the channel level, and then the stacked features are input into a 1×1 convolution to map to 3 channels. The softmax function is used to obtain the proportion of the three features. Finally, the formula F1*W1 is used. The F2*W2+F3*W3 fusion method combines multi-scale features. F1, F2, and F3 correspond to three layers of features, and W1, W2, and W3 correspond to three spatial attention weights. Channel attention fusion involves first summing the aligned three-layer features element-wise, then extracting the max-pooling and average-pooling features of that feature. Borrowing from the SENet compression and dilation principle, a weight feature corresponding to (B, 3*C, 1, 1) is obtained. The feature is then adjusted to the form (B, C, 3, 1), and the softmax function is applied to the third channel. Finally, F1*CW1 is used... The +F2*CW2+F3*CW3 fusion feature is used, where CW1, CW2, and CW3 correspond to spatial channel attention weights. After obtaining spatial and channel fusion features, the fusion of the two features is achieved by referencing the C2f approach. First, a 1×1 convolution is used to initially fuse the features and divide them into two parts. One part is retained unchanged, while the other part is input into four consecutive 3×3 channels compressed into 1 / 4 convolutions to obtain rich receptive field information. Features are retained after each convolution, and then stacked into complete features in the channel layer. Finally, the retained features and rich receptive field features are fused by element-wise addition. S3: The fused features and P3 are stacked at the channel level to achieve residual connection, and then the residual features are input into the C2f module proposed by the YOLOv8 detector to optimize feature information; S4: The features obtained from the final step S3 are input into the region feature extraction module. The region feature extraction module consists of three steps: first, all regions are calculated and candidate boxes are obtained; second, region features are extracted from image features using the candidate boxes; and finally, comprehensive features are extracted based on the features of each region. Combining the sliding window algorithm in image segmentation, the candidate boxes of the original image segments are mapped to the feature level, and the ROI-Align algorithm is used to extract features (B*N,C,h,w) of different candidate regions, where B is the image batch, C is the channel feature, N is the corresponding candidate region, and h and w represent the feature scale of the candidate region. After obtaining the features, it is necessary to extract the comprehensive features of each candidate region. For this purpose, a three-branch feature pooling module is proposed. This module includes three feature extraction methods: max pooling, average pooling, and adaptive pooling. Finally, the three pooling features are aggregated (B,3*C,N,1). S5: Input the aggregated features into the classification head to obtain the judgment result for each region, and set the corresponding threshold according to the loss function; S6: Filter the original image based on the prediction results of the adaptive region filtering model, and retain only the image patches corresponding to the prediction results that are higher than the threshold. S7: Retain the filtered image patch information and input the image patch into the backbone network for traffic sign detection based on image segmentation technology to extract effective information related to traffic signs; S8: Input the multi-scale features extracted by the backbone network into the multi-scale segmentation map extraction module (CAM) to obtain multi-scale segmentation maps that address features at different resolutions, and use them as spatial attention (1+M)*F enhancement features. S9: Input the enhanced features into the improved FPN+PAN feature fusion module to achieve multi-scale feature fusion; S10: Input the fused multi-scale features into the corresponding feature refinement module Tex module to refine the features, and then input them into the detector to obtain the detection results; S11: Collect the detection results for each image patch, use the retained image patch information to map the detection results to the original image, and use the non-maximum suppression algorithm (NMS) to eliminate redundant detection results.

2. The method for rapid detection of high-resolution traffic signs based on adaptive region filtering according to claim 1, characterized in that: In the three-branch feature pooling module, adaptive pooling uses the principle of spatial attention to obtain the attention level of each pixel. First, the maximum feature and average feature are obtained at the channel level and then the two features are stacked. Then, a 7×7 large kernel convolution is used to map the channels from 2 to 1. Then, softmax activation is applied to the spatial layer to obtain a probability map representing the probability of each pixel. Finally, the feature information is aggregated at the spatial level using the probability map.

3. The method for rapid detection of high-resolution traffic signs based on adaptive region filtering according to claim 1, characterized in that: The multi-scale segmentation map extraction module (CAM) first inputs multi-scale feature information into the module, uses FPN to aggregate multi-scale feature information from bottom to top to obtain optimized shallow features, then uses ELAN to extract feature information with rich receptive fields, then inputs the features into a 3×3 convolution to reduce the resolution and maps the 3×3 convolution to channel 1 to obtain the segmentation map, and repeats the above process to obtain multi-scale segmentation maps corresponding to features of different resolutions.

4. The method for rapid detection of high-resolution traffic signs based on adaptive region filtering according to claim 1, characterized in that: The improved FPN+PAN feature fusion module incorporates the influence of segmentation maps into the multi-scale feature fusion part of the original FPN+PAN network. Different scale features require different regions of interest. The segmentation maps are used to adjust the bottom-up and top-down fusion processes, aligning the regions of interest for the fused features.

5. The method for rapid detection of high-resolution traffic signs based on adaptive region filtering according to claim 1, characterized in that: The feature refinement module, Tex module, while the CAM module is responsible for focusing on the region of interest, in contrast to CAM, the Tex module extracts feature information of the non-interest regions for different output results using cascaded 3×3 dilated convolutions (1, 2, 3) and refines the input features.

Images