A deep learning-based partially-occluded traffic sign recognition method

Abstract

The application discloses a kind of local occluded traffic sign recognition method based on deep learning, for the fine-grained object detection needs model to extract rich discriminant feature, an improved backbone network based on MobileNetv2 is proposed;For the traffic sign itself is small, is occluded and resolution is low, ordinary linear interpolation mode cannot capture higher order, more detailed feature, an improved dynamic weight upsampling module is proposed, the channel information after channel attention weighting is integrated to rearrange pixels, generate high-resolution image;For the problem that CIoU and other loss functions are sensitive to small target position changes, the normalized Gaussian wassertein distance is used to optimize the bounding box regression loss. It can be proved by experimental verification that the accuracy and recall rate of the method for local occluded traffic sign recognition have been further improved, and it has good recognition application effect in complex environment where traffic signs are partially occluded.

Description

Technical Field

[0001] This invention relates to the fields of deep learning technology and image classification technology, specifically to a method for recognizing partially occluded traffic signs based on deep learning. Background Technology

[0002] With the development of artificial intelligence technology, object detection algorithms have been widely used in fields such as intelligent driving and video indexing. Traffic signs on urban roads contain rich navigation information, which can instruct and warn drivers about their driving behavior. For the application of intelligent driving technology in vehicles, accurately detecting and recognizing traffic signs during driving is also an important technical component, helping intelligent driving assistance vehicles improve their road driving safety.

[0003] However, in real-world road environments, traffic signs can be obscured by trees, buildings, vehicles, and other objects. Obscured traffic signs can affect the recognition and accuracy of intelligent driving assistance vehicles, thus impacting driving safety. Therefore, accurate location and classification of traffic signs are crucial for driving safety and road sign maintenance.

[0004] In recent years, with the continuous development of artificial intelligence technology, deep learning technology has been widely applied to the intelligent detection and recognition of traffic signs. Compared with traditional detection methods, deep learning technology has better model generalization ability and robustness, and higher detection accuracy, making it suitable for various complex and diverse road environments. Although deep learning has achieved good results in the intelligent detection and recognition of traffic signs, the following shortcomings still exist:

[0005] (1) There is a lack of open-source datasets for traffic signs that are obscured. Most of them are traffic sign images without background, which do not meet the complex scenarios captured by vehicles in autonomous driving.

[0006] (2) Traffic sign recognition is a fine-grained image processing task, which requires the model to extract rich discriminative features;

[0007] (3) When traffic signs are obscured, some information is lost, which will increase the false negative rate of the model;

[0008] (4) Loss functions such as CIoU (Complete-IoU) are more sensitive to changes in the position of small targets.

[0009] Therefore, in complex environments where traffic signs are partially obscured, how to further improve the recognition accuracy and recall rate of partially obscured traffic signs is a new technical problem that needs to be solved. Summary of the Invention

[0010] To address the shortcomings of the existing technology, this invention provides a deep learning-based method for recognizing partially occluded traffic signs, thereby further improving the accuracy and recall rate of recognizing partially occluded traffic signs.

[0011] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0012] A deep learning-based method for recognizing partially occluded traffic signs involves acquiring a test image containing traffic signs, inputting it into a pre-trained occluded traffic sign recognition network model, and obtaining the traffic sign recognition result in the test image.

[0013] The occluded traffic sign recognition network model includes a backbone network based on an inverted residual network architecture and a high-resolution image reconstruction and recognition module based on channel attention weighting. The backbone network combines the inverted residual network structure with full-dimensional dynamic convolution to extract discriminative features from the input image, resulting in a discriminative feature map. The high-resolution image reconstruction and recognition module performs channel attention weighting on the obtained discriminative feature map, and then performs pixel rearrangement processing through sub-pixel convolution to obtain a reconstructed high-resolution image. The traffic sign recognition result is then obtained from the reconstructed high-resolution image and output as the traffic sign recognition result of the image to be tested.

[0014] Specifically, the backbone network includes a CBL module, an inverse residual module, a full-dimensional dynamic convolution module ODConv, a batch normalization unit (BN), a ReLU activation function unit, and a fast spatial pyramid pooling unit (SPPF) cascaded in sequence.

[0015] Specifically, the processing procedure of the backbone network is as follows:

[0016] ;

[0017] in, and These represent the input image and the output discriminant feature map of the backbone network, respectively. Represents the reverse residual operator; Represents the reverse residual operator; This represents a full-dimensional dynamic convolution operator; This indicates the batch normalization operator; Represents the ReLU activation function operator; This represents the fast space pyramid pooling operator.

[0018] Specifically, the operation process of the full-dimensional dynamic convolution module ODConv is represented as follows:

[0019] ;

[0020] in, and These represent the input and output features of the full-dimensional dynamic convolution module ODConv, respectively. , These represent the number of channels for the input feature and the output feature, respectively. This represents the first dimensionality of the full-dimensional dynamic convolution module ODConv. One convolutional kernel, , This indicates the number of convolutional kernels contained in the full-dimensional dynamic convolution module ODConv; For the first The spatial location of each convolution kernel and its convolution parameters; For the first Each convolutional kernel corresponds to The attention scalar allocated to each input channel; For the first Each convolutional kernel corresponds to The attention scalar allocated to each output channel filter; For the first Each convolutional kernel is assigned an attention scalar to the entire kernel space; This represents multiplication operations along different dimensions of the kernel space.

[0021] Specifically, the high-resolution image reconstruction and recognition module includes two encoding layers and two decoding layers connected in sequence;

[0022] Each coding layer consists of a cascaded 1×1 convolutional layer, a dynamic weight upsampling module (DEUM), a fully connected layer, and a 3×3 convolutional layer; each decoding layer consists of a cascaded 1×1 convolutional layer, a fully connected layer, and a 3×3 convolutional layer.

[0023] Specifically, the discriminant feature map input to the high-resolution image reconstruction and recognition module serves as the input to the first encoding layer, the output of the first encoding layer serves as the input to the second encoding layer, the output of the second encoding layer serves as the input to the second decoding layer, and the output of the second decoding layer serves as the input to the first decoding layer. Furthermore, the output of the 1×1 convolutional layer in the first encoding layer is also transmitted to the fully connected layer in the first decoding layer, and the output of the 1×1 convolutional layer in the second encoding layer is also transmitted to the fully connected layer in the second decoding layer. The output of the first decoding layer serves as the output of the high-resolution image reconstruction and recognition module.

[0024] Specifically, the dynamic weight upsampling module DEUM includes a channel attention unit, a two-dimensional convolution unit, a subpixel convolution unit, and a ReLU activation function unit cascaded in sequence.

[0025] Specifically, in the dynamic weight upsampling module DEUM, the processing procedure of the sub-pixel convolution unit is as follows:

[0026] ;

[0027] in and This represents the input and output feature maps of the sub-pixel convolutional layer. and These are the parameters of the two convolution kernels for the sub-pixel convolutional layer. This indicates a pixel rearrangement operation.

[0028] Specifically, the occluded traffic sign recognition network model is trained in the following manner:

[0029] S101: Prepare a traffic sign sample image dataset, which is divided into a training image dataset and a test image dataset. The traffic sign sample images in the traffic sign sample image dataset are all pre-marked with the real location of the traffic signs.

[0030] S102: Input the training image dataset into the occluded traffic sign recognition network model for training, and use the normalized Gaussian Wasserstein distance loss function to construct the bounding box regression loss. The occluded traffic sign recognition network model is trained and optimized with the goal of minimizing the bounding box regression loss until the occluded traffic sign recognition network model converges, and the trained occluded traffic sign recognition network model is obtained.

[0031] S103: Test the occluded traffic sign recognition network model using a test image dataset.

[0032] Specifically, the bounding box regression loss is expressed as:

[0033] ;

[0034] in, This represents the bounding box regression loss; The Gaussian distribution model representing the traffic sign recognition results output by the traffic sign recognition network model when the traffic sign is obscured. A Gaussian distribution model representing the true location markings of traffic signs;

[0035] Gaussian distribution model representing traffic sign recognition results Gaussian distribution model of traffic sign location markings Normalized Wasserstein distance between them:

[0036] ;

[0037] Where C represents the number of pixels contained in the average area of ​​the traffic sign's true location marker for each traffic sign sample in the training image dataset. Gaussian distribution model representing traffic sign recognition results Gaussian distribution model of traffic sign location markings The second-order Wasserstein distance between them.

[0038] Specifically, the Gaussian distribution model of traffic sign recognition results Gaussian distribution model of traffic sign location markings The second-order Wasserstein distance between them is expressed as :

[0039] ;

[0040] in, These represent the coordinates of the Gaussian distribution center of the traffic sign recognition results. And the pixel width and pixel height of the traffic sign recognition result box; These represent the coordinates of the center of the Gaussian distribution of the actual location marker of the traffic sign. And the pixel width and pixel height of the actual location marker box of the traffic sign; This represents L2 norm operations.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] This invention presents a deep learning-based method for recognizing partially occluded traffic signs. Addressing the need for rich discriminative features in fine-grained object detection, it proposes an improved backbone network based on MobileNetv2. To address the issue that traffic signs are small, occluded, and have low resolution, making it difficult for ordinary linear interpolation to capture higher-order, more detailed features, an improved dynamic weight upsampling module is proposed. This module integrates channel information after channel attention weighting for pixel rearrangement, generating a high-resolution image. To address the sensitivity of loss functions such as CIoU to changes in the position of small targets, normalized Gaussian Wasserstein distance is used to optimize the bounding box regression loss. Experimental results demonstrate that this method further improves the accuracy and recall of partially occluded traffic sign recognition, exhibiting excellent performance in complex environments where traffic signs are partially occluded. Attached Figure Description

[0043] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0044] Figure 1 This is a schematic diagram of the overall architecture of the occluded traffic sign recognition network model in the method of the present invention;

[0045] Figure 2 This is a diagram illustrating the operational principle of the full-dimensional dynamic convolution module;

[0046] Figure 3 This is a schematic diagram of the operation principle of the dynamic weight upsampling module DEUM in the high-resolution image reconstruction and recognition module;

[0047] Figure 4 This example shows the categories and sample images of the partially obscured traffic signs dataset in the embodiment.

[0048] Figure 5 This is a comparison chart of the traffic sign prediction and recognition results of different algorithm models in the embodiments. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but only to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0050] The terms "first," "second," and similar words used in the specification and claims of this patent application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, unless the context clearly indicates otherwise, the singular forms of "an," "a," or "the," etc., do not indicate a quantity limitation, but rather indicate the presence of at least one. Terms such as "comprising" or "including" mean that the element or object preceding "comprising" encompasses the features, integrals, steps, operations, elements, and / or components listed following "comprising" or "including," and do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; these relative positional relationships may change accordingly when the absolute position of the described object changes.

[0051] This invention provides a deep learning-based method for recognizing partially occluded traffic signs. It involves acquiring a test image containing traffic signs, inputting it into a pre-trained occluded traffic sign recognition network model, and obtaining the traffic sign recognition result in the test image. A key improvement of this invention lies in providing an improved occluded traffic sign recognition network model, such as... Figure 1 As shown, the occluded traffic sign recognition network model includes a backbone network based on an inverted residual network architecture and a high-resolution image reconstruction and recognition module based on channel attention weighting. The backbone network combines the inverted residual network structure with full-dimensional dynamic convolution to extract the discriminative features of the input image, resulting in a discriminative feature map. The high-resolution image reconstruction and recognition module performs channel attention weighting on the obtained discriminative feature map and then performs pixel rearrangement processing through sub-pixel convolution to obtain a reconstructed high-resolution image. The traffic sign recognition result is then obtained from the reconstructed high-resolution image and output as the traffic sign recognition result of the image to be tested.

[0052] The method for recognizing partially obscured traffic signs of the present invention, and the network model architecture for recognizing obscured traffic signs used therein, will be described in detail below with reference to the accompanying drawings.

[0053] The occluded traffic sign recognition network model used in the method of this invention includes a backbone network based on an inverted residual network architecture and a high-resolution image reconstruction and recognition module based on channel attention weighting. The following sections will elaborate on each part of the model.

[0054] I. Backbone Network

[0055] This invention employs a backbone network based on an improved MobileNetv2, integrating full-dimensional dynamic convolution into the inverted residual module to extract rich discriminative features from the input image and obtain the discriminative feature map of the input image.

[0056] The MobileNetv2 model is a lightweight convolutional neural network for mobile devices, using an inverted residual block in its backbone. The inverted residual block uses residual connections to perform dimensionality upscaling, feature extraction, and then dimensionality reduction on the input. Dimensionality upscaling introduces more channels and captures richer feature information. Furthermore, depthwise separable convolutions are used after dimensionality upscaling to reduce the number of parameters and computational complexity. However, its drawbacks include insufficient information exchange and limited representational capabilities.

[0057] However, traffic sign recognition is a fine-grained image processing task, characterized by large intra-class differences and small inter-class differences. Therefore, we designed a backbone network based on an improvement of MobileNetv2, which combines the inverted residual structure with omni-dimensional dynamic convolution (ODconv). Omni-dimensional dynamic convolution learns complementary attention in parallel along the four dimensions of the kernel space, and can dynamically adjust the size and shape of the convolution kernel according to the features of the input data, thus having a stronger feature extraction capability than traditional convolution.

[0058] like Figure 1 As shown, the improved backbone network of this invention includes a CBL module, an inverse residual module, a full-dimensional dynamic convolution module ODConv, a batch normalization unit (BN), a ReLU activation function unit, and a fast spatial pyramid pooling unit (SPPF) cascaded in sequence; the processing of the backbone network can be represented as follows:

[0059] ;

[0060] in, and These represent the input image and the output discriminant feature map of the backbone network, respectively. Represents the reverse residual operator; Represents the reverse residual operator; This represents a full-dimensional dynamic convolution operator; This indicates the batch normalization operator; Represents the ReLU activation function operator; This represents the fast space pyramid pooling operator.

[0061] In the backbone network, the CBL module, inverse residual module, batch normalization unit (BN), ReLU activation function unit, and fast spatial pyramid pooling unit (SPPF) are commonly used network module units in deep neural networks. For example, the CBL module consists of a convolutional layer (Conv), a batch normalization layer (BN), and a Leaky ReLU activation function; the inverse residual module consists of a CBL module, a depthwise separable convolution (DConv), a batch normalization layer (BN), a ReLU activation function, a regular convolution (Conv), another batch normalization layer (BN), and a fully connected layer; and so on.

[0062] The operation process of the full-dimensional dynamic convolution module ODConv can be represented as follows:

[0063] ;

[0064] The operational principle diagram of the full-dimensional dynamic convolution module is as follows: Figure 2 As shown, where, and These represent the input and output features of the full-dimensional dynamic convolution module ODConv, respectively. , These represent the number of channels for the input feature and the output feature, respectively. This represents the first dimensionality of the full-dimensional dynamic convolution module ODConv. One convolutional kernel, , This indicates the number of convolutional kernels contained in the full-dimensional dynamic convolution module ODConv; For the first The spatial location of each convolution kernel and its convolution parameters; For the first Each convolutional kernel corresponds to The attention scalar allocated to each input channel; For the first Each convolutional kernel corresponds to The attention scalar allocated to each output channel filter; For the first Each convolutional kernel is assigned an attention scalar to the entire kernel space; This represents multiplication operations along different dimensions of the kernel space.

[0065] II. High-resolution image reconstruction and recognition module

[0066] The high-resolution image reconstruction and recognition module of the present invention integrates the channel information after channel attention weighting to perform pixel rearrangement on the discrimination feature map, reconstructs the original input feature map into a high-resolution image, and then obtains the traffic sign recognition result based on the reconstructed high-resolution image.

[0067] The architecture of the high-resolution image reconstruction and recognition module of the present invention is as follows: Figure 1 The system comprises two sequentially connected encoding layers and two sequentially connected decoding layers. Each encoding layer includes a cascaded 1×1 convolutional layer, a dynamic weight upsampling module (DEUM), a fully connected layer, and a subpixel convolutional layer. Each decoding layer includes a cascaded 1×1 convolutional layer, a fully connected layer, and a subpixel convolutional layer. The discriminant feature map input to the high-resolution image reconstruction and recognition module serves as the input to the first encoding layer. The output of the first encoding layer serves as the input to the second encoding layer, the output of the second encoding layer serves as the input to the second decoding layer, and the output of the second decoding layer serves as the input to the first decoding layer. Furthermore, the output of the 1×1 convolutional layer in the first encoding layer is also transmitted to the fully connected layer in the first decoding layer, and the output of the 1×1 convolutional layer in the second encoding layer is also transmitted to the fully connected layer in the second decoding layer. The output of the first decoding layer serves as the output of the high-resolution image reconstruction and recognition module.

[0068] In the high-resolution image reconstruction and recognition module, the module architecture of the Dynamic Weight Upsampling Module (DEUM) is as follows: Figure 1 As shown, it includes channel attention units, two-dimensional convolution units, subpixel convolution units, and ReLU activation function units cascaded in sequence.

[0069] For occluded traffic signs in autonomous driving scenarios, there are issues such as low resolution and loss of some pixel values. Traditional upsampling methods, such as linear interpolation, cannot capture higher-order features, leading to image smoothing. For partially occluded traffic signs, subpixel convolution can leverage inter-channel correlation to help improve image detail and edge information.

[0070] In the dynamic weight upsampling module DEUM, the processing of sub-pixel convolution units is represented as follows:

[0071] ;

[0072] in and This represents the input and output feature maps of the sub-pixel convolutional layer. and These are the parameters of the two convolution kernels for the sub-pixel convolutional layer. This indicates a pixel rearrangement operation.

[0073] The operational principle of the dynamic weighted upsampling module DEUM is as follows: Figure 3 As shown. In the Dynamic Weight Upsampling Module (DEUM), for an input feature map with width W, height H, and number of channels c, before pixel rearrangement, the input feature map is first processed by a channel attention unit. During this process, the input feature map undergoes spatial dimension average pooling and channel max pooling respectively. Then, after passing through a fully connected layer, convolutional filters are applied for feature extraction. Finally, the outputs are stacked to obtain the channel attention feature map. Then, after a series of convolutions on the output channel attention feature map, an H×W×c result is obtained. 2 The intermediate feature map is then input into a sub-pixel convolutional unit for pixel rearrangement. For the position of each pixel in the intermediate feature map, its corresponding c is... 2 The pixels of each channel are rearranged into a c×c region, ultimately resulting in an H×W×c region. 2 The intermediate feature maps are rearranged into a high-resolution image I of size cH×cW×1. out .

[0074] From the perspective of channel weight allocation, during pixel rearrangement, not all channels of information for the target pixel are useful. Therefore, a dynamic weight upsampling module is designed to assign weights to each channel using a channel attention mechanism before subpixel convolution.

[0075] III. Model Training and Bounding Box Regression Loss

[0076] In the training of the high-resolution image reconstruction and recognition module, this invention uses a normalized Gaussian Wasserstein distance loss function to optimize the bounding box regression loss, thereby completing the training of the high-resolution image reconstruction and recognition module and improving training efficiency.

[0077] In this invention, the network model for recognizing occluded traffic signs is trained in the following manner:

[0078] S101: Prepare a traffic sign sample image dataset, which is divided into a training image dataset and a test image dataset. The traffic sign sample images in the traffic sign sample image dataset are all pre-marked with the real location of the traffic signs.

[0079] S102: Input the training image dataset into the occluded traffic sign recognition network model for training, and use the normalized Gaussian Wasserstein distance loss function to construct the bounding box regression loss. The occluded traffic sign recognition network model is trained and optimized with the goal of minimizing the bounding box regression loss until the occluded traffic sign recognition network model converges, and the trained occluded traffic sign recognition network model is obtained.

[0080] S103: Test the occluded traffic sign recognition network model using a test image dataset.

[0081] In practice, the occluded traffic sign recognition network model can be built using PyTorch and then trained. Both the training and test sets have a batch size of 16, and the dataset is divided into training and test sets. The weight file with the minimum loss on the test set is saved and used to predict occluded traffic signs. Finally, the weight file with the minimum loss is used to test with actual occluded traffic sign images, predicting each image individually.

[0082] In real-world autonomous driving scenarios, traffic signs occupy only a small portion of the image and are not strictly rectangular. Foreground and background pixels within the bounding box are concentrated at the center and boundary, respectively. The Normalized Gaussian Wasserstein Distance (NWD) loss function is used to optimize the bounding box regression loss. This model the bounding box as a two-dimensional Gaussian distribution, with the center pixel having the highest weight and pixels farther from the center having lower weights. For the horizontal bounding box R=(c x ,c y ,w,h), where (c x ,c y Let w and h represent the center coordinates, width, and height, respectively. The equation of its inscribed ellipse can be expressed as:

[0083] ;

[0084] in The coordinates of the center of the ellipse It is the length of the semi-axis along the x and y axes, therefore:

[0085] , , , .

[0086] The probability density function of a two-dimensional Gaussian distribution can be expressed as:

[0087] ;

[0088] Where x, Represent the coordinates (x, y), mean vector, and covariance matrix of the Gaussian distribution. Define the horizontal bounding box R=(c x ,c y Modeling w,h) as a two-dimensional Gaussian distribution ,in:

[0089] ;

[0090] Therefore, the similarity between the predicted bounding box and the ground truth bounding box can be converted into the distance between two Gaussian distributions.

[0091] For two two-dimensional Gaussian distributions and The second-order Wasserstein distance between them is defined as:

[0092] ;

[0093] The above formula can be simplified to:

[0094] ;

[0095] in, It is the L2 norm. It is the Frobenius norm.

[0096] For bounding box A(cx) a ,cy a ,w a ,h a ) and B(cx b ,cy b ,w b ,h b The Gaussian distribution N modeled a and N b The above formula can be simplified to:

[0097] ;

[0098] But W2 2 (N a N b Since is a distance metric, it cannot be directly used for similarity measurement. Therefore, it is normalized exponentially to obtain the normalized Wasserstein distance, where the constant C is the average absolute size of the targets in the dataset.

[0099] .

[0100] Based on the above approach, in this application, the bounding box regression loss is expressed as:

[0101] ;

[0102] in, This represents the bounding box regression loss; The Gaussian distribution model representing the traffic sign recognition results output by the traffic sign recognition network model when the traffic sign is obscured. A Gaussian distribution model representing the true location markings of traffic signs;

[0103] Gaussian distribution model representing traffic sign recognition results Gaussian distribution model of traffic sign location markings Normalized Wasserstein distance between them:

[0104] ;

[0105] Where C represents the number of pixels contained in the average area of ​​the traffic sign's true location marker for each traffic sign sample in the training image dataset. Gaussian distribution model representing traffic sign recognition results Gaussian distribution model of traffic sign location markings The second-order Wasserstein distance between them.

[0106] Gaussian distribution model of traffic sign recognition results Gaussian distribution model of traffic sign location markings The second-order Wasserstein distance between them is expressed as :

[0107] ;

[0108] in, These represent the coordinates of the Gaussian distribution center of the traffic sign recognition results. And the pixel width and pixel height of the traffic sign recognition result box; These represent the coordinates of the center of the Gaussian distribution of the actual location marker of the traffic sign. And the pixel width and pixel height of the actual location marker box of the traffic sign; This represents L2 norm operations.

[0109] Example

[0110] In this embodiment, the datasets used are the self-made Chinese occluded traffic sign dataset COTSD and the publicly available CCTSDB dataset.

[0111] The self-made Chinese Occluded Traffic Signs Dataset COTSD: This dataset is a self-made private dataset for traffic signs that are occluded. It contains 3,612 images of traffic signs of different types, covering 16 common traffic sign categories, with a total of 8,734 labels. It is divided into a training set of 2,927 images and a test set of 685 images.

[0112] The publicly available CCTSDB dataset is one of the commonly used public datasets for road traffic sign experiments. It provides 17,856 images containing traffic sign instances, and includes three categories: warning signs, prohibition signs, and instruction signs. It includes complex environments such as rainy days, foggy days, and dark days. In experiments, the dataset is randomly divided into training and test sets at a ratio of 9:1.

[0113] The dataset categories and sample images of partially occluded traffic signs used in this embodiment are as follows: Figure 4 As shown.

[0114] In this embodiment, the performance of the method proposed in this invention is evaluated on two datasets, along with other comparative models.

[0115] To objectively evaluate the performance of the network model, precision (P), recall (R), F1-Score, and mean average precision (mAP50) were used as evaluation metrics. In the backbone network comparison experiments, parameter count, GFLOPs, precision, recall (R), and mAP50 were used as evaluation metrics. Parameter count and GFLOPs are commonly used performance metrics for lightweight model tasks. FPS is an evaluation metric for the real-time performance of object detection algorithms. Autonomous driving has high requirements for model real-time performance, and the model's FPS needs to reach above 30. The implementation results are shown in Tables 1, 2, and 3 below. Figure 5 As shown.

[0116] Table 1 Evaluation metrics of each backbone network on the self-made COTSD dataset

[0117]

[0118] Table 2 Evaluation metrics for each model on the self-made COTSD dataset

[0119]

[0120] Table 3 Evaluation metrics for each model on the publicly available CCTSDB dataset

[0121]

[0122] at the same time, Figure 5 A comparison chart of the traffic sign prediction and recognition results of different algorithm models in this embodiment is provided. Figure 5 (a) shows the traffic sign recognition bounding box results of the YOLOv3-tiny model. Figure 5 (b) Traffic sign recognition bounding box results from the YOLOv5 model. Figure 5 (c) Traffic sign recognition bounding box results from the YOLOv7 model Figure 5 (d) Traffic sign recognition bounding box results from the YOLOv8 model Figure 5 (e) Traffic sign recognition bounding box results of the SSD model. Figure 5 (f) Traffic sign recognition bounding box results from the RetinaNet model Figure 5 (g) Traffic sign recognition bounding box results of the Faster R-CNN model Figure 5(h) Traffic sign recognition bounding box results of the method of the present invention; The annotation information in Figure 5 represents the label classification, confidence level, and predicted bounding box position, respectively. The color of the annotation information varies depending on the category of the sign. The confidence level is a measure of whether the algorithm accurately predicts the target, with a value ranging from 0 to 1, and a higher value is more ideal. For example, for the first test set sample, except for the SSD and RetinaNet models, all other model categories were correctly predicted, but the confidence levels of the predictions of each model were different. By comparison, it can be seen that the method proposed in this invention has a confidence level as high as 0.99 on the first test set sample, and the recognition effect is the best.

[0123] On the other hand, the above comparisons also show that, after a series of experimental verifications, the algorithm of this invention outperforms other algorithms in terms of detection accuracy and recall on open-source datasets. On a self-made dataset targeting occluded traffic signs, the algorithm of this invention outperforms other algorithms in recall, except for YOLOv3-tiny. Furthermore, the improved algorithm exhibits the best performance compared to other algorithms in terms of comprehensive metrics such as accuracy, F1 score, and mAP50, achieving an FPS of 120.48. On the self-made occluded COTSD dataset, the method of this invention achieves an accuracy of 93.60%, a recall of 72.50%, an F1 score of 81.71%, and an mAP50 of 79%. On the publicly available CCTSDB dataset, which contains a small number of occluded traffic signs, the accuracy is 92.2%, the recall is 78.8%, the F1 score is 85%, and the mAP50 is 88.5%. The experimental results on both datasets demonstrate that the improved algorithm of this invention can effectively improve the recognition accuracy of occluded traffic signs. Therefore, the algorithm of this invention improves the detection and recognition capabilities of traffic signs that are obscured, and can meet the application requirements in real driving scenarios.

[0124] In summary, to improve the accuracy of partially occluded traffic sign recognition in autonomous driving scenarios, this invention proposes an improved backbone network based on MobileNetv2 to address the need for models to extract rich discriminative features for fine-grained object detection. To address the issue that ordinary linear interpolation methods cannot capture higher-order, more detailed features due to the small size, occlusion, and low resolution of traffic signs, an improved dynamic weight upsampling module is proposed. This module integrates channel information after channel attention weighting for pixel rearrangement, generating high-resolution images. Furthermore, to address the sensitivity of loss functions such as CIoU to changes in the position of small targets, normalized Gaussian Wasserstein distance is used to optimize the bounding box regression loss. Experimental results demonstrate that the proposed method further improves both the accuracy and recall of partially occluded traffic sign recognition, exhibiting excellent recognition performance in complex environments where traffic signs are partially occluded.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A method for recognizing partially occluded traffic signs based on deep learning, characterized in that, The test image containing traffic signs is obtained and input into a pre-trained occluded traffic sign recognition network model to obtain the traffic sign recognition result in the test image. The occluded traffic sign recognition network model includes a backbone network based on an inverted residual network architecture and a high-resolution image reconstruction and recognition module based on channel attention weighting. The backbone network combines the inverted residual network structure with full-dimensional dynamic convolution to extract discriminative features from the input image, resulting in a discriminative feature map. The high-resolution image reconstruction and recognition module performs channel attention weighting on the obtained discriminative feature map, followed by pixel rearrangement processing through sub-pixel convolution to obtain a reconstructed high-resolution image. The traffic sign recognition result is then obtained from the reconstructed high-resolution image and output as the traffic sign recognition result of the image to be tested. The backbone network comprises a cascaded CBL module, an inverse residual module, a full-dimensional dynamic convolution module ODConv, a batch normalization unit (BN), a ReLU activation function unit, and a fast spatial pyramid pooling unit (SPPF); the processing procedure of the backbone network is as follows: ； in, and These represent the input image and the output discriminant feature map of the backbone network, respectively. This represents the reverse residual operator; This represents a full-dimensional dynamic convolution operator; This indicates the batch normalization operator; Represents the ReLU activation function operator; This represents the fast space pyramid pooling operator; The obscured traffic sign recognition network model is trained in the following manner: S101: Prepare a traffic sign sample image dataset, which is divided into a training image dataset and a test image dataset. The traffic sign sample images in the traffic sign sample image dataset are all pre-marked with the real location of the traffic signs. S102: Input the training image dataset into the occluded traffic sign recognition network model for training, and use the normalized Gaussian Wasserstein distance loss function to construct the bounding box regression loss. The occluded traffic sign recognition network model is trained and optimized with the goal of minimizing the bounding box regression loss until the occluded traffic sign recognition network model converges, and the trained occluded traffic sign recognition network model is obtained. S103: Test the occluded traffic sign recognition network model using a test image dataset.

2. The method for recognizing partially occluded traffic signs based on deep learning according to claim 1, characterized in that, The operation process of the full-dimensional dynamic convolution module ODConv can be represented as follows: ； in, and These represent the input and output features of the full-dimensional dynamic convolution module ODConv, respectively. , These represent the number of channels for the input feature and the output feature, respectively. This represents the first dimensionality of the full-dimensional dynamic convolution module ODConv. One convolutional kernel, , This indicates the number of convolutional kernels contained in the full-dimensional dynamic convolution module ODConv; For the first The spatial location of each convolution kernel and its convolution parameters; For the first Each convolutional kernel corresponds to The attention scalar allocated to each input channel; For the first Each convolutional kernel corresponds to The attention scalar allocated to each output channel filter; For the first Each convolutional kernel is assigned an attention scalar to the entire kernel space; This represents multiplication operations along different dimensions of the kernel space.

3. The method for recognizing partially occluded traffic signs based on deep learning according to claim 1, characterized in that, The high-resolution image reconstruction and recognition module includes two encoding layers and two decoding layers connected in sequence; Each coding layer consists of a cascaded 1×1 convolutional layer, a dynamic weight upsampling module (DEUM), a fully connected layer, and a 3×3 convolutional layer; each decoding layer consists of a cascaded 1×1 convolutional layer, a fully connected layer, and a 3×3 convolutional layer. Specifically, the discriminant feature map input to the high-resolution image reconstruction and recognition module serves as the input to the first encoding layer, the output of the first encoding layer serves as the input to the second encoding layer, the output of the second encoding layer serves as the input to the second decoding layer, and the output of the second decoding layer serves as the input to the first decoding layer. Furthermore, the output of the 1×1 convolutional layer in the first encoding layer is also transmitted to the fully connected layer in the first decoding layer, and the output of the 1×1 convolutional layer in the second encoding layer is also transmitted to the fully connected layer in the second decoding layer. The output of the first decoding layer serves as the output of the high-resolution image reconstruction and recognition module.

4. The method for recognizing partially occluded traffic signs based on deep learning according to claim 3, characterized in that, The dynamic weighted upsampling module DEUM includes a channel attention unit, a two-dimensional convolution unit, a subpixel convolution unit, and a ReLU activation function unit, which are cascaded in sequence.

5. The method for recognizing partially occluded traffic signs based on deep learning according to claim 4, characterized in that, In the dynamic weight upsampling module DEUM, the processing procedure of the sub-pixel convolution unit is as follows: ； in and This represents the input and output feature maps of a sub-pixel convolutional layer. and These are the parameters of the two convolution kernels for the sub-pixel convolutional layer. This indicates a pixel rearrangement operation.

6. The method for recognizing partially occluded traffic signs based on deep learning according to claim 1, characterized in that, The bounding box regression loss is expressed as: ； in, This represents the bounding box regression loss; The Gaussian distribution model representing the traffic sign recognition results output by the traffic sign recognition network model when the traffic sign is obscured. A Gaussian distribution model representing the true location markings of traffic signs; Gaussian distribution model representing traffic sign recognition results Gaussian distribution model of traffic sign location markings Normalized Wasserstein distance between them: ； Where C represents the number of pixels contained in the average area of ​​the traffic sign's true location marker for each traffic sign sample in the training image dataset. Gaussian distribution model representing traffic sign recognition results Gaussian distribution model of traffic sign location markings The second-order Wasserstein distance between them.

7. The method for recognizing partially occluded traffic signs based on deep learning according to claim 6, characterized in that, Gaussian distribution model of traffic sign recognition results Gaussian distribution model of traffic sign location markings The second-order Wasserstein distance between them is expressed as : ； in, These represent the coordinates of the Gaussian distribution center of the traffic sign recognition results. And the pixel width and pixel height of the traffic sign recognition result box; These represent the coordinates of the center of the Gaussian distribution of the actual location marker of the traffic sign. And the pixel width and pixel height of the actual location marker box of the traffic sign; This represents L2 norm operations.

Images