Locally-shielded traffic sign recognition method based on attention-enhanced CNN
By using dynamic occlusion data augmentation and attention-enhanced CNNs, the feature robustness and adaptability of the traffic sign recognition model in occluded scenarios are improved, solving the problem of decreased recognition accuracy of existing models under occlusion conditions and achieving higher recognition accuracy.
Patent Information
- Application Number
- CN202511878655.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-12
- Publication Date
- 2026-03-20
AI Technical Summary
Existing traffic sign recognition models have not been adequately exposed to occluded samples of various types, scales, and forms during the training phase. This results in a lack of sufficient feature robustness and adaptability when faced with partially occluded images during the inference phase, leading to a significant decrease in recognition accuracy.
A local occlusion traffic sign recognition method based on attention-enhanced CNN is adopted. Through techniques such as dynamic occlusion data augmentation, lightweight convolutional neural network, pyramid-style multi-level feature decoupling module, channel attention mechanism and depth-separable convolution with holes, the feature extraction and classification capabilities of the model are improved.
It improves the feature extraction quality and class discrimination ability of the model under occlusion conditions, maintains stable recognition ability on partially occluded images, and improves recognition accuracy.
Smart Images
Figure CN121708575A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image recognition technology, and in particular to a method for recognizing partially occluded traffic signs based on attention-enhanced CNNs. Background Technology
[0002] With the rapid development of technologies such as intelligent transportation, autonomous driving, and smart cities, computer vision-based traffic sign recognition systems play an irreplaceable and crucial role in road traffic management and intelligent vehicle control. Traffic sign recognition technology requires systems capable of accurately and in real-time identifying different types of traffic signs from continuous road images captured by vehicle-mounted cameras, providing critical decision-making support for vehicle route planning, speed control, and safety strategies.
[0003] In existing technologies, traffic sign recognition typically relies on deep convolutional neural networks to automatically extract and classify features from images. For example, traditional support vector machine (SVM) methods rely on manually selected features such as color, shape, and texture for classification, which suffers from severe feature loss and a significant drop in accuracy in occluded scenarios. Lightweight networks, such as mobile models based on depthwise separable convolutions, reduce computational costs but lack dedicated mechanisms for handling occlusion.
[0004] During the research process, the inventors of this application discovered that existing traffic sign recognition models fail to be adequately exposed to occluded samples of various types, scales, and forms during the training phase. This results in the model lacking sufficient feature robustness and adaptability when facing partially occluded images during the inference phase, leading to a significant decrease in recognition accuracy. Summary of the Invention
[0005] To overcome the above shortcomings, this invention provides a traffic sign recognition method based on attention-enhanced CNN, which aims to improve the problem that existing traffic sign recognition models fail to be fully exposed to occluded samples of various types, scales and forms during the training phase, resulting in the model lacking sufficient feature robustness and adaptability when facing partially occluded images during the inference phase.
[0006] This invention provides the following technical solution: a method for recognizing partially occluded traffic signs based on attention-enhanced CNNs, comprising the following steps:
[0007] S1. Perform dynamic occlusion data augmentation processing on the input traffic sign image to obtain the enhanced traffic sign image;
[0008] S2. Input the enhanced traffic sign image into a lightweight convolutional neural network;
[0009] S3. The lightweight convolutional neural network sequentially performs primary feature extraction, intermediate feature fusion, and high-level feature abstraction through a pyramid-style multi-level feature decoupling module.
[0010] S4. In the intermediate feature fusion process, a channel attention mechanism is introduced to weight each channel of the feature map;
[0011] S5. In the advanced feature abstraction process, dilated depth-separable convolution is used to process the feature map;
[0012] S6. After global average pooling, the feature map after abstracting the high-level features is input into three fully connected layers in sequence.
[0013] S7. The predicted probabilities of various traffic signs are output through the last fully connected layer to obtain the recognition results.
[0014] Preferably, the dynamic occlusion data enhancement processing of S1 includes the following steps in sequence:
[0015] Randomly overlay blocky occlusion regions or Gaussian noise occlusion regions onto the input traffic sign image;
[0016] The superimposed occluded image is subjected to random rotation, random translation, random cropping, and random scaling operations in sequence.
[0017] Perform random brightness adjustment and random contrast adjustment operations sequentially on the image after geometric transformation;
[0018] The images that have undergone all enhancement operations are uniformly adjusted to a size of 224×224 pixels to obtain the enhanced traffic sign images.
[0019] Preferably, the convolutional part of the lightweight convolutional neural network is constructed entirely using depthwise separable convolutions, comprising five convolutional stages, with the number of output channels for the five convolutional stages being 32, 64, 64, 128, and 128, respectively.
[0020] Preferably, the primary feature extraction in the pyramid-shaped multi-level feature decoupling module is completed by the first to third convolutional stages, and the first to third convolutional stages are all followed by a max pooling layer after depthwise separable convolution.
[0021] Preferably, the specific steps of the channel attention mechanism in S4 are as follows:
[0022] Global average pooling is performed on the input intermediate feature map to obtain a single value for each channel;
[0023] The vector after global average pooling is input into the first fully connected layer for dimensionality reduction, and then input into the second fully connected layer for dimensionality increase.
[0024] Apply the Sigmoid activation function to the output of the second fully connected layer to obtain the weighting coefficients for each channel;
[0025] The weighted coefficients of each channel are multiplied by the corresponding channels of the original intermediate feature map to obtain the weighted intermediate feature map.
[0026] Preferably, in the channel attention mechanism, the number of neurons in the first fully connected layer is 1 / 16 of the number of input channels, and the number of neurons in the second fully connected layer is restored to the original number of channels.
[0027] Preferably, the hole depth of the S5 is separable, the dilation rate of the convolution is 2, the kernel size is 3×3, and the number of output channels is 128.
[0028] Preferably, the number of neurons in the three fully connected layers of the lightweight convolutional neural network is 512, 256, and the number of traffic sign categories, respectively, and a dropout layer with a dropout rate of 0.3 is set after the first two fully connected layers.
[0029] Preferably, the lightweight convolutional neural network is trained in the following manner:
[0030] Use cross-entropy as the loss function;
[0031] The network parameters are updated using stochastic gradient descent with momentum, and the momentum coefficient is 0.9.
[0032] Training is terminated early if the validation set loss does not decrease for 10 consecutive training rounds.
[0033] Preferably, during the reasoning phase, the following order should be followed:
[0034] The input traffic sign images are only normalized and resized to 224×224 pixels.
[0035] The preprocessed image is input into a lightweight convolutional neural network to complete all forward computation processes from S2 to S7, and the recognition result is output.
[0036] The present invention has the following beneficial effects:
[0037] 1. In this invention, dynamic occlusion data augmentation processing is used to apply block occlusion, Gaussian noise occlusion, geometric transformation, and photometric transformation to the input image during the training phase, so that the training samples cover different occlusion types and different imaging conditions. This enables the convolutional neural network to encounter multiple occlusion forms during training, thereby maintaining a stable recognition ability for partially occluded images during the inference phase.
[0038] 2. This invention employs depthwise separable convolutions to construct a convolutional network structure, combined with a pyramid-style multi-level feature decoupling module and a channel attention mechanism, to achieve independent feature processing at different scales and weighting of channel dimensions. This reduces the number of network parameters while enhancing the discriminative power of feature representation, enabling the network to obtain more complete image structural information with a fixed computational load, and improving feature extraction quality under occlusion conditions.
[0039] 3. This invention employs dilated, depthwise separable convolutions with an expansion rate of 2 in the advanced feature abstraction stage, and generates classification results after feature aggregation through global average pooling and a multi-layer fully connected structure. Dilated convolutions expand the effective receptive region of the convolution kernel without increasing the number of convolution parameters, enabling the network to extract a wider range of contextual information while maintaining stable computational load, thereby improving the class discrimination ability in the final recognition stage. Attached Figure Description
[0040] Figure 1 This is a flowchart of the method for recognizing traffic signs with local occlusion based on attention-enhanced CNN proposed in this invention;
[0041] Figure 2 This is a diagram of the pyramid-shaped multi-level feature decoupling convolutional architecture model of the local occlusion traffic sign recognition method based on attention-enhanced CNN proposed in this invention;
[0042] Figure 3 The images show the accuracy and loss rate of the local occlusion traffic sign recognition method based on attention-enhanced CNN proposed in this invention at each stage.
[0043] Figure 4 This is a heatmap showing the recognition accuracy of various traffic signs after 20 rounds of training of the attention-enhanced CNN-based local occlusion traffic sign recognition method proposed in this invention.
[0044] Figure 5 This is a heatmap showing the recognition accuracy of various traffic signs after 40 rounds of training of the attention-enhanced CNN-based local occlusion traffic sign recognition method proposed in this invention. Detailed Implementation
[0045] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0046] Reference Figures 1-5In the first embodiment of the present invention, the present invention provides a method for recognizing partially occluded traffic signs based on attention-enhanced CNN, comprising the following steps:
[0047] S1. Perform dynamic occlusion data augmentation processing on the input traffic sign image to obtain the enhanced traffic sign image;
[0048] S2. Input the enhanced traffic sign image into a lightweight convolutional neural network;
[0049] S3, the lightweight convolutional neural network sequentially performs primary feature extraction, intermediate feature fusion and high-level feature abstraction through a pyramid-style multi-level feature decoupling module;
[0050] S4. In the intermediate feature fusion process, a channel attention mechanism is introduced to weight each channel of the feature map;
[0051] S5. In the advanced feature abstraction process, dilated depth-separable convolution is used to process the feature map;
[0052] S6. After global average pooling, the feature map after abstracting the high-level features is input into three fully connected layers in sequence.
[0053] S7. The predicted probabilities of various traffic signs are output through the last fully connected layer to obtain the recognition results.
[0054] Specifically, S1 involves performing dynamic occlusion data augmentation on the input traffic sign image to obtain an enhanced traffic sign image. Dynamic occlusion data augmentation applies occlusion regions, geometric transformations, and photometric transformations to the input image, enabling the network training phase to receive image data containing different occlusion patterns. S2 involves inputting the enhanced traffic sign image into a lightweight convolutional neural network. This convolutional neural network includes multiple convolutional operation units, and the convolutional kernels employ a depth-separable structure.
[0055] S3, the lightweight convolutional neural network performs a pyramid-style multi-level feature decoupling operation on the input image. This operation divides feature extraction into three stages: primary feature extraction, intermediate feature fusion, and high-level feature abstraction, with each stage corresponding to a different scale of feature representation. S4, a channel attention mechanism is introduced during the intermediate feature fusion process to perform weighted processing on each channel of the feature map. The channel attention mechanism generates channel weights based on the statistics of each channel, and these weights are used to adjust the relative importance of the channels.
[0056] S5, during the advanced feature abstraction process, dilated depthwise separable convolution is used to perform convolution operations on the feature map. Dilated convolution adjusts the sampling interval of the convolution kernel using a dilation rate, thereby expanding the receptive field without changing the number of parameters. S6, global average pooling is performed on the feature map obtained from the advanced feature abstraction, and the pooling results are sequentially input into three fully connected layers to generate intermediate feature vectors and the final classification vector. Dropout units are set between fully connected layers to reduce the risk of overfitting.
[0057] S7 outputs the predicted probabilities of various traffic signs through the last fully connected layer, obtaining the recognition result. The predicted probability vector is calculated using the Softmax function, the expression of which is:
[0058] ;
[0059] in, For the output probability of category iii, This represents the real value corresponding to category i output by the fully connected layer. This represents the total number of traffic sign categories. This is for exponential operations.
[0060] The above steps constitute the traffic sign recognition process. After enhancement processing, the input image enters the convolutional neural network, which generates classification results according to a fixed computational process.
[0061] Furthermore, the dynamic occlusion data enhancement processing of S1 includes the following steps:
[0062] Randomly overlay blocky occlusion regions or Gaussian noise occlusion regions onto the input traffic sign image;
[0063] The superimposed occluded image is subjected to random rotation, random translation, random cropping, and random scaling operations in sequence.
[0064] Perform random brightness adjustment and random contrast adjustment operations sequentially on the image after geometric transformation;
[0065] The images that have undergone all enhancement operations are uniformly adjusted to a size of 224×224 pixels to obtain the enhanced traffic sign images.
[0066] Specifically, the input image is denoted as: ;in, Image height, 3 represents the image width, and 3 represents the three RGB color channels.
[0067] During the occlusion region overlay process, block-shaped occlusion regions are first applied to the input image according to a preset probability. These block-shaped occlusion regions have a rectangular structure, with their height and width defined by a range of... and The occlusion area is randomly selected within the image coordinate system. The occlusion pixel value can be set to a fixed value or a random value sampled from a uniform distribution.
[0068] In another occlusion method, Gaussian noise is used to cover local regions of the input image. The Gaussian noise matrix is represented as: Where σ is the standard deviation of Gaussian noise. The noise matrix is added element-wise to the input image within a specified local region to obtain the noise-occluded image. The two occlusion methods mentioned above are selected and executed independently according to a preset ratio.
[0069] After the occlusion overlay is completed, a geometric transformation operation is performed on the image. The random rotation angle is from the interval... The rotation matrix is obtained from the data and denoted as:
[0070] ;
[0071] in, Rotation angle. Image coordinate points are transformed according to the rotation matrix.
[0072] Then a translation operation is performed, with the horizontal and translation offsets respectively from the interval... and Independent sampling in the middle, the translation vector is represented as: ;in, and These represent the offsets in the horizontal and vertical directions, respectively.
[0073] In the shearing operation, the shearing factor is from the interval The selected element has the following shear matrix:
[0074] ;
[0075] in, For shearing parameters.
[0076] The scaling factor for the scaling operation is selected from the interval [0.8, 1.2]. The scaling matrix is:
[0077] ;
[0078] Where s is the scaling factor.
[0079] The above geometric transformations are combined to form an affine transformation matrix: The affine matrix is used to map the coordinates of the input image and perform geometric transformations.
[0080] During the photometric transformation process, the image brightness is first adjusted. The brightness increment is selected from the interval [0.1, 0.3], and the brightness transformation formula is as follows:
[0081] ;
[0082] in, This is the brightness adjustment amount. For color channel indexes.
[0083] Then, contrast adjustment is performed, with the contrast change selected from the interval [0.2, 0.4]. The contrast transformation formula is as follows:
[0084] ;
[0085] in, For contrast factor, For the current image in color channels The average pixel value.
[0086] The image, after geometric and photometric transformations, is resized to 224×224×3 using bilinear interpolation to form the enhanced traffic sign image. This size matches the input interface of the subsequent convolutional neural network. This dynamic occlusion data augmentation step provides input samples with different occlusion structures, poses, and photometric conditions for the network training phase. It generates input image data that conforms to real-world occlusion scenarios during training, supporting the stability of the subsequent network training process.
[0087] Furthermore, the convolutional part of the lightweight convolutional neural network is constructed entirely using depthwise separable convolutions, and contains a total of five convolutional stages. The number of output channels for the five convolutional stages are 32, 64, 64, 128, and 128, respectively.
[0088] The primary feature extraction in the pyramid-style multi-level feature decoupling module is completed by the first to third convolutional stages. The first to third convolutional stages are all followed by a max pooling layer after depthwise separable convolution.
[0089] Specifically,
[0090] The pyramid-style multi-level feature decoupling module processes the features output from the first to fifth convolutional stages in layers according to spatial scale and semantic depth. (Primary feature map) These correspond to three spatial dimensions: 112×112, 56×56, and 28×28, respectively. These characteristics are denoted as:
[0091] ;
[0092] To achieve feature decoupling, channel-wise normalization is performed on features at each scale, as calculated below:
[0093] ;
[0094] in and Feature maps The mean and standard deviation of the c-th channel.
[0095] The features at different scales were then converted to the same size of 28×28 using bilinear interpolation, resulting in...
[0096] ;
[0097] And then splice them together along the channel dimension: ;in, .
[0098] The aforementioned feature concatenation structure allows semantic information at different scales to be processed independently and then combined to form a decoupled feature set in the pyramid structure, providing input for subsequent attention weighting and dilated convolution processing.
[0099] The lightweight convolutional neural network takes an enhanced image of size 224×224×3 as input, and the input data is denoted as... Depthwise separable convolution consists of depthwise convolution and pointwise convolution. Depthwise convolution performs spatial convolution operations independently on each channel, while pointwise convolution linearly combines channels using a 1×1 convolution kernel. Taking the input feature map X as an example, the formula for calculating depthwise convolution is:
[0100] ;
[0101] in, The kernel size is the convolution kernel size. represents the weights of the depthwise convolution, (x, y) represents the spatial coordinates, and c represents the channel index.
[0102] The formula for pointwise convolution is:
[0103] ;
[0104] in, Input the number of channels. For output channel index, The weights are 1×1 convolutional kernel weights. The output of a depthwise separable convolution is denoted as... The first convolution stage is applied to the input Performing a 3×3 depthwise separable convolution with 32 output channels yields the following result. .
[0105] The convolution is followed by a 2×2 max-pooling layer with a 2×2 window and a stride of 2. The max-pooling operation formula is:
[0106]
[0107] Obtain the feature map after pooling .
[0108] The second convolution stage uses As input, perform a 3×3 depthwise separable convolution, with an output of 64 channels. The effective stride of the convolution kernel in the spatial dimension is set to 1. The output feature map is denoted as: ;
[0109] Convolution followed by max pooling layer yields .
[0110] The third convolution stage Taking the input as input, perform a 3×3 depthwise separable convolution, with an output channel count of 64, to obtain... Convolution followed by max pooling layer yields... The convolution parameters in the three stages mentioned above are independent of each other, and the model obtains edge information, contour information, and local pattern information at different spatial scales.
[0111] The first to third convolutional stages constitute the primary feature extraction part. This part gradually reduces the spatial dimension of the input image from 224×224 to 28×28, and obtains low-level features in the process. All three stages perform a combination of depthwise separable convolution and max pooling, reducing the number of convolution parameters, improving the computational speed of convolution operations, and maintaining feature expressiveness.
[0112] The fourth convolution stage Perform a depthwise separable convolution with 128 output channels to obtain... In the fifth convolutional stage, depthwise separable convolution is performed, with an output channel count of 128, resulting in... The output features of the fourth and fifth convolutional stages are used in the intermediate feature fusion and high-level feature abstraction stages. All five convolutional stages use the same kernel size and activation function, and the weight parameters for the convolutional operations are learned from the training data.
[0113] The sequential combination of depthwise separable convolution and max pooling constitutes the primary feature extraction structure of this invention. This structure maps different regions of the input image to response values of different channels, progressively forming low-dimensional, multi-channel feature maps, which facilitates subsequent processing by the pyramid-shaped feature decoupling module.
[0114] Furthermore, the specific steps of S4's channel attention mechanism are as follows:
[0115] Global average pooling is performed on the input intermediate feature map to obtain a single value for each channel;
[0116] The vector after global average pooling is input into the first fully connected layer for dimensionality reduction, and then input into the second fully connected layer for dimensionality increase.
[0117] Apply the Sigmoid activation function to the output of the second fully connected layer to obtain the weighting coefficients for each channel;
[0118] The weighting coefficients of each channel are multiplied by the corresponding channels of the original intermediate feature map to obtain the weighted intermediate feature map;
[0119] In the channel attention mechanism, the number of neurons in the first fully connected layer is 1 / 16 of the number of input channels, and the number of neurons in the second fully connected layer is restored to the original number of channels.
[0120] Specifically, the feature map input in the intermediate feature fusion stage is denoted as... ;in, and These represent the height and width of the feature map, respectively. This represents the number of channels in the feature map.
[0121] First, global average pooling is performed on the feature map, mapping the two-dimensional feature of each channel to a single scalar. The value of the c-th channel after global average pooling is denoted as...
[0122] ;
[0123] The output vector of global average pooling is denoted as: The vector z is then input into the first fully connected layer for dimensionality reduction. The weight matrix of the first fully connected layer is denoted as... ;in, This represents the dimension after dimensionality reduction. The resulting vector after dimensionality reduction is: ;in, This represents the bias term of the first fully connected layer. The dimensionality-reduced vector u is input into the second fully connected layer for dimensionality upscaling. The weight matrix of the second fully connected layer is denoted as... The result of the dimensionality upgrade is ,in, This is the bias term for the second fully connected layer.
[0124] Applying the Sigmoid activation function to the dimensionality-upgraded result v yields a channel-weighted vector: ;in, For the Sigmoid function, Let be the weighting coefficient for the c-th channel. The weighting coefficient vector is: The weighting coefficients are multiplied channel by channel of the original feature map to form the weighted feature map.
[0125] Weighted operations are expressed as ;in, The feature map is then weighted by channel attention. The above operations impart learnable weighting coefficients to the feature map along the channel dimension, with the weight of each channel determined by the statistics of the input features. The input, parameter calculation, and output of the channel attention mechanism all consist of deterministic linear transformations and element-wise operations, with clearly defined computational relationships across all matrix dimensions.
[0126] The weighted feature map output by the channel attention mechanism Based on this, the feature map is input into the advanced feature abstraction module. The advanced abstraction stage adopts a serial structure, first... Perform a 1×1 convolution to adjust the channel distribution so that its dimensions remain constant. Convolution operation is:
[0127] ;
[0128] in,
[0129] The above operations are used to unify the channel structure before performing dilated convolution, so that the input dimensions of subsequent layers are consistent.
[0130] The weight parameters of the first and second fully connected layers are obtained through gradient updates during network training. These two fully connected layers correspond to dimensionality reduction and dimensionality increase operations, respectively, with their output dimensions matching the input dimensions. This ensures that the number of channels in the weighted vector matches the number of channels in the feature map, allowing for direct channel-level multiplication. The channel-attention-weighted feature map then serves as the input for subsequent high-level feature abstraction stages. The weighting process does not change the spatial dimensions; it only adjusts the feature response values along the channel dimension.
[0131] Furthermore, the dilation rate of the S5's depth-separable convolution is 2, the kernel size is 3×3, and the number of output channels is 128.
[0132] The number of neurons in the three fully connected layers of the lightweight convolutional neural network are 512, 256, and the number of traffic sign categories, respectively. After the first two fully connected layers, a dropout layer with a dropout rate of 0.3 is set.
[0133] Specifically, after completing the intermediate feature fusion, the input feature map is denoted as... Where H and W are the spatial dimensions, and 128 is the number of channels. This feature map serves as the input to a dilated depthwise separable convolution. The dilated depthwise separable convolution consists of a depthwise dilated convolution and a pointwise convolution. The depthwise dilated convolution performs a convolution operation with a sampling interval independently on each channel, with an dilation rate denoted as r=2. For the input channel c, the output of the depthwise dilated convolution is:
[0134] ;
[0135] Where k=3 is the kernel size. This is the weight tensor of the depthwise convolution kernel. The coordinates (x, y) represent the convolution output position.
[0136] Then, pointwise convolution is performed to linearly combine the 128 channels of the depthwise convolution output. The formula for pointwise convolution is:
[0137] ;
[0138] in, This is the output channel index; the number of output channels is set to 128. The weights are 1×1 convolution kernel weights.
[0139] The output feature map of a depth-separable convolution with holes is denoted as: The dilation rate of deep-dilated convolution is 2, expanding the sampling range of the convolution kernel from the original 3×3 to a 5×5 receptive region, while keeping the number of kernel weights unchanged. This structure allows the network to acquire a wider range of contextual information with less computation. Pointwise convolution combines feature channels while maintaining the same spatial size, providing a fixed number of input features for subsequent global pooling.
[0140] After performing separable convolution with dilated depth, global average pooling is performed on the feature map. Global average pooling averages all spatial locations of each channel, and the output vector is:
[0141] ;
[0142] The output vector of global average pooling is denoted as: The above vector serves as the input to the first fully connected layer. The weight matrix of the first fully connected layer is denoted as: The bias vector is denoted as The output of the first fully connected layer is: ;in, A dropout layer is set after the fully connected layer with a dropout rate of 0.3. The dropout operation independently and randomly masks the output of some neurons during the training phase.
[0143] The input to the second fully connected layer is The weight matrix of the second fully connected layer is denoted as: ;
[0144] The bias vector is denoted as Its output is: ;in, This layer is followed by another drop-out layer with a drop-out rate of 0.3.
[0145] The third fully connected layer serves as the classification layer, and its weight matrix is as follows: Where C represents the number of traffic sign categories. The bias vector is denoted as... The output of the third fully connected layer is: ,in .
[0146] The vector z above represents the unnormalized score for each category and serves as the input to the Softmax classification function to generate the final predicted probability. The input dimension, output dimension, weight matrix shape, and dropout parameter of the three fully connected layers are all fixed values, which ensures that the output dimension remains constant and can be used for final classification when all input images are of the same size.
[0147] Furthermore, lightweight convolutional neural networks are trained as follows:
[0148] Use cross-entropy as the loss function;
[0149] The network parameters are updated using stochastic gradient descent with momentum, and the momentum coefficient is 0.9.
[0150] Training is terminated early if the validation set loss does not decrease for 10 consecutive training rounds.
[0151] Specifically, during the network training phase, the enhanced traffic sign images and their corresponding label data are used to form training samples. The label of each sample is represented using one-hot encoding. If the total number of traffic sign categories is C, then the label vector for category i is represented as:
[0152] ;
[0153] Where, when the sample belongs to category i, The corresponding value for the other category indices is 0.
[0154] The predicted probability vector output by the network forward computation is denoted as...
[0155] ;
[0156] in, Let be the predicted probability that the sample belongs to class i. The predicted probability is calculated using the Softmax function, the form of which has been given in the preceding sections.
[0157] The cross-entropy loss function is used during training, and the loss value for a single sample is defined as:
[0158] ;
[0159] When using batch training, the batch loss is the average loss of all samples in that batch. The gradient of the loss function L with respect to the network weights is calculated using the backpropagation algorithm.
[0160] The stochastic gradient descent method with momentum is used for weight update optimization. Let the current network weights be... The gradient of the loss function with respect to the weights is Let the momentum vector be v. The momentum update formula is: ,in, This is the momentum coefficient (value 0.9). Let be the learning rate. The weight update formula is: The aforementioned momentum mechanism, by combining the current gradient and the previous update amount in each iteration, provides smoothness to the gradient direction, making the trend of weight update sequence more stable. To avoid overfitting where the network continuously optimizes on the training set but degrades on the validation set during training, an early termination strategy is introduced. The validation set loss is calculated after each training epoch, denoted as... Where t is the current training round. When the condition is met for 10 consecutive rounds... Training is stopped when the validation set loss does not decrease. Early termination occurs when the validation loss no longer improves, at which point the current network parameters are fixed for the inference phase.
[0161] Training data is input into the network in fixed batch sizes. Each training round includes forward computation of all training samples, loss calculation, and parameter updates. After training, the network parameters remain unchanged, and forward computation is performed directly based on these fixed parameters during the inference phase.
[0162] Furthermore, during the reasoning phase, the following steps are performed:
[0163] The input traffic sign images are only normalized and resized to 224×224 pixels.
[0164] The preprocessed image is input into a lightweight convolutional neural network to complete all forward computation processes from S2 to S7, and the recognition result is output.
[0165] Specifically, the input image for the reasoning stage is denoted as: ;in, and 3 represents the original image size, and 3 represents the number of color channels. During the inference phase, the input image content remains unchanged; only numerical normalization and resizing are performed. Occlusion enhancement and geometric or photometric transformations as performed during the training phase are not executed.
[0166] First, a normalization operation is performed on the input image, mapping pixel values from integers to floating-point values. The normalization method is pixel-wise linear scaling, defined as:
[0167] ;
[0168] The pixel value range is uniformly mapped to the interval [0,1].
[0169] The normalized image is adjusted to a fixed size using bilinear interpolation. During resampling, interpolation is performed independently on all color channels, and the output is a tensor with the same size as the input during training. Then... The input to the lightweight convolutional neural network sequentially passes through five depthwise separable convolutional stages, a channel attention module, a dilated depthwise separable convolution, a global average pooling layer, and three fully connected layers. The network's forward computation flow remains consistent with the training phase, with the computation order corresponding to S2 to S7.
[0170] During convolution, depthwise separable convolution is performed as follows:
[0171]
[0172] The process iterates through the convolutional stages until the fifth-stage output feature map is generated. In the channel attention module stage, the intermediate feature map is subjected to global average pooling to form channel vectors. Separable convolution with dilation r=2, the corresponding convolution operation is:
[0173] ;
[0174] The number of output channels is fixed at 128. After performing separable convolution with dilatancy depth, global average pooling is performed on the feature map to obtain a vector of length 128. ;
[0175] The above vector is input into three fully connected layers, and the following linear transformation is performed sequentially:
[0176] ;
[0177] in, CCC represents the number of traffic sign categories.
[0178] The final output vector z is input into the Softmax function to obtain the predicted probability vector.
[0179] ;
[0180] The final output of the inference phase is p, with dimension C, representing the probability that the input image belongs to each traffic sign category. The network uses the category corresponding to the highest probability as the recognition result.
[0181] The input, preprocessing, convolution operation, weight matrix operation, and probability calculation in the inference stage are all deterministic mathematical operations, which do not involve randomness or variability in the training process, and can stably output traffic sign classification results.
[0182] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for recognizing locally occluded traffic signs based on attention-enhanced CNNs, characterized in that, Includes the following steps: S1. Perform dynamic occlusion data enhancement processing on the input traffic sign image to obtain the enhanced traffic sign image; S2. Input the enhanced traffic sign image into a lightweight convolutional neural network; S3. The lightweight convolutional neural network sequentially performs primary feature extraction, intermediate feature fusion, and high-level feature abstraction through a pyramid-style multi-level feature decoupling module. S4. In the intermediate feature fusion process, a channel attention mechanism is introduced to weight each channel of the feature map; S5. In the advanced feature abstraction process, dilated depth-separable convolution is used to process the feature map; S6. After global average pooling, the feature map after abstracting the high-level features is input into three fully connected layers in sequence. S7. The predicted probabilities of various traffic signs are output through the last fully connected layer to obtain the recognition results.
2. The method for recognizing partially occluded traffic signs based on attention-enhanced CNN according to claim 1, characterized in that, The dynamic occlusion data enhancement processing of S1 includes the following steps in sequence: Randomly overlay blocky occlusion regions or Gaussian noise occlusion regions onto the input traffic sign image; The superimposed occluded image is subjected to random rotation, random translation, random cropping, and random scaling operations in sequence. Perform random brightness adjustment and random contrast adjustment operations sequentially on the image after geometric transformation; The images that have undergone all enhancement operations are uniformly adjusted to a size of 224×224 pixels to obtain the enhanced traffic sign images.
3. The method for recognizing partially occluded traffic signs based on attention-enhanced CNN according to claim 1, characterized in that, The convolutional part of the lightweight convolutional neural network is constructed entirely using depthwise separable convolutions, and contains five convolutional stages. The number of output channels for the five convolutional stages are 32, 64, 64, 128, and 128, respectively.
4. The method for recognizing partially occluded traffic signs based on attention-enhanced CNN according to claim 3, characterized in that, The primary feature extraction in the pyramid-shaped multi-level feature decoupling module is completed by the first to third convolutional stages, and the first to third convolutional stages are all followed by a max pooling layer after depthwise separable convolution.
5. The method for recognizing partially occluded traffic signs based on attention-enhanced CNN according to claim 1, characterized in that, The specific steps of the channel attention mechanism in S4 are as follows: Global average pooling is performed on the input intermediate feature map to obtain a single value for each channel; The vector after global average pooling is input into the first fully connected layer for dimensionality reduction, and then input into the second fully connected layer for dimensionality increase. Apply the Sigmoid activation function to the output of the second fully connected layer to obtain the weighting coefficients for each channel; The weighted coefficients of each channel are multiplied by the corresponding channels of the original intermediate feature map to obtain the weighted intermediate feature map.
6. The method for recognizing partially occluded traffic signs based on attention-enhanced CNN according to claim 5, characterized in that, In the channel attention mechanism, the number of neurons in the first fully connected layer is 1 / 16 of the number of input channels, and the number of neurons in the second fully connected layer is restored to the original number of channels.
7. The method for recognizing partially occluded traffic signs based on attention-enhanced CNN according to claim 1, characterized in that, The S5 has a hole depth separable convolution with an expansion rate of 2, a kernel size of 3×3, and 128 output channels.
8. The method for recognizing partially occluded traffic signs based on attention-enhanced CNN according to claim 1, characterized in that, The lightweight convolutional neural network has 512, 256 neurons in its three fully connected layers, and the number of traffic sign categories are respectively. After the first two fully connected layers, a dropout layer with a dropout rate of 0.3 is set.
9. The method for recognizing partially occluded traffic signs based on attention-enhanced CNN according to claim 1, characterized in that, The lightweight convolutional neural network is trained in the following manner: Use cross-entropy as the loss function; The network parameters are updated using stochastic gradient descent with momentum, and the momentum coefficient is 0.
9. Training is terminated early if the validation set loss does not decrease for 10 consecutive training rounds.
10. The method for recognizing partially occluded traffic signs based on attention-enhanced CNN according to claim 1, characterized in that, During the reasoning phase, the following steps are performed in order: The input traffic sign images are only normalized and resized to 224×224 pixels. The preprocessed image is input into a lightweight convolutional neural network to complete all forward computation processes from S2 to S7, and the recognition result is output.