An adaptive image enhancement method for traffic sign detection in complex weather

Abstract

The present invention discloses an adaptive image enhancement method for complex weather traffic sign detection. This method designs a Micro CNN model based on a soft attention mechanism, and obtains the parameters of the image processing filter by learning the characteristics of the noise image. The model output parameters are compared with the threshold value set in advance to determine the image processing filter used for this noise image. The filters are sorted in order from large to small according to the parameters, and the noise image is filtered in sequence. This method determines the filter parameters through a neural network model, and dynamically selects the filter combination through an adaptive joint filtering strategy. It utilizes the permutations and combinations of multiple low-computation image filters to improve the flexibility of image optimization in complex environments. Applying the processed images for traffic sign detection can greatly improve the recognition accuracy of the image recognition algorithm for traffic signs affected by complex weather.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, relates to traffic sign image processing, and particularly relates to an adaptive image enhancement method for traffic sign detection in complex weather. Background Art

[0002] Traffic signs are fundamental to traffic management and road safety. Accurately detecting and identifying traffic signs on the road is crucial for autonomous driving. However, natural weather conditions are complex and changeable. The visibility of traffic signs on the road can be significantly affected in rainy, snowy, foggy, or under strong sunlight, posing a challenge to traffic sign detection and recognition. To address this, the captured image must be enhanced before recognition. Existing image enhancement methods typically address only a single issue, for example, enhancing images only in rainy or low-light conditions. However, in real-world driving scenarios, due to the complexity and diversity of weather conditions, drivers must contend with adverse conditions caused by multiple factors, such as rain or fog at night. Furthermore, traditional neural network-based image enhancement methods employ complex network structures to ensure detection accuracy, resulting in lengthy computational times that cannot meet the real-time requirements for traffic sign detection in autonomous driving, resulting in low practical value. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this paper proposes an adaptive image enhancement method for traffic sign detection in complex weather conditions. This method optimizes images based on weather changes while simultaneously controlling computational and memory costs to ensure the real-time performance required by subsequent tasks in practical application scenarios.

[0004] An adaptive image enhancement method for traffic sign detection in complex weather conditions, comprising the following steps:

[0005] Step 1: Collect unobstructed traffic sign images under well-lit conditions as training labels. Add noise to the original images to simulate the effects of complex weather conditions on traffic sign images, and use them as training samples.

[0006] Step 2: Construct multiple different image processing filters, initialize the parameters at the same time, and set the filter parameter threshold T.

[0007] Preferably, the image processing filters include a white balance filter, a gamma filter, a defogging filter, a tone filter, a histogram equalization filter, and a Laplace filter.

[0008] Step 3: Construct a Micro CNN network, input the training sample obtained in Step 1, and output the parameters of the image processing filter described in Step 2. The Micro CNN network obtains the feature map corresponding to the training sample through the feature extraction module. The soft attention module assigns weights to features of different dimensions in the feature map. After two cascaded linear layers, the weighted feature map is output.

[0009] Preferably, the feature extraction module includes 5 consecutive convolution and batch normalization layers, outputting a feature map m. The soft attention module assigns weights to features of different dimensions in the feature map m:

[0010]

[0011]

[0012] z n =avp(m n )

[0013] Among them, m n Represents the feature map under the nth channel, α n Indicates m n The corresponding soft attention weights, N represents the number of channels of feature map m. att() represents the soft attention module, and avp() represents the global average pooling, which can effectively reduce the dimension of m.

[0014] Step 4: Normalize the image processing filter parameters obtained in Step 3 so that their values fall within the interval [0, 1]. Then compare the parameters of each image processing filter with a threshold value T, and retain all image processing filters whose parameters are greater than the threshold value T. Calculate the parameter mean of each image processing filter in turn, and sort the retained image processing filters in descending order of parameter mean value for use.

[0015] Step 5: Denormalize the parameters of the image processing filter retained after step 4, and filter the training samples according to the order of use in step 4.

[0016] Step 6: Calculate the mean absolute error L between the filtered image and the training label MAE , as the loss function, update the Micro CNN network parameters.

[0017] As an advantage, the mean absolute error L MAE The calculation formula is:

[0018]

[0019] Where I represents the number of pixels in the input image, y irepresents the i-th pixel of the filtered image, represents the i-th pixel of the training label.

[0020] Step 7: Input the traffic sign image to be processed into the trained Micro CNN network, select the corresponding image processing filter to process the traffic sign image, eliminate the influence of complex weather conditions, and use it for the downstream traffic sign recognition task.

[0021] The present invention has the following beneficial effects:

[0022] This method uses a convolutional neural network to train image filter parameters. To further control computational costs, a phase-adaptive joint filtering strategy is designed. This strategy sets a filter parameter threshold and compares the weight parameters trained by the network with the filter parameter threshold to select the image processing filter to use. The weights are then used to prioritize the filters, maximizing the image optimization effect. Experimental data demonstrates that this method significantly improves the accuracy of image recognition algorithms for traffic signs affected by complex weather conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is the overall architecture diagram of the adaptive image enhancement method;

[0024] Figure 2 Schematic diagram of the qualitative recognition effect under complex weather conditions in the embodiment. DETAILED DESCRIPTION

[0025] The present invention will be further explained below with reference to the accompanying drawings;

[0026] This embodiment constructs six image processing filters to perform adaptive filtering on images affected by complex weather conditions. These filters include a white balance filter, a gamma filter, a dehazing filter, a pitch filter, a histogram equalization filter, and a Laplace filter. These filters are computationally simple and require few parameters, but through different combinations, they can achieve cost-effective image optimization. The working mechanism is as follows:

[0027] (1) White balance filter: I WB (x,y)=(W b B(x,y),W g G(x,y),W r R(x,y)

[0028] Among them, (x, y) represents the pixel position in the image, I(x, y) represents the original image, and I WB (x,y) represents the image after the white balance filter, B(x,y), G(x,y) and R(x,y) represent the three color channels in the original image I(x,y).b 、W g 、W r Represents the three parameters required for the white balance filter,

[0029] (2) Gamma filter: I ga (x,y)=I(x,y) γ

[0030] I ga (x, y) represents the image processed by the gamma filter, and γ is the parameter of the gamma filter.

[0031] (3) Dehazing filter: I(x,y)=J(x,y)t(x,y)+A(1-t(x,y))

[0032] I(x,y) represents the original image, J(x,y) represents the target image, t(x,y) is the transmittance, and A is the global atmospheric light component.

[0033] (4) Tone filter:

[0034] The pitch filter is designed as a monotonic and piecewise linear function, where I tone (x,y) represents the image after being processed by the pitch filter, and the clamp() function is used to limit the input to the range of (0,1). The 8 filter parameters t output by Micro-CNN k , k = 0, 1, ... 7, calculate the points on the pitch curve.

[0035] (5) Histogram equalization filter: I HE (x,y)=α·HE(I(x,y))+(1-α)·I(x,y)

[0036] The histogram equalization filter makes the image grayscale value of the image approximately uniformly distributed through nonlinear transformation, thereby enhancing the contrast of the image. HE (x, y) represents the image after being processed by the histogram equalization filter, HE() represents histogram equalization, and α represents the parameter of the histogram equalization filter.

[0037] (6) Laplace filter:

[0038] The Laplacian filter can highlight the details of the object, where I L (x,y) represents the image after Laplace filter processing, Represents the Laplace operator, λ represents the positive scale parameter, that is, the parameter of the Laplace filter, which can be used to adjust the sharpness of the filter.

[0039] The parameters of the above 6 image processing filters are output through the Micro CNN network. The Micro CNN network includes a feature extraction module, a soft attention module and two cascaded linear layers. The soft attention module can help the model calculate the attention weight of each pixel, thereby determining the area that needs attention and improving the model's analysis of important parts in the noisy image. Figure 1 As shown, the parameters of the filter are determined based on the network output, and the filter to be used for the input image is used to implement adaptive filtering. The pseudo code corresponding to this strategy is as follows:

[0040]

[0041] Traffic sign images from the public datasets CURE-TSD and DFG are used to verify the image processing effect of this method. The CURE-TSD dataset contains 14 categories, and the DFG dataset contains 200 categories. The categories in the DFG dataset cover the categories in the CURE-TSD dataset. First, the images in the DFG dataset are divided into training set, test set and validation set according to the ratio of 7:2:1. The above model is trained and tested. The hardware environment configuration is: CPU (Intel(R) Xeon(R) Gold 6242R CPU@3.10GHz), memory (256GB), graphics card (8×GeForce RTX 3090) and video memory (8×24GB). Distributed training is adopted. 8 graphics cards are equipped with 32Batch-size data respectively. The hyperparameters of the initial training neural network are set as: Batch-size (8×32) and training round Epochs (200). Then, after adaptive enhancement processing, the images in the CURE-TSD dataset are input into the YOLOv5 target detection method to identify the traffic sign categories in the processed images. The recognition results are shown in Figure 2. Figure 2 As shown in the figure, for high-exposure, dim, and rainy scenes, this method can accurately locate the position of traffic signs in the image and accurately identify them.

[0042] At the same time, the images in the CURE-TSD dataset processed by this method were input into the target detection method commonly used in the prior art and compared with the test results of this method to verify the enhancement and generalization capabilities of this method for images affected by complex weather conditions. The test results are shown in Table 1:

[0043]

[0044] Table 1

[0045] The test results in Table 1 show that in dim and foggy scenes, that is, in conditions affected by complex weather conditions, image enhancement using this method can significantly improve detection accuracy. Furthermore, image enhancement using this method also helps improve detection accuracy in clear weather conditions, that is, when not affected by complex weather conditions. This demonstrates that this method can improve the accuracy of traffic sign detection in complex weather conditions and exhibits strong generalization capabilities.

[0046] The above specific embodiments are used to illustrate the present invention rather than to limit the present invention. Any modifications and changes made to the present invention within the spirit of the present invention and the protection scope of the claims shall fall within the protection scope of the present invention.

Claims

1. An adaptive image enhancement method for traffic sign detection in complex weather conditions. This method collects unobstructed traffic sign images under well-lit conditions as training labels. It also adds noise to the original images to simulate the effects of complex weather conditions on the traffic sign images, which serve as training samples. The method is characterized by: The specific steps include: Step 1: Set the filter parameter threshold T; Step 2: Construct multiple different image processing filters and initialize their parameters at the same time; Step 3: Construct a Micro CNN network, input the training sample, and output the parameters of the image processing filter described in step 2. The Micro CNN network obtains the feature map corresponding to the training sample through the feature extraction module, and then assigns weights to the features of different dimensions in the feature map through the soft attention module. Finally, after two cascaded linear layers, the weighted feature map is output. Step 4: Normalize the image processing filter parameters obtained in step 3; then compare the parameters of each image processing filter with a threshold value T, and retain all image processing filters whose parameters are greater than the threshold value T; calculate the parameter means of the retained image processing filters in turn, and arrange the image processing filters in descending order of the parameter means as the order of use; Step 5: Denormalize the parameters of the image processing filter retained in step 4, and filter the training samples in the order of use in step 4; Step 6: Calculate the mean absolute error L between the filtered image and the training label MAE , as the loss function, update the Micro CNN network parameters; Step 7: Input the traffic sign image to be processed into the trained Micro CNN network, select the corresponding image processing filter to process the traffic sign image, eliminate the influence of complex weather conditions, and use it for the downstream traffic sign recognition task.

2. The adaptive image enhancement method for complex weather traffic sign detection according to claim 1, characterized in that: The image processing filters include a white balance filter, a gamma filter, a defogging filter, a tone filter, a histogram equalization filter, and a Laplace filter.

3. The adaptive image enhancement method for complex weather traffic sign detection according to claim 2, characterized in that: The working mechanism of the image processing filter is: (1)For the purpose:I WB (x,y)=(W b B(x,y), W g G(x,y), W r R(x,y)) Among them, (x, y) represents the pixel position in the image, I(x, y) represents the original image, and I WB (x, y) represents the image after the white balance filter, B(x, y), G(x, y) and R(x, y) represent the three color channels in the original image I(x, y); W b 、W g 、W r Represents the three parameters required for the white balance filter, (2) Gamma filter: I ga (x,y)=I(x,y) γ I ga (x, y) represents the image after being processed by the gamma filter, and γ is the parameter of the gamma filter; (3) Dehazing filter: I(x,y)=J(x,y)t(x,y)+A(1-t(x,y)) I(x,y) represents the original image, J(x,y) represents the target image, t(x,y) is the transmittance, and A is the global atmospheric light component; (4) Tone filter: The pitch filter is designed as a monotonic and piecewise linear function, where I tone (x, y) represents the image after being processed by the pitch filter. The clamp() function is used to limit the input to the range of (0, 1). The 8 filter parameters t output by Micro-CNN are: k , k = 0, 1, ... 7, calculate the points on the pitch curve; (5) Histogram equalization filter: I HE (x,y)=α·HE(I(x,y))+(1-α)·I(x,y) The histogram equalization filter makes the image grayscale value of the image approximately uniformly distributed through nonlinear transformation, thereby enhancing the contrast of the image; wherein I HE (x, y) represents the image after being processed by the histogram equalization filter, HE() represents histogram equalization, and α represents the parameter of the histogram equalization filter; (6) Laplace filter: The Laplacian filter can highlight the details of the object, where I L (x,y) represents the image after Laplace filter processing, Represents the Laplace operator, λ represents the positive scale parameter, that is, the parameter of the Laplace filter, which can be used to adjust the sharpness of the filter.

4. The adaptive image enhancement method for complex weather traffic sign detection according to claim 1, characterized in that: The feature extraction module includes 5 consecutive convolution and BN layers.

5. The adaptive image enhancement method for complex weather traffic sign detection according to claim 1, characterized in that: The soft attention module assigns weights to features of different dimensions in the feature map m output by the feature extraction module: z n =avp(m n ) Among them, m n Represents the feature map under the nth channel, α n Indicates m n The corresponding soft attention weight, N represents the number of channels of the feature map m; att() represents the soft attention module, and avp() represents the global average pooling, which can effectively reduce the dimension of m.

6. The adaptive image enhancement method for complex weather traffic sign detection according to claim 1, characterized in that: The mean absolute error L MAE The calculation formula is: Where I represents the number of pixels in the input image, y i represents the i-th pixel of the filtered image, represents the i-th pixel of the training label.

7. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 6.

Images