A traffic image defogging method and intelligent traffic image processing system based on deep learning

Abstract

The present invention discloses a traffic image defogging method and a smart traffic image processing system based on deep learning. The method comprises acquiring real-time traffic image data; performing color contrast, calculating the average intensity, variance and average gradient of the image, judging whether the image is foggy, and outputting a foggy image; inputting the foggy image into a deep learning model to obtain features of the image at multiple scales; receiving features at multiple scales and fusing them to restore the image. The traffic image defogging method and a smart traffic image processing system based on deep learning of the present invention fuse the features obtained by the deep learning model, effectively improving the clarity and contrast of the image obtained by the camera in foggy scenes.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a traffic image defogging method based on deep learning, and also to an intelligent traffic image processing system adopting the method. Background Art

[0002] Intelligent Traffic Systems (ITS) are systems that use advanced information and communication technologies, as well as sensors, computers, communications and control technologies, to comprehensively and accurately monitor, manage and control traffic systems.

[0003] Computer Vision (CV) is an important branch of artificial intelligence, dedicated to enabling computer systems to understand, interpret, and process image and video data. Its technical background involves many aspects, including image processing, pattern recognition, machine learning, and deep learning.

[0004] In CV, ITS is crucial to improving intelligent monitoring, intelligent navigation, autonomous driving, traffic efficiency, etc. However, due to the scattering and absorption of ambient light by turbid media in the atmosphere, the quality of captured videos or images will inevitably decay under hazy imaging conditions. This phenomenon will seriously reduce the contrast and color fidelity of the entire image. At the same time, the foreground and background will become blurred and deformed, resulting in the hiding of key target information. Summary of the invention

[0005] The purpose of the present invention is to overcome the above-mentioned defects in the prior art and provide a traffic image defogging method and an intelligent traffic image processing system based on deep learning, which fuse the features obtained by the deep learning model to effectively improve the image clarity and contrast obtained by the camera in foggy scenes.

[0006] To achieve the above object, the present invention provides a traffic image defogging method based on deep learning, comprising the following steps:

[0007] S1. Obtain real-time traffic image data;

[0008] S2. Perform color contrast, calculate the average intensity, variance and average gradient of the image, determine whether the image is foggy, and output the foggy image;

[0009] S3. Input the foggy image into the deep learning model to obtain features of the image at multiple scales;

[0010] S4. Receive features at multiple scales and fuse them to restore the image;

[0011] S5. Output the restored image.

[0012] In the above technical solution, S2 includes:

[0013] S21. Perform color comparison: according to the three primary colors RGB, extract a series of values ​​of the three channels of the obtained image for data enhancement, and compare the obtained image with the original image to determine whether it is foggy;

[0014] S22. Calculate the average intensity of the image: calculate the average grayscale value of the pixels of the three channels of the image;

[0015] S23. Calculate the variance of the image: calculate the gray value of each pixel minus the sum of squares of the average gray value of the image divided by the total number of pixels;

[0016] S24. Calculate the average gradient of the image: calculate the average intensity of the change in pixel values ​​in the image;

[0017] S25. Determine whether the image is foggy by combining color contrast, average intensity, variance and average gradient of the image, and output a foggy image.

[0018] In the above technical solution, the deep learning model includes a global-local attention mechanism and a multi-scale convolutional neural network. The global-local attention mechanism obtains the attention features of the image, and then inputs the image and the attention features into the multi-scale convolutional neural network.

[0019] In the above technical solution, assuming that the input features are X and Z, and the attention feature of the global-local attention mechanism is Y, it can be expressed as:

[0020] Y=α·X(1-α)·Z

[0021] Among them, X is global information; Z is local information; α is the weight learned by the deep learning model.

[0022] In the above technical solution, it is assumed that the input feature map is X, and the sizes of the convolution kernels are k1, k2, …, k n , the corresponding weights are W1, W2, …, W n , the output feature map of the multi-scale convolutional neural network is Y, which can be expressed as:

[0023]

[0024] in, Indicates that the input feature map X is of size k i Convolution operation; W i Represents the same as size k i The weights associated with the convolution kernel; It means summing the convolution kernels of all sizes.

[0025] In the above technical solution, in S4, assuming that the two received features are A and B, the fusion of A and B can be expressed as:

[0026] Attention Fusion(A,B)=α·A+β·B

[0027] α,β=Softmax(FC([A,B]))

[0028] Among them, FC represents the fully connected layer, and α and β are the attention weights obtained by normalization of the Softmax function.

[0029] In the above technical solution, S3 also includes preprocessing the foggy image and then inputting the preprocessed image into the deep learning model; S4 also includes post-processing the image after fusion of multiple scale features.

[0030] The present invention also provides an intelligent traffic image processing system based on a traffic image defogging method based on deep learning, comprising:

[0031] An image acquisition module, used to obtain real-time traffic image data as input data;

[0032] An image detection module is used to determine whether the real-time traffic image data is foggy and output a foggy image;

[0033] Image processing module, including deep learning model, used to obtain and output features of multiple scales of images;

[0034] Image restoration module, used to fuse features of multiple scales to obtain restored images;

[0035] An image output module, used for outputting the restored image;

[0036] The above modules complete the logical link in a progressive relationship.

[0037] In the above technical solution, the image processing module also includes a preprocessing unit, which is used to perform preliminary processing on the received foggy image and then output it to the deep learning model.

[0038] In the above technical solution, the image restoration module also includes a post-processing unit for post-processing the restored image and then outputting it to the image output module.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. S2 in this technical solution adds the judgment of whether the image is foggy before processing the image by comparing the color and calculating the average intensity, variance and average gradient of the image, thereby improving the effectiveness and accuracy of defogging.

[0041] 2. In this technical solution, S3 obtains features of multiple scales of images and S4 fuses features of multiple scales to more comprehensively and accurately restore foggy traffic images, improve the clarity and visibility of traffic monitoring images, and have a positive impact on traffic control, autonomous driving, traffic safety and other aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is a flow chart of a traffic image defogging method based on deep learning provided by the present invention;

[0044] Figure 2 It is a structural block diagram of an intelligent traffic image processing system for a traffic image defogging method based on deep learning provided by the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] refer to Figure 1 As shown, the present invention provides a traffic image defogging method based on deep learning, comprising the following steps:

[0047] S1. Obtain real-time traffic image data;

[0048] S2. Perform color contrast, calculate the average intensity, variance and average gradient of the image, determine whether the image is foggy, and output the foggy image;

[0049] S3. Preprocess the foggy image and then input the preprocessed image into the deep learning model to obtain features of the image at multiple scales; the preprocessing includes operations such as denoising, removing artifacts, and scaling;

[0050] S4. receiving features at multiple scales and fusing them to restore an image, and then post-processing the image after the fusion of features at multiple scales; the post-processing includes color correction, contrast adjustment, etc. on the image to further improve the image quality;

[0051] S5. Output the post-processed image.

[0052] In this embodiment, the specific steps of S2 are:

[0053] S21. Perform color contrast: according to the three primary colors RGB, extract a series of values ​​of the three channels of the obtained image for data enhancement, and then compare the obtained image with the original image to determine whether it is foggy; wherein, color contrast refers to comparing the degree of difference between two or more colors in an image or design.

[0054] S22. Calculate the average intensity of the image: calculate the average grayscale value of the pixels in the three channels of the image; the average grayscale value reflects the brightness of the image. The larger the average grayscale value, the brighter the image, and vice versa.

[0055] S23. Calculate the variance of the image: calculate the gray value of each pixel minus the sum of the squares of the average gray value of the image and divide it by the total number of pixels; the variance reflects the degree of dispersion between the gray value of the image pixel and the average gray value. The larger the variance, the more colorful or noisy the image may be.

[0056] S24. Calculate the average gradient of an image: Calculate the average intensity of the change in pixel values ​​in an image; usually used to measure the edge information of an image. The gradient represents the rate of change of the pixel values ​​in an image, so the average gradient can reflect the density of the texture and edges in the image. The larger the average gradient, the more edge and texture information there is in the image, while a smaller average gradient may indicate that the image is relatively smooth or lacks obvious texture and edges.

[0057] S25. Determine whether the image is foggy by combining color contrast, average intensity, variance and average gradient of the image, and output a foggy image.

[0058] In this embodiment, the deep learning model in S3 includes a global-local attention mechanism, so that the method of the present invention can dynamically adjust the degree of attention to different parts of the image, thereby restoring the image more accurately; wherein the global attention mechanism is used to capture the contextual information of the entire image, such as the overall traffic scene and road environment, and the local attention mechanism is used to focus on local important areas, such as key vehicles, pedestrians or traffic signs. This combination helps to improve the accuracy of image restoration. Specifically, assuming that the input feature is X and the output feature of the global-local attention mechanism is Y, it can be expressed as:

[0059] Y=α·X(1-α)·Z

[0060] Among them, X is the global information, which is usually obtained by global pooling of the input; Z is the local information, which is obtained by local convolution or other local perception operations on the input; α is a learned weight used to balance the importance of global information and local information.

[0061] In this embodiment, the deep learning model also includes a multi-scale convolutional neural network. Using convolution kernels of different scales at different levels can help the network effectively capture traffic elements such as vehicles and pedestrians in foggy scenes and improve the accuracy of image restoration. Specifically, assuming that the input feature map is X, the sizes of the convolution kernels are k1, k2, ..., k n , the corresponding weights are W1,W2,…,W n , the output feature map of the multi-scale convolution is Y, then the calculation of the multi-scale convolution can be expressed as:

[0062]

[0063] in, Indicates that the input feature map X is of size k i Convolution operation; W i Represents the same as size k i The weights associated with the convolution kernel; It means summing the convolution kernels of all sizes.

[0064] In this embodiment, S4 uses multi-feature fusion technology to merge multiple features from different sources, which can improve the performance and generalization ability of the deep learning model to restore clear traffic images more comprehensively and accurately; multi-feature fusion can weightedly fuse features of different scales according to the calculated weights to ensure that the model considers information of different scales more comprehensively in the final feature representation. Specifically, assuming that the two received features are A and B, the fusion of A and B can be expressed as:

[0065] Attention Fusion(A,B)=α·A+β·B

[0066] α,β=Softmax(FC([A,B]))

[0067] Among them, FC represents the fully connected layer, and α and β are the attention weights obtained by normalization of the Softmax function.

[0068] refer to Figure 2 As shown, the present invention also provides an intelligent traffic image processing system based on a traffic image defogging method based on deep learning, comprising:

[0069] The image acquisition module is used to obtain real-time traffic image data as input data; the image acquisition module can be selected as a traffic camera or a vehicle-mounted camera, etc.

[0070] An image detection module is used to determine whether the real-time traffic image data is foggy and output a foggy image;

[0071] An image processing module, including a preprocessing unit and a deep learning model. The preprocessing unit performs preliminary processing on the received foggy image and then outputs it to the deep learning model. The deep learning model calculates and outputs features of multiple scales of the image.

[0072] An image restoration module is used to fuse features of multiple scales to obtain a restored image, and also includes a post-processing unit to post-process the restored image to further improve the image quality, and then output it to the image output module;

[0073] The image output module is used to output the restored image; the output image can be used for traffic control, autonomous driving, violation detection, etc.

[0074] The above modules complete the logical link in a progressive relationship.

[0075] In summary, the traffic image defogging method based on deep learning of the present invention effectively improves the clarity and contrast of images acquired by the camera in foggy scenes by fusing the features obtained by the deep learning model.

[0076] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A traffic image defogging method based on deep learning, characterized in that: The steps include: S1. Obtain real-time traffic image data; S2. Perform color contrast, calculate the average intensity, variance and average gradient of the image, determine whether the image is foggy, and output the foggy image; S21. Perform color comparison: according to the three primary colors RGB, extract a series of values ​​of the three channels of the obtained image for data enhancement, and compare the obtained image with the original image to determine whether it is foggy; S22. Calculate the average intensity of the image: calculate the average grayscale value of the pixels of the three channels of the image; S23. Calculate the variance of the image: calculate the gray value of each pixel minus the sum of squares of the average gray value of the image divided by the total number of pixels; S24. Calculate the average gradient of the image: calculate the average intensity of the change in pixel values ​​in the image; S25. Determine whether the image is foggy by combining color contrast, average intensity, variance and average gradient of the image, and output a foggy image; S3. Input the foggy image into the deep learning model to obtain features of the image at multiple scales; The deep learning model includes a global-local attention mechanism and a multi-scale convolutional neural network; the global-local attention mechanism obtains the attention features of the image, and then inputs the image and the attention features into the multi-scale convolutional neural network; The input features are X and Z, and the attention feature of the global-local attention mechanism is Y, which can be expressed as: Among them, X is global information; Z is local information; are the weights learned by the deep learning model; The input feature is Y, and the sizes of the convolution kernels are , the corresponding weight is , the output feature of the multi-scale convolutional neural network is T, which can be expressed as: in, It means that the input feature Y is sized Convolution operation; Representation and size The weights associated with the convolution kernel; represents the summation of convolution kernels of all sizes; S3 also includes preprocessing the foggy image, and then inputting the preprocessed image into the deep learning model; S4. Receive features at multiple scales and fuse them to restore the image; The S4 also includes post-processing the image after the fusion of multiple scale features; S5. Output the restored image.

Images