High-speed scene visibility evaluation method based on fast defogging neural network algorithm
Through a high-speed scene visibility evaluation method based on the fast defog neural network algorithm, the problem of insufficient accuracy and robustness of visibility evaluation in the prior art is solved, and high accuracy and real-time visibility evaluation is achieved, which is suitable for a variety of complex scenarios.
Patent Information
- Application Number
- CN202411863221.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-17
- Publication Date
- 2025-04-11
AI Technical Summary
Existing visibility assessment methods have shortcomings in terms of accuracy and robustness, especially in complex weather conditions, which are difficult for traditional methods to effectively evaluate the visibility of images.
A high-speed scene visibility evaluation method based on a fast defog removal neural network algorithm is used to remove the fog treatment on the pictures, compare the image feature changes before and after defog removal, and calculate the visibility value. This method uses shallow neural network for defogging, and combines technologies such as image scaling and Sobel operators to improve the accuracy and real-time evaluation.
It achieves more accurate visibility assessment, has good real-time and adaptability, and is suitable for a variety of application scenarios. The evaluation consistency of visibility coefficient can reach 99%, and the processing speed is significantly improved.
Smart Images
Figure CN119810051A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an evaluation method, in particular to a high-speed scene visibility evaluation method based on a fast defogging neural network algorithm, and belongs to the technical field of digital image processing. Background Art
[0002] With the development of modern society, digital images are increasingly used in various fields, especially in environmental monitoring, traffic safety and weather forecasting. Visibility, as an important indicator to measure image clarity, has a direct impact on image quality and application effects. Under complex weather conditions, such as fog, smoke, haze, etc., reduced visibility will seriously affect the quality and application effects of images. Therefore, the development of an effective image visibility assessment method is of great significance to improving the accuracy and reliability of image processing technology. ), severe weather such as rain, snow, sand and dust are the "first killer" and "mobile killer" of road traffic, which poses a great threat to traffic safety.
[0003] There are two main categories of traditional visibility assessment methods: one is to rely on direct measurements from meteorological instruments. Although these methods are accurate, they have the following defects: (1) poor real-time performance, (2) high equipment deployment costs, and (3) single application scenarios; the other is image-based visibility assessment methods, which estimate visibility by analyzing features in images, such as contrast, brightness, color saturation, etc. However, the existing image-based visibility assessment methods still need to be improved in terms of accuracy and robustness.
[0004] In recent years, deep learning technology has made significant progress in the field of image processing, especially in image recognition, classification and enhancement. The image defogging algorithm based on deep learning can build a deep learning network that focuses on the details of foggy images, learns the intrinsic relationship between foggy images and clear weather images, and achieves the defogging effect. The advantage of the fast defogging neural network algorithm we designed is that it can process images with different fog concentrations and complex scenes, and has low computational complexity. Therefore, through the effectiveness of the defogging algorithm, a measurement mechanism can be established to quantify visibility. Summary of the invention
[0005] The present invention aims at the technical problems existing in the prior art and provides a high-speed scene visibility assessment method based on a fast defogging neural network algorithm. The method calculates the visibility value by defogging the image and comparing the changes in image features before and after defogging. This method can not only provide a more accurate visibility assessment, but also has good real-time performance and adaptability, and is suitable for a variety of application scenarios.
[0006] In order to achieve the above object, the technical solution of the present invention is as follows: a high-speed scene visibility assessment method based on a fast defogging neural network algorithm, the method is as follows:
[0007] Step 1: Image input: The image to be evaluated is acquired through a high-speed camera.
[0008] Step 2: Image preprocessing: Perform necessary preprocessing on the input image, including cropping, scaling, etc.
[0009] Step 3: Dehazing:
[0010] 3-1 Collect high-speed monitoring images, collect foggy and non-fog images in pairs according to the point positions, and convert the non-fog images into grayscale images as reconstruction labels, and use the foggy images as input data to form a defogging reconstruction data set.
[0011] 3-2 Build a fast image dehazing network,
[0012] For an image of input size H×W×3, the processing order is as follows,
[0013] Use a convolution layer with a kernel size of 5x5, an output dimension of 16, and a stride of 2, a Silu activation layer, a convolution layer with a kernel size of 3x3, an output dimension of 16, and a stride of 2, and a Silu activation layer to achieve one downsampling and feature extraction (network layers 1 to 4) to obtain a feature map of H2×W2×16;
[0014] Use a convolution layer with a kernel size of 3x3, an output dimension of 32, and a stride of 2, a Silu activation layer, a convolution layer with a kernel size of 3x3, an output dimension of 32, and a stride of 1, and a Silu activation layer to achieve one-time downsampling and feature extraction (network 4 to 8 layers) to obtain a feature map of H4×W4×32;
[0015] A convolutional layer with a kernel size of 3x3, an output dimension of 64, and a stride of 2 is used, and a Silu activation layer is stacked to achieve the third downsampling and feature extraction (network layers 9 to 10), and a feature map of H8×W8×64 is obtained;
[0016] Use a transposed convolution layer with a convolution kernel size of 3x3, an output dimension of 16, and a stride of 2, a Silu activation layer, a Concat layer, a convolution layer with a convolution kernel size of 3x3, an output dimension of 32, and a stride of 1, and a Silu activation layer to achieve one upsampling and feature fusion, and obtain a feature map of H4×W4×32;
[0017] Use a transposed convolution layer with a convolution kernel size of 3x3, an output dimension of 16, and a stride of 2, a Silu activation layer, a Concat layer, a convolution layer with a convolution kernel size of 3x3, an output dimension of 16, and a stride of 1, and a Silu activation layer to achieve one upsampling and feature fusion, and obtain a feature map of H2×W2×16;
[0018] A transposed convolution layer with a convolution kernel size of 3x3, an output dimension of 16, and a stride of 2, a convolution layer with a convolution kernel size of 3x3, an output dimension of 3, and a stride of 1, and a Sigmoid activation layer are stacked to achieve one upsampling and obtain a H×W×3 result image.
[0019] 3-3 Use the data set to train and test the fast dehazing network to obtain the optimal weights.
[0020] 3-4 Use the trained fast defogging network weights to defog the input image to obtain the reconstructed image. Step 4: Image texture extraction.
[0021] 4-1 Use bilinear interpolation method to scale the image, reduce the image to 1 / 16 of the original image, and then restore it to the original size to obtain a smooth image, instead of Gaussian smoothing.
[0022] 4-2 Use the Sobel operator to perform convolution on the image to obtain the horizontal and vertical gradient matrices of the image, and then obtain the gradient matrix and angle matrix of the image.
[0023] Horizontal Sobel operator:
[0024] Vertical Sobel operator:
[0025] Horizontal gradient of the image:
[0026] Image vertical gradient:
[0027] The image gradient is:
[0028] 4-3 Use the non-maximum suppression method to process the image angle matrix and compare the gradient strength of the current pixel with the two pixels along the positive and negative gradient directions. If the gradient strength of the current pixel is the largest compared with the other two pixels, the pixel point is retained as an edge point, otherwise the pixel point will be suppressed.
[0029] 4-4 Use double thresholds to remove noise points and obtain the image texture binary matrix.
[0030] Step 5: Calculation of visibility estimation:
[0031] 5-1 Extract the problem from the input image and obtain the original image texture binary matrix M s ,
[0032] 5-2 Extract the problem from the dehazed image and obtain the original texture binary matrix M t ,
[0033] 5-3 A quick visibility evaluation formula is designed to calculate the visibility evaluation coefficient p, which is between 0 and 1. The smaller the value, the lower the visibility, as shown below
[0034]
[0035] Step 6: Result output: Output the calculated visibility estimate;
[0036] This technical solution is highly accurate, highly automated, and highly practical. Due to the use of shallow neural networks, the defogging image reconstruction performance is greatly improved, and the speed can reach 102simples / s on the Atlas 300 server, which is much higher than the existing defogging network. In addition, the consistency of the visibility coefficient evaluation at fixed points can reach 99% accuracy.
[0037] Compared with the prior art, the present invention has the following advantages: 1) It uses a shallow neural network to reduce the amount of computation, speed up the processing speed, and is suitable for edge deployment; 2) It uses image scaling instead of Gaussian smoothing to further reduce the amount of computation, and is independent of the parameter configuration of the Gaussian operator, so it has a wider range of applications; 3) The technical solution has high accuracy, and can more accurately restore the real details of the image by defogging the image through a defogging deep learning network, thereby providing a reliable basis for visibility assessment; 4) The solution has a high degree of automation. The method reduces manual intervention and improves the efficiency and consistency of the assessment through an automated image processing process; 5) It has strong practicality and can be used not only for meteorological monitoring and environmental assessment, but also in many fields such as traffic safety, aviation navigation, and ocean shipping. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 It is the overall flow chart of the present invention,
[0039] Figure 2 A schematic diagram of the network structure. DETAILED DESCRIPTION
[0040] In order to deepen the understanding of the present invention, the present embodiment is described in detail below with reference to the accompanying drawings.
[0041] Example 1: See Figure 1 , Figure 2, a high-speed scene visibility assessment method based on a fast defogging neural network algorithm, the method is as follows:
[0042] Step 1: Image input: The image to be evaluated is acquired through a high-speed camera.
[0043] Step 2: Image preprocessing: Perform necessary preprocessing on the input image, including cropping, scaling, etc.
[0044] Step 3: Dehazing:
[0045] 3-1 Collect high-speed monitoring images, collect foggy and non-fog images in pairs according to the point positions, and convert the non-fog images into grayscale images as reconstruction labels, and use the foggy images as input data to form a defogging reconstruction data set.
[0046] 3-2 Build a fast image dehazing network,
[0047] For an image of input size H×W×3, the processing order is as follows,
[0048] Use a convolution layer with a kernel size of 5x5, an output dimension of 16, and a stride of 2, a Silu activation layer, a convolution layer with a kernel size of 3x3, an output dimension of 16, and a stride of 2, and a Silu activation layer to achieve one downsampling and feature extraction (network layers 1 to 4) to obtain a feature map of H2×W2×16;
[0049] Use a convolution layer with a kernel size of 3x3, an output dimension of 32, and a stride of 2, a Silu activation layer, a convolution layer with a kernel size of 3x3, an output dimension of 32, and a stride of 1, and a Silu activation layer to achieve one-time downsampling and feature extraction (network 4 to 8 layers) to obtain a feature map of H4×W4×32;
[0050] A convolutional layer with a kernel size of 3x3, an output dimension of 64, and a stride of 2 is used, and a Silu activation layer is stacked to achieve the third downsampling and feature extraction (network layers 9 to 10), and a feature map of H8×W8×64 is obtained;
[0051] Use a transposed convolution layer with a convolution kernel size of 3x3, an output dimension of 16, and a stride of 2, a Silu activation layer, a Concat layer, a convolution layer with a convolution kernel size of 3x3, an output dimension of 32, and a stride of 1, and a Silu activation layer to achieve one upsampling and feature fusion, and obtain a feature map of H4×W4×32;
[0052] Use a transposed convolution layer with a convolution kernel size of 3x3, an output dimension of 16, and a stride of 2, a Silu activation layer, a Concat layer, a convolution layer with a convolution kernel size of 3x3, an output dimension of 16, and a stride of 1, and a Silu activation layer to achieve one upsampling and feature fusion, and obtain a feature map of H2×W2×16;
[0053] A transposed convolution layer with a convolution kernel size of 3x3, an output dimension of 16, and a stride of 2, a convolution layer with a convolution kernel size of 3x3, an output dimension of 3, and a stride of 1, and a Sigmoid activation layer are stacked to achieve one upsampling and obtain a H×W×3 result image.
[0054] 3-3 Use the data set to train and test the fast dehazing network to obtain the optimal weights.
[0055] 3-4 Use the trained fast defogging network weights to defog the input image to obtain the reconstructed image. Step 4: Image texture extraction.
[0056] 4-1 Use bilinear interpolation method to scale the image, reduce the image to 1 / 16 of the original image, and then restore it to the original size to obtain a smooth image, instead of Gaussian smoothing.
[0057] 4-2 Use the Sobel operator to perform convolution on the image to obtain the horizontal and vertical gradient matrices of the image, and then obtain the gradient matrix and angle matrix of the image.
[0058] Horizontal Sobel operator:
[0059] Vertical Sobel operator:
[0060] Horizontal gradient of the image:
[0061] Image vertical gradient:
[0062] The image gradient is:
[0063] 4-3 Use the non-maximum suppression method to process the image angle matrix and compare the gradient strength of the current pixel with the two pixels along the positive and negative gradient directions. If the gradient strength of the current pixel is the largest compared with the other two pixels, the pixel point is retained as an edge point, otherwise the pixel point will be suppressed.
[0064] 4-4 Use double thresholds to remove noise points and obtain the image texture binary matrix.
[0065] Step 5: Calculation of visibility estimation:
[0066] 5-1 Extract the problem from the input image and obtain the original image texture binary matrix M s ,
[0067] 5-2 Extract the problem from the dehazed image and obtain the original texture binary matrix M t ,
[0068] 5-3 A quick visibility evaluation formula is designed to calculate the visibility evaluation coefficient p, which is between 0 and 1. The smaller the value, the lower the visibility, as shown below
[0069]
[0070] Step 6: Result output: Output the calculated visibility estimate;
[0071] This technical solution is highly accurate, highly automated, and highly practical. Due to the use of shallow neural networks, the defogging image reconstruction performance is greatly improved, and the speed can reach 102simples / s on the Atlas 300 server, which is much higher than the existing defogging network. In addition, the consistency of the visibility coefficient evaluation at fixed points can reach 99% accuracy.
[0072] It should be noted that the above embodiments are not intended to limit the protection scope of the present invention, and equivalent changes or substitutions made on the basis of the above technical solutions all fall within the protection scope of the claims of the present invention.
Claims
1. A high-speed scene visibility assessment method based on a fast defogging neural network algorithm, characterized in that: The method is as follows: Step 1: Image input: The image to be evaluated is acquired through a high-speed camera. Step 2: Image preprocessing: Perform necessary preprocessing on the input image, including cropping, scaling, Step 3: Dehazing Step 4: Image texture extraction, Step 5: Calculation of visibility estimation, Step 6: Result output: Output the calculated visibility estimate.
2. A high-speed scene visibility assessment method based on a fast defogging neural network algorithm, characterized in that: Step 3: Defogging, as follows: 3-1 Collect high-speed monitoring images, collect foggy and non-fog images in pairs according to the point positions, and convert the non-fog images into grayscale images as reconstruction labels, and use the foggy images as input data to form a defogging reconstruction data set. 3-2 Build a fast image dehazing network, A shallow neural network is designed. For an image with an input size of H×W×3, the processing order is as follows: Use a convolution layer with a kernel size of 5x5, an output dimension of 16, and a stride of 2, a Silu activation layer, a convolution layer with a kernel size of 3x3, an output dimension of 16, and a stride of 2, and a Silu activation layer to achieve one downsampling and feature extraction (network layers 1 to 4) to obtain a feature map of H / 2×W / 2×16; Use a convolution layer with a kernel size of 3x3, an output dimension of 32, and a stride of 2, a Silu activation layer, a convolution layer with a kernel size of 3x3, an output dimension of 32, and a stride of 1, and a Silu activation layer to achieve one-time downsampling and feature extraction (network 4 to 8 layers) to obtain a feature map of H / 4×W / 4×32; A convolutional layer with a kernel size of 3x3, an output dimension of 64, and a stride of 2 is used, and a stack of Silu activation layers is used to implement the third downsampling and feature extraction (network layers 9 to 10), and a feature map of H / 8×W / 8×64 is obtained; Use a transposed convolution layer with a convolution kernel size of 3x3, an output dimension of 16, and a stride of 2, a Silu activation layer, a Concat layer, a convolution layer with a convolution kernel size of 3x3, an output dimension of 32, and a stride of 1, and a Silu activation layer to achieve one upsampling and feature fusion, and obtain a feature map of H / 4×W / 4×32; Use a transposed convolution layer with a convolution kernel size of 3x3, an output dimension of 16, and a stride of 2, a Silu activation layer, a Concat layer, a convolution layer with a convolution kernel size of 3x3, an output dimension of 16, and a stride of 1, and a Silu activation layer to achieve one upsampling and feature fusion, and obtain a feature map of H / 2×W / 2×16; Use a transposed convolution layer with a convolution kernel size of 3x3, an output dimension of 16, and a stride of 2, a convolution layer with a convolution kernel size of 3x3, an output dimension of 3, and a stride of 1, and a stack of Sigmoid activation layers to achieve one upsampling and get a H×W×3 result image. 3-3 Use the data set to train and test the fast dehazing network to obtain the optimal weights. 3-4 Use the trained fast dehazing network weights to dehaze the input image to obtain the reconstructed image.
3. A high-speed scene visibility assessment method based on a fast defogging neural network algorithm, characterized in that: Step 4: Image texture extraction, as follows: 4-1 Use bilinear interpolation method to scale the image, reduce the image to 1 / 16 of the original image, and then restore it to the original size to obtain a smooth image, instead of Gaussian smoothing. 4-2 Use the Sobel operator to perform convolution operation on the image to obtain the horizontal and vertical gradient matrices of the image, and then obtain the gradient matrix and angle matrix of the image. Horizontal Sobel operator: Vertical Sobel operator: Horizontal gradient of the image: Image vertical gradient: The image gradient is: 4-3 Use the non-maximum suppression method to process the image angle matrix and compare the gradient strength of the current pixel with the two pixels along the positive and negative gradient directions. If the gradient strength of the current pixel is the largest compared with the other two pixels, the pixel point is retained as an edge point, otherwise the pixel point will be suppressed. 4-4 Use double thresholds to remove noise points and obtain the image texture binary matrix.
4. A high-speed scene visibility assessment method based on a fast defogging neural network algorithm, characterized in that: Step 5: Calculation of visibility estimation: The details are as follows: 5-1 Extract the problem from the input image and obtain the original image texture binary matrix M s , 5-2 Extract the problem from the dehazed image and obtain the original texture binary matrix M t , 5-3 A quick visibility evaluation formula is designed to calculate the visibility evaluation coefficient p, which is between 0 and 1. The smaller the value, the lower the visibility, as shown below 5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the high-speed scene visibility assessment method based on the defogging neural network algorithm as described in any one of claims 1 to 4 above.
6. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the high-speed scene visibility assessment method based on the defogging neural network algorithm as described in any one of claims 1 to 4 above is implemented.