Video image rain and fog removal enhancement processing method and system for security monitoring

Abstract

The present application belongs to the technical field of image processing, and particularly relates to a video image rain and fog removal enhancement processing method and system for security monitoring, comprising the following steps: S1, acquiring a current frame image of a monitoring video, and using a linear iterative clustering algorithm to segment the current frame image into a plurality of mutually non-overlapping superpixel data blocks; S2, traversing each superpixel data block, and calculating the regional average brightness and regional gradient standard deviation of all pixel points in the superpixel data block; S3, constructing a local texture fog feeling index, which is positively correlated with the regional average brightness and negatively correlated with the regional gradient standard deviation. The present application effectively solves the problems of low contrast of long-range targets, easy noise interference in security monitoring, and over-darkness of long-range caused by traditional fog removal algorithms by introducing a fog feeling index and a gain compensation term, realizes the balance between near-range fidelity and long-range fog removal, and improves the visual distance of the monitoring picture.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method and system for enhancing video images by removing rain and fog for security monitoring. Background Technology

[0002] In urban security, border and coastal defense, and traffic monitoring, cameras often need to cover an ultra-wide depth of field, ranging from tens of meters to several kilometers. In rainy or foggy weather, the absorption and scattering of light by suspended particles in the atmosphere increases exponentially with distance, resulting in non-uniform degradation of the monitoring image: near objects may be slightly blurred but retain some color, while distant objects are completely shrouded in grayish-white fog with extremely low contrast, severely impacting the monitoring system's ability to identify and track distant targets.

[0003] Existing dehazing techniques typically employ global processing methods, such as global histogram equalization or single-parameter dark channel dehazing algorithms. These methods often assume that the fog density is uniform across the entire image, applying the same dehazing intensity directly to the whole image. The drawback of this approach is that objects that are relatively clear in the foreground are over-enhanced, resulting in color distortion and noise explosion; while distant targets that truly need dehazing remain blurry, failing to achieve balanced processing for scenes with large depth of field.

[0004] Furthermore, in long-distance surveillance, high-frequency texture details in images are easily attenuated due to atmospheric scattering. In such cases, the camera's thermal noise is numerically very similar to these subtle textures. Existing algorithms struggle to distinguish noise from real textures, often misinterpreting noise as texture details for enhancement. This results in an enhanced image that appears dirty and not only fails to recover useful information but also introduces new interference. Therefore, accurately distinguishing fog from background textures and avoiding noise amplification while removing fog is a pressing technical challenge in the field of security surveillance image enhancement. Summary of the Invention

[0005] This invention provides a method and system for enhancing video images by removing rain and fog for security monitoring, in order to solve the technical problems of low contrast of distant targets and susceptibility to noise interference in security monitoring scenarios, as well as the tendency of traditional defogging algorithms to cause oversaturation of near objects and darkness of distant objects.

[0006] In a first aspect, the present invention provides a method for enhancing video images by removing rain and fog for security monitoring, comprising the following steps: S1, acquire the current frame image of the surveillance video, and use a linear iterative clustering algorithm to segment the current frame image into multiple non-overlapping superpixel data blocks; S2, traverse each superpixel data block and calculate the regional average brightness and regional gradient standard deviation of all pixels in the superpixel data block; S3. Construct a local texture haze index. The local texture haze index is positively correlated with the average brightness of the region and negatively correlated with the standard deviation of the region gradient. Establish a nonlinear depth transmittance mapping model to map the local texture haze index to the estimated transmittance. S4 combines the estimated transmittance and global atmospheric light value, and uses an adaptive gain compensation formula that includes distant brightness compensation gain to perform pixel-level restoration of the current frame image, resulting in the final output image.

[0007] The benefits are as follows: This invention addresses the non-uniform degradation issues in existing security monitoring scenarios, such as near-field oversaturation due to large depth of field and incomplete dehazing of distant objects. It proposes an adaptive enhancement scheme based on superpixel segmentation and local texture statistics. Unlike traditional global histogram equalization or single dark channel prior algorithms, which easily lead to color distortion of near objects and unclear distant targets, this invention segments video frames into statistically significant superpixel blocks through linear iterative clustering and constructs a local texture fog index using the physical characteristics of high brightness and low gradient standard deviation. This allows for precise differentiation of real fog from white background walls or noise, much like the human eye. More importantly, this invention innovatively introduces an adaptive gain compensation mechanism into the physical restoration model, automatically injecting energy into distant areas with extremely low transmittance. This effectively solves the "distant black hole effect" often seen after dehazing using traditional physical models, achieving significant improvement in contrast and visibility distance for distant, weak targets while preserving near-field color fidelity. This significantly enhances the practical effectiveness of the monitoring system in adverse weather conditions.

[0008] Furthermore, the average brightness and standard deviation of the gradient of all pixels within the superpixel data block are calculated, specifically including: The average brightness of the region is obtained by summing the pixel values ​​of all pixels within the superpixel data block and dividing the sum of pixel values ​​by the total number of pixels in the superpixel data block. The Sobel operator is used to calculate the horizontal and vertical gradients of each pixel in the superpixel data block. The gradient magnitude of each pixel is calculated based on the horizontal and vertical gradients. The standard deviation of the gradient magnitudes of all pixels in the superpixel data block is calculated to obtain the regional gradient standard deviation.

[0009] The effect is that by combining the Sobel operator with statistical methods to calculate the average brightness and standard deviation of the gradient within a superpixel block, the complex visual information of the image is transformed into two core statistical features that can be processed quickly by a computer. This calculation method can not only effectively quantify the richness of texture within a region through the standard deviation of the gradient magnitude, but also reflect the overall illumination of the region through the average brightness. This provides a solid data foundation with low computational cost for the subsequent accurate construction of the fog index, avoiding the waste of computing power caused by pixel-by-pixel calculation.

[0010] Furthermore, the local texture haze index The calculation formula is:

[0011] In the formula, For the first Average brightness of the region of each superpixel data block For the first Regional gradient standard deviation of each superpixel data block High light sensitivity coefficient, As a noise suppression factor, is the numerical stability constant.

[0012] The benefits are as follows: This invention constructs a nonlinear local texture fog index formula that includes a high light sensitivity coefficient and a noise suppression factor. Utilizing the physical principle that fog typically exhibits high brightness and loss of texture details, a robust fog concentration quantification standard is established. This index effectively suppresses the interference of camera thermal noise on defogging judgment, preventing the misclassification of high-brightness non-fog areas as dense fog, thereby ensuring the accuracy of depth transmittance mapping and solving the problems of incorrect or uneven defogging in complex dynamic scenes.

[0013] Furthermore, the transmittance is estimated. The calculation formula is:

[0014] In the formula, The minimum transmittance threshold, This is the sensitivity control coefficient. For depth of field adjustment index, Represented by natural constant An exponential function with base 0. This represents the local texture haze index.

[0015] The effect is that by establishing a nonlinear depth transmittance mapping model based on an exponential function, the dimensionless fog perception index is smoothly transformed into the transmittance parameters required by the physical model. This formula introduces a minimum transmittance threshold and a sensitivity control coefficient, which not only prevents mathematical computational failures caused by zero transmittance but also simulates the physical characteristic of light attenuating exponentially with distance in the atmosphere. This ensures a natural and smooth transition of transmittance from foreground to background, avoiding obvious discontinuities or blocky effects in the enhanced image.

[0016] Furthermore, the final output value of each pixel in the final output image. The calculation formula is:

[0017] In the formula, For the current frame image at pixel point Pixel value at that location, This represents the global atmospheric light value. For pixels The estimated transmittance corresponding to the superpixel data block. Gain is used to compensate for the brightness of distant scenes.

[0018] The effect is as follows: This invention proposes a final output formula that includes a distant scene brightness compensation gain, breaking through the limitations of the classic atmospheric scattering model. This formula utilizes the inverse characteristic of transmittance as a trigger condition. In near-field areas with high transmittance, almost no compensation is introduced to maintain the original image quality. However, in distant foggy areas with extremely low transmittance, the compensation term is automatically activated to replenish brightness energy for severely attenuated light signals. This mathematically solves the inherent defect of traditional defogging algorithms that darken the image upon defogging, making details of distant pedestrians and vehicles clearly visible.

[0019] Furthermore, the method for obtaining the global atmospheric light value is as follows: sort the local texture fog index of all pixels in the entire image, select the pixels with the largest local texture fog index in the preset percentile range, and calculate the average brightness of these pixels as the global atmospheric light value.

[0020] The effect is that it uses a strategy based on local texture fog index sorting to obtain global atmospheric light values, rather than simply selecting the brightest pixel in the entire image. This method avoids mistaking nearby white objects for atmospheric light, ensuring that the selected atmospheric light values ​​truly originate from dense fog areas. This improves the accuracy of the physical restoration model's parameters, resulting in a more natural tone in the final restored image and reducing color cast.

[0021] Furthermore, when using a linear iterative clustering algorithm to segment the current frame image into multiple non-overlapping superpixel data blocks, at least two different segmentation scales are set, each corresponding to a different initial seed spacing.

[0022] Its effect is that by setting multiple different initial seed spacings for multi-scale superpixel segmentation, it effectively addresses the perspective changes of objects appearing larger when closer and smaller when farther away in the monitoring image. This multi-scale strategy ensures that both large vehicles in the foreground and small pedestrians in the distance are tightly surrounded by superpixel blocks of appropriate size, thereby better preserving the edge information of targets at different scales in subsequent processing and reducing edge jaggedness or loss of detail caused by a single segmentation scale.

[0023] Furthermore, after obtaining the estimated transmittance, the method further includes: Using the current frame image as the guide image, the estimated transmittance is subjected to guided filtering to obtain a smoothed transmittance map, which is then used for subsequent image restoration calculations.

[0024] Furthermore, before calculating the average brightness of the region, the following steps are also included: The current frame image is converted from the red-green-blue color space to the hue-saturation-brightness color space, and the brightness component channel is extracted as the basic data source for calculating the average brightness of the region and the standard deviation of the region gradient.

[0025] Secondly, the present invention provides a video image de-raining and de-fogging enhancement processing system for security monitoring, including a memory and a processor. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned video image de-raining and de-fogging enhancement processing method for security monitoring is implemented.

[0026] The beneficial effects are: 1. Strong scene adaptability: This invention does not use universal global parameters, but instead innovatively constructs a local texture fog index based on multi-scale superpixel segmentation, combining regional average brightness and gradient standard deviation. This index can automatically distinguish between white walls and dense fog, just like the human eye, avoiding misjudgments by traditional algorithms against solid color backgrounds, and enabling the algorithm to maintain stable judgment in various complex monitoring scenarios.

[0027] 2. Significant long-range enhancement effect: Addressing the large depth of field characteristic of security monitoring, this invention proposes an image restoration formula incorporating a square root gain compensation term. This formula utilizes the inverse property of transmittance to maintain original brightness in near-field areas while automatically replenishing energy in far-field areas with severe signal attenuation. This design not only solves the black hole effect after defogging in the physical model but also significantly improves the contrast of distant targets, making vehicles and pedestrians previously invisible in fog clearly discernible, greatly enhancing the practical value of the monitoring system. Attached Figure Description

[0028] Figure 1 This is a flowchart of the video image de-raining and de-fogging enhancement processing method for security monitoring according to the present invention.

[0029] Figure 2 This is a heatmap of the fog index distribution analysis based on superpixel features according to the present invention.

[0030] Figure 3 This is a comparison diagram of image signal contrast waveforms in a long-distance monitoring scenario according to the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] An embodiment of the video image de-raining and de-fogging enhancement method for security monitoring provided by this invention: like Figure 1 As shown, the video image de-raining and de-fogging enhancement method for security monitoring includes the following steps: S1: Obtain the current frame image of the surveillance video and use a linear iterative clustering algorithm to segment the current frame image into multiple non-overlapping superpixel data blocks.

[0033] Specifically, to accurately capture the fog distribution characteristics at different depths of field, it is first necessary to convert continuous video frames into discrete, statistically significant data units. Since targets in a monitoring scene vary in size, single-grid segmentation can easily disrupt object edges. This embodiment employs a simple linear iterative clustering algorithm to analyze the current frame image of the acquired monitoring video. The image is then processed. During segmentation, at least two different segmentation scales are set; for example, the initial seed intervals are 1 / 20 and 1 / 50 of the total pixel width, respectively. This multi-scale segmentation divides the image into segments. A collection of non-overlapping superpixel data blocks. Each data block has similar color and brightness characteristics, and its edges closely match the outlines of vehicles, buildings, or pedestrians.

[0034] Multi-scale superpixel segmentation can better adapt to changes in object scale in monitoring scenarios, preserving object edge information while providing accurate spatial units for subsequent local feature statistics.

[0035] S2, traverse each superpixel data block and calculate the regional average brightness and regional gradient standard deviation of all pixels within the superpixel data block.

[0036] Specifically, for any number of segments obtained from the division... One superpixel data block Iterate through all the pixels contained within it and calculate the following two basic metrics: (1) Regional average brightness : The average brightness of a superpixel data block is calculated by summing the grayscale values ​​of all pixels within that block and then dividing by the total number of pixels in the block. This metric reflects the overall brightness of the area. For example, a superpixel data block containing 100 pixels with a total pixel value of 15000 indicates the average brightness of that area. .

[0037] (2) Regional gradient standard deviation : First, the horizontal gradient of each pixel within the superpixel data block is calculated using the Sobel operator. and vertical gradient The gradient magnitude is obtained. Then, the standard deviation of the gradient magnitude of all pixels within the superpixel data block is calculated. This index is used to characterize the richness and dispersion of the texture in the region.

[0038] For example: If the area is a smooth wall or a foggy area, the pixel values ​​change gradually, the gradient values ​​are generally small and close to the mean, and the calculated standard deviation... Lower, for example If the area contains leaves, grass, or a vehicle grille, with rich texture details and large gradient value differences, the calculated standard deviation will be... Higher, for example .

[0039] By calculating the average brightness and standard deviation of the regional gradient, the visual features of the image are transformed into computable statistical features, which can effectively distinguish between flat and textured regions, laying a data foundation for the subsequent construction of the fog index.

[0040] S3. Construct a local texture haze index. The local texture haze index is positively correlated with the average brightness of the region and negatively correlated with the standard deviation of the region gradient. Establish a nonlinear depth transmittance mapping model to map the local texture haze index to the estimated transmittance.

[0041] Specifically, in security scenarios, dense fog areas typically exhibit high brightness and loss of texture details. Based on this physical principle, this embodiment constructs a dimensionless local texture fog perception index. To evaluate the first The fog effect of each data block. The calculation formula is as follows:

[0042] In the formula, This is the high light sensitivity coefficient, used to amplify the contribution of bright areas to the haze index; This is a noise suppression factor used to dynamically increase the reference value of the denominator using the brightness value, preventing random noise in bright areas from being misjudged as texture. The value is a numerical stability constant, which is fixed at 1.0 to ensure that the denominator is not zero.

[0043] Calculation example: Assume , , ; Situation A (Dense Fog Area): , ; A high index indicates dense fog.

[0044] Situation B (Close-up Sharp Area): , ; A low index indicates light fog.

[0045] Next, in order to address the effect of distant black holes, a mapping model needs to be established to predict the transmittance. The calculation formula is:

[0046] In the formula, The range of values ​​is ; This is set as the minimum transmittance threshold to prevent the operation from crashing if the transmittance is 0. This is the sensitivity control coefficient; This is the depth-of-field adjustment index.

[0047] Calculation example: Assume , , ; Regarding situation A: , , ; (Extremely low transmittance, corresponding to distant views).

[0048] For situation B: , , ; (High transmittance, corresponding to close-up views).

[0049] By constructing a local texture fog index and a nonlinear depth transmittance mapping model, the scene depth can be accurately inferred from a monocular image. The nonlinear mapping relationship ensures the smoothness of the transmittance changes between the near and far scenes, avoiding blocky effects in the image.

[0050] S4 combines the estimated transmittance and global atmospheric light value, and uses an adaptive gain compensation formula that includes distant brightness compensation gain to perform pixel-level restoration of the current frame image, resulting in the final output image.

[0051] Specifically, the first step is to obtain the global atmospheric light value. The method is to select The average brightness of pixels within the preset percentile range with the highest values ​​is calculated. Then, the calculated transmittance map is used to restore the image. To address the issue of an overly dark image after dehazing using traditional physical models, dynamic gain compensation is embedded. The brightness of each pixel is calculated... The final output value :

[0052] In the formula, Gain compensation for distant scene brightness; Fog concentration factor For the current frame image at pixel point Pixel value at that location, This represents the global atmospheric light value.

[0053] Logical explanation: The first term of the formula It is a classic inverse model of atmospheric scattering physics, used to remove fog obstruction. The second term is an innovative compensation term, when... When it approaches 1, When the value is close to 0, the compensation term does not work, maintaining the original brightness of the foreground; when... When smaller, The compensation value is increased, automatically injecting brightness into distant areas.

[0054] Calculation example: Assume , , , ; First item (defogging): ; Second item (compensation): ; Final output .

[0055] Without compensation, the result is only ,at this time Obviously more Brighter, effectively improving the visibility of distant scenery.

[0056] refer to Figure 2 This figure illustrates the distribution analysis of the haze index based on superpixel features. The horizontal axis represents the local gradient standard deviation, and the vertical axis represents the average brightness of the region. The clustered points in the upper left corner represent high-brightness, low-gradient, high-haze regions, while the clustered points in the lower right corner represent low-brightness, high-gradient, clear-texture regions, visually verifying... The index's ability to distinguish between fog and texture.

[0057] refer to Figure 3 The figure illustrates a comparison of image signal contrast waveforms in a long-distance monitoring scenario. The horizontal axis represents the monitoring distance, and the vertical axis represents the contrast amplitude. The shaded waveform shows that the original signal attenuates rapidly with increasing distance. The signal processed by this invention maintains a high amplitude in distant areas, demonstrating that this invention significantly recovers the contrast information of distant targets.

[0058] Through the adaptive gain compensation formula, it is possible to intelligently replenish the energy of distant light that has been attenuated due to scattering while removing fog, effectively solving the industry pain point of being able to see close objects but not distant ones, and greatly improving the usability of surveillance images.

[0059] An embodiment of the video image de-raining and de-fogging enhancement processing system for security monitoring provided by the present invention: The video image de-raining and de-fogging enhancement system for security monitoring includes a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, implement the aforementioned video image de-raining and de-fogging enhancement method for security monitoring.

[0060] The video image de-raining and de-fogging enhancement system used for security monitoring also includes other components well known to those skilled in the art, such as communication interfaces. Their settings and functions are known in the art and will not be described in detail here.

[0061] In this invention, the aforementioned memory can be any tangible medium containing or storing a program that can be used or combined with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc., or any other medium that can be used to store desired information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device. Any application or module described in this invention can be implemented using computer-readable / executable instructions stored or otherwise maintained by such a computer-readable medium.

[0062] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for enhancing video images by removing rain and fog for security monitoring, characterized in that, Includes the following steps: S1, acquire the current frame image of the surveillance video, and use a linear iterative clustering algorithm to segment the current frame image into multiple non-overlapping superpixel data blocks; S2, traverse each superpixel data block and calculate the regional average brightness and regional gradient standard deviation of all pixels in the superpixel data block; S3. Construct a local texture haze index. The local texture haze index is positively correlated with the average brightness of the region and negatively correlated with the standard deviation of the region gradient. Establish a nonlinear depth transmittance mapping model to map the local texture haze index to the estimated transmittance. S4 combines the estimated transmittance and global atmospheric light value, and uses an adaptive gain compensation formula that includes distant brightness compensation gain to perform pixel-level restoration of the current frame image, resulting in the final output image.

2. The video image de-raining and de-fogging enhancement method for security monitoring according to claim 1, characterized in that, Calculate the average brightness and standard deviation of the gradient of all pixels within a superpixel data block, specifically including: The average brightness of the region is obtained by summing the pixel values ​​of all pixels within the superpixel data block and dividing the sum of pixel values ​​by the total number of pixels in the superpixel data block. The Sobel operator is used to calculate the horizontal and vertical gradients of each pixel in the superpixel data block. The gradient magnitude of each pixel is calculated based on the horizontal and vertical gradients. The standard deviation of the gradient magnitudes of all pixels in the superpixel data block is calculated to obtain the regional gradient standard deviation.

3. The video image de-raining and de-fogging enhancement method for security monitoring according to claim 1, characterized in that, Local texture haze index The calculation formula is: In the formula, For the first Average brightness of the region of each superpixel data block For the first Regional gradient standard deviation of each superpixel data block High light sensitivity coefficient, As a noise suppression factor, is the numerical stability constant.

4. The video image de-raining and de-fogging enhancement method for security monitoring according to claim 3, characterized in that, Predicted transmittance The calculation formula is: In the formula, The minimum transmittance threshold, This is the sensitivity control coefficient. For depth of field adjustment index, Represented by natural constant An exponential function with base 0. This represents the local texture haze index.

5. The video image de-raining and de-fogging enhancement method for security monitoring according to claim 4, characterized in that, The final output value of each pixel in the final output image The calculation formula is: In the formula, For the current frame image at pixel point Pixel value at that location, This represents the global atmospheric light value. For pixels The estimated transmittance corresponding to the superpixel data block. Gain is used to compensate for the brightness of distant scenes.

6. The video image de-raining and de-fogging enhancement method for security monitoring according to claim 5, characterized in that, The method for obtaining the global atmospheric light value is as follows: sort the local texture fog index of all pixels in the whole image, select the pixels with the largest local texture fog index in the preset percentile range, and calculate the average brightness of these pixels as the global atmospheric light value.

7. The video image de-raining and de-fogging enhancement method for security monitoring according to claim 1, characterized in that, When using a linear iterative clustering algorithm to segment the current frame image into multiple non-overlapping superpixel data blocks, at least two different segmentation scales are set, each corresponding to a different initial seed spacing.

8. The video image de-raining and de-fogging enhancement method for security monitoring according to claim 4, characterized in that, After obtaining the estimated transmittance, the method further includes: Using the current frame image as the guide image, the estimated transmittance is subjected to guided filtering to obtain a smoothed transmittance map, which is then used for subsequent image restoration calculations.

9. The video image de-raining and de-fogging enhancement method for security monitoring according to claim 2, characterized in that, Before calculating the average brightness of the region, the following steps are also included: The current frame image is converted from the red-green-blue color space to the hue-saturation-brightness color space, and the brightness component channel is extracted as the basic data source for calculating the average brightness of the region and the standard deviation of the region gradient.

10. A video image de-raining and de-fogging enhancement processing system for security monitoring, characterized in that, The system includes a memory and a processor. The memory stores computer program instructions, which, when executed by the processor, implement the video image de-raining and de-fogging enhancement processing method for security monitoring as described in any one of claims 1-9.

Images