Statistical prior based multi-scene image enhancement method and system

Abstract

The application discloses a kind of multi-scene image enhancement method and system based on statistical prior, comprising: inputting fog image into atmospheric scattering model, according to fog image, obtain fog image and atmospheric light after normalization by calculation;By reading the pixel value of fog image, the statistical degree component and standard deviation of each local image block in the normalized image after fog image and atmospheric light value are divided are calculated, and the statistical line is fitted to determine the transmission diagram;Wherein the statistical degree component is the ratio of the average value and standard deviation of local pixel, and local pixel is the pixel value of reading fog image;Introduce upper and lower regularization and guide filter to optimize transmission diagram, obtain optimized transmission diagram;Based on optimized transmission diagram and atmospheric light, input atmospheric scattering model, obtain the final defogging image.The application constructs statistical line for each local image, so as to better estimate transmittance, and then achieve the visual effect of defogging and retaining details and clarity simultaneously.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image enhancement, and particularly relates to a multi-scene image enhancement method and system based on statistical prior. BACKGROUND

[0002] In recent years, due to the significant progress of CNN in large-scale image processing such as detection and recognition, methods for dehazing of blurred images using deep learning have also emerged. Cai Bolun et al. analyzed the previous features and established a Dehaze-Net model to achieve more accurate prediction of transmission images. Although Dehaze-Net can achieve dehazing to a certain extent, there are inevitably problems such as color distortion, color distortion, blocking effect, etc. The details and texture structure level are not high enough, and the dehazing effect is poor. The generative adversarial network (GAN) is a deep learning model developed and proposed by Goodfellow et al. The generative adversarial network mainly consists of two parts: the generator G and the discriminator D. Based on the idea of zero-sum game, the generator G tries to achieve the most accurate result when generating the mode, while the discriminator D tries to accurately distinguish the generated mode from the real mode, that is, to accurately distinguish between false images and real images. The collapse and gradient dispersion problems of the GAN model have existed and have not been effectively solved so far. The above two methods are dehazing algorithms based on deep learning model, which highly depend on the accurate prediction of intermediate variables. There are errors in the prediction of intermediate variables to varying degrees. These errors will be amplified when reconstructing the haze-free image, resulting in image distortion. In order to alleviate the above problems, researchers use an end-to-end method based on DCNN, which directly encodes the dehazing into a research network based on the haze-free image generated from the blurred image. Although it has achieved remarkable results, the performance is limited. Recently, various end-to-end CNN-based methods have been proposed to simplify the dehazing problem by directly converting the dehazing network, but there are still problems such as false positives. In general, haze will cause the information quality of images taken in haze weather to decrease, so various methods have been proposed to eliminate the blur caused thereby. These methods have achieved significant results in dehazing. However, the intrinsic correlation of local pixels is crucial for restoring fine details. However, few people have explored it, resulting in limited deblurring performance of most methods in highly degraded scenes, or poor robustness in various types of haze scenes. SUMMARY

[0003] To solve the above technical problems, the present application proposes a multi-scene image enhancement method and system based on statistical prior, which constructs a statistical line for each local image, thereby better estimating the transmittance, and further achieving the visual effect of dehazing and preserving details and clarity at the same time.

[0004] In one aspect, to achieve the above object, the present application provides a multi-scene image enhancement method based on statistical prior, comprising:

[0005] inputting the haze image into an atmospheric scattering model, and obtaining an image after haze image and atmospheric light normalization by calculation according to the haze image;

[0006] reading pixel values of the haze image, and obtaining statistical degree components and standard deviations of each local image block in a normalized image after haze image and atmospheric light value division by calculation, and fitting a statistical line to determine a transmission map; wherein the statistical degree component is a ratio of an average value of local pixels to the standard deviation, and the local pixels are the pixel values read from the haze image;

[0007] the method for obtaining the statistical degree components and the standard deviations of each local image block in the image after haze image and atmospheric light normalization is as follows:

[0008]

[0009]

[0010] wherein, the ratio of the pixel average value to the standard deviation of the local image block after atmospheric light normalization, the pixel average value of the local image block after atmospheric light normalization, the pixel standard deviation of the local image block after atmospheric light normalization, the ratio of the local image block of the haze image to the atmospheric light value of the image block, the transmission map, the intercept, the ratio of the pixel average value to the standard deviation of the local image block of the haze-free image after processing, the ratio of the local image block of the haze-free image to the atmospheric light value of the image block, the atmospheric light value, the 15*15 image block in the image;

[0011] introducing upper and lower regularization and guide filter to optimize the transmission map, and obtaining an optimized transmission map;

[0012] inputting the optimized transmission map and the atmospheric light into the atmospheric scattering model to obtain a final haze-free image.

[0013] Optionally, the haze image is input into a known atmospheric scattering model and represented as follows:

[0014]

[0015] wherein, the atmospheric light value, is a pixel position, is a haze-free image after haze removal processing, is a haze image requiring haze removal processing, is a transmission map.

[0016] Optionally, the method for obtaining the haze image and the image after atmospheric light normalization from the haze image is:

[0017]

[0018]

[0019] wherein, is a 15*15 image block in the image, is a perspective view, is a local image block of the haze image requiring haze removal processing, is an atmospheric light value of the local image block in the haze image, is a local image block of the haze-free image, is a ratio of the local image block of the haze image and the atmospheric light value of the image block, is a ratio of the local image block of the haze-free image and the atmospheric light value of the image block.

[0020] Optionally, the method for fitting a statistical line to determine the transmission map comprises:

[0021] a statistical line model is constructed, and the statistical degree component and the standard deviation of each local image block in the haze image and the image after atmospheric light normalization are input into the statistical line model to obtain the transmission map.

[0022] Optionally, the method for introducing upper and lower regularization and guide filter to optimize the transmission map to obtain an optimized transmission map is:

[0023]

[0024]

[0025] wherein, is the transmission map after upper and lower boundary limitation, is a color channel, is a red channel, is a green channel, is a blue channel, is an atmospheric light value of a local image of the haze image in different channels, is a local image of the haze image in different color channels, is a lower boundary of different color channels, is an upper boundary of different color channels, A guided filter operator is used to guide the filtering operation. The optimized transmission map.

[0026] Optionally, based on the optimized transmission map and the atmospheric light, the atmospheric scattering model is input to obtain a final defogging image.

[0027]

[0028] Wherein, The final defogging image, The haze image that needs to be defogged, The atmospheric light value, The optimized transmission map.

[0029] In order to achieve the above object, the present application also provides a multi-scene image enhancement system based on statistical prior, comprising:

[0030] A haze image normalization module, a perspective map determination module, a perspective map optimization module and a defogging image acquisition module;

[0031] The haze image normalization module is used to input the haze image into the atmospheric scattering model, and calculate the normalized image of the haze image and the atmospheric light according to the haze image.

[0032] The perspective map determination module is used to read the pixel value of the haze image, calculate the statistical component and the standard deviation of each local image block in the normalized image after the haze image is divided by the atmospheric light value, and fit the statistical line to determine the transmission map; wherein the statistical component is the ratio of the average value of the local pixel to the standard deviation, and the local pixel is the pixel value of the haze image.

[0033] The method for obtaining the statistical component and the standard deviation of each local image block in the normalized image of the haze image and the atmospheric light is as follows:

[0034]

[0035]

[0036] Wherein, The ratio of the average value of the pixel of the local image block after the atmospheric light normalization to the standard deviation, The average value of the pixel of the local image block after the atmospheric light normalization, The standard deviation of the pixel of the local image block after the atmospheric light normalization, The ratio of the local image block of the haze image to the atmospheric light value of the image block, The transmission map, The intercept, a ratio of a mean value of pixels of a local image block of the haze-free image to a standard deviation of the pixels, a ratio of a local image block of the haze-free image to an atmospheric light value of the image block, an atmospheric light value, a 15*15 image block in the image;

[0037] the perspective diagram optimization module is used for introducing up-down regularization and a guided filter to optimize the transmission diagram, and obtaining an optimized transmission diagram;

[0038] the haze-removed image acquisition module is used for inputting the optimized transmission diagram and the atmospheric light into the atmospheric scattering model, and obtaining a final haze-removed image.

[0039] Technical effects of the present application: the present application is based on an atmospheric scattering model, and uses a simple and effective single-image haze-removal model, namely a statistical line model (SLM). A statistical measure (SM) is proposed in the present application to quantify the haze density of different scene regions in a haze image. The statistical line model reveals a linear relationship between the statistical component of a local image in the haze and the reciprocal of the pixel standard deviation, thereby converting the estimation of a transmission diagram into the construction of a statistical line. On this basis, the present application uses an adaptive transmission constraint, which contains information from the haze scene itself. This constraint can not only remove haze, but also significantly suppress the amplification of noise, thereby improving the ability to process large haze scenes. The present application applies a guided filter to smooth the transmission diagram generated thereby, thereby providing high-quality image restoration for a haze scene. The final haze-removed image not only has a good haze-removal effect, but also has clearer details, and is superior to the most advanced technology in terms of color preservation. In addition, the method proposed in the present application can also process other complex scenes, namely image restoration under different weather and imaging conditions, for example, sand-dust weather, underwater imaging conditions and low-light imaging conditions, and has high robustness. The present application can be further applied to other image processing tasks as image processing software, such as target recognition and detection systems in underwater exploration, low-light conditions at night, road monitoring systems and the like, and in complex scenes, improves the recognition and clarity of the obtained image. BRIEF DESCRIPTION OF DRAWINGS

[0040] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of the present application illustrated in the drawings and their descriptions are used to explain the present application and are not intended to limit the present application. In the drawings:

[0041] Figure 1 a flowchart of a statistical-prior-based multi-scene image enhancement method according to an embodiment of the present application;

[0042] Figure 2 a structural diagram of a statistical-prior-based multi-scene image enhancement system according to an embodiment of the present application;

[0043] Figure 3 The following are experimental results of fog in embodiments of the present invention, wherein (a) is a fog image, (b) is an image restored using the dark channel method (DCP), (c) is an image restored using the rank-one prior method (ROP), and (d) is a visually enhanced image obtained by the method (SLM) used in the present invention;

[0044] Figure 4 The following are experimental results of dense fog in embodiments of the present invention, wherein (a) is a dense fog image, (b) is an image restored using the dark channel method (DCP), (c) is an image restored using the rank-one prior method (ROP), and (d) is a visually enhanced image obtained by the method (SLM) used in the present invention;

[0045] Figure 5 The following are the experimental results of image restoration under underwater conditions in the embodiments of the present invention, wherein (a) is the underwater image processing result under multiple colors, and (b) is the underwater image processing result under high scene depth.

[0046] Figure 6 The images shown are the results of the image restoration experiment under low illumination conditions according to the embodiments of the present invention, wherein (a) is the image enhancement result under global low illumination, and (b) is the image enhancement result under local low illumination. Detailed Implementation

[0047] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0048] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0049] like Figure 1 As shown, this embodiment provides a multi-scene image enhancement method based on statistical priors, including: Step S1, using a known atmospheric scattering model, inputting a haze image. And obtain haze images Image after normalization with atmospheric light ;

[0050] Step S2: Based on the defined statistical components Haze images were obtained respectively. Each local image patch Statistical components and standard deviation In order to fit a statistical line, the transmission map is determined. ;

[0051] Step S3, optimizing the transmission map by introducing upper and lower regularization and guided filter , so as to obtain a more refined transmission map ;

[0052] Step S4, using the atmospheric scattering model and the obtained transmission map to obtain the final defogging image .

[0053] Step S1 includes the following steps:

[0054] The mathematical model describing the degradation caused by the absorption and scattering of light by particles in the atmosphere is: (1)

[0055] wherein, is the pixel position, is the haze-free image after defogging processing, is the haze image that needs to be defogged, is the atmospheric light value, is the transmission map.

[0056] The atmospheric scattering model and the atmospheric light are normalized:

[0057] (2)

[0058] wherein is a 15*15 image block in the image.

[0059] Formula (2) is rewritten as:

[0060] (3)

[0061] wherein , .

[0062] Step S2 includes the following steps:

[0063] Formula (3) is rearranged as:

[0064] (4)

[0065] Substituting formula (4) into formula (1) gives:

[0066] (5)

[0067] wherein is the statistical component, and the ratio of the average value of the pixels of the image to the standard deviation of the pixels of the image is defined as the statistical component, and the specific formula is as follows:

[0068] (6)

[0069] At the same time, formula (5) is rewritten as

[0070] (7)

[0071] (8)

[0072] Combining formula (7) and formula (8), we get:

[0073] (9)

[0074] Because the average value of the pixel value of the local image in the haze image and the standard deviation are fixed values, formula (9) is rewritten as:

[0075] (10)

[0076] For the sake of clarity, formula (10) is rewritten as:

[0077] (11)

[0078] where and respectively represent the slope and intercept, which are obtained in the following way:

[0079] (12)

[0080] (13)

[0081] According to formula (13), once the slope is obtained, the corresponding local image block transmission map can be directly inferred: (14)

[0082] where, is the slope of the linear relationship described by and .

[0083] Step S3 specifically includes:

[0084] In order to ensure effective defogging in the local area of the haze image, boundary conditions are introduced, which are usually used to generate the initial transmission map and provide the necessary transmission map for the modified area.

[0085] For a given hazy pixel, the defogging process consists in removing these pixels from the atmospheric light pixels in all color channels. Without loss of generality, consider the case of a color channel . Given a lower bound and an upper bound , express and as color channel representations of the lower and upper bounds, respectively. The purpose of the bounds conditions is to obtain a transmission value that fully satisfies the given lower bound or upper bound of the defogged pixel.

[0086] Moreover, obtaining the maximum value of the transmission of the three color channels represents the transmission value that allows the corrected pixel to fully satisfy the lower bound or upper bound in the color channel. In this case, the corresponding transmission value is determined as follows:

[0087] (15)

[0088] where the recommended bounds values, i.e. and are adopted. The introduction of the bounds conditions ensures an effective defogging of the SLM local area, thus improving the reliability.

[0089] Moreover, the proposed method assumes that the transmission map is constant, and a direct defogging can lead to block artifacts. Therefore, the obtained transmission map is smoothed using a guided filter:

[0090] (16)

[0091] where is the guided filter operator.

[0092] The present invention improves the use of local pixels by introducing a bounds-constrained and guided-filtered effective transmission mapping refinement method. It lays the foundation for the subsequent application of the atmospheric scattering model in defogging.

[0093] Step S4 specifically comprises:

[0094] The atmospheric light can be obtained by various methods. In fact, the atmospheric light values estimated by these methods have little difference in most scenes, while they all tend to take the pixels of the farthest area as the reference of the atmospheric light value. In the present invention, this method is used for estimation.

[0095] According to the atmospheric scattering model in formula (1), the defogged image is obtained:

[0096] (17)​

[0097] At this point, according to the above statistical line model (SLM) construction process, the defogging after fog-free image is obtained .

[0098] As shown in Figure 3 and Figure 4 , the famous dark channel prior method (DCP) and the latest rank one prior method (ROP) are compared under thin fog and thick fog weather respectively. Among them, the method described in the application is shown in Figure 3 (d) and Figure 4 (d), in the defogging process, no block artifacts and fog appear, and the recovered image details are clearer, and the recovered image is closer to the original color. As can be seen from Figure 3 (b)-(c) and Figure 4 (b)-(c), DCP and ROP cannot display details at a distance, resulting in obvious haze residue and detail loss. In contrast, the statistical line model proposed in the application can completely remove the haze and restore the true color while reducing artifacts and suppressing noise.

[0099] Figure 5 and Figure 6 respectively show the results of image recovery processing under underwater conditions and low light conditions according to the application. Figure 5 (a) in the figure is the result of underwater image processing under various colors, Figure 5 (b) in the figure is the result of underwater image processing under high scene depth; Figure 6 (a) in the figure is the result of image enhancement under global low light, Figure 6 (b) in the figure is the result of image enhancement under local low light.

[0100] Under the above conditions, the method of the application can still have good effect and can recover more details.

[0101] As shown in Figure 2 , the embodiment also provides a multi-scene image enhancement system based on statistical prior, which comprises: a haze image normalization module, a perspective view determination module, a perspective view optimization module and a defogging image acquisition module.

[0102] The haze image normalization module is used to input the haze image into the atmospheric scattering model, and obtain the haze image and the image normalized by atmospheric light according to the haze image by calculation.

[0103] The perspective view determination module is used for calculating statistical degree components and standard deviations of each local image block in a normalized image obtained by dividing the haze image by the atmospheric light value by reading pixel values of the haze image, and fitting a statistical line to determine the transmission diagram; wherein the statistical degree component is a ratio of an average value of local pixels to the standard deviation, and the local pixels are pixel values read from the haze image;

[0104] The perspective view optimization module is used for introducing upper and lower regularization and a guided filter to optimize the transmission diagram, and obtaining an optimized transmission diagram.

[0105] The defogging image acquisition module is used for inputting the optimized transmission diagram and the atmospheric light into an atmospheric scattering model to obtain a final defogging image.

[0106] The present application is based on an atmospheric scattering model, and uses a simple and effective single image defogging model, i.e., a statistical line model (SLM). In the present application, a statistical measure (SM) is proposed to quantify the haze density of different scene regions in a haze image. The statistical line model reveals a linear relationship between the statistical component of a local image in the haze and the inverse of the pixel standard deviation, thereby converting the estimation of the transmission diagram into the construction of a statistical line. On this basis, an adaptive transmission constraint is used, which contains information from the haze scene itself. This constraint can not only remove the haze, but also significantly suppress the amplification of noise, thereby improving the ability to process large haze scenes. Then, a guided filter is applied to smooth the transmission diagram generated thereby, thereby providing high-quality image restoration for the haze scene. The final defogging image not only has a good defogging effect, but also has clearer details, and is superior to the most advanced techniques in color preservation. In addition, the method proposed in the present application can also process other complex scenes, i.e., image restoration under different weather and imaging conditions, such as sand-dust weather, underwater imaging conditions and low-light imaging conditions, and has high robustness. The present application can be further applied to other image processing tasks as image processing software, such as target recognition and detection systems in underwater exploration, low-light conditions at night, road monitoring systems and the like, and in complex scenes, the recognition degree and clarity of the obtained images are improved.

[0107] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any changes or replacements easily thought of by those skilled in the art within the technical range disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A multi-scene image enhancement method based on statistical prior, characterized in that, include: The haze image is input into the atmospheric scattering model, and the haze image and the image after atmospheric light normalization are obtained by calculation based on the haze image. By reading the pixel values ​​of a haze image, the statistical component and standard deviation of each local image patch in the normalized image obtained after dividing the haze image by the atmospheric light value are calculated, and a statistical line is fitted to determine the transmission map; wherein the statistical component is the ratio of the average value of the local pixels to the standard deviation, and the local pixels are the pixel values ​​of the haze image. The method for obtaining the statistical components and standard deviation of each local image patch in the haze image and the image after atmospheric light normalization is as follows: in, This is the ratio of the pixel mean to the standard deviation of a local image patch after atmospheric light normalization. This represents the average pixel value of a local image patch after atmospheric light normalization. This represents the pixel standard deviation of a local image patch after atmospheric light normalization. This represents the ratio of a local image patch in a haze image to the atmospheric light value of that patch. This is a transmission image. The intercept is... This represents the ratio of the pixel mean to the standard deviation of a local image patch in the processed, haze-free image. This is the ratio of a local image patch in a fog-free image to the atmospheric light value of that patch. Atmospheric light value, This refers to a 15x15 image block within the image; The transmission map is optimized by introducing upper and lower regularization and a guided filter to obtain an optimized transmission map. Based on the optimized transmission map and atmospheric light input to the atmospheric scattering model, the final dehazed image is obtained.

2. The multi-scene image enhancement method based on statistical prior as described in claim 1, characterized in that, The haze image is represented as follows when input into a known atmospheric scattering model: in, Atmospheric light value, For pixel position, This is the haze-free image after dehazing. For haze images that need dehazing processing, This is a transmission image.

3. The multi-scene image enhancement method based on statistical prior as described in claim 1, characterized in that, The method for obtaining the haze image and the atmospheric light-normalized image from the haze image is as follows: in, For a 15x15 image patch in the image, This is a perspective view. For a local image patch of a hazy image that needs dehazing, This represents the atmospheric light value of a local image patch within a haze image. A local image patch of a fog-free image. This represents the ratio of a local image patch in a haze image to the atmospheric light value of that patch. It is the ratio of a local image patch in a fog-free image to the atmospheric light value of that image patch.

4. The multi-scene image enhancement method based on statistical prior as described in claim 1, characterized in that, Methods for determining the transmission map by fitting statistical lines include: Construct a statistical line model to integrate each local image patch in the haze image with the image after atmospheric light normalization. The statistical components and standard deviations are input into the statistical line model to obtain the transmission map.

5. The multi-scene image enhancement method based on statistical prior as described in claim 1, characterized in that, The method for optimizing the transmission map by introducing upper and lower regularization and a guided filter to obtain the optimized transmission map is as follows: in, This is the transmission image obtained after being constrained by the upper and lower boundaries. For color channels, It is a red channel. For green channel, For the blue channel, These are the atmospheric light values ​​of local images of haze images from different channels. These are partial images of haze from different channels. This represents the lower boundary of different color channels. The upper boundary of different color channels. For guided filtering operators; This is the optimized transmission map.

6. The multi-scene image enhancement method based on statistical prior as described in claim 1, characterized in that, The method for obtaining the final dehazed image based on the optimized transmission map and atmospheric light input to the atmospheric scattering model is as follows: in, For the final dehazed image, For haze images that need dehazing processing, Atmospheric light value, This is the optimized transmission map.

7. A system for a multi-scene image enhancement method based on statistical priors according to any one of claims 1-6, characterized in that, include: Haze image normalization module, perspective determination module, perspective optimization module, and dehaze image acquisition module; The haze image normalization module is used to input the haze image into the atmospheric scattering model and calculate the image after the haze image and atmospheric light are normalized. The perspective determination module is used to calculate the statistical component and standard deviation of each local image block in the normalized image after dividing the haze image by the atmospheric light value by the pixel value of the haze image, and fit a statistical line to determine the transmission map; wherein the statistical component is the ratio of the average value of the local pixels to the standard deviation, and the local pixels are the pixel values ​​of the haze image. The method for obtaining the statistical components and standard deviation of each local image patch in the haze image and the image after atmospheric light normalization is as follows: in, This is the ratio of the pixel mean to the standard deviation of a local image patch after atmospheric light normalization. This represents the average pixel value of a local image patch after atmospheric light normalization. This represents the pixel standard deviation of a local image patch after atmospheric light normalization. This represents the ratio of a local image patch in a haze image to the atmospheric light value of that patch. This is a transmission image. The intercept is... This represents the ratio of the pixel mean to the standard deviation of a local image patch in the processed, haze-free image. This is the ratio of a local image patch in a fog-free image to the atmospheric light value of that patch. Atmospheric light value, This refers to a 15x15 image block within the image; The perspective optimization module is used to introduce upper and lower regularization and guide filters to optimize the transmission map and obtain an optimized transmission map. The dehazing image acquisition module is used to input the optimized transmission map and atmospheric light into the atmospheric scattering model to obtain the final dehazing image.

Images