Atmospheric scattering degraded image restoration method and system based on scene depth information

Abstract

The invention discloses an atmospheric scattering degraded image restoration method based on scene depth information. The method comprises the following steps: acquiring a to-be-processed image; obtaining a first transmission image through dark channel prior hypothesis based on the to-be-processed image, wherein the first transmission image is a roughly estimated transmission image; obtaining scene depth relative estimation based on the to-be-processed image; obtaining a second transmission image based on the first transmission image and scene depth relative estimation, wherein the second transmission image is an optimized transmission image; and based on the second transmission image, the to-be-processed image obtains the scene radiance through an atmospheric scattering model. According to the method, the transmission image estimation of the dark channel prior hypothesis optimized by the method of optimizing the relative estimation of the scene depth can be obtained, the real scene information is recovered, the degraded image caused by atmospheric scattering is effectively recovered, and the method has the advantages of simplicity, convenience and flexibility in calculation. The invention further discloses an atmospheric scattering degraded image restoration system based on the scene depth information, electronic equipment and a computer readable storage medium.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and in particular to a method, system, device and computer storage medium for restoring an atmospheric scattering degraded image based on scene depth information. Background Art

[0002] As light propagates through the atmosphere, it is scattered by atmospheric particles, causing attenuation related to its propagation distance. This significantly degrades the quality of images and videos captured by computer systems, significantly compromising subsequent computer vision tasks. Therefore, compensating for the image quality degradation caused by atmospheric scattering through restoration or enhancement methods is of great significance in computer vision.

[0003] Currently, there are two mainstream approaches to compensate for the image quality degradation caused by atmospheric scattering: image enhancement-based methods and image restoration-based methods.

[0004] Image enhancement methods treat atmospheric scattering degradation as a low-contrast, low-saturation image enhancement problem. Representative methods include histogram equalization, Retinex-based algorithms, and image fusion-based algorithms. These algorithms can enhance low-contrast, low-saturation atmospheric scattering degradation images into high-contrast, high-saturation enhanced images. However, due to their lack of physical foundations, their stability is relatively poor, and the enhanced results cannot be guaranteed to be consistent with the actual scene, often resulting in over-enhancement.

[0005] The image restoration method introduces a priori assumptions through statistical analysis of natural images, and eliminates some unknowns from the atmospheric scattering model (Formula (1)(2)(3), where J is the scene radiance, t is the transmission map, β is the scattering coefficient, d is the distance, λ is the wavelength, A is the atmospheric light intensity, I is the low-quality image caused by atmospheric scattering, x is the pixel, and the scattering coefficient β is inversely proportional to the γth power of the wavelength λ, where 0≤γ≤4, and the value of γ is related to the size of atmospheric particles. The smaller the particle, the smaller the value.) and reversely solves the image quality degradation process caused by atmospheric scattering to restore the image. Representative algorithms include dark channel prior algorithm and color line prior algorithm. Due to its certain physical support, the image restoration method can often recover scene information from degraded images more stably than the image enhancement method.

[0006] I(x)=J(x)t(x)+A(1-t(x)) Formula (1)

[0007] t(x)=e -β(λ)d(x) Formula (2)

[0008]

[0009] The dark channel prior is one of the most widely used prior assumptions in image restoration methods degraded by atmospheric scattering. This prior assumes that, in an outdoor color natural image unaffected by atmospheric scattering, there will always be a pixel whose minimum value of the red, green, and blue channels approaches zero (i.e., at least one color channel at that pixel). Based on this assumption, a transmission map can be derived from an image degraded by atmospheric scattering, thereby restoring the scene's radiance.

[0010] However, statistical analysis of natural images shows that only approximately 70 percent of pixels in a natural image meet this assumption. For the remaining 30 percent of pixels that do not meet the assumption, dehazing based on the dark channel prior will lead to an inaccurate estimate of the transmission map, resulting in a distorted restoration result. While a large number of methods for restoring images degraded by atmospheric scattering based on the dark channel prior have emerged in recent years, this fundamental flaw of the dark channel prior assumption has remained unaddressed. Summary of the Invention

[0011] The purpose of the present invention is to solve the problem of erroneous estimation of transmission map and distortion of restoration result caused by defogging based on dark channel prior assumption.

[0012] In the first aspect, the present invention provides a method for restoring degraded images caused by atmospheric scattering based on scene depth information. The method can optimize the transmission map estimation of the dark channel prior hypothesis by optimizing the relative estimation method of the scene depth, restore the real scene information, and effectively restore the degraded image caused by atmospheric scattering. It has the advantages of simple and flexible calculation.

[0013] To solve the above technical problems, an embodiment of the present invention discloses a method for restoring an image degraded by atmospheric scattering based on scene depth information, comprising: obtaining an image to be processed; obtaining a first transmission map based on the image to be processed through a dark channel prior hypothesis, the first transmission map being a roughly estimated transmission map; obtaining a relative estimation of the scene depth based on the image to be processed; obtaining a second transmission map based on the first transmission map and the relative estimation of the scene depth, the second transmission map being an optimized transmission map; and obtaining the scene emissivity through an atmospheric scattering model based on the second transmission map and the image to be processed.

[0014] This technical solution optimizes the relative estimation of scene depth to optimize the first transmission map based on the dark channel prior hypothesis, obtaining an optimized second transmission map. This optimizes scene radiance and restores true scene information. This solution effectively addresses the 30 percent of pixels in natural scenes that do not satisfy the dark channel prior hypothesis, restoring images degraded by outdoor atmospheric scattering. Furthermore, optimizing scene radiance by optimizing the relative estimation of scene depth makes the calculations simple and convenient.

[0015] According to another specific embodiment of the present invention, obtaining a relative estimation of scene depth based on the image to be processed includes: obtaining a relative estimation of scene depth based on the image to be processed by a monocular depth estimation method.

[0016] According to another specific embodiment of the present invention, the monocular depth estimation method includes: at least one of a linear perspective method, an atmospheric scattering prior method, and a shadow plane method.

[0017] According to another specific embodiment of the present invention, the atmospheric scattering degraded image restoration method based on scene depth information also includes: obtaining a sky area and a non-sky area based on the image to be processed; and obtaining a second transmission map based on the sky area and the non-sky area.

[0018] According to another specific embodiment of the present invention, obtaining a second transmission map based on the sky area or the non-sky area includes: for all pixels in the non-sky area, obtaining a transmission map of the non-sky area according to all pixels in the non-sky area based on relative estimation of scene depth, and using the transmission map of the non-sky area as the second transmission map; for all pixels in the sky area, obtaining the minimum value of the transmission map of the non-sky area as the transmission map of the sky area, and using the transmission map of the sky area as the second transmission map.

[0019] According to another specific embodiment of the present invention, a sky area and a non-sky area are obtained based on the image to be processed, including: obtaining a first pixel area through a clustering method based on the first transmission map; obtaining a second pixel area based on a relative estimation of the scene depth; obtaining a sky area based on the first pixel area and the second pixel area; and other areas in the image to be processed except the sky area are non-sky areas.

[0020] According to another specific embodiment of the present invention, obtaining the first pixel area by a clustering method based on the first transmission image includes: using K-means clustering.

[0021] According to another specific embodiment of the present invention, K-means clustering is used, including: taking 0 as the first cluster center and 1 as the second cluster center; clustering all pixel points of the first transmission image according to the first cluster center and the second cluster center to divide all pixel points into a first group and a second group, and calculating the ratio of the first cluster center to the second cluster center: judging whether the ratio of the first cluster center to the second cluster center exceeds 2, and if so, recording a group of pixel points with smaller center point values ​​as a first pixel area; if not, taking the mean of the pixel points in the first group as the first cluster center, and taking the mean of the pixel points in the second group as the second cluster center, and repeatedly clustering all pixel points of the first transmission image according to the first cluster center and the second cluster center to divide all pixel points into a first group and a second group, and calculating the ratio of the first cluster center to the second cluster center until the first pixel area is obtained.

[0022] According to another specific embodiment of the present invention, in the first pixel area, a clustering method is obtained based on the first transmission map, and the clustering method includes: a partitioning method, a hierarchical method, a density-based method, a grid-based method, a model-based method, a transitive closure method, a Boolean matrix method, a direct clustering method, a correlation analysis clustering, and a statistics-based clustering method.

[0023] According to another specific embodiment of the present invention, obtaining a second pixel area based on the relative estimation of the scene depth includes: obtaining the median and standard deviation of the relative estimation of the scene depth, and recording the pixel points whose relative estimation of the scene depth is greater than the standard deviation of a preset multiple of the median as the second pixel area.

[0024] According to another specific embodiment of the present invention, obtaining the sky area based on the first pixel area and the second pixel area includes: obtaining a union of the first pixel area and the second pixel area as the sky area.

[0025] According to another specific embodiment of the present invention, obtaining the second transmission map based on the first transmission map and the relative estimation of scene depth includes: obtaining a scattering coefficient based on the first transmission map and the relative estimation of scene depth.

[0026] According to another specific embodiment of the present invention, a scattering coefficient is obtained based on a first transmission map and a relative estimation of scene depth, including: for all pixels in a non-sky area, calculating the average value of the ratio of the inverse natural logarithm of the first transmission map to the relative estimation of scene depth to obtain the scattering coefficient.

[0027] According to another specific embodiment of the present invention, the method for restoring an atmospheric scattering degraded image based on scene depth information also includes: obtaining atmospheric light intensity based on the sky area; obtaining scene emissivity through an atmospheric scattering model based on the second transmission map, the image to be processed, and the atmospheric light intensity.

[0028] According to another specific embodiment of the present invention, obtaining the atmospheric light intensity based on the sky area includes: calculating the average value of all pixels in the sky area to obtain the atmospheric light intensity.

[0029] According to another specific embodiment of the present invention, after obtaining the image to be processed, the method further includes regularizing the image to be processed.

[0030] According to another specific embodiment of the present invention, regularizing the image to be processed includes: dividing the image to be processed by the maximum value of pixels in the image to be processed.

[0031] According to another specific embodiment of the present invention, obtaining a first transmission map based on the image to be processed through a dark channel prior hypothesis includes: determining the minimum value of each pixel of the image to be processed in the red, green, and blue color channels; determining the mean value of a set of pixel points in a preset area of ​​the image to be processed; and obtaining the first transmission map based on the minimum value and mean value of each pixel in the R, G, and B channels.

[0032] According to another specific embodiment of the present invention, the brightness of pixels in the preset area is higher than the brightness of pixels in the non-preset area in the image to be processed.

[0033] According to another specific embodiment of the present invention, the pixel point set in the preset area is a set of the brightest 10% of pixel points in the image to be processed.

[0034] According to another specific embodiment of the present invention, after obtaining the scene radiance based on the second transmission map, the image to be processed, and the atmospheric light intensity, the method further includes: outputting the scene radiance.

[0035] In the second aspect, an embodiment of the present invention discloses an atmospheric scattering degradation image restoration system based on scene depth information, including: an image acquisition module for acquiring an image to be processed; a dark channel prior processing module for obtaining a first transmission map based on the image to be processed through a dark channel prior hypothesis, the first transmission map being a roughly estimated transmission map; a scene depth relative estimation module for obtaining a relative estimation of the scene depth based on the image to be processed; a transmission map optimization processing module for obtaining a second transmission map based on the first transmission map and the relative estimation of the scene depth, the second transmission map being an optimized transmission map; a scene radiance processing module for obtaining the scene radiance based on the second transmission map and the image to be processed through an atmospheric scattering model.

[0036] Using the above technical solution, the transmission map optimization processing module optimizes the scene radiance using the scene depth relative estimation module based on the first transmission map obtained by the dark channel prior processing module to obtain an optimized second transmission map. The scene radiance processing module then obtains the scene radiance based on the optimized second transmission map. The image acquisition module, dark channel prior processing module, scene depth relative estimation module, transmission map optimization processing module, and scene radiance processing module work together to optimize the first transmission map based on the dark channel prior hypothesis using the scene depth relative estimation to obtain the optimized second transmission map, thereby optimizing the scene radiance. This effectively addresses the 30 percent of pixels in natural scenes that do not meet the dark channel prior hypothesis, restoring true scene information and recovering images degraded by outdoor atmospheric scattering.

[0037] According to another specific embodiment of the present invention, the atmospheric scattering degradation image restoration system based on scene depth information also includes: a pixel area processing module, which is used to obtain a first pixel area through a clustering method based on the first transmission map, and obtain a second pixel area based on the relative estimation of the scene depth; a sky area acquisition module, which is used to obtain a sky area based on the first pixel area and the second pixel area.

[0038] According to another specific embodiment of the present invention, the atmospheric scattering degradation image restoration system based on scene depth information further includes: an atmospheric light intensity acquisition module for obtaining atmospheric light intensity based on a sky area.

[0039] According to another specific embodiment of the present invention, the atmospheric scattering degradation image restoration system based on scene depth information also includes: a scattering coefficient optimization module, which is used to calculate the average value of the ratio of the natural logarithm of the first transmission image to the relative estimated scene depth after taking the inverse of the natural logarithm of the first transmission image for all pixel points in the non-sky area to obtain the scattering coefficient.

[0040] According to another specific embodiment of the present invention, the atmospheric scattering degraded image restoration system based on scene depth information further includes: a regularization processing module, configured to perform regularization processing on the image to be processed.

[0041] According to another specific embodiment of the present invention, the atmospheric scattering degraded image restoration system based on scene depth information further includes: an image output module, configured to output scene radiance.

[0042] In a third aspect, an embodiment of the present invention discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, the method for restoring an atmospheric scattering-degraded image based on scene depth information in any embodiment of the first aspect is implemented.

[0043] Using the above technical solution, the electronic device optimizes the first transmission map of the dark channel prior hypothesis through relative estimation of scene depth to obtain an optimized second transmission map. Based on the second transmission map, it optimizes the scene emissivity, effectively processing the 30% of pixels in the natural scene that do not meet the dark channel prior hypothesis, restoring the real scene information, and restoring the outdoor atmospheric scattering degraded image.

[0044] In a fourth aspect, an embodiment of the present invention discloses a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the atmospheric scattering degradation image restoration method based on scene depth information in any embodiment of the first aspect.

[0045] By adopting the above technical solution, a computer-readable storage medium realizes the optimization of the first transmission map of the dark channel prior hypothesis through relative estimation of scene depth, obtains the optimized second transmission map, optimizes the scene emissivity based on the second transmission map, effectively processes the 30% of pixels in the natural scene that do not meet the dark channel prior hypothesis, restores the real scene information, and restores the outdoor atmospheric scattering degraded image. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 1 ;

[0047] Figure 2 The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 2 ;

[0048] Figure 3a Flowchart 3 showing a method for restoring an image degraded by atmospheric scattering based on scene depth information according to an embodiment of the present invention;

[0049] Figure 3b The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 4 ;

[0050] Figure 3c The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 5 ;

[0051] Figure 4 The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 6 ;

[0052] Figure 5 The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 7 ;

[0053] Figure 6 The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 8 ;

[0054] Figure 7 The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 9 ;

[0055] Figure 8 The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 10 ;

[0056] Figure 9 The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 10 one;

[0057] Figure 10 The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 10 two;

[0058] Figure 11 The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 10 three;

[0059] Figure 12 The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 10 Four;

[0060] Figure 13 The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 10 five;

[0061] Figure 14 The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 10 six;

[0062] Figure 15 The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 10 seven;

[0063] Figure 16 The process of the atmospheric scattering degradation image restoration method based on scene depth information in an embodiment of the present invention is shown as follows Figure 10 eight;

[0064] Figure 17 Schematic diagram showing the structure of the atmospheric scattering degradation image restoration system based on scene depth information in an embodiment of the present invention Figure 1 ;

[0065] Figure 18 Schematic diagram showing the structure of the atmospheric scattering degradation image restoration system based on scene depth information in an embodiment of the present invention Figure 2 ;

[0066] Figure 19 A third structural diagram of a system for restoring an image degraded by atmospheric scattering based on scene depth information according to an embodiment of the present invention is shown;

[0067] Figure 20Schematic diagram showing the structure of the atmospheric scattering degradation image restoration system based on scene depth information in an embodiment of the present invention Figure 4 ;

[0068] Figure 21 Schematic diagram showing the structure of the atmospheric scattering degradation image restoration system based on scene depth information in an embodiment of the present invention Figure 5 ;

[0069] Figure 22 Schematic diagram showing the structure of the atmospheric scattering degradation image restoration system based on scene depth information in an embodiment of the present invention Figure 6 ;

[0070] Figure 23 A schematic structural diagram of an electronic device for restoring an image degraded by atmospheric scattering based on scene depth information in an embodiment of the present invention is shown;

[0071] Figure 24 shows an image to be processed in an embodiment of the present invention;

[0072] Figure 25 shows a first perspective view in an embodiment of the present invention;

[0073] Figure 26 shows a second perspective view in an embodiment of the present invention;

[0074] Figure 27 A scene radiance image in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0075] The following is an explanation of the embodiments of the present invention by specific specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Although the description of the present invention will be introduced in conjunction with the preferred embodiment, this does not mean that the features of this invention are limited to this embodiment. On the contrary, the purpose of introducing the invention in conjunction with the embodiment is to cover other options or modifications that may be extended based on the claims of the present invention. In order to provide a deep understanding of the present invention, the following description will include many specific details. The present invention can also be implemented without using these details. In addition, in order to avoid confusion or blurring the focus of the present invention, some specific details will be omitted in the description. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0076] It should be noted that in this specification, similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0077] The terms “first”, “second”, etc. are only used for distinguishing descriptions and should not be understood as indicating or implying relative importance.

[0078] In the description of this embodiment, it should be noted that, unless otherwise specified or limited, the terms "disposed," "connected," and "connected" should be understood broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in this embodiment based on specific circumstances.

[0079] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0080] In its first aspect, the present invention provides a method for restoring an image degraded by atmospheric scattering based on scene depth information. All embodiments of the present invention utilize floating-point numbers for calculations. Compared to integers, floating-point numbers can represent values ​​between integers, and their range of representation is larger than that of integers. Using floating-point numbers for calculations avoids precision loss due to rounding.

[0081] The atmospheric scattering model (Formula (1) (2) (3)) mentioned in the present invention is:

[0082] I(x)=J(x)t(x)+A(1-t(x)) (1)

[0083] t(x)=e -β(λ)d(x) (2)

[0084]

[0085] Where J(x) is the scene radiance, t(x) is the transmission map, β is the scattering coefficient, d(x) is the distance (i.e., scene depth), λ is the wavelength, A is the atmospheric light intensity, I(x) is the low-quality image (i.e., the image to be processed) caused by atmospheric scattering, x is the pixel, and the scattering coefficient β is inversely proportional to the wavelength λ raised to the power of γ, where 0≤γ≤4. The value of γ is related to the size of the atmospheric particles, and the smaller the particle, the smaller the value.

[0086] refer to Figure 1 The present invention provides a method for restoring an image degraded by atmospheric scattering based on scene depth information, comprising:

[0087] S1: Obtain the image to be processed I(x).

[0088] As mentioned above, the image to be processed is represented by I(x) in the atmospheric scattering model.

[0089] The image to be processed is a degraded image. For example, Figure 24 The image shown is a blurry image captured under hazy weather.

[0090] In this embodiment, the image to be processed may be an existing image obtained from a database, or an instant image obtained by shooting.

[0091] S2: Obtain the first transmission image based on the image to be processed I(x) through the dark channel prior assumption First transmission image is a rough estimate of the transmission map.

[0092] As mentioned above, the transmission map is represented by t(x) in the atmospheric scattering model. Here, the first transmission map is a rough estimated transmission map, and express.

[0093] For example, Figure 25 The first transmission diagram is shown

[0094] The dark channel prior hypothesis algorithm is an image prior algorithm in computer vision, which aims to improve the effect of image dehazing. In this algorithm, the dark channel information of the image is analyzed and combined with the atmospheric light intensity estimation to calculate the scattering coefficient in the original image and further obtain the dehazed image.

[0095] The dark channel prior hypothesis refers to the minimum value of all pixels within a local area. In other words, the darkest pixel value within a region is called the dark channel of that region. The smaller the dark channel value of a region, the greater the probability that the region is similar in color to fog. By analyzing the dark channel of the image to be processed, the location and magnitude of atmospheric light intensity can be accurately estimated. In the dark channel prior algorithm, it is assumed that for each pixel in the image to be processed, at least one of the three channels (R (red channel), G (green channel), and B (blue channel)) has a maximum value. This maximum value is the atmospheric light intensity of that pixel.

[0096] In this embodiment, the image to be processed is processed using an existing dark channel a priori hypothesis algorithm to obtain a first transmission map. Only approximately 70 percent of the pixels in the first transmission map satisfy the dark channel a priori hypothesis, while 30 percent do not. Therefore, dehazing the image to be processed using the dark channel a priori hypothesis algorithm will result in an inaccurate estimate of the first transmission map, resulting in a distorted restoration result.

[0097] S3: Obtain relative scene depth estimation based on the image to be processed I(x)

[0098] As mentioned above, the scene depth is expressed as d(x) in the atmospheric scattering model, and the scene depth relative estimation is used here. express.

[0099] For example, Figure 2 As shown, step S3 includes: S31: obtaining a relative estimation of scene depth based on the image to be processed I(x) by a monocular depth estimation method

[0100] For example, relative estimation of scene depth Available via the ZoeDepth method:

[0101]

[0102] Among them, p x (k) is the probability weight of the pixel depth container k, c x (k) is the center value of the pixel depth container k, N total is the total number of depth containers for this pixel.

[0103] The distance information from each point in the scene to the camera is obtained. This distance information is called a depth map. Monocular depth estimation uses a single RGB image or a single viewpoint to estimate the distance of each pixel in the image relative to the source, thereby inferring three-dimensional space from a two-dimensional image.

[0104] Through the above technical solution, the relative estimation of scene depth is obtained from a single image through the monocular depth estimation method. Be reliable.

[0105] S4: Based on the first transmission map Relative scene depth estimation A second transmission map t(x) is obtained, and the second transmission map t(x) is the optimized transmission map.

[0106] In this embodiment, by optimizing the scene depth relative estimation The first transmission image processed by the dark channel prior hypothesis Further optimization, specifically optimization of the first transmission map Thirty percent of the pixels in the dark channel prior hypothesis algorithm are still not satisfied, thereby obtaining the second transmission map t(x). In other words, the second transmission map t(x) is estimated by the relative depth of the scene For the first transmission image Optimized transmission map after processing.

[0107] S5: Based on the second transmission map t(x) and the image to be processed I(x), the scene radiance J(x) is obtained by using formula (1) of the atmospheric scattering model.

[0108] I(x)=J(x)t(x)+A(1-t(x)) Formula (1)

[0109] In this embodiment, the second transmission image t(x) is obtained by relative estimation of scene depth. The optimized transmission map, the scene radiance J(x) obtained from the second transmission map t(x), is an image that restores the true scene information. The processed image I(x) is also called the degraded image, and the scene radiance J(x) is the restored image after removing the degradation factors.

[0110] This technical solution obtains a reliable scene depth estimate from a single image. By optimizing the relative scene depth estimate, the first transmission map based on the dark channel prior hypothesis is optimized, resulting in an optimized second transmission map. This optimizes the scene radiance and restores true scene information. This technical solution effectively addresses the 30 percent of pixels in natural scenes that do not satisfy the dark channel prior hypothesis, enabling the restoration of images degraded by outdoor atmospheric scattering. Furthermore, optimizing the scene radiance by optimizing the relative scene depth estimate is computationally simple and convenient.

[0111] refer to Figure 2 Combined with Figure 1 In some possible embodiments provided by the present invention, S3: obtaining a relative estimation of scene depth based on the image to be processed I(x) include:

[0112] S31: Obtain relative scene depth estimation based on the image to be processed I(x) using a monocular depth estimation method

[0113] Depth estimation involves obtaining the distance from each point in the scene to the camera. This distance information is called a depth map. Monocular depth estimation uses a single RGB image or a single viewpoint to estimate the distance of each pixel in the image relative to the source, thereby inferring three-dimensional space from a two-dimensional image.

[0114] In some possible embodiments provided by the present invention, the monocular depth estimation method includes at least one of a linear perspective method, an atmospheric scattering prior method, and a shadow plane method.

[0115] refer to Figure 3a 、 Figure 3b and Figure 3c In some possible embodiments provided by the present invention, the method for restoring an image degraded by atmospheric scattering based on scene depth information further includes:

[0116] S6: Obtain the sky area Γ based on the image to be processed I(x) sky and non-sky regions;

[0117] S8: Based on sky region Γ sky The second transmission map t(x) is obtained for the non-sky area.

[0118] In this embodiment, the sky area Γ is divided into sky The second transmission map t(x) is obtained from the non-sky area, so that the subsequently obtained scene radiance J(x) is better and the degraded image I(x) is further restored.

[0119] refer to Figure 4 and combined Figure 3a 、 Figure 3b and Figure 3c In some possible embodiments provided by the present invention, S8: obtaining a second transmission map t(x) based on the sky area and the non-sky area includes:

[0120] S81: For all pixels in the non-sky area, relative estimation based on scene depth The transmission map of the non-sky area is obtained based on all the pixels in the non-sky area, and the transmission map of the non-sky area is used as the second transmission map t(x). The calculation formula is:

[0121]

[0122] Using the above technical solution, the transmission map of the non-sky area is based on the optimized scene depth relative estimation Acquisition makes the subsequently acquired scene radiance J(x) better, and can achieve better restoration of the degraded image I(x).

[0123] S82: For all pixels in the sky area, obtain the minimum value of the transmission map of the non-sky area as the transmission map of the sky area, and use the transmission map of the sky area as the second transmission map t(x). The calculation formula is:

[0124]

[0125] That is, combining step S81 and step S82, the calculation formula of the second transmission image t(x) is:

[0126]

[0127] Using the above technical solution, the transmission map of the sky area and the non-sky area (i.e., the second transmission map t(x)) is a relative estimation of the scene depth based on the optimization Acquisition makes the subsequently acquired scene radiance J(x) better, and can achieve better restoration of the degraded image I(x).

[0128] refer to Figure 5 and combined Figure 3a 、 Figure 3b and Figure 3c In some possible embodiments provided by the present invention, S6: obtaining a sky area Γ based on the image to be processed I(x) sky and non-sky areas, including:

[0129] S61: Based on the first transmission image The first pixel region Γ is obtained by clustering method t ;

[0130] In this embodiment, a clustering method is used according to the first transmission map. The similarity between the pixels in is used to divide the pixels into two different categories.

[0131] S62: Based on scene depth relative estimation Get the second pixel area Γ d ; S63: Based on the first pixel area Γ t , the second pixel area Γ d Get the sky area Γ sky ;

[0132] S64: excluding the sky area Γ in the image to be processed I(x) sky The rest of the area is the non-sky area.

[0133] In this embodiment, the optimized scene depth relative estimation is used in the division of sky area and non-sky area. This makes the subsequently acquired scene radiance J(x) better and further restores the degraded image I(x).

[0134] For example, refer to Figure 6 and combined Figure 5 In some possible embodiments provided by the present invention, S61: based on the first transmission image The first pixel region Γ is obtained by clustering method t , including: S610: based on the first transmission image The first pixel region Γ is obtained by using the K-means clustering method t .

[0135] refer to Figure 7 and combined Figure 6 In some possible embodiments provided by the present invention, S610: using K-means clustering includes:

[0136] S6101: 0 is used as the first cluster center and 1 is used as the second cluster center;

[0137] S6102: The first transmission image is analyzed based on the first cluster center and the second cluster center. Clustering all pixel points to divide all pixel points into the first group and the second group;

[0138] S6103: Taking the mean of the pixels in the first group as the first cluster center, taking the mean of the pixels in the second group as the second cluster center, and performing clustering on the first transmission image. Re-cluster all pixel points to divide all pixel points into the third group and the fourth group;

[0139] S6104: Determine whether the ratio of the first cluster center to the second cluster center exceeds 2.

[0140] S6105: If yes, then record a group of pixels with smaller center point values ​​as the first pixel region Γ t ;

[0141] S6106: If not, the first pixel region Γ t is an empty set; taking the third group as the first group and the fourth group as the second group, and repeating step S6103 until the first pixel area Γ is obtained. t .

[0142] The above technical solution is adopted. Based on the characteristics of K-means algorithm, which is one of the clustering algorithms with the lowest computational complexity, K-means algorithm is relatively simple, easy to understand and implement, K-means algorithm is applicable to large data sets and can perform large-scale data classification, K-means clustering results are easy to interpret, etc., based on the first transmission map Obtaining the first pixel region through the K-means clustering algorithm has the advantages of being fast, easy to understand and implement, applicable to large data sets, and interpretable results.

[0143] In some other possible embodiments provided by the present invention, S61: based on the first transmission image The first pixel region Γ is obtained by clustering method t (like Figure 5 As shown), clustering methods include: partitioning method, hierarchical method, density-based method, grid-based method, model-based method, transitive closure method, Boolean matrix method, direct clustering method, correlation analysis clustering, and one of the statistical-based clustering methods.

[0144] In order to make the predicted scene depth and the real scene depth consistent in magnitude, the median of the two scene depths is generally used as the scale to shrink the predicted scene depth information.

[0145] refer to Figure 8 Combined with Figure 5 In some possible embodiments provided by the present invention, S62: Based on the relative estimation of scene depth Get the second pixel area Γd ,include:

[0146] S620: Obtaining relative scene depth estimation The median and standard deviation of the scene depth relative estimation The pixel points with a standard deviation greater than the preset multiple of the median are recorded as the second pixel area Γ d .

[0147] In this embodiment, the scene depth is relatively estimated The median and standard deviation of are used to find the point with the largest depth in the scene as the possible sky area.

[0148] For example, the preset multiple is 5, that is, the scene depth is estimated relative to The pixel points with a standard deviation greater than 5 times the median are recorded as the second pixel area Γ d .

[0149] refer to Figure 9 and combined Figure 5 In some possible embodiments provided by the present invention, S63: based on the first pixel area Γ t , the second pixel area Γ d Get the sky area Γ sky ,include:

[0150] S630: Acquire the first pixel region Γ t , the second pixel area Γ d The union of sky , the calculation formula is:

[0151] Γ sky =Γ t ∪Γ d Formula (6)

[0152] In this embodiment, the first pixel region Γ t From the first transmission diagram Estimated sky area, second pixel area Γ d is estimated relative to the scene depth The estimated sky area is taken as the union of the two to ensure that the possible sky area is found as much as possible.

[0153] Based on relative estimation of scene depth Optimize the acquisition of the second pixel area Γ d , and based on the second pixel area Γ d Get the sky area Γ sky That is, by relative estimation of scene depth Optimize the acquisition of sky area Γ sky , and then based on the optimized sky area Γsky Obtain a better scene radiance J(x) and further restore the degraded image I(x).

[0154] refer to Figure 10 and combined Figure 1 In some possible embodiments provided by the present invention, S4: based on the first transmission image Relative scene depth estimation Obtaining a second transmission map t(x) includes:

[0155] S41: Based on the first transmission image Relative scene depth estimation Obtain the scattering coefficient β.

[0156] refer to Figure 11 and combined Figure 10 In some possible embodiments provided by the present invention, S41: based on the first transmission image Relative scene depth estimation Obtain the scattering coefficient β, including:

[0157] S411: Calculate the first transmission map for all pixels in the non-sky area After taking the inverse of the natural logarithm, it is estimated relative to the scene depth. The average value of the ratio is used to obtain the scattering coefficient β. The calculation formula is:

[0158]

[0159] In this embodiment, compared with step S4: based on the first transmission image Relative scene depth estimation To obtain the second transmission map t(x), the scene depth is relatively estimated based on Optimize and obtain the scattering coefficient β, and based on the scattering coefficient β, pass step S81: for all pixels in the non-sky area, based on the scene depth relative estimation The transmission map of the non-sky area is obtained according to all the pixels in the non-sky area, and the transmission map of the non-sky area is used as the second transmission map t(x) (e.g. Figure 4 As shown). Through formula (9) The transmission map of the non-sky area is optimized as the second transmission map t(x). Subsequently, a better scene radiance J(x) is obtained based on the optimized second transmission map t(x), and the degraded image I(x) is further restored.

[0160] refer to Figure 3a 、 Figure 3b and Figure 3c In some possible embodiments provided by the present invention, the method for restoring an image degraded by atmospheric scattering based on scene depth information further includes:

[0161] S7: Based on sky region Γ sky Obtain atmospheric light intensity A;

[0162] S9: Obtain the scene radiance J(x) through the atmospheric scattering model based on the second transmission map t(x), the image to be processed I(x), and the atmospheric light intensity A.

[0163] In this embodiment, the atmospheric light intensity A is obtained based on the sky area Γ sky Therefore, step S7 must be located in step S6: obtaining the sky area Γ based on the image to be processed sky At the same time, the acquisition of the second transmission map t(x) is also based on the sky area Γ sky Therefore, step S8: based on the sky area Γ sky Obtaining the second transmission map t(x) for the non-sky area must occur after step S6. However, the placement of steps S7 and S8 is not limited. Furthermore, the scene radiance J(x) in step S9 is obtained based on the second transmission map t(x) and the atmospheric light intensity A, so both steps S7 and S8 precede step S9.

[0164] For example, Figure 3a As shown, step S7 and step S8 are both located after step S6, and step S7 and step S8 are processed in parallel, and step S9 is located after step S7 and step S8; as shown in FIG. Figure 3b As shown, step S7 and step S8 are both located after step S6, and step S7 is located after step S8, and step S9 is located after step S7; Figure 3c As shown, both step S7 and step S8 are located after step S6, step S7 is located before step S8, and step S9 is located after step S8.

[0165] refer to Figure 12 3, in some possible embodiments provided by the present invention, S7: based on the sky area Γ sky Obtain atmospheric light intensity A, including:

[0166] S71: Calculate the sky area Γ sky The average value of all pixels is used to obtain the atmospheric light intensity A, which is calculated as follows:

[0167]

[0168] In this embodiment, based on the relative estimation of scene depth Optimize the acquisition of the sky area and obtain the atmospheric light intensity A based on the sky area, realizing the relative estimation of scene depth. Based on this, the atmospheric light intensity A is further optimized, and then a better scene radiance J(x) is obtained based on the optimized atmospheric light intensity A, and the degraded image I(x) is further restored.

[0169] refer to Figure 13 and combined Figure 1 In some other possible embodiments provided by the present invention, after S1: obtaining the image to be processed I(x), the method for restoring an image degraded by atmospheric scattering based on scene depth information further includes:

[0170] S10: Perform regularization processing on the image to be processed I(x).

[0171] refer to Figure 14 and combined Figure 13 In this embodiment, S10: regularizing the image to be processed I(x) includes:

[0172] S101: Divide the image to be processed by the maximum value of the pixels in the image to be processed to obtain a regularized image to be processed.

[0173] The calculation formula is:

[0174] I(x)=I(x) / MAX(I(x)) Formula (4)

[0175] The above technical solution is adopted to ensure that the maximum value of the image to be processed I(x) does not exceed 1 in subsequent calculations, thereby simplifying the calculations.

[0176] refer to Figure 15 and combined Figure 1 In some other possible embodiments provided by the present invention, S2: obtaining a first transmission image based on the image to be processed I(x) by a dark channel priori assumption include:

[0177] S21: Determine the minimum value of each pixel of the image to be processed I(x) in the three color channels of red, green and blue.

[0178] As mentioned above, the dark channel prior hypothesis refers to the minimum value of all pixels in a local area.

[0179] S22: Determine the mean value of a set of pixel points in a preset area in the image to be processed I(x).

[0180] As previously mentioned, in the dark channel prior algorithm, it is assumed that for each pixel in the image to be processed, at least one of the three channels (R (red channel), G (green channel), and B (blue channel)) has a maximum value. This maximum value is the atmospheric light intensity of that pixel. The mean value obtained in step S22 is the atmospheric light intensity of the image to be processed I(x).

[0181] S23: Obtain a first transmission map based on the minimum value and average value of each pixel in the three channels R, G, and B

[0182] In this embodiment, the value of the red channel of the image to be processed I(x) at pixel x is I r (x), the value of the green channel is 1 g (x), the value of the blue channel is I b (x).

[0183] Let the image to be processed be centered around pixel x and the surrounding 15×15 pixel area be Ω(x), let the set of the brightest 10% pixels in the image to be processed I(x) be z, and the mean of these pixels be I r (y) is the minimum value of the pixel of the red channel, I g (y) is the minimum value of the pixel of the green channel, I b (y) is the minimum value of the pixel in the blue channel.

[0184] First transmission image The calculation formula is:

[0185]

[0186] Using the above technical solution, the first transmission image is obtained based on the dark channel prior hypothesis based on the image to be processed I(x)

[0187] In some possible embodiments provided by the present invention, the brightness of pixels in the preset area is higher than the brightness of pixels in the non-preset area in the image to be processed I(x).

[0188] In this embodiment, the pixel set of the preset area is the set of the brightest 10% of pixels in the image to be processed I(x). In some other possible embodiments, the pixel set of the preset area may be the set of the brightest 0.1% of pixels in the image to be processed I(x).

[0189] refer to Figure 16 In some possible embodiments provided by the present invention, after S9: obtaining the scene radiance J(x) based on the second transmission image t(x), the image to be processed I(x), and the atmospheric light intensity A, the process further includes:

[0190] S11: Outputting the scene radiance J(x). That is, outputting an image that has restored the true scene information or outputting a restored image after removing degradation factors.

[0191] By adopting the above technical solution, the atmospheric scattering degraded image restoration method based on scene depth information provided by the present invention realizes the effective restoration of the degraded image (ie, the image to be processed I(x)) caused by atmospheric scattering.

[0192] The following are preferred embodiments of the present invention:

[0193] refer to Figure 1 , execute step S1: obtain the image to be processed I(x).

[0194] For example, Figure 24 The image shown is a blurry image captured under hazy weather.

[0195] refer to Figure 13 , execute step S10: perform regularization processing on the image to be processed I(x).

[0196] refer to Figure 14 Combined with Figure 13 , executing step S101: dividing the image to be processed by the maximum value of the pixels in the image to be processed to obtain a regularized image to be processed.

[0197] The calculation formula is:

[0198] I(x)=I(x) / MAX(I(x)) Formula (4)

[0199] refer to Figure 1 , execute step S2: obtain the first transmission image based on the image to be processed I(x) through the dark channel prior hypothesis First transmission image is a rough estimate of the transmission map.

[0200] refer to Figure 15 , execute step S21: determine the minimum value of each pixel of the image to be processed I(x) in the three color channels of red, green and blue.

[0201] In this embodiment, the value of the red channel of the image to be processed I(x) at pixel x is I r (x), the value of the green channel is 1 g (x), the value of the blue channel is I b (x).

[0202] Execute step S22: determine the mean value of a set of pixel points in a preset area in the image to be processed I(x).

[0203] Let the image to be processed be centered around pixel x and the surrounding 15×15 pixel area be Ω(x), let the set of the brightest 10% pixels in the image to be processed I(x) be z, and the mean of these pixels be

[0204] Execute step S23: obtain the first transmission map based on the minimum value and average value of each pixel in the three channels R, G, and B

[0205] I r (y) is the minimum value of the pixel of the red channel, I g (y) is the minimum value of the pixel of the green channel, I b (y) is the minimum value of the pixel in the blue channel.

[0206] First transmission image The calculation formula is:

[0207]

[0208] like Figure 1 As shown, step S3 is executed: obtaining a relative estimation of scene depth based on the image to be processed I(x)

[0209] like Figure 2 As shown, step S31 is executed: a relative estimation of scene depth is obtained based on the image to be processed I(x) by a monocular depth estimation method.

[0210] For example, the scene depth relative estimation is obtained by the ZoeDepth method

[0211]

[0212] Among them, p x (k) is the probability weight of the pixel depth container k, c x (k) is the center value of the pixel depth container k, N total is the total number of depth containers for this pixel.

[0213] like Figure 3a 、 Figure 3b and Figure 3c As shown, step S6 is executed: obtaining the sky area Γ based on the image to be processed I(x) sky and non-sky areas.

[0214] like Figure 5 As shown, step S61 is executed: based on the first transmission image The first pixel region Γ is obtained by clustering method t ;

[0215] like Figure 6 As shown, step S610 is executed: based on the first transmission image The first pixel region Γ is obtained by using the K-means clustering method t .

[0216] refer to Figure 7 , execute step S6101: use 0 as the first cluster center and 1 as the second cluster center;

[0217] S6102: The first transmission image is analyzed based on the first cluster center and the second cluster center. Clustering all pixel points to divide all pixel points into the first group and the second group;

[0218] S6103: Taking the mean of the pixels in the first group as the first cluster center, taking the mean of the pixels in the second group as the second cluster center, and performing clustering on the first transmission image. Re-cluster all pixel points to divide all pixel points into the third group and the fourth group;

[0219] S6104: Determine whether the ratio of the first cluster center to the second cluster center exceeds 2.

[0220] S6105: If yes, then record a group of pixels with smaller center point values ​​as the first pixel region Γ t ;

[0221] S6106: If not, the first pixel region Γ t is an empty set; taking the third group as the first group and the fourth group as the second group, and repeating step S6103 until the first pixel area Γ is obtained. t .

[0222] refer to Figure 5 , execute step S62: based on the relative estimation of scene depth Get the second pixel area Γ d .

[0223] refer to Figure 8 , execute step S620: obtain relative scene depth estimation The median and standard deviation of the scene depth relative estimation The pixel points with a standard deviation greater than the preset multiple of the median are recorded as the second pixel area Γ d .

[0224] refer to Figure 5 , execute step S63: based on the first pixel area Γ t , the second pixel area Γ d Get the sky area Γ sky .

[0225] refer to Figure 9 , execute step S630: obtain the first pixel area Γ t , the second pixel area Γ d The union ofsky , the calculation formula is:

[0226] Γ sky =Γ t ∪Γ d Formula (6)

[0227] refer to Figure 5 , execute step S64: remove the sky area Γ from the image to be processed I(x) sky The rest of the area is the non-sky area.

[0228] refer to Figure 3a , execute step S7: based on the sky area Γ sky Obtain the atmospheric light intensity A.

[0229] refer to Figure 12 , execute step S71: calculate the sky area Γ sky The average value of all pixels is used to obtain the atmospheric light intensity A, which is calculated as follows:

[0230]

[0231] refer to Figure 1 , execute step S4: based on the first transmission image Relative scene depth estimation A second transmission map t(x) is obtained, and the second transmission map t(x) is the optimized transmission map.

[0232] refer to Figure 11 , execute step S411: calculate the first transmission map for all pixels in the non-sky area After taking the inverse of the natural logarithm, it is estimated relative to the scene depth. The average value of the ratio is used to obtain the scattering coefficient β. The calculation formula is:

[0233]

[0234] refer to Figure 3a , while executing step S7, execute step S8: based on the sky area Γ sky The second transmission map t(x) is obtained for the non-sky area.

[0235] refer to Figure 4 , execute step S81: for all pixels in the non-sky area, based on the relative estimation of scene depth The transmission map of the non-sky area is obtained based on all the pixels in the non-sky area, and the transmission map of the non-sky area is used as the second transmission map t(x). The calculation formula is:

[0236]

[0237] S82: For all pixels in the sky area, obtain the minimum value of the transmission map of the non-sky area as the transmission map of the sky area, and use the transmission map of the sky area as the second transmission map t(x). The calculation formula is:

[0238]

[0239] That is, combining step S81 and step S82, the calculation formula of the second transmission image t(x) is:

[0240]

[0241] For example, the second transmission image t(x) is as follows: Figure 26 shown.

[0242] refer to Figure 3a , execute step S9: obtain the scene radiance J(x) through the atmospheric scattering model based on the second transmission map t(x), the image to be processed I(x), and the atmospheric light intensity A.

[0243] I(x)=J(x)t(x)+A(1-t(x)) Formula (1)

[0244] For example, Figure 27 The scene radiance J(x) in this embodiment is shown.

[0245] refer to Figure 16 , executing step S11: outputting the scene radiance J(x). That is, outputting an image that has restored the real scene information or outputting a restored image after removing degradation factors.

[0246] The method for restoring degraded images from atmospheric scattering based on scene depth information provided by the present invention has the advantages of simple computation and flexibility. Unlike previous methods based on dark channel priors, the method not only restores pixels in outdoor color natural images where the minimum values ​​of the red, green, and blue channels approach zero in the surrounding local area, but also correctly processes the minority of pixels in the scene that do not meet the dark channel prior assumption, obtaining low-distortion images that reflect real scene information and facilitating subsequent computer vision tasks.

[0247] Second, reference Figure 17 The present invention provides an atmospheric scattering degradation image restoration system 1 based on scene depth information, including: an image acquisition module 11, a dark channel priori processing module 12, a scene depth relative estimation module 13, a transmission map optimization processing module 14 and a scene radiance processing module 15.

[0248] Among them, the image acquisition module 11 (data output end) is connected to the dark channel prior processing module 12 (data input end), the dark channel prior processing module 12 (data output end) is connected to the scene depth relative estimation module 13 (data input end), the scene depth relative estimation module 13 (data output end) is connected to the transmission map optimization processing module 14 (data input end), and the transmission map optimization processing module 14 (data output end) is connected to the scene radiance processing module 15 (data input end).

[0249] Continue to refer Figure 17 The image acquisition module 11 is used to acquire the image to be processed. The dark channel priori processing module 12 is used to obtain a first transmission map based on the image to be processed using a dark channel priori hypothesis. The first transmission map is a roughly estimated transmission map. The scene depth relative estimation module 13 is used to obtain a relative estimation of the scene depth based on the image to be processed. The transmission map optimization processing module 14 is used to obtain a second transmission map based on the first transmission map and the relative estimation of the scene depth. The second transmission map is an optimized transmission map. The scene radiance processing module 15 is used to obtain the scene radiance based on the second transmission map and the image to be processed using an atmospheric scattering model.

[0250] Using the above technical solution, the transmission map optimization processing module 14 optimizes the scene radiance based on the first transmission map obtained by the dark channel prior processing module 12 through the scene depth relative estimation module 13 to obtain an optimized second transmission map. The scene radiance processing module 15 then obtains the scene radiance based on the optimized second transmission map. The image acquisition module 11, the dark channel prior processing module 12, the scene depth relative estimation module 13, the transmission map optimization processing module 14, and the scene radiance processing module 15 cooperate with each other to optimize the first transmission map based on the dark channel prior hypothesis using the scene depth relative estimation to obtain the optimized second transmission map, thereby optimizing the scene radiance. This effectively processes the 30 percent of pixels in natural scenes that do not meet the dark channel prior hypothesis, restores true scene information, and recovers images degraded by outdoor atmospheric scattering.

[0251] refer to Figure 18 In some possible embodiments of the present invention, a system 01 for restoring an image degraded by atmospheric scattering based on scene depth information is provided. In addition to the image acquisition module 11, dark channel prior processing module 12, scene depth relative estimation module 13, transmission map optimization processing module 14, and scene radiance processing module 15 in the previous embodiment, the system further includes a pixel region processing module 21 and a sky region acquisition module 22.

[0252] In the previous embodiment, the pixel region processing module 21 (the data input end) is connected to the scene depth relative estimation module 13 (the data output end), the pixel region processing module 21 (the data output end) is connected to the sky region acquisition module 22 (the data input end), and the sky region acquisition module 22 (the data output end) is connected to the transmission map optimization processing module 14 (the data input end).

[0253] In this embodiment, the pixel region processing module 21 is used to obtain a first pixel region by clustering based on the first transmission map, and obtain a second pixel region based on relative scene depth estimation. The sky region acquisition module 22 is used to obtain a sky region based on the first pixel region and the second pixel region.

[0254] Using the above technical solution, the pixel area processing module 21 estimates the scene depth relative to the scene depth. Optimize the acquisition of sky area Γ sky , and then the subsequent scene radiance processing module 15 is based on the optimized sky area Γ sky Obtain a better scene radiance J(x) and further restore the degraded image I(x).

[0255] refer to Figure 19 In some possible embodiments of the present invention, a system 03 for restoring an image degraded by atmospheric scattering based on scene depth information is provided. In addition to the image acquisition module 11, dark channel priori processing module 12, scene depth relative estimation module 13, transmission map optimization processing module 14, scene radiance processing module 15, pixel region processing module 21, and sky region acquisition module 22 of the previous embodiment, the system further includes an atmospheric light intensity acquisition module 31.

[0256] In the above embodiment, the atmospheric light intensity acquisition module 31 (data input end) is connected to the sky area acquisition module 22 (data output end), and the atmospheric light intensity acquisition module 31 (data output end) can be connected to the scene radiance image processing module 15 (data input end) (e.g. Figure 20 shown).

[0257] In this embodiment, the atmospheric light intensity acquisition module 31 is used to obtain the atmospheric light intensity based on the sky area.

[0258] Using the above technical solution, the sky area acquisition module 22 estimates the scene depth relative to the scene depth. Optimize the acquisition of the sky area, the atmospheric light intensity acquisition module 31 acquires the atmospheric light intensity A based on the sky area, and realizes the relative estimation of the scene depth. Based on this, the atmospheric light intensity A is further optimized, and then the subsequent scene radiance image processing module 15 obtains a better scene radiance J(x) based on the optimized atmospheric light intensity A, and further restores the degraded image I(x).

[0259] refer to Figure 20 In some possible embodiments provided by the present invention, an atmospheric scattering degradation image restoration system 04 based on scene depth information is provided. In addition to including the image acquisition module 11, dark channel prior processing module 12, scene depth relative estimation module 13, transmission map optimization processing module 14, scene radiance processing module 15, pixel area processing module 21, sky area acquisition module 22 and atmospheric light intensity acquisition module 31 in the previous embodiment, it also includes: a scattering coefficient optimization module 41.

[0260] On the basis of the previous embodiment, the scattering coefficient optimization module 41 (the data input end) is connected to the sky area acquisition module 22 (the data output end), and the scattering coefficient optimization module 41 (the data output end) is connected to the transmission map optimization processing module 14 (the data input end).

[0261] In this embodiment, the scattering coefficient optimization module 41 is used to calculate the average value of the ratio of the inverse natural logarithm of the first transmission image to the relative estimated scene depth for all pixels in the non-sky area to obtain the scattering coefficient.

[0262] Using the above technical solution, the scattering coefficient optimization module 41 also estimates the scene depth relative to the scene depth. The scattering coefficient β is optimized so that the subsequent transmission map optimization processing module 14 can obtain an optimized second transmission map t(x), and the subsequent scene radiance processing module 15 can obtain a better scene radiance J(x) based on the optimized second transmission map t(x), thereby further restoring the degraded image I(x).

[0263] refer to Figure 21 In some possible embodiments provided by the present invention, an atmospheric scattering degradation image restoration system 05 based on scene depth information is provided. In addition to including the image acquisition module 11, dark channel prior processing module 12, scene depth relative estimation module 13, transmission map optimization processing module 14, scene radiance processing module 15, pixel area processing module 21, sky area acquisition module 22, atmospheric light intensity acquisition module 31 and scattering coefficient optimization module 41 in the previous embodiment, it also includes: a regularization processing module 51.

[0264] On the basis of the previous embodiment, the regularization processing module 51 (the data input end) is connected to the image acquisition module 11 (the data output end), and the regularization processing module 51 (the data output end) is connected to the dark channel prior processing module (the data input end).

[0265] In this embodiment, the regularization processing module 51 is used to perform regularization processing on the image to be processed.

[0266] By adopting the above technical solution, the regularization processing module 51 ensures that the maximum value of the image to be processed I(x) does not exceed 1 in subsequent calculations, thereby simplifying the calculations.

[0267] refer to Figure 22 In some possible embodiments provided by the present invention, an atmospheric scattering degradation image restoration system 06 based on scene depth information is provided. In addition to including the image acquisition module 11, dark channel prior processing module 12, scene depth relative estimation module 13, transmission map optimization processing module 14, scene radiance processing module 15, pixel area processing module 21, sky area acquisition module 22, atmospheric light intensity acquisition module 31, scattering coefficient optimization module 41 and regularization processing module 51 in the previous embodiment, it also includes: an image output module 61.

[0268] The image output module 61 (the data input end thereof) is connected to the scene radiance image processing module 15 (the data output end thereof).

[0269] In this embodiment, the image output module 61 is used to output the scene radiance.

[0270] Thirdly, reference Figure 23 The present invention provides an electronic device 2, comprising a memory 201, a processor 202, and a computer program stored in the memory 201 and executable on the processor 202. When the processor 202 executes the computer program, the method for restoring an image degraded by atmospheric scattering based on scene depth information in any of the aforementioned embodiments is implemented. The memory 201 may include, for example, a system memory, a fixed non-volatile storage medium, and the like. The system memory may store, for example, an operating system, application programs, a boot loader, and other programs.

[0271] In this embodiment, the electronic device optimizes the first transmission map of the dark channel prior hypothesis through relative estimation of scene depth to obtain an optimized second transmission map, and optimizes the scene emissivity based on the second transmission map, effectively processing the 30% of pixels in the natural scene that do not meet the dark channel prior hypothesis, restoring the real scene information, and restoring the outdoor atmospheric scattering degraded image.

[0272] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the atmospheric scattering degradation image restoration method based on scene depth information in any of the aforementioned embodiments.

[0273] By adopting the above technical solution, a computer-readable storage medium realizes the optimization of the first transmission map of the dark channel prior hypothesis through relative estimation of scene depth, obtains the optimized second transmission map, optimizes the scene emissivity based on the second transmission map, effectively processes the 30% of pixels in the natural scene that do not meet the dark channel prior hypothesis, restores the real scene information, and restores the outdoor atmospheric scattering degraded image.

[0274] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable non-transitory storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0275] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0276] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0277] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0278] Although the present invention has been illustrated and described with reference to certain preferred embodiments thereof, it should be understood by those skilled in the art that the above description is provided as a further detailed description of the present invention in conjunction with specific embodiments thereof, and that the specific implementation of the present invention is not limited to these descriptions. Those skilled in the art may make various changes in form and details, including simple deductions or substitutions, without departing from the spirit and scope of the present invention.

Claims

1. A method for restoring an image degraded by atmospheric scattering based on scene depth information, characterized in that: include: Get the image to be processed; Obtaining a first transmission map based on the image to be processed by a dark channel priori assumption, where the first transmission map is a roughly estimated transmission map; Obtaining a relative estimation of scene depth based on the image to be processed; Obtaining a second transmission map based on the first transmission map and the relative estimation of scene depth, wherein the second transmission map is an optimized transmission map; The scene radiance is obtained based on the second transmission map and the image to be processed through an atmospheric scattering model.

2. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 1, wherein: Obtaining a relative estimation of scene depth based on the image to be processed includes: obtaining a relative estimation of scene depth based on the image to be processed by a monocular depth estimation method.

3. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 2, wherein: The monocular depth estimation method includes at least one of a linear perspective method, an atmospheric scattering prior method, and a shadow plane method.

4. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 1, wherein: Also includes: Obtaining a sky area and a non-sky area based on the image to be processed; The second transmission map is obtained based on the sky area and the non-sky area.

5. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 4, wherein: The obtaining of the second transmission map based on the sky area and the non-sky area includes: For all pixels in the non-sky area, obtaining a transmission map of the non-sky area according to all pixels in the non-sky area based on the relative scene depth estimation, and using the transmission map of the non-sky area as the second transmission map; For all pixel points in the sky area, the minimum value of the transmission map of the non-sky area is obtained as the transmission map of the sky area, and the transmission map of the sky area is used as the second transmission map.

6. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 5, wherein: The obtaining of the sky area and the non-sky area based on the image to be processed includes: Obtaining a first pixel area by a clustering method based on the first transmission map; Obtaining a second pixel region based on the relative estimation of scene depth; Obtain a sky area based on the first pixel area and the second pixel area; Other areas in the image to be processed except the sky area are non-sky areas.

7. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 6, wherein: The obtaining the first pixel area by a clustering method based on the first transmission image includes: adopting K-means clustering.

8. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 7, wherein: The K-means clustering method includes: Take 0 as the first cluster center and 1 as the second cluster center; According to the first cluster center and the second cluster center, the first transmission map Clustering all pixel points to divide all pixel points into the first group and the second group; The mean value of the pixels in the first group is used as the first cluster center, and the mean value of the pixels in the second group is used as the second cluster center. Re-cluster all pixel points to divide all pixel points into the third group and the fourth group; Determine whether the ratio of the first cluster center to the second cluster center exceeds 2, If so, a group of pixels with smaller center point values ​​is recorded as the first pixel area Γ t ; If not, the first pixel area Γ t is an empty set; the third group is used as the first group, the fourth group is used as the second group, and the mean of the pixels in the first group is used as the first cluster center, and the mean of the pixels in the second group is used as the second cluster center. Re-cluster all pixel points to divide all pixel points into the third group and the fourth group until the first pixel area Γ is obtained t .

9. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 5, wherein: The first pixel area is obtained by a clustering method based on the first transmission map, and the clustering method includes: a partitioning method, a hierarchical method, a density-based method, a grid-based method, a model-based method, a transitive closure method, a Boolean matrix method, a direct clustering method, a correlation analysis clustering, and a statistics-based clustering method.

10. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 6, wherein: The obtaining of the second pixel region based on the relative scene depth estimation includes: obtaining a median and a standard deviation of the relative scene depth estimation, and recording pixels whose standard deviation of the relative scene depth estimation is greater than a preset multiple of the median as the second pixel region.

11. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 6, wherein: The obtaining of the sky area based on the first pixel area and the second pixel area includes: obtaining a union of the first pixel area and the second pixel area as the sky area.

12. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 5, wherein: The obtaining of the second transmission map based on the first transmission map and the relative estimation of the scene depth includes: obtaining a scattering coefficient based on the first transmission map and the relative estimation of the scene depth.

13. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 12, wherein: The obtaining of a scattering coefficient based on the first transmission map and the relative estimation of the scene depth includes: For all pixels in the non-sky area, an average value of the ratio of the inverse natural logarithm of the first transmission image to the relative estimated scene depth is calculated to obtain the scattering coefficient.

14. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 4, wherein: Also includes: Obtaining atmospheric light intensity based on the sky area; The scene radiance is obtained through an atmospheric scattering model based on the second transmission map, the image to be processed, and the atmospheric light intensity.

15. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 14, wherein: The obtaining of the atmospheric light intensity based on the sky area includes: calculating the average of all pixels in the sky area to obtain the atmospheric light intensity.

16. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 1, wherein: After obtaining the image to be processed, the method further includes performing regularization processing on the image to be processed.

17. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 16, wherein: Regularizing the image to be processed includes: dividing the image to be processed by the maximum value of pixels in the image to be processed.

18. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 1, wherein: The obtaining of a first transmission map based on the image to be processed by using a dark channel priori assumption includes: Determine the minimum value of each pixel of the image to be processed in the three color channels of red, green and blue; Determining the mean value of a set of pixel points in a preset area in the image to be processed; The first transmission map is obtained based on the minimum value and the average value of each pixel in the three channels R, G, and B.

19. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 18, wherein: The brightness of the pixels in the preset area is higher than the brightness of the pixels in the non-preset area in the image to be processed.

20. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 19, wherein: The pixel point set in the preset area is a set of the brightest 10% of pixel points in the image to be processed.

21. The method for restoring an image degraded by atmospheric scattering based on scene depth information according to claim 1, wherein: After obtaining the scene radiance based on the second transmission map, the image to be processed, and the atmospheric light intensity, the method further includes: outputting the scene radiance.

22. An atmospheric scattering degradation image restoration system based on scene depth information, characterized in that: include: An image acquisition module, used for acquiring an image to be processed; a dark channel priori processing module, configured to obtain a first transmission map based on the image to be processed by using a dark channel priori hypothesis, wherein the first transmission map is a roughly estimated transmission map; A scene depth relative estimation module, configured to obtain a scene depth relative estimation based on the image to be processed; a transmission map optimization processing module, configured to obtain a second transmission map based on the first transmission map and the relative estimation of scene depth, wherein the second transmission map is an optimized transmission map; The scene radiance processing module is used to obtain the scene radiance through an atmospheric scattering model based on the second transmission map and the image to be processed.

23. The atmospheric scattering degradation image restoration system based on scene depth information according to claim 22, characterized in that: Also includes: a pixel region processing module, configured to obtain a first pixel region by a clustering method based on the first transmission map, and obtain a second pixel region based on the relative estimation of the scene depth; The sky area acquisition module is configured to obtain a sky area based on the first pixel area and the second pixel area.

24. The atmospheric scattering degradation image restoration system based on scene depth information according to claim 23, characterized in that: Also includes: The atmospheric light intensity acquisition module is used to obtain the atmospheric light intensity based on the sky area.

25. The atmospheric scattering degradation image restoration system based on scene depth information according to claim 24, characterized in that: Also includes: The scattering coefficient optimization module is used to calculate the average value of the ratio of the inverse natural logarithm of the first transmission image to the relative estimated scene depth for all pixel points in the non-sky area to obtain the scattering coefficient.

26. The atmospheric scattering degradation image restoration system based on scene depth information according to claim 25, characterized in that: Also includes: The regularization processing module is used to perform regularization processing on the image to be processed.

27. The atmospheric scattering degradation image restoration system based on scene depth information according to claim 26, characterized in that: Also includes: Image output module, used to output scene radiance.

28. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for restoring an image degraded by atmospheric scattering based on scene depth information according to any one of claims 1 to 21 is implemented.

29. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for restoring an image degraded by atmospheric scattering based on scene depth information according to any one of claims 1 to 21 is implemented.

Images