An image restoration method based on physical scattering model

Abstract

The present invention relates to an image restoration method based on a real physical parameter scattering physical model. A new scattering model different from the general atmospheric light scattering model is established, which can give a corresponding physical explanation for each scattering environment, explain the relationship between the light source position and the scattering situation, and solve the problem of scattering removal under non-uniform illumination. First, an integrating sphere is used to remove the camera dark corner, and then the pure scattering part is found by an improved dark channel prior method. Finally, the zenith angle, azimuth angle, extinction coefficient and other related physical quantities are calculated according to the light attenuation and backscattering distribution of the scattering medium to eliminate the backscattering and light attenuation. In theory, the model can be used for any uniform scattering medium, such as underwater and foggy environments, and works well in high-density media. The final experimental results show that the model solves the problem of backscattering removal under strong scattering media, is correct and effective, and has practical physical significance.

Description

Technical Field

[0001] The invention relates to a scattering model, in particular to an image restoration method based on physical parameters, which explains the relationship between the light source position and backscattering. Background Art

[0002] In recent years, advances in technology have allowed people to capture images and videos in strong scattering environments. However, in actual engineering environments, removing backscatter has always been a persistent problem. Backscattering causes image color distortion and blurring, among other things. This severely affects the performance of standard computer vision algorithms, because generally speaking, these algorithms are developed to work in clear air. Therefore, in order to obtain reasonable results using these algorithms, the image must first be de-scattered. The well-known atmospheric scattering model is based on the Beer-Lambert law:

[0003] I(x)=J(x)t(x)+A(1-t(x)) (1)

[0004] t(x)=e -σz (2)

[0005] Among them, the atmospheric light A depends on the depth map, t(x) is related to the attenuation coefficient σ and the depth z, the input image is I(x), and the clear image is J(x). This model is relatively simple and only requires a few parameters to remove scattering in air and water. This also brings some disadvantages, that is, it cannot cover all complex situations.

[0006] When the midday light shines from due south, the zenith angle is close to the minimum value of the day, and the overall brightness of the image is brighter than at other times. The zenith angle is the angle between the incident direction of the light and the zenith direction, and its change cycle can be a whole year. When shooting towards the light source, the image brightness on the photo is symmetrical about the middle axis. Similarly, when the shooting angle and the illumination angle are 90°, the image brightness is monotonic about the long axis of the image. In an image, the farther away from the light source, the lower the brightness. Similarly, the farther the lens is from the light source, the lower the overall brightness of the image. When underwater, this is reflected by the distance of the water surface from the lens. Camera settings, such as lens aperture size, focal length, shutter speed, etc., will also affect the overall brightness of the image and the distribution of pixels. These phenomena caused by light will affect the removal of backscatter. Therefore, it is necessary and physical to consider all these factors.

[0007] As can be seen from the above, changes in the light source position will cause changes in image pixel values ​​and backscatter distribution, while the atmospheric scattering model cannot consider the information of the light source. Considering the relationship between the light source position and scattering is the core idea of ​​the method proposed by the present invention. Summary of the invention

[0008] In view of the above technical deficiencies, the present invention provides an image restoration method based on a physical parameter scattering model. 1) A new model based on physical parameters is proposed to eliminate the image blurring problem caused by backscattering and light attenuation of images taken in scattering media. 2) The limitations of the well-known atmospheric light attenuation model are discovered, and a completely different model method is proposed to solve the image blurring problem caused by non-uniform illumination. The relationship between the atmospheric scattering environment and the position of the light source is explained. In theory, the model can be used for images taken in any uniform scattering medium (such as underwater and smoky environments). 3) The model is particularly effective in solving images with high-density scattering problems.

[0009] The technical solution adopted by the present invention to solve its technical problems is: an image restoration method based on a physical scattering model, establishing a model mapping relationship between the physical position of a light source and scattering, dividing the attenuation image into backscattering components and direct radiation components for calculation; sampling points of the image, approximating the most realistic physical parameters through multiple fittings; substituting the calculated physical parameters back into the model to calculate the attenuation value and backscattering of the entire image, thereby restoring a clear image.

[0010] The method comprises the following steps:

[0011] Step 1. Use an integrating sphere to calibrate the camera vignetting on the attenuated image;

[0012] Step 2. Convert the image pixel coordinates into coordinates in the world coordinate system;

[0013] Step 3. Establish a descattering mathematical model, divide the attenuated image into two parts: direct radiation and backscattering, and derive the model mapping relationship respectively;

[0014] Step 4. Sample the corrected image after removing the vignetting, perform OSTU binarization, and obtain the partial pixel area of ​​pure scattering for fitting and parameter calculation;

[0015] Step 5. Use the calculated parameters to remove backscattering and attenuation from the entire image.

[0016] The method of using an integrating sphere to calibrate camera vignetting comprises:

[0017] Create a vignetting table;

[0018] The original attenuation image is photographed using an integrating sphere, and the correction coefficient at each pixel position is obtained. The coefficient is used to eliminate vignetting by using a lookup table method pixel by pixel on the original attenuation image to obtain a corrected image.

[0019] The direct radiation term can be expressed as follows:

[0020]

[0021] The backscatter term can be expressed as follows:

[0022]

[0023] The clear image is expressed by the following formula:

[0024]

[0025] Among them, I c (x) is the input image to be restored, L b c (x,y) is the backscattering term, is the direct radiation term.

[0026] The method of obtaining a partial pixel area of ​​pure scattering is to extract pixels containing only scattering components and having similar depths for estimating physical parameters:

[0027] a. Convert the input image from RGB space to HSV space and extract its saturation channel and intensity channel;

[0028] b. Use OSTU maximum inter-class variance method to calculate the optimal global threshold and obtain the image area with high intensity and saturation;

[0029] c. Further sample the image in the x and y directions to generate a point set V;

[0030] d. Apply the improved DCP-like rule in the image point set V:

[0031]

[0032] That is, J c The image region where (x)≈0.

[0033] The physical parameters include the zenith angle of the sun, the direction angle, the focal length of the camera, the illumination, the extinction coefficient, the depth of the object from the camera, the anisotropy constant, and the distance from the object in the water to the water surface.

[0034] During fitting and parameter calculation, since different parameters have different effects on the function, fitting and solving are performed in sequence according to the fitting priority order of the parameters.

[0035] This method restores images under non-uniform illumination based on the relationship between the light source position and the scattering distribution.

[0036] The present invention has the following beneficial effects and advantages:

[0037] 1. The method of the present invention establishes a new image descattering model method. Different from the currently popular image enhancement method, this model infers the physical parameters of the shooting scene based on the backscattering conditions on the image, and makes a physical explanation for the process of removing backscattering. The problem is solved by solving the inverse process of the physical method, making the image restoration effect more natural.

[0038] 2. The method of the present invention innovatively explains the relationship between the light source position and backscattering, and solves the image descattering problem under non-uniform illumination.

[0039] 3. The method of the present invention is superior to the existing algorithms in terms of descattering and has stable performance, especially when strong scattering occurs in the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of an image restoration method for a physical parameter scattering physical model;

[0041] Figure 2 It is an image taken in pure scattering without an object;

[0042] Figure 3 It is caused by non-uniform lighting;

[0043] Figure 4 are the zenith angle and azimuth angle of the light source;

[0044] Figure 5 is the relevant calculation parameter of backscattering;

[0045] Figure 6 It is an experimental environment;

[0046] Figure 7 is about θ A The change of pixel value;

[0047] Figure 8 is about θ Z The change of pixel value;

[0048] Fig. 9 It is the change of pixel value with respect to the change of g and σ;

[0049] Fig.10 It is the change of pixel value with the change of f;

[0050] Fig.11 It's about Z t ,M,Y u The change of pixel value;

[0051] Fig.12 It is the restored image of the high scattering image corresponding to Table 2;

[0052] Fig.13 This is the comparative experimental result of the present invention and other methods. DETAILED DESCRIPTION

[0053] The present invention is further described in detail below in conjunction with the embodiments. The method steps are described in conjunction with the accompanying drawings.

[0054] A scattering model based on physical parameters includes the following steps:

[0055] 1. Use a uniform light source (such as an integrating sphere) to calibrate the camera's vignetting and remove image noise;

[0056] 2. Establish a descattering mathematical model and divide the attenuated image into two parts: direct radiation and backscattering, and calculate them separately;

[0057] 3. Perform OSTU binarization on the collected images, and perform pixel fitting and parameter calculation on some of the pure scattering pixels.

[0058] 4. Use the calculated parameters to remove backscattering and attenuation from the entire image.

[0059] The vignetting calibration comprises:

[0060] Establish a de-vignetting table corresponding to all camera settings including focal lengths; use the lookup table method for the processed image and use the corresponding de-vignetting table to remove vignetting. Vignetting is a position-dependent illumination attenuation phenomenon that is inevitable for all camera lenses. Generally, vignetting occurs when the light beam passing through the aperture from the object point on the optical axis is larger than the imaging beam of the off-axis object point. As far as vignetting elimination is concerned, the ideal fourth cosine law cannot be verified in practice, so there are many approximate works like curves, as well as improvements to the fourth cosine law. In this article, we use a lookup table (LUT) method to eliminate vignetting. This operation should be performed in a dark room, with a uniform light source with low specular reflection and known input intensity. We use an integrating sphere with a diameter of 1000mm, 12 built-in lamps, and a light outlet diameter of 100mm to ensure uniform light source. The correction coefficient at each pixel position is:

[0061]

[0062] Among them, I ref (i,j) represents the pixel value at position (i,j), I ref,max Indicates I ref The maximum value of (i,j), I LUT (i,j) represents the correction factor at coordinate (i,j). The correction factors for all positions are calculated and combined into the de-vignetting table. Afterwards, images taken with the same camera settings can be corrected using the table lookup method:

[0063] I′(i,j)=I(i,j)I LUT (i,j) (4)

[0064] Where I(i,j) represents the input intensity value at pixel (i,j), and I′(i,j) represents the output intensity value of the pixel after vignetting correction.

[0065] The backscattering component is an optical effect caused by light being randomly scattered by suspended particles in the scattering medium to reach the camera lens without being reflected by the object. The direct radiation component is an optical effect caused by light being reflected by the surface of the object and then being scattered by suspended particles in the scattering medium to reach the camera lens.

[0066] The descattering mathematical model includes:

[0067] 1. Overview of non-uniform lighting

[0068] Figure 2 These are two pure scattering images without objects taken underwater (a) and (e). In terms of atmospheric light scattering, the curves of the two marked lines should tend to be constant. However, Figure 2 As shown, the shape of the curve of the marked pixel value is closely related to the position of the light source. For the vertical lines (d) and (h) marked in (b) and (f), the pixel value decreases as the ordinate decreases, because the light source is farther away from the position with a larger ordinate. For the horizontal lines (c) and (g) marked in (b) and (f), the conditions are more complicated. We found that when the angle between the shooting direction and the light source direction on the earth plane is kπ (k=1,2,3…), the shape of the curve remains symmetrical about the central axis. When the angle between the shooting direction and the light source direction on the earth plane is kπ+π / 2 (k=1,2,3…), the shape of the curve remains monotonically increasing or monotonically decreasing. When the shooting angle is between the above two cases, the pixel value of its curve will have a maximum or minimum value. The above problem can be simply summarized as the closer to the light source, the higher the illumination, and the larger the pixel value. In addition, we also found that this phenomenon becomes more serious as the turbidity of the medium increases, because light attenuates faster in a denser medium. These issues are non-negligible and present problems when performing depth estimation and image restoration in dense media.

[0069] like Figure 3 As shown, light propagation results in uneven illumination and intensity. Figure 3 (a) shows that the light received by the camera consists of two parts: the direct attenuation of the object and the scattered light. However, the widely used atmospheric light scattering model does not consider the light transmission from the light source to the object. Figure 3As shown in (b), light D2 travels a longer distance than light D1, which means that light D2 is attenuated more severely. The atmospheric light scattering model also assumes that the scattered light is parallel and travels the same distance to the scattering point. However, Figure 3 As shown in (c), at different camera angles (O1 and O2), the scattered light from the light source to the scattering point (L1 and L2) is also different. Figure 3 (b) and Figure 3 (c) shows two practical factors that lead to the fact that the lighting is not uniform.

[0070] 2. Model building

[0071] The camera's charge-coupled device (CCD) is used as the origin of the world coordinate system. Let the x-axis and y-axis on the image plane be parallel to the world's x-axis and y-axis, and the z-axis be aligned with the camera's optical axis. For a point x = (x0, y0) on the image plane, let its corresponding point X = (X0, Y0, Z0) on the object surface, and the relationship between them is: Where f is the focal length of the camera.

[0072] Then for a point (x, y) in the image coordinate system, its corresponding coordinates in the world coordinate system can be expressed as:

[0073]

[0074] Where x0 and y0 are the center coordinates of the image, w x and w y It is the physical size of the pixel calculated based on the CCD width, and f is the focal length.

[0075] Using the single scattering model, forward scattering is ignored, so the radiation reaching the camera can be divided into direct radiation and backscattering:

[0076]

[0077] Among them, I b is the backscattering, I d is the direct radiation component. Therefore, by accurately calculating the backscattering intensity and attenuation of each pixel, a clear image can be obtained.

[0078] According to the Beer-Lambert law, light is considered to attenuate with the distance from the light exit point to the lens, and the propagation of light is related to the extinction coefficient. Among them, the extinction coefficient varies based on the wavelength of light. In water, visible light attenuates at the longest wavelength and eventually appears blue in the human eye, while red tends to decay faster than green and blue, so the extinction coefficients of different channels should not be the same. In addition, when sunlight shines from a distance, the part scattered in each direction should also be taken into account, which is determined by the phase function and the scattering coefficient.

[0079] 3. Direct radiation component

[0080] For a point x=(x0,y0) on the image plane, assume that it corresponds to the coordinate X=(X0,Y0,Z0) on the object in the world coordinate system. Then for this point, the direct radiation component is:

[0081]

[0082] Among them, ‖D(X)‖ represents the distance that the light propagates to the object, I0 represents the radiation intensity of the light source before entering the scattering medium, and c represents the color channel. The distance ‖D(X)‖ includes the distance in water ‖τD(X)‖ (0≤τ≤1) and the distance in air ‖1-τD(X)‖. After that, the light is reflected by the object, and the reflectivity is ρ c (X), and reaches the camera lens after a distance of ‖X‖. After all the processes, the direct radiance reaching the camera lens can be written as:

[0083] I d (x) = E c e -σ(‖τD(X)‖+‖X‖) (8)

[0084] Let Y u is the distance from the camera to the horizontal plane. Figure 5 As shown, considering Figure 4 The zenith angle θ shown Z , the distance light travels in water before being reflected by an object is equal to:

[0085]

[0086] Let r(x,y) be equal to the distance ‖X‖ from the object to the camera lens, then we get:

[0087]

[0088]

[0089] Finally, the direct radiation component can be represented by physical parameters:

[0090]

[0091] Among them, J c (x) represents the clear image, and r represents the distance from the pixel point r(x,y) to the origin.

[0092] When the light source is in a scattering medium, Y u Refers to the distance from the object to the light source; otherwise Y u Refers to the distance of an object from the horizontal plane or the distance to the top of the atmosphere.

[0093] 4. Backscattering component

[0094] Backscattering is an optical response caused by random scattering of suspended particles in water at small angles when light does not reflect from the surface of an object but only scatters through the medium to reach the camera lens. The phase function is used to describe the degree of scattering of light in a specific direction, and the scattering coefficient β is used to represent the degree of light scattering. In terms of phase function, the most classic is the Heyney-Greenstein phase function, which can be represented by two parameters: molecular anisotropy g and scattering angle α. The relative amount of forward and backward scattering is described by g∈(-1,1):

[0095]

[0096] Among them, the scattering angle α ranges from [0,π], which is the angle between the object point direction and the light source direction:

[0097]

[0098]

[0099] u=[sin(θ Z ) sin(θ A ) cos(θ Z ) sin(θ Z ) cos(θ A )] (16)

[0100] Among them, θ Z is the zenith angle, θ A Indicates the azimuth.

[0101] According to the principle u·X=‖u‖‖X‖cosα, the scattering angle can be expressed as a real physical quantity:

[0102]

[0103] From formula (10), we get

[0104]

[0105] Backscatter is obtained by integrating the depth from the camera to the scattering point:

[0106]

[0107] Among them, Z s represents the depth, β c represents the extinction coefficient, σ c represents the scattering coefficient.

[0108] Simplify the formula, let Mc =E c β c ,The backscattering component is obtained as follows, which is composed of focal length, scattering angle, depth, extinction coefficient, etc.

[0109]

[0110] The pixel fitting and parameter calculation include:

[0111] According to equations (6), (12), (20), the formula for a clear image can be obtained as follows:

[0112]

[0113] Among them, I c (x) is the input image to be restored, L b c (x,y) is the backscattering term, is the direct radiation term.

[0114] After removing the vignetting in the image, the image plane coordinates are converted to the real coordinate system for descattering calculation. The image is sampled with a step size of 50 pixels, and a method similar to the dark channel prior is used for the sampling points to find the part where the direct radiance is close to 0, which is set as set V. For set V, the unknown parameter M in the formula is calculated using the optimization fitting method c ,σ,Y u ,θ z ,f,θ z ,θ A ,Z s Finally, the backscattering part of the whole image is calculated by the obtained parameters, and the backscattering term is subtracted and divided by the attenuation term to get the final result.

[0115] 1. Experimental Preparation

[0116] In order to verify the accuracy of the model, the experimental environment can be arranged as follows: Figure 6 As shown. We used a water tank filled with scattering medium (diluted milk) to conduct experiments. The light source is the sun. A water tank with a size of 90×90×45 cm is used and filled with clean water. Milk is added to increase the turbidity of the scattering environment multiple times. We designed experiments to test and verify our image restoration model. Since the equations are related to actual physical values, the following settings can be made to avoid interference from objective factors in the calculation of physical parameters:

[0117] Except for the side where the lens is placed, use black shading cloth to cover all the inside and outside surfaces of the water tank, including the bottom of the water tank, to prevent sunlight from reflecting between the glass and causing interference in the calculation;

[0118] Use a compass to calibrate the direction of the water tank so that the north-south direction of the water tank is parallel to the north and south poles of the earth's axis; place the water tank on a horizontal roof to prevent inaccurate calculation of the zenith angle;

[0119] Add the milk and stir evenly, let it sit until the water surface is still before shooting;

[0120] Turn off the camera's automatic fill light, automatic adjustment, peripheral light correction and other functions, and use the most original raw data for processing;.

[0121] When taking photos, try to remove or avoid water droplets adhering to the glass surface on the side you are taking photos.

[0122] When taking photos, hold the camera horizontally and avoid tilting it. Otherwise, the direction angle and other parameters may be calculated inaccurately.

[0123] 2. Experimental Procedure

[0124] 2.1 Software Environment Configuration

[0125] The computer platform configuration of this experiment is Intel(R) Core(TM) i9-10900K CPU@3.70GHz, 32G cache, and Nvidia GeForce RTX2080 SUPER graphics card. The software environment is Windows 10 Professional Edition. The software development environment of the experiment is Matlab2020b.

[0126] 2.2 Parameter calculation

[0127] In order to calculate the backscattered component I b , we need to find some areas whose pixel values ​​are close to the backscattering values ​​and the direct radiation component is equal to 0.

[0128]

[0129]

[0130] Therefore, it is necessary to extract some pixels that only contain scattered components and have similar depths to estimate physical parameters. To this end, the input image is first converted from RGB space to HSV space, and its saturation channel and intensity channel are extracted. Then, the graythresh function in Matlab, that is, the OSTU maximum inter-class variance method, is used to calculate the optimal global threshold and extract some areas with high intensity and saturation. In these areas, the image is further sampled at intervals of 50 pixels in the x and y directions to generate a point set V. For each point belonging to V, the minimum value of its five pixel adjacent areas is taken as follows.

[0131]

[0132] Formula (24) is inspired by the dark channel prior method proposed by He et al. The difference is that the present invention does not extract the minimum value of different color channels, because such calculation can adapt to more light source types. In addition, in order to ensure that the points used for calculation only contain backscattering terms, that is, J c (x)≈0, we apply this improved DCP-like law on the point set V instead of on the entire image.

[0133] 2.3 Curve fitting

[0134] The extracted area containing only backscattering is used to calculate the corresponding physical parameters. For the backscattering term we proposed in formula (20), it contains eight parameters, M c ,σ c ,Y u ,g,f,θ z ,θ A ,Z t . Theoretically, for an equation with eight parameters, the analytical solution of all parameters can be calculated by taking more than or equal to ten points. However, solving them one by one is difficult and time-consuming, and the point selection is greatly affected by noise. Fitting is a more time-saving method. Due to the large number of underwater disturbance factors that lead to a large amount of noise, and because there are too many parameters, the solution of the equation can easily shift during the fitting process, and parameter optimization is necessary. In the fitting calculation, each parameter has three corresponding values, namely the initial value, the fitting upper limit, and the fitting lower limit. It is particularly important to choose the right initial point.

[0135] In the experiment, we use milk as the scattering medium. Based on the calculation of the anisotropy parameters of milk in previous work, we can roughly reduce the value range of g from [-1, 1] to [0, 1]. Since the amount of milk dripped is much smaller than the volume of the water tank, the initial value of g can be set to a small value of 0.1. As shown in the figure, g mainly affects the slope along the vertical axis in the three-dimensional coordinates, which is reflected in the brightness difference, which gradually changes with the increase of water depth.

[0136] Table 1

[0137]

[0138] Using the control variable method, we can know the mathematical relationship obtained by substituting the parameters shown in Table 1 into the backscattering function. A Represents the azimuth angle from the light source to the camera, which mainly controls the tilt of the pixel value in the x-axis direction. Figure 7 (a)(b)(c) shows that when the θ of the two functions AWhen they are symmetric about 180°, their pixel values ​​z are about x=x s / 2 symmetry, and the tilt direction is also different between [0,π] and [0,π]. s is the maximum value in the image coordinate system. The mathematical derivation is as follows.

[0139] Defining functions

[0140]

[0141] but

[0142] f(-x)=f(2π-θ A ) (26)

[0143] The phase function

[0144]

[0145] Apparently

[0146] P(-x)=P(2π-θ A ) (28)

[0147] In equation (20), the phase function is the only one that contains x and θ A part, so the function of backscattering

[0148] L b (-x) = L b (2π-θ A ) (29)

[0149] Because formula (20) takes the center of the image as the coordinate origin, and the calculation is usually based on the first pixel in the image as the coordinate origin, the above formula can be changed to

[0150] L b (x s -x)=L b (2π-θ A ) (30)

[0151] In addition, if Figure 7 As shown in (d)(e)(f), when θ A = kπ, the overall image pixel value is about x = x s / 2 symmetry. So theoretically, for the same image, θ A There is only one solution. When fitting, θ AThe value is between 0° and 360°. The initial value is set to 10° as a step length, traversing 10° to 350°, the lower limit is set to 10° as a step length, traversing 0° to 340°, and the upper limit is set to 10° as a step length, traversing 20° to 360°. Compare all the fitting values ​​in the traversal process to determine θ A In the interval and its initial value.

[0152] θ Z Indicates the zenith angle of the light source. Figure 8 As shown, θ Z The change of mainly controls the change of the function amplitude. However, when θ Z When the value increases, both the numerator and denominator of the fraction in formula (20) will increase, so it is related to θ Z There is no qualitative relationship between θ and the overall pixel value. Z The value is between 0° and 90°. The initial value is set to 5° as a step length, traversing 5° to 85°, the lower limit is set to 5° as a step length, traversing 0° to 80°, and the upper limit is set to 5° as a step length, traversing 10° to 90°. Compare all the fitting values ​​in the traversal process to determine θ Z In the interval and its initial value.

[0153] Parameter σ c , g is related to the scattering medium itself. Friscad et al. proposed a theoretical model for calculating the scattering properties of participating media, and Narasimhan et al. measured the scattering parameters of 40 common materials, including g, by increasing the concentration of the diluent. Fig. 9 As shown in (a), g mainly affects the slope along the z-axis on the yz coordinate plane. When fitting, set the step size of traversing g to 0.1 and traverse [0,1] to find the most suitable initial value. Fig. 9 As shown in (b), σ c represents the attenuation coefficient, which varies with the overall amplitude of the function and has almost no effect on L b The gradient of x, y, and z. According to the research of Narasimhan and Jerlov, this value is generally 10 -5 / mm to 10 -3 / mm has different orders of magnitude, so the initial value of this value is set to 10 -4 The order of magnitude is 10. -5 and 1, and the traversal step size is 0.0001.

[0154] The focal length f mainly controls the curvature of the function on the x-axis and y-axis, such as Fig.10As shown in the figure, it is expressed as the gradient size of y or z, and the difference between the maximum and minimum backscattering values. This can be explained by the mechanism of camera shooting. When the camera position is fixed, the larger the focal length, the narrower the field of view, and the brightness changes in a curve, so the change of focal length will cause the change of brightness difference in the image. For the camera Cannon 5D Mark2, it can directly take the upper and lower limits as [24, 105] and the step size as 10 in all fitting calculations for fitting. In addition, it can be considered to read directly from the EXIF ​​tag of the photo.

[0155] like Fig.11 As shown, the parameter Z t ,Y u ,M mainly affects the overall amplitude and rarely changes the slope. t Indicates the depth from the object to the camera. Z t The larger it is, the more turbid the image is as a whole, and the deeper the backscattering is. M is usually not a large value, and its product with f is close to the value of the background light. In addition, the upper and lower limits need to be large enough to prevent the failure to obtain all high-fit solutions.

[0156] When fitting, different parameters have different effects on the function, so the fitting priority should be set. c ,Y u ,σ c ,f,g,θ A ,θ Z ,Z t Set the priorities to [5 8 1 4 7 6 2 3] respectively, perform fitting in order of priority, and select the result with the highest degree of fitting.

[0157] The overall algorithm flow is given in Algorithm 1.

[0158]

[0159] 3 Fitting results and comparative experiments

[0160] For Fig.12 (a) is an image with high scattering, and the contents of the object are almost invisible to the naked eye. The corresponding real physical parameters are calculated as shown in Table 2:

[0161] Table 2

[0162]

[0163] Its restored image is as follows Fig.12 (b) It can be clearly seen that the effect of the present invention is to remove the scattering of a considerable concentration, and the estimated physical parameters are also very close to the true values, which can verify the correctness of the present model.

[0164] Fig.13 The experimental results of the comparison between the present method and other methods are shown, and the comparison methods are Peng's method (IBLA), Huang's method (RGHS), UDCP method, and Song's method (ULAP). It can be seen with the naked eye that the present invention is superior in improving image vision and is more effective than other methods.

[0165] 4 Summary

[0166] This paper proposes a new physics-based method to restore densely scattered images. This method allows for superior handling of non-uniform illumination, which is ignored in commonly used atmospheric light scattering models. The present invention uses completely realistic physical parameters to describe the distribution of backscattering and direct radiation, and uses the idea of ​​curve fitting to remove backscattering. It is believed that this work will open up a new path for research on descattering, underwater enhancement, and image restoration in turbid media.

Claims

1. An image restoration method based on a physical scattering model, characterized in that: Establish a model mapping relationship between the physical position of the light source and scattering, and divide the attenuation image into backscattering component and direct radiation component for calculation; sample the image points and approximate the most realistic physical parameters through multiple fittings; Substitute the calculated physical parameters into the model to calculate the attenuation value and backscattering of the entire image, thereby restoring a clear image; The method comprises the following steps: Step 1. Use an integrating sphere to calibrate the camera vignetting on the attenuated image; Step 2. Convert the image pixel coordinates into coordinates in the world coordinate system; Step 3. Establish a descattering mathematical model, divide the attenuated image into two parts: direct radiation and backscattering, and derive the model mapping relationship respectively; The direct radiation term is expressed as follows: The backscatter term is expressed as follows: Step 4. Sample the corrected image after removing the vignetting, perform OSTU binarization, and obtain the partial pixel area of ​​pure scattering for fitting and parameter calculation; Step 5. Use the calculated parameters to remove backscattering and attenuation of the entire image; The clear image is expressed by the following formula: Among them, I c (x) is the input image to be restored, L b c (x,y) is the backscattering term, is the direct radiation term.

2. The image restoration method based on the physical scattering model according to claim 1, characterized in that: The method of using an integrating sphere to calibrate camera vignetting comprises: Create a vignetting table; The original attenuation image is photographed using an integrating sphere, and the correction coefficient at each pixel position is obtained. The coefficient is used to eliminate vignetting by using a lookup table method pixel by pixel on the original attenuation image to obtain a corrected image.

3. The image restoration method based on the physical scattering model according to claim 1, characterized in that: The method of obtaining a partial pixel area of ​​pure scattering is to extract pixels containing only scattering components and having similar depths for estimating physical parameters: a. Convert the input image from RGB space to HSV space and extract its saturation channel and intensity channel; b. Use OSTU maximum inter-class variance method to calculate the optimal global threshold and obtain the image area with high intensity and saturation; c. Further sample the image in the x and y directions to generate a point set V; d. Apply the improved DCP-like rule in the image point set V: That is, J c The image region where (x)≈0.

4. The image restoration method based on the physical scattering model according to claim 1, characterized in that: The physical parameters include the zenith angle of the sun, the direction angle, the focal length of the camera, the illumination, the extinction coefficient, the depth of the object from the camera, the anisotropy constant, and the distance from the object in the water to the water surface.

5. The image restoration method based on the physical scattering model according to claim 4, characterized in that: During fitting and parameter calculation, since different parameters have different effects on the function, fitting and solving are performed in sequence according to the fitting priority order of the parameters.

6. The image restoration method based on the physical scattering model according to any one of claims 1 to 5, characterized in that: This method is used to restore images under non-uniform illumination based on the relationship between the light source position and the scattering distribution.

Images