Transmittance rapid valuation model based on atmospheric visibility

Abstract

The invention discloses a transmissivity rapid valuation model based on atmospheric visibility, belongs to intelligent mobile terminal image sharpening processing content in the technical field of machine vision, and particularly relates to an image sharpening processing technology that an intelligent video image terminal arranged on a mobile device is in a moving state in a rain and fog environment. According to one content of the technology, the atmospheric transmissivity of an image sharpening processing model is required to be rapidly estimated, and a sharpening processing flow based on an atmospheric transmissivity rapid estimation model is constructed, so that the sharpening processing model can be applied to a mobile device carrying a video processing terminal in real time. The provided atmospheric transmissivity rapid valuation algorithm model is simple in form, simple in process, low in calculation complexity, good in real-time performance and good in image sharpening processing effect, and is not only suitable for vehicle traffic application scenes, but also suitable for vehicle traffic application scenes. And the method can also be applied to video image sharpening processing scenes of intelligent mobile terminals configured by ships, industrial intelligent trolleys, port loading and unloading vehicles, scenic spot tourist buses, airport passenger transfer vehicles, unmanned aerial vehicles, military mobile equipment and the like in a rain and fog environment.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent mobile terminal image sharpening processing technology of machine vision, and particularly relates to a transmittance fast estimation model for rain and fog degenerated video image sharpening processing of an intelligent mobile terminal by using actual measurement values of atmospheric visibility. TECHNICAL BACKGROUND

[0002] The intelligent mobile terminal of machine vision image processing, such as a video image central terminal of a vehicle, a ship, a drone, etc., often faces the video image sharpening processing in a rain and fog environment. According to the optical imaging theory, there are two analysis models for the visible light object scene imaging mechanism, one is the attenuation model, and the other is the atmospheric optical model, and their theoretical basis is the atmospheric scattering theory.

[0003] Based on the atmospheric scattering theory, the gray level of any point of the visible light object scene imaging is expressed as:

[0004] E=I ∞ ρe -β(λ)d +I ∞ (1-e -β(λ)d ) (1)

[0005] The above formula can be usually converted as:

[0006] I(x)=t(x)J(x)+A(1-t(x)) (2)

[0007] Wherein: I(x) is the original image obtained by the video terminal, i.e. the actual image to be removed rain and fog

[0008] J(x) is the clear image without rain and fog influence

[0009] A is the object image ambient light

[0010] t(x)=e -β(λ)d is the atmospheric transmittance

[0011] As can be seen from the above formula (2), the physical meaning is that the original image I(x) obtained by the video terminal is mainly composed of the image t(x)J(x) of the clear image J(x) attenuated by the atmospheric transmittance t(x), and the object image ambient light attenuated by the atmospheric transmittance t(x), so that the rain and fog image restoration processing is to obtain the clear target image J(x) by estimating t(x) and A of the video terminal original image I(x):

[0012] J(x)=[I(x)-A] / t(x)+A (3)

[0013] According to the beer law of atmospheric light radiation, the initial radiation intensity of the object scene light is weakened by the scattering coefficient β(λ) of the rain and fog layer with a depth of d, resulting in unclear object scene image acquired by the image terminal. The attenuation degree of the object scene light caused by the rain and fog layer is referred to as atmospheric transmittance, which is expressed as follows:

[0014] t(x) = e-β(λ)d (4)

[0015] According to the atmospheric light radiation theory, the attenuation effect of the rain and fog layer suspended particles on the object scene light is affected by various factors such as atmospheric molecular absorption coefficient, atmospheric molecular attenuation coefficient, rain and fog particle extinction coefficient, etc. under different environments, and the estimation and calculation of t(x) are very complex, which brings great difficulty to the application of formula (3).

[0016] To this end, the application provides a transmittance fast estimation model based on atmospheric visibility, which solves the problem of real-time processing of video images by a mobile intelligent terminal. SUMMARY

[0017] The application mainly takes Figure 1 The vehicle-mounted video terminal in the rain and fog environment is used as an example to process the object scene in front of the vehicle, the atmospheric extinction coefficient of the rain and fog suspended particle layer is taken as the research object, the influence of the object scene visual range depth d in front of the video terminal and the atmospheric visibility on the atmospheric transmittance is discussed, and then the transmittance fast estimation model based on atmospheric visibility is provided.

[0018] DESCRIPTION OF DRAWINGS

[0019] The application mainly takes Figure 1 : An example of the vehicle-mounted video terminal in the rain and fog environment shooting the object scene in front of the vehicle

[0020] The application mainly takes Figure 2 : Relationship diagram of the object scene scattering light intensity and the ratio of the atmospheric particle size to the wavelength

[0021] The application mainly takes Figure 3 : Spectral diagram of various types of light waves

[0022] The application mainly takes Figure 4 : Functional architecture model of the rain and fog video image sharpening processing

[0023] The application mainly takes Figure 5 : Image sharpening processing flow constructed by using the Matlab simulink integrated environment

[0024] The application mainly takes Figure 6 : Sharpened image of the fog visibility being 60 m and the object scene visual range distance being 150 m

[0025] The application mainly takes Figure 7Rain and fog visibility is 95m, the clear processing image of the visual range distance of 200m of object scene is realized

[0026] The technical principle of the "transmittance fast evaluation model based on atmospheric visibility" proposed in the application is described as follows in combination with the drawings.

[0027] (1) Noun explanation and definition

[0028] Depth of view: also called object scene view thickness or scene depth, that is, the distance of the camera terminal along the shooting direction to the object scene, denoted by d, unit m, see Figure 1 .

[0029] Visibility: atmospheric visibility in rain and fog environment, denoted by L, unit m.

[0030] Transmittance: the attenuation coefficient of object scene light by atmospheric particles, denoted by t(x).

[0031] Scattering coefficient: the scattering coefficient of object scene light by atmospheric particles, denoted by β(λ).

[0032] Light wavelength: that is, object scene light wavelength, denoted by λ, unit μm (micron).

[0033] Pixel coordinate: image pixel coordinate, denoted by x.

[0034] (2) Basic theory

[0035] In rain and fog environment, fog, dust and other particles in the atmosphere will form aerosol particles, which will produce light scattering and light absorption effects on the object scene, which is called atmospheric extinction or atmospheric attenuation. The strength of atmospheric extinction is generally represented by the extinction coefficient (i.e. scattering coefficient), which is related to the properties, size, density, etc. of aerosol particles. Generally, the total extinction coefficient μ of rain and fog environment atmosphere is the sum of the absorption coefficient β a and the scattering coefficient β s :

[0036] μ = β a + β s (5)

[0037] When the size of atmospheric particles is much smaller than the wavelength of incident light, that is, 1 / 10 of the wavelength λ, the scattering light intensity is inversely proportional to the fourth power of the wavelength (λ 4 ), which is the famous Rayleigh scattering theory. In the early 20th century, the German physicist G. Mie found that when the size of atmospheric particles is comparable to or even larger than the wavelength of incident light (such as rain and fog, smoke, dust environment), the relationship between the intensity of scattered light and the wavelength is weakened, see Figure 2 . G. Mie's discovery is known as Mie scattering in the field of physical optics.

[0038] Mie scattering theory tells us that the total extinction coefficient μ of rain, fog, smoke, haze, and other atmospheric environments on the light of the scene, mainly by scattering coefficient β s Determination, so:

[0039] μ≈β s (6)

[0040] The scattering coefficient β in the above formula (6) s , is the transmittance formula (4) we need to quickly obtain β(λ) in the video image processing.

[0041] However, Mie scattering theory assumes that atmospheric suspended particles are ideal spheres, and proposes a scattering coefficient calculation model:

[0042] e β(λ) =f(α+n) (7)

[0043] In the formula: α = πd / λ, the size parameter of the particle

[0044] n = n re +i*n im , the real part n re is the refractive index, and the imaginary part i*n im is the scattering property

[0045] It can be seen that to obtain the scattering coefficient β(λ) of rain and fog atmosphere through formula (7), the calculation process is very complex, which brings great difficulty to its practical application.

[0046] (3) Simplified method

[0047] ① Since the atmospheric transmittance described by formula (4) represents the degree of attenuation of the atmospheric suspended layer on the light of the scene, the scattering coefficient β s in the extinction coefficient of Mie scattering theory can be used to replace β(λ) in formula (4), that is:

[0048]

[0049] In the formula, d is the distance of the light passing through the atmospheric suspended particle layer, that is, the optical scattering thickness.

[0050] ② In meteorological optics theory, the following empirical formula is usually used to estimate the extinction coefficient:

[0051]

[0052] In the formula, q is the visibility L-wavelength λ correction factor, which is related to the visibility, and the value is:

[0053]

[0054] Typically, the safe visibility range of vehicle-mounted video terminals in rainy or foggy environments is limited to within 2km; therefore, the correction factor q can be set to 0.585L. 1 / 3 .

[0055] ③ Most vehicular traffic is along greenways, with vehicles primarily traveling horizontally. The ambient light along the roads is predominantly greenish-yellow visible light, with wavelengths generally between 0.50 and 0.58 μm, while the average wavelength of visible light from sunlight is 0.53 μm (see Appendix). Figure 3 Therefore, the comprehensive reference value of the object scene light wavelength λ can be taken as 0.55μm, and the extinction coefficient estimation formula (9) can be modified as follows:

[0056]

[0057] ④ Based on formulas (8) and (11), and from section ①, the formula for calculating the relationship between atmospheric transmittance and atmospheric visibility L can be obtained:

[0058]

[0059] Formula (12) is a rapid estimation model for atmospheric transmittance under rain and fog conditions.

[0060] (4) Workflow structure for rain and fog video image processing

[0061] According to atmospheric scattering theory, the calculation formula for sharpening rain and fog images acquired by mobile video terminals is: [see formula (3) in the Technical Background section]

[0062] J(x)=[I(x)-A] / t(x)+A (13)

[0063] Where: I(x) is the original rain and fog image acquired by the video terminal, captured by the video terminal.

[0064] J(x) is the clear graph that needs to be obtained.

[0065] A represents atmospheric light, obtained from the estimation model of the guided filter map based on haze images (derivation omitted).

[0066] t(x)=e -β(λ)d Atmospheric transmittance

[0067] d represents the depth of vision, which is the thickness of the rain or fog layer through which light passes.

[0068] First, we designed a functional architecture model for rain and fog video image sharpening processing, see attached. Figure 4 This functional architecture model describes the logical relationships in the rain and fog video image sharpening process.

[0069] Then, using Matlab's visualization machine vision toolkit, and Matlab's Simulink providing an integrated environment for dynamic simulation of image processing, we constructed the workflow for video image sharpening processing. (See appendix.) Figure 5 This allows us to obtain a clearer video after processing.

[0070] Appendix Figure 5 The constructed process performs the following steps within a processing cycle—

[0071] ① Capture a single original image from the original video in real time, display it, and transmit the original image data;

[0072] ② In the Matlab Fundion1 module, the original image is converted to RGB color space to obtain a grayscale image;

[0073] ③ Use the grayscale image in the Matlab Fundion module to perform the following processing:

[0074] — Using grayscale images as guide maps for box filter processing

[0075] —Estimation calculation of atmospheric light A using atmospheric light homogenization map.

[0076] —Atmospheric transmittance t(x) is calculated using atmospheric visibility L and the object's field of view thickness d.

[0077] ④ Use the Matlab Fundion1 module to perform rain and fog image enhancement processing on formula (13);

[0078] ⑤ After processing by the Matlab Fundion 1 module, a cleared and restored image is output.

[0079] After one processing cycle is completed, the above steps are repeated until the sharpening process is finished. The constructed sharpening process achieves a running speed of 35 frames per second, achieving the goal of fast processing. If the attached... Figure 5 The algorithm flow is converted into an algorithm chip, and its computing speed can reach more than 60 frames per second.

[0080] (5) Rain and fog video image processing effects

[0081] Appendix Figure 6 This is the processing effect when the visibility L in heavy fog is 60m and the object sight distance d is set to 150m.

[0082] Appendix Figure 7 This is the processing effect when the visibility L in rain and fog is 95m and the object sight distance d is set to 200m.

[0083] (6) Conclusion

[0084] The transmittance rapid estimation model based on atmospheric visibility proposed in this invention solves the problem of rapid and clear processing of rain and fog video images by intelligent video terminals in rain and fog environments. The technology of this invention can be directly applied to various application scenarios for clear processing of video images in rain and fog environments.

Claims

1. Based on Mie scattering theory, image sharpening transmission rate, and the safe visibility range regulations for vehicle-mounted video terminals in rain and fog environments, a formula for calculating the relationship between atmospheric transmittance and atmospheric visibility L in rain and fog environments is obtained, i.e., a rapid estimation model for atmospheric transmittance in rain and fog environments: Where t(x) represents the attenuation coefficient of atmospheric particles on the scene light, which is the atmospheric transmittance; β s Atmospheric scattering coefficient; d is the distance that the light from the scene passes through the atmospheric suspended particle layer, i.e., the optical scattering thickness, also known as the depth of vision. In practical applications, this invention defines d as the distance at which the target scene can be clearly seen; L represents the atmospheric visibility in rain and fog environments.

2. Construct a video image sharpening process based on a rapid atmospheric transmittance estimation model. Based on atmospheric scattering theory, the calculation formula for sharpening rain and fog images acquired by mobile video terminals is as follows: in, I(x) represents the original rain and fog image acquired by the video terminal; J(x) represents the clear image obtained after processing; A is atmospheric light, obtained from the estimation model of the fog and haze image guided by the filter image; t(x) represents atmospheric transmittance; d represents the depth of vision, that is, the thickness of the object light passing through the rain and fog layer. Based on formula (2), a rain and fog video image sharpening processing architecture model is designed, and a rain and fog video image processing flow is constructed. This flow performs frame-by-frame sharpening processing on the image within one image acquisition cycle and then outputs the result. The following steps are performed on each frame of the image: ① Capture a single frame from the original video in real time; ② Convert the image to GBR color space to obtain a grayscale image; ③ Perform box filtering on the grayscale image; ④ Calculate the atmospheric light estimate of A for the image; ⑤ Calculate atmospheric transmittance t(x) using atmospheric visibility L and object field thickness d; ⑥ Use formula (2) to sharpen the rain and fog image and output the image.

Images