Adaptive polarization dark channel defogging and spectral prior fusion enhancement system and method

Abstract

The application provides an adaptive polarized dark original color defogging and spectral prior fusion enhancement system and method, and an electro-optical fog-penetrating imaging system comprises a detection imaging module, a laser ranging module, a power module, a signal processing module and a main control module. By using the system, the synchronous acquisition of polarized visible light and near-infrared spectral images is realized. In combination with the system, the application discloses an adaptive polarized dark original color defogging and spectral prior fusion enhancement algorithm, which is used for adaptive defogging in different concentration fog scenes, overcomes the limitations of the traditional dark original color method, and restores the color of the image background sky area. In combination with the near-infrared spectrum and polarized visible light information, multi-modal image fusion enhancement is carried out, the utilization rate of light intensity is improved, and the texture details of the invisible area of the scene are restored. In terms of image defogging effect and recovery quality, the application shows excellent performance, and provides an efficient and comprehensive solution for the field related to image defogging.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of optical imaging and image information processing, in particular to an adaptive polarized dark channel defogging and spectral prior fusion enhancement system and method. BACKGROUND

[0002] In the image acquisition imaging process, it is easy to be disturbed by the surrounding environment, such as rain, snow, dust, haze, and dust particles in the atmosphere and stray light emitted by objects. The collected image is usually covered with a layer of gray mask, which is similar to the area covered with fog, which is summarized as the haze effect. The visibility of the haze covered area is poor and blurred, which brings great inconvenience to people's driving and travel; it causes trouble to the research of imaging disciplines such as military detection and target recognition; and it affects the normal work of security monitoring equipment such as camera and the operation of various heading detection systems. Therefore, the through-fog imaging technology is particularly important.

[0003] The current image defogging technology mainly includes the following categories: image defogging algorithm based on physical model, image enhancement algorithm based on non-model, and deep learning image defogging algorithm based on convolutional neural network. Among them, the defogging algorithm based on physical model is mainly based on the atmospheric scattering physical model, the mapping relationship is summarized by observing a large number of foggy and non-foggy images, and the clear defogging image is obtained by inverse operation, and the typical algorithm is dark channel prior defogging algorithm, which has good defogging effect and stability. The image enhancement defogging algorithm is to improve the contrast of the foggy image and highlight the details to realize the defogging function, and the representative algorithms include histogram equalization, wavelet transform and Retinex algorithm. The deep learning defogging algorithm based on convolutional neural network establishes an end-to-end model to generate atmospheric scattering model parameters, and obtains the defogging image according to the parameters; or directly uses the convolutional neural network to generate a clear defogging image according to the foggy image.

[0004] In recent years, single image defogging has been widely concerned and researched, but there are still some problems, such as single defogging method, numerous defogging parameters, complicated adjustment steps, and poor defogging effect for different fog concentration scenes. SUMMARY

[0005] Therefore, the purpose of the present application is to provide an adaptive polarized dark channel defogging and spectral prior fusion enhancement system and method, which improves the field of view utilization rate and the stability of the imaging system; in the imaging mode, the multi-angle polarized visible light and near-infrared spectral image are collected synchronously, and the light intensity information of the image is fully utilized; in the defogging algorithm, the laser ranging inversion information and the dark channel priori principle are used for defogging, combined with the polarization prior information, the color of the image background area is restored, and at the same time, the near-infrared spectral prior information is used to overcome the limitation of the dark channel, enhance the polarized light intensity information, and restore the high-quality and clear target scene.

[0006] To achieve the above object, the application adopts the following technical solution: an adaptive polarization dark original color defogging and spectral prior fusion enhancement system, comprising a detection imaging module, a laser ranging module, a power module, a signal processing module and a main control module; wherein the detection imaging module is composed of a polarization filter array, a visible light and snapshot multispectral camera array and a visible light and snapshot multispectral camera lens array.

[0007] In a preferred embodiment, the snapshot spectral camera located at the center of the detection imaging module collects near-infrared band spectral images; the visible light cameras located around the detection imaging module have polarization filters of different angles covered at the front ends of the lenses, which are used to collect visible light images containing different polarization information, and then transmit the signals to the signal processing module for pre-processing; the laser ranging module located above the detection imaging module detects the actual distance from the imaging end to the target scene through a laser range finder and transmits the signal to the signal processing module; the fog and haze area proportion coefficient of different fog concentration images is calculated, and the fog degree parameter is mapped out to realize the adaptive parameter adjustment function; the range of transmittance is inverted through laser ranging, and the dark original color prior information is integrated to refine the transmittance; the polarization dark original color defogging imaging is performed by combining the polarization prior information, optimizing the image background area and generating a new atmospheric scattering model; the defogging image and the near-infrared spectral image are multi-scale fused.

[0008] The application also provides an adaptive polarization dark original color defogging and spectral prior fusion enhancement method, which adopts the adaptive polarization dark original color defogging and spectral prior fusion enhancement system described above, and comprises image registration, adaptive polarization dark original color defogging and near-infrared spectral fusion enhancement algorithm; the Sift registration algorithm is adopted to eliminate the image field sub-pixel displacement deviation caused by the difference in camera aperture position; the visible light cameras with polarization filters covered at the front ends of the lenses in the detection imaging module are used for imaging detection, and four polarization visible light fog images of I0(x), I 45 (x), I 90 (x) and I 135 (x) are collected; the atmospheric scattering model is optimized by combining the dark original color ambient light matrix L A (x) and the calculated polarization ambient light matrix L P (x), which can enhance the contrast of the image background area and restore the clear texture details of the image.

[0009] In a preferred embodiment, the main steps of the defogging algorithm are as follows:

[0010] 1) The threshold segmentation principle is used to calculate the fog and haze area proportion Ratio of the foggy image I(x):

[0011]

[0012] Ratio = Haze(x) / I(x) (2)

[0013] Where I(x) is the haze image of any selected polarization angle; White(x) represents the gray-white and pure white area; Gray(x) represents the gray haze covered area; Other(x) represents other areas; t1 and t2 represent threshold coefficients; Since the part close to gray-white in White(x) area contains haze, it is considered to be integrated with Gray(x) as the haze covered area Haze(x) of the whole image; Ratio represents the proportion coefficient of the haze covered area;

[0014] 2) Establishing an adaptive mapping relationship between the proportion coefficient of the haze covered area and the dehazing degree parameter p

[0015] For different scene haze images, after multiple experimental calculations, the haze area proportion is a specific constant, which is related to the haze concentration. The thicker the haze, the greater the density of bright pixels in the image, the more the number of pixels, the larger the interval span, and the higher the value. Test multiple groups of haze scenes with different haze concentrations, combine mathematical induction, and establish the mapping relationship expression as follows:

[0016] p = 1.054e 0.00059*Ratio (0≤R≤1) (3)

[0017] 3) Fine transmission t(x)

[0018] Based on the dark color prior principle, the mean pixel and guided filter operation are used to smooth the image background noise. The calculation steps are shown in the following formulas (4) and (5). The transmission range is further refined by laser ranging inversion, as shown in the following formulas (6) and (7):

[0019]

[0020]

[0021] t2(x) = e -β*d(x) (6)

[0022]

[0023] Where J dark (x) is the dark channel of the haze image; c represents the r, g, b color channels; Ω(x) is the local block area centered on pixel x; d is the dark channel guided filter image G guidedfilterpixel mean of (x), representing the average brightness of the image; ω is the brightness adjustment factor, whose upper limit is 0.9; β is the atmospheric attenuation coefficient, which is a global constant; d(x) is the real-world light distance of the laser measurement target area; t1(x) is the transmittance after dark channel solving; t2(x) is the transmittance after distance inversion; t(x) is the final refined transmittance map, which has clear texture and prominent edge details;

[0024] 4) Refined global atmospheric light value A

[0025] Estimate the bright channel map J of the image bright (x), and combine the guided filter map G of the dark channel guidedfilter (x), select the brightest pixel value of the image as the refined global atmospheric light value A; the refined global atmospheric light value A after solving is as shown in the following formula:

[0026]

[0027] 5) According to the refined transmittance t(x) calculated jointly by the dark color prior information and the laser ranging inversion, and combined with the refined global atmospheric light value A, the atmospheric attenuation term in the atmospheric scattering model can be effectively estimated, and the calculation steps are as shown in the following formula:

[0028] I(x) = J(x)t(x) + A(1-t(x)) (9)

[0029] LA(x) = A(1-t(x)) = min(min[d, ω]·G guidedfilter (x), 1-J dark (x)) (10)

[0030] Wherein, formula (8)-(9) is the atmospheric scattering model, J(x)t(x) is the direct attenuation term, A(1-t(x)) is the atmospheric attenuation term, L A (x) is the atmospheric attenuation term after solving, which is defined as the dark color ambient light matrix by us;

[0031] 6) Solve the defogging image jointly by the atmospheric scattering model, formula (9) can be changed, as shown in the following formula:

[0032]

[0033] 7) Calculate the Stokes vector for the four different angle polarized foggy images to represent the polarization degree of the whole foggy image, and the calculation steps are as shown in the following formula:

[0034]

[0035]

[0036] where S0(x) represents the total light intensity; S1(x) represents the linearly polarized light component along the x-axis direction; S2(x) represents the linearly polarized light component along the 45° axis direction; and DoLP(x) is the degree of polarization of the whole foggy image.

[0037] 8) The sky region of the registered multi-angle polarized foggy image is intercepted, and the degree of polarization of the sky region of the whole image is solved by using the Stokes vector relationship formula. The calculation steps are shown in the following formula:

[0038]

[0039] wherein, S0sky(x) represents the total light intensity of the sky region; S1sky(x) represents the linearly polarized light component of the sky region along the x-axis direction; S2sky(x) represents the linearly polarized light component of the sky region along the 45° axis direction; and DoLP sky (x) is the degree of polarization of the sky region of the foggy image solved;

[0040] 9) According to the calculated polarization prior information, the calculation steps are shown in the following formula:

[0041]

[0042] wherein, I total (x) represents the total light intensity, which is the same as S0(x); L P (x) is the atmospheric attenuation term solved after introducing the polarization information, which is defined as the polarized ambient light matrix;

[0043] 10) Combined with formula (10) and formula (15), the multi-angle polarization information: the polarized ambient light matrix L P (x) is introduced, and combined with the dark color ambient light matrix L A (x), the atmospheric scattering model is optimized, as shown in the following formula:

[0044]

[0045] wherein, γ is an image background adjustment factor introduced for optimizing the polarized ambient light matrix; and J(x) is an adaptive polarized dark color defogging image, which integrates the dark color visible light information and the multi-angle polarization information in the calculation process.

[0046] In a preferred embodiment, the main steps of the adaptive polarized dark color defogging image and near-infrared spectral image fusion algorithm are as follows:

[0047] (1) Select a near-infrared spectral image of a certain waveband, up-sample the near-infrared spectral image, and then register the adaptive polarized dark color defogging image;

[0048] (2) The up-sampled and registered near-infrared spectral image is fused and enhanced with the adaptive polar dark channel dehazing image, and a GFCE guided filter context enhancement and multi-scale decomposition algorithm is adopted, and the calculation steps are as shown in the following formula:

[0049]

[0050]

[0051]

[0052]

[0053] Wherein, D R represents a near-infrared spectral image, D V represents an adaptive polar dark channel dehazing image; is a large-scale layer; j represents a layer number; i=0 represents texture details, and i=1 represents edge features; C represents a fusion weight; is a base layer; represents cascaded high-frequency texture details, represents cascaded high-frequency edge features; is a small-scale layer; M(x) is a fused and enhanced image after combining information of each scale layer;

[0054] According to actual use requirements, the visibility of a target region is calculated before and after image dehazing, the visibility is greatly improved, it is indicated that a visible distance that can be seen is farther, and the dehazing effect of the image is represented.

[0055]

[0056] Wherein I(x,y) max represents a brightest pixel value in a scene; I(x,y) min represents a darkest pixel value in the scene; V is the visibility of a target scene; V I(x) is the visibility of the scene before dehazing; V M(x) is the visibility of the scene after dehazing, V I(x) <V M(x) .

[0057] Compared with the prior art, the present application has the following beneficial effects:

[0058] 1.The multi-aperture polarization visible and near-infrared spectral fog imaging system of the present application: the multi-aperture structure of the system is compact and stable, the multi-dimensional light field information utilization rate is high, and the synchronous detection of the polarization information and the multi-band near-infrared spectral information of the scene at multiple angles can be realized. Compared with a single-aperture camera, it has more dimensions of light field information because the detector can simultaneously acquire information in three dimensions of polarization, visible light, and near-infrared spectrum, which is beneficial to fog imaging.

[0059] 2.The adaptive polarization dark channel defogging algorithm designed by the present application: for different fog concentration scenes, laser ranging inversion and dark channel prior information are used, and the polarization prior information is fully utilized, the dark channel ambient light and polarization ambient light matrix are combined to improve the overall dark situation of the image after dark channel processing, and the color and brightness of the image background area are restored, so that the defogging image is clearer.

[0060] 3.The multi-scale image fusion algorithm used by the present application integrates near-infrared spectral information into the adaptive polarization dark channel defogging image, improves the polarization light intensity information, and restores the overall contrast of the image; the invisible area caused by the residual local dark pixel block of the dark channel defogging image is effectively restored, so that the clarity and contrast of the defogging image are greatly improved.

[0061] 4.The detection imaging module and laser ranging module designed by the present application include a mechanical bearing platform of the camera and the laser range finder, which is used to fix the position of the camera and the laser range finder, and realizes the functions of synchronous image acquisition of multiple cameras and real-time detection of the actual distance of the scene target. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 It is a functional module schematic diagram of the multi-aperture polarization visible and near-infrared spectral fog imaging system;

[0063] Figure 2 It is a plane structure schematic diagram of the multi-aperture polarization visible and near-infrared spectral fog imaging system;

[0064] Figure 3 It is a three-dimensional structure simulation diagram of the multi-aperture polarization visible and near-infrared spectral fog imaging system;

[0065] Figure 4 It is a specific implementation step schematic diagram of the system and algorithm;

[0066] Figure 5 It is an adaptive polarization dark channel, near-infrared spectral fusion defogging algorithm flowchart;

[0067] Figure 6 It is a polarization visible light foggy image of different fog concentrations collected by the system;

[0068] Figure 7The binary image of the foggy area of the fog image (take the image with medium fog concentration as an example);

[0069] Figure 8 The refined transmittance image;

[0070] Figure 9 The adaptive polarized dark channel dehazing image;

[0071] Figure 10 The spectral image of multiple bands (show the spectral image of part of the collected near-infrared band);

[0072] Figure 11 The spectral image of the near-infrared 820nm band after upsampling and registration;

[0073] Figure 12 The fusion dehazing enhanced image (including thin fog, medium fog concentration, thick fog, etc.)

[0074] Figure 13 The dehazing visibility test process (including the physical diagram of the laser ranging instrument, the position diagram measured on site, the target area diagram after fog and dehazing (take the image with medium fog concentration as an example)). DETAILED DESCRIPTION

[0075] The application will be further described below in conjunction with the accompanying drawings and examples.

[0076] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0077] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application; as used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and furthermore, it should be understood that when the terms "comprise" and / or "include" are used in the specification, there is a feature, step, operation, device, component and / or combination thereof.

[0078] An adaptive polarized dark channel dehazing and spectral prior fusion enhancement system, referring to Figures 1-13The photoelectric fog-penetrating imaging system is composed of a detection imaging module, a laser ranging module, a power module, a signal processing module and a master control module; the detection imaging module is composed of a polarized filter array, a visible light camera and a multi-spectral camera array, and a visible light and multi-spectral camera lens array; the visible light camera array with different angle polarized filters at the front end of the lens is used to construct a polarized visible light fog-penetrating imaging layout; and the multi-band near-infrared spectrum fusion enhancement layout is combined to jointly realize a multi-aperture field of view fog-penetrating imaging enhancement layout.

[0079] According to actual use requirements, the detection imaging module is used for collecting external image information and transmitting signals to the signal processing module; the laser ranging module uses a laser range finder to detect the distance of a target scene and transmits signals to the signal processing module; the power module provides power supply for the normal operation of the system; the signal processing module provides a normal working signal frequency for the detection imaging module and pre-processes received signals; and the master control module is used for receiving pre-processed image signals transmitted by the signal processing module, then performing image registration, fog removal and fusion algorithm processing on the image through a matching special software, and presenting the final fog-removed image on a window interface.

[0080] According to actual use requirements, in order to effectively remove the haze and restore a clear target scene, the application designs a multi-aperture novel photoelectric imaging system, which has the advantage of multi-dimensional detection imaging compared with a single-aperture imaging system, and the multi-aperture structure improves the stability of the imaging equipment and the utilization rate of the imaging field of view.

[0081] According to actual use requirements, in combination with the imaging system, the application designs a self-adaptive polarized dark original color fog removal algorithm, which is based on laser inversion and dark original color prior information of visible light and increases multi-angle polarized information, solves the problem of color distortion of an image after dark original color prior fog removal processing by combining a dark original color ambient light matrix and a polarized ambient light matrix, and restores the background area detail information of a fog-removed image well. The algorithm combines the advantages of model fog removal and non-model enhancement, accurately identifies the fog and haze area of a scene with different fog concentrations through region segmentation, and adaptively restores a clear and fog-free scene.

[0082] According to the actual use requirements, the image pre-processing is carried out before the image defogging: image upsampling and registration, so as to eliminate the displacement deviation of the target scene in the polarized visible light image and the near-infrared spectrum image, keep the image field of view consistent, and the registration accuracy is high.

[0083] According to the actual use requirements, the main steps of the adaptive polarized dark original color defogging algorithm are as follows:

[0084] (1) The proportion of fog and haze area of the fog image I(x) is calculated by using the threshold segmentation principle:

[0085]

[0086] Ratio=Haze(x) / I(x) (2)

[0087] Wherein, I(x) is a fog image of any selected polarization angle; White(x) represents the gray-white and pure white area; Gray(x) represents the gray fog and haze covered area; Other(x) represents other areas; t1 and t2 represent threshold coefficients; Since the part close to gray-white in White(x) area contains fog and haze, it is considered to be integrated with Gray(x) as the whole fog and haze covered area Haze(x); Ratio represents the proportion coefficient of the fog and haze covered area.

[0088] (2) Establishing the adaptive mapping relationship between the proportion coefficient of the fog and haze covered area and the defogging degree parameter p

[0089] For different scenes of fog images, after multiple experimental calculations, the proportion of the fog and haze area is a specific constant, which is related to the fog and haze concentration. The brighter the pixel density in the image, the more the number of pixels, the larger the interval span, and the higher the value. Test multiple groups of fog scenes with different fog concentrations, combine mathematical induction method, and establish the mapping relationship expression as follows:

[0090] p=1.054e 0.00059*Ratio (0≤R≤1) (3)

[0091] (3) Fine transmission t(x)

[0092] Based on the dark original color prior principle, the mean pixel and guided filter operation are adopted to smooth the image background noise, retain the edge detail information of the image, and reduce the operation time. The calculation steps are shown in the following formula (4) and (5). The transmission range is further refined by laser ranging inversion, as shown in the following formula (6) and (7):

[0093]

[0094]

[0095] t2(x) = e -β*d(x) (6)

[0096]

[0097] where J dark (x) is the dark channel of the foggy image; c represents the r, g, b color channels; Ω(x) is a local block region centered at pixel x; d is the pixel mean of the dark channel guided filter map G guidedfilter (x), representing the average brightness of the image; ω is a brightness adjustment factor, whose upper limit is 0.9; β is the atmospheric attenuation coefficient, which is a global constant; d(x) is the real-world light distance of the target area measured by laser; t1(x) is the transmittance after dark channel solving; t2(x) is the transmittance inverted by distance; t(x) is the final refined transmittance map, which has clear texture and prominent edge details.

[0098] (4) Refined global atmospheric light value A

[0099] The calculation of the traditional global atmospheric light value is often limited to the 0.1% range of the brightest pixels in the image, which can only reflect the brightness of the foggy area in the air to a certain extent. Therefore, according to the inspiration of the dark color prior, the bright channel map J bright (x) of the image is estimated, and the guided filter map G guidedfilter (x) of the dark channel is combined, and the brightest pixel value in the image is selected as the refined global atmospheric light value A. This avoids the misestimation of the dark color prior for bright sky, pure white, and color vivid regions, and reduces the time-consuming of calculation. The refined global atmospheric light value A after solving is shown in the following formula:

[0100]

[0101] (5) According to the refined transmittance t(x) calculated by the dark color prior information and laser ranging inversion, and combined with the refined global atmospheric light value A, the atmospheric attenuation term in the atmospheric scattering model can be effectively estimated, and the calculation steps are shown in the following formula:

[0102] I(x) = J(x)t(x) + A(1-t(x)) (9)

[0103] L A (x) = A(1-t(x)) = min(min[d, ω]·G guidedfilter (x), 1-J dark (x)) (10)

[0104] where formula (9) is the atmospheric scattering model, J(x)t(x) is the direct attenuation term, A(1-t(x)) is the atmospheric attenuation term, L A(x) The atmospheric attenuation term after solving, we define it as the dark color ambient light matrix.

[0105] (6) Joint atmospheric scattering model to solve the fog image, formula (9) can be changed, as shown in the following formula:

[0106]

[0107] Where, in order to improve the image defogging effect, the denominator part of the ambient light matrix, we need to introduce the multi-angle polarization prior information part.

[0108] (7) Four different angle of polarization fog image Stokes vector calculation, to characterize the polarization degree of the whole fog image, the calculation steps are shown in the following formula:

[0109]

[0110]

[0111] Where S0(x) represents the total light intensity; S1(x) represents the linearly polarized light component along the x-axis direction; S2(x) represents the linearly polarized light component along the 45° axis direction; DoLP(x) is the polarization degree of the fog image.

[0112] (8) The sky area is intercepted after the registration of the multi-angle polarization fog image, and the polarization degree of the sky area is solved by using the relationship of Stokes vector, and the calculation steps are shown in the following formula:

[0113]

[0114] Where, represents the total light intensity of the sky area; represents the linearly polarized light component along the x-axis direction of the sky area; represents the linearly polarized light component along the 45° axis direction of the sky area; DoLP sky (x) is the polarization degree of the sky area of the fog image.

[0115] (9) According to the calculated polarization prior information: the total light intensity and polarization degree of the fog image and the sky area, the atmospheric attenuation term in the atmospheric scattering model can be effectively estimated, and the calculation steps are shown in the following formula:

[0116]

[0117] Where, I total (x) represents the total light intensity, which is the same as S0(x); L P (x) is the atmospheric attenuation term solved by introducing polarization information, which is defined as the polarization ambient light matrix.

[0118] (10) Combined with formula (10) and formula (15), the multi-angle polarization information is introduced: the polarized ambient light matrix L P (x), and combined with the dark ambient light matrix L A (x), the atmospheric scattering model is optimized, as shown in the following formula:

[0119]

[0120] Wherein, γ is the introduced image background adjustment factor, used to optimize the polarization ambient light matrix; J(x) is the adaptive polarization dark original color defogging image, which integrates the dark original color visible light information and multi-angle polarization information in the calculation process.

[0121] As preferred, according to the actual use requirement, the system is suitable for multi-dimensional defogging imaging scene. Among them, the camera array in the detection imaging module is used for synchronous acquisition of 4 different angle polarization visible light foggy images and multiple waveband near-infrared spectrum foggy images. For the polarization dark original color defogging image, by integrating the near-infrared spectrum prior information, the limitation of dark original color prior can be improved, and the clear scene can be restored for local black brown and invisible area; At the same time, the polarization light intensity information is enhanced, the color of the image background sky area is improved, and the texture detail information of the image is reserved. The main steps of near-infrared spectrum fusion enhancement algorithm are as follows:

[0122] 1) Select a certain waveband near-infrared spectrum image, such as 820nm waveband, and the spectrum image size is 410×218. Since the adaptive polarization dark original color image size is 1920×1080, there is a big difference in the spatial resolution of two dimensions, so the near-infrared waveband spectrum image is up-sampled and registered with the adaptive polarization dark original color defogging image.

[0123] 2) The up-sampled and registered near-infrared 820nm waveband spectrum image is fused and enhanced with the adaptive polarization dark original color defogging image, and the GFCE guided filtering context enhancement and multi-scale decomposition algorithm is adopted, and the calculation steps are as follows:

[0124]

[0125]

[0126]

[0127]

[0128] Wherein, D R represents the near-infrared spectrum image, D V represents the adaptive polarization dark original color defogging image. is a large-scale layer; j represents a layer number; i=0 represents texture details, and i=1 represents edge features; C represents a fusion weight value; is a base layer; represents cascaded high-frequency texture details, represents cascaded high-frequency edge features; is a small-scale layer; M(x) is a fusion enhanced image after information of each scale layer is combined.

[0129] As preferred, according to actual use requirements, the visibility of the target region is calculated before and after image defogging, the visibility is greatly improved, the visible distance that can be seen is farther, and the defogging effect of the image is represented.

[0130]

[0131] wherein I(x,y) max represents a brightest pixel value in the scene; I(x,y) min represents a darkest pixel value in the scene; V is the visibility of the target scene; V I(x) is the visibility of the scene before defogging; V M(x) is the visibility of the scene after defogging, V I(x) <V M(x) .

[0132] After the above steps, the defogged image is clearer, the target contrast is bright, the texture details are prominent, the color fidelity is better, the local dark pixel block is restored in brightness, the overall defogging effect and the image quality are improved, and the defogging efficiency of the system is further verified.

[0133] In summary, the present application has innovation in the system level, for the first time, a multi-angle polarization visible light imaging device and a multispectral imaging device are used for cooperative collection of a foggy target scene, and together form a multi-aperture polarization visible light and near-infrared spectral fog-penetrating imaging system. The imaging method is multi-dimensional, and the designed mechanical bearing platform has stability in structure. The present application also has innovation in the defogging algorithm level, and the defogging degree parameter is adaptively calculated for different concentrations of foggy scenes; the transmittance is calculated by laser ranging and is integrated into the dark color prior defogging algorithm, combined with polarization information, the dark color ambient light and polarization ambient light matrix are calculated, so that the overall color and background region brightness of the defogged image are restored well; at the same time, the near-infrared spectral information is integrated, the invisible area caused by the residual local dark pixel block after dark color defogging is effectively restored, the polarization light intensity information is enhanced, and the overall clarity and quality of the defogged image are greatly improved.

Claims

1. An adaptive polarization dark primary color dehazing and spectral prior fusion enhancement method, characterized in that, This includes image registration, adaptive polarization dark primary color dehazing, and near-infrared spectral fusion enhancement algorithms; the Sift registration algorithm is used to eliminate sub-pixel displacement deviation of the image field of view caused by differences in camera aperture position; Imaging and detection are performed by a visible light camera with a polarizing filter covering the front of the lens in the detection imaging module, acquiring I0(x) and I... 45 (x), I 90 (x) and I 135 (x) Hazy visible light images with polarization at four different angles; combined with the dark primary ambient light matrix L A (x) and the calculated polarization ambient light matrix L P (x), by optimizing the atmospheric scattering model, can enhance the contrast of the background region of the image and restore the clear texture details of the image. The steps of the dehazing algorithm are as follows: 1) The proportion of hazy areas (Ratio) in the hazy image I(x) is calculated using the threshold segmentation principle: Ratio=Haze(x) / I(x) (2) Where I(x) is a hazy image with an arbitrarily selected polarization angle; White(x) represents the gray and pure white regions; Gray(x) represents the gray haze-covered region; Other(x) represents other regions; t1 and t2 represent threshold coefficients; since the near-gray-white part of the White(x) region contains haze, it is considered to integrate it with Gray(x) to form the haze-covered region Haze(x) of the entire image; Ratio represents the ratio coefficient of the haze-covered region. 2) Establish an adaptive mapping relationship between the proportion coefficient of haze-covered area and the defogging degree parameter p. For foggy images in different scenarios, after multiple experimental calculations, the proportion of foggy areas is a specific constant, which is related to the fog concentration. The denser the fog, the greater the density of bright pixels in the image, the more pixels there are, the larger the range, and the higher the value. After testing multiple groups of foggy scenes with different fog concentrations, and combining mathematical induction, the following mapping relationship expression was established: p=1.054e 0.00059*Ratio (0≤R≤1) (3) 3) Refined transmittance t(x) Based on the dark primary color prior principle, average pixel and guided filtering operations are used to smooth the background noise of the image. The calculation steps are shown in equations (4) and (5) below. The transmittance range is obtained by laser ranging, and the transmittance is further refined as shown in equations (6) and (7) below: t2(x)=e -β*d(x) (6) Among them, J dark (x) represents the dark channel of a hazy image; c represents the r, g, and b color channels; Ω(x) is a local block region centered on pixel x; d is the dark channel guided filter map G. guidedfilter (x) represents the average pixel value, indicating the average brightness of the image; ω is the brightness adjustment factor, with an upper limit of 0.9; β t1(x) is the atmospheric attenuation coefficient, which is a global constant; d(x) is the real-world light distance of the laser measurement target area; t1(x) is the transmittance after solving the dark channel; t2(x) is the transmittance obtained by distance inversion; t(x) is the final refined transmittance map, which has clear texture and prominent edge details. 4) Refined global atmospheric light value A Estimated brightness channel map J of the image bright (x), and combined with the guided filter graph G of the dark channel guidedfilter (x), the brightest pixel value in the image is selected as the refined global atmospheric light value A; the refined global atmospheric light value A after solving is shown in the following formula: 5) Based on the refined transmittance t(x) calculated jointly from the dark primary color prior information and laser ranging inversion, and combined with the refined global atmospheric light value A, the atmospheric attenuation term in the atmospheric scattering model can be effectively estimated. The calculation steps are shown in the following formula: I(x)=J(x)t(x)+A(1-t(x)) (9) L A (x)=A(1-t(x))=min(min[d,ω]·G guidedfilter (x),1-J dark (x) (10) In this context, formulas (8)-(9) represent the atmospheric scattering model, J(x)t(x) is the direct attenuation term, A(1-t(x)) is the atmospheric attenuation term, and L... A (x) is the atmospheric attenuation term after the solution is obtained, which we define as the dark primary color ambient light matrix; 6) Solve the dehazed image using the combined atmospheric scattering model. Formula (9) can be modified as shown in the following formula: 7) Perform Stokes vector calculations on the four polarized hazy images at different angles to characterize the degree of polarization of the entire hazy image. The calculation steps are shown in the following formula: Where S0(x) represents the total light intensity; S1(x) represents the linearly polarized light component along the x-axis; S2(x) represents the linearly polarized light component along the 45° axis; DoLP(x) is the degree of polarization of the entire foggy image. 8) For the registered multi-angle polarized hazy image, the sky region is cropped. Similarly, the Stokes vector relation formula is used to solve for the degree of polarization of the sky region of the entire image. The calculation steps are shown in the following formula: in, This represents the total luminous intensity of the sky region; This represents the linearly polarized light component of the sky region along the x-axis. DoLP represents the linearly polarized light component of the sky region along the 45° axis. sky (x) is the degree of polarization of the sky region in the foggy image obtained by solving; 9) Based on the calculated polarization prior information: The calculation steps are shown in the following formula: Among them, I total (x) represents the total light intensity, which is the same as S0(x); L P (x) is the atmospheric attenuation term obtained after introducing polarization information, which we define as the polarization ambient light matrix; 10) Combining formulas (10) and (15), multi-angle polarization information is introduced: polarization ambient light matrix L P (x), and combined with the dark primary ambient light matrix L A (x), the atmospheric scattering model is optimized as shown in the following equation: Here, γ is an introduced image background adjustment factor used to optimize the polarization ambient light matrix; J(x) is an adaptive polarization dark primary color dehazing image, which incorporates dark primary color visible light information and multi-angle polarization information during the calculation process.

2. The adaptive polarization dark primary color dehazing and spectral prior fusion enhancement method according to claim 1, characterized in that, The main steps of the algorithm for fusing adaptive polarization dark primary dehazed images with near-infrared spectral images are as follows: (1) Select a near-infrared spectral image of a certain band, upsample the near-infrared spectral image, and then register it with the adaptive polarization dark primary color dehazing image. (2) The upsampled and registered near-infrared spectral image is fused and enhanced with the adaptive polarization dark primary dehazing image, and the GFCE guided filtering context enhancement and multi-scale decomposition algorithm is used. The calculation steps are shown in the following formula: Among them, D R Represents a near-infrared spectral image, D V This represents an adaptive polarization dark primary color dehazing image; It is a large-scale layer; j represents the layer number; i=0 represents texture details, i=1 represents edge features; C represents the fusion weight; It is at the grassroots level; Indicates cascaded high-frequency texture details. Indicates cascaded high-frequency edge features; M(x) is a small-scale layer; M(x) is a fused and enhanced image that combines information from various scale layers. Based on actual usage requirements, the visibility of the target area is calculated before and after image dehazing. The visibility is greatly improved, indicating that the visible distance is farther, which characterizes the dehazing effect of the image. Where I(x,y) max Represents the value of the brightest pixel in the scene; I(x,y) min V represents the darkest pixel value in the scene; V is the visibility of the target scene; V I(x) It refers to the visibility of the scene before the fog clears; V M(x) It refers to the visibility of the scene after defogging, V I(x) <V M(x) .

3. An adaptive polarization dark primary color dehazing and spectral prior fusion enhancement system, characterized in that, The adaptive polarization dark primary color dehazing and spectral prior fusion enhancement method described in claim 1 or 2 includes a detection imaging module, a laser ranging module, a power supply module, a signal processing module, and a main control module: wherein the detection imaging module consists of a polarization filter array, a visible light and snapshot multispectral camera array, and a visible light and snapshot multispectral camera lens array.

4. The adaptive polarization dark primary color dehazing and spectral prior fusion enhancement system according to claim 3, characterized in that, A snapshot-type spectroscopic camera located at the center of the detection and imaging module acquires near-infrared spectral images. Visible light cameras located around the detection and imaging module have polarization filters at different angles covering their lenses to acquire visible light images containing different polarization information. The signals are then transmitted to the signal processing module for preprocessing. A laser ranging module located above the detection and imaging module uses a laser rangefinder to detect the actual distance from the imaging end to the target scene and transmits the signal to the signal processing module. The module calculates the proportion coefficient of the haze area in images with different fog concentrations and maps it to the defogging degree parameters to achieve adaptive parameter adjustment. The range of transmittance is derived through laser ranging and incorporated with dark spectral prior information to refine the transmittance. Combined with polarization prior information, the background area of ​​the image is optimized to generate a new atmospheric scattering model for polarization dark spectral defogging imaging. The defogging image is then fused with the near-infrared spectral image at multiple scales.

Images