An adaptive shadow enhancement method and device for reconstructing images of smoke infrared digital holograms

Abstract

The application discloses a kind of self-adapting shadow enhancement method and device of smoke-passed infrared digital holographic reconstructed image, belong to image processing technical field.The application proposes a new image processing algorithm to enhance the shadow area caused by smoke-passed infrared digital holographic reconstructed image smoke sheltering, the algorithm is first using bilateral filter to remove noise to image, in combination with binaryzation, edge extraction, morphological algorithm divides shadow area, and in combination with bright area to achieve the purpose of enhancing the brightness of shadow area.The method disclosed in the application solves the problems of severe speckle noise, obvious shadow area, uneven brightness, missing object detail information and other problems of smoke-passed infrared holographic reconstructed image, can effectively enhance the image quality of smoke-passed infrared holographic reconstructed image, has significant effect in the aspects of shadow area enhancement and speckle noise removal, and belongs to the field of image processing technology.

Description

Technical Field

[0001] This invention specifically relates to an adaptive shadow enhancement method and apparatus for infrared digital holographic reconstruction images over smoke, belonging to the field of image processing technology. Background Technology

[0002] In recent years, holography has been considered a significant step forward and has been applied in many scientific and technological fields, including shape and strain measurement, optical image encryption, quantitative analysis of biological samples, real-time modeling and high resolution, and its widespread use as an experimental setup and light source in 3D imaging and display. Digital holography in the visible light band has become a crucial observation and detection method in the imaging field after years of research and development. However, in industrial applications, the limitations of traditional visible light digital holography are becoming increasingly apparent in environments with smoke or flame interference. Research has revealed that infrared lasers have a significantly higher ability to penetrate high-density object fields than visible light, allowing infrared holography to capture images through smoke and flames, opening up new application areas for holographic technology.

[0003] In 2003, Lambertus Hesselink and Sergei S. Orlov first used an infrared CCD to capture an infrared hologram. In 2009, Zhong Jie, Li Zuoyou, and others from the China Academy of Engineering Physics studied the application of infrared holography to particle fields, concluding that infrared laser coaxial holography has certain advantages in experimental diagnosis of small-size, high-density particle fields. In 2013, M. Locatelli, 1E. Pugliese, and others successfully imaged smoke and flames in the infrared range using lensless digital holography. Experiments verified that continuous laser digital holography allows for the recording of dynamic human-sized targets. Previous studies had not yet enhanced the reconstructed infrared holographic images; nor had they performed brightness correction for shadowed areas obscured by smoke.

[0004] Through experiments with infrared digital holography in a smoke field, it was found that the reconstructed image had severe speckle noise and obvious shadow areas. To address the speckle noise, this invention uses a bilateral filter to remove noise and preserve image details. Through multiple experiments, it was found that the distribution of shadow areas in the infrared holographic reconstructed image is relatively continuous and there is a clear boundary between the shadow areas and the bright areas. Based on this characteristic, an edge extraction algorithm and a morphological algorithm are proposed to extract the shadow areas and perform brightness correction. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, this invention provides an adaptive shadow enhancement method for infrared digital holographic reconstructed images through smoke, which solves many problems such as severe speckle noise, obvious shadow areas, uneven brightness, and lack of object detail information in infrared holographic reconstructed images through smoke. It can effectively enhance the image quality of infrared holographic reconstructed images through smoke, and has significant effects in enhancing shadow areas and removing speckle noise.

[0006] The technical solution of this invention is: an adaptive shadow enhancement method for infrared digital holographic reconstructed images through smoke, specifically including the following steps:

[0007] Bilateral filtering is used to remove speckle noise from the reconstructed image of the original infrared hologram. The formula for the bilateral filter is:

[0008]

[0009] ω(x,y,k,l)=d(x,y,k,l)r(x,y,k,l) ​​(25)

[0010] In the formula, f(x, y) is the input image, g(x, y) is the output image, f(k, l) is the neighboring pixel of the filter, (x, y) and (k, l) are both pixel coordinates, ω(x, y, k, l) is the weight coefficient, which is the product of the domain kernel d(x, y, k, l) and the range kernel r(x, y, k, l), σ d Let σ be the global variance. r This represents the local variance.

[0011] The formula for reconstructing the foreground and background of an image using binarization segmentation is shown below:

[0012] ω0=N0 / (M×N) (26)

[0013] ω1=N1 / (M×N) (27)

[0014] N0+N1=M×N (28)

[0015] ω0+ω1=1 (29)

[0016] μ=ω0×μ0+ω1×μ1 (30)

[0017]

[0018] σ 2 =ω0(μ0-μ) 2 +ω1(μ1-μ) 2 (33)

[0019] In the formula, N0 represents the number of pixels in the foreground, N1 represents the number of pixels in the background, M×N represents the total number of pixels, ω0 represents the proportion of foreground pixels, ω1 represents the proportion of background pixels, μ0 represents the foreground variance, μ1 represents the background variance, T represents the binarization threshold, L=255, μ represents the inter-class variance, and p i σ represents the ratio of pixel i to the total number of pixels. 2 n represents the maximum inter-class variance. i Let i be the number of pixels with pixel value i, and let Thr be the image binarization threshold, calculated by the following formula:

[0020]

[0021] Let the binarized image be g2(x,y), which can be obtained by the following formula.

[0022]

[0023] The Sobel operator is used to extract edge information from the binarized image, and then dilation and padding operations are performed on the edge-extracted image. The Sobel operator is calculated as follows:

[0024]

[0025] g3(x, y) = |g2(x, y)*G x |+|g2(x,y)*G y | (38)

[0026] In the formula G x G is the horizontal convolution template. y The vertical convolution template is g3(x, y), and the edge extraction image is g3(x, y).

[0027] Furthermore, by combining the maximum connected component algorithm, the contours of the target image for morphological segmentation are labeled according to the maximum connected components. Let the region with the largest area be A1, then the target object f is segmented. o (x, y) can be calculated from equation (39).

[0028] f o (x, y) = f(x, y) * A1 (39)

[0029] In the formula, f(x, y) is the input image.

[0030] Edge information of the target object is extracted using the Sobel operator, followed by dilation and padding. Similarly, the maximum connected component algorithm is used to extract the maximum connected component, labeled A2, to segment the shaded region f. s (x, y) is

[0031] f s (x, y) = f o(x, y)(A1-A1∩A2) (40)

[0032] bright area f in the target object b (x, y) is

[0033] f b (x, y) = f o (x, y)(A1∩A2) (41).

[0034] The algorithm of this invention performs a shadow removal method to ensure global uniform illumination, which has better robustness in complex scenes; it removes shadows according to the pixel illuminance ratio of each region of the image, and removes shadows by calculating the distribution of shadows in bright regions, resulting in a reconstructed image with adaptive shadow enhancement.

[0035]

[0036] In the formula E(f S (x, y)), E(f) b (x, y) represents the mean of the shaded and bright areas, and SD(f) represents the mean of the shaded and bright areas. S (x, y)), SD(f) b (x, y) represents the variance of the shaded region and the bright region, f′ S (x, y) represents the shadow area after brightness adjustment.

[0037] Another object of the present invention is to provide an adaptive shadow enhancement device for infrared digital holographic reconstruction over smoke, including H e -N e The system comprises a laser 1, an infrared laser 2, a total reflection mirror 3, a first beam splitter 4, a first beam expander 5, a first collimating lens 6, a second beam splitter 7, a second beam expander 8, a second collimating lens 9, an object under test 10, an attenuator 11, a smoke generator 12, a CCD image sensor 13, and a computer 14. The laser beam is output from the infrared laser 2 and splits at the first beam splitter 4 into a horizontal reference beam path and a vertical object beam path with an intensity ratio of 1:1. The vertical object beam is reflected by the total reflection mirror 3 and then passes through the first beam expander 5, the first collimating lens 6, and the smoke generator 12 before illuminating the object under test. The horizontal reference beam, after passing through the attenuator 11, the second beam expander 8, and the second collimating lens 9, interferes with the object beam at the second beam splitter 7. This interference is recorded by the CCD image sensor 13 and input into the computer 14. Finally, after processing using the aforementioned algorithm, an adaptive shadow-enhanced image after infrared digital holographic reconstruction with smoke is obtained.

[0038] Compared with the prior art, the beneficial effects of the present invention are:

[0039] This invention first uses bilateral filtering to remove noise from the image, then combines binarization, edge extraction, and morphological algorithms to segment the shadow region, and combines bright regions to enhance the brightness of the shadow region; while suppressing speckle noise, the overall contrast is significantly improved, and the brightness correction effect of the shadow region is good. Attached Figure Description

[0040] Figure 1 This is a process flow diagram of the method described in this invention;

[0041] Figure 2 This is a schematic diagram of the device described in this invention, wherein the straight lines with arrows indicate the direction of light propagation;

[0042] In the diagram: 1-He-Ne laser; 2-Infrared laser; 3-Total reflection mirror; 4-First beam splitter; 5-First beam expander; 6-First collimating lens; 7-Second beam splitter; 8-Second beam expander; 9-Second collimating lens; 10-Object under test; 11-Attenuator; 12-Smoke generator; 13-CCD image sensor; 14-Computer.

[0043] Figure 3 To simulate the effect of smoke imaging; a - the object being imaged; b - image through smoke;

[0044] Figure 4 The original infrared hologram of the smoke;

[0045] Figure 5 The results are processed step by step;

[0046] Figure 6 Comparison of shadow area enhancement processing results: a-Original reconstructed image; b-Enhanced shadow area image. Detailed Implementation

[0047] The present invention will be further described in detail below with reference to specific embodiments, but the scope of protection of the present invention is not limited to the content described.

[0048] The apparatus used in the embodiments of the present invention is as follows: Figure 2As shown, the system includes a He-Ne laser 1, an infrared laser 2, a total reflection mirror 3, a first beam splitter 4, a first beam expander 5, a first collimating lens 6, a second beam splitter 7, a second beam expander 8, a second collimating lens 9, an object under test 10, an attenuator 11, a smoke generator 12, a CCD image sensor 13, and a computer 14. The laser beam is output from the infrared laser 2 and splits into a horizontal reference beam path and a vertical object beam path with an intensity ratio of 1:1 at the first beam splitter 4. The vertical object beam is reflected by the total reflection mirror 3 and then passes through the first beam expander 5, the first collimating lens 6, and the smoke generator 12 before illuminating the object under test. The horizontal reference beam passes through the attenuator 11, the second beam expander 8, and the second collimating lens 9 and interferes with the object beam at the second beam splitter 7. The interference is recorded by the CCD image sensor 13 and input into the computer 14. Finally, after processing by the above algorithm, an adaptive shadow enhancement image after infrared digital holographic reconstruction with smoke is obtained. The infrared laser 2 described in this embodiment is model MIL-III, with a center wavelength of 1064nm. It is a semiconductor infrared laser that operates in continuous mode, with a maximum adjustable laser power of 115.3mW and a transverse mode of TEM00. The emitted beam has a divergence angle of 1.5mrad, a beam diameter of 1.329mm, and a coherence length of 20cm.

[0049] Example 1

[0050] Step 1: In this embodiment, the object to be tested is a medal with a radius of 3cm, and the simulated smoke state is as follows: Figure 3 As shown in a and b; the optical lenses used in the optical path device are all ordinary infrared-transmitting lenses, and S is a smoke generator with a smoke output of 6000 cuft / min; this invention selects a CCD with a resolution of 1920×10880 and a pixel size of 2.9μm; the infrared hologram of this object captured by the above device is shown in Figure 1. Figure 4 As shown.

[0051] Step 2: Bilateral filtering is used to remove speckle noise from the reconstructed image of the original infrared hologram. Speckle noise is a type of black and white speckle, and its impact on the grayscale value of the image is uncertain. This invention uses bilateral filtering to denoise the reconstructed image. The results show that the algorithm can retain more detailed information while removing speckle noise.

[0052] Step 3: This invention uses the maximum inter-class variance method for image binarization; after image binarization segmentation according to the threshold obtained by Otsu's method, the inter-class variance between the foreground and background images is maximized.

[0053] Step 4: Use the Sobel operator to extract edge information from the binarized image.

[0054] Step 5: Perform dilation and filling operations on the edge-extracted image.

[0055] Step 6: Combining the maximum connected component algorithm, the contours of the morphologically segmented target image are marked according to the maximum connected component. Finally, the image is filled according to the marked contours, and the area of ​​each filled region is calculated. The region with the largest area is the target object. In this embodiment, it is a medal, which is denoted as A1.

[0056] Step 7: Next, the shadow region in the target object needs to be segmented and enhanced. The steps are as follows: use the Sobel operator to extract edge information of the target object, then perform dilation and filling; similarly, use the maximum connected component algorithm to extract the maximum connected component, and then segment out the shadow region.

[0057] The processing results of steps two through seven are as follows: Figure 5 As shown.

[0058] Step 8: Finally, the algorithm of this invention executes a shadow removal method to ensure globally uniform illumination, resulting in better robustness in complex scenes; shadows are removed based on the pixel illuminance ratio of each region of the image, by calculating the distribution of shadows in bright regions:

[0059]

[0060] In the formula E(f S (x,y)), E(f) b (x,y)) represents the mean of the shaded and bright areas, E(f) S (x,y)), SD(f b (x,y) represents the variance of the shaded and bright regions, f′ S (x,y) represents the shadow area after brightness adjustment, f S (x,y) represents the grayscale value of the shaded area, f b (x,y) represents the grayscale value of the bright area.

[0061] Figure 6 The image below shows a comparison between the original reconstructed hologram and the reconstructed hologram after shadow region enhancement. Through subjective judgment and analysis of the enhancement effect of the reconstructed image, it can be found that the algorithm proposed in this invention basically eliminates the speckle noise of the image, preserves the texture details, and the comparison shows that the shadow region correction effect is quite good.

[0062] Example 2

[0063] The method and apparatus described in this invention can be used for fire search and rescue, and can image through flames or smoke. It can also enhance the shadow areas of the reconstructed image, which can effectively improve the rapid search and rescue of people who are obscured by flames or smoke in fire scenes.

[0064] The method described in this embodiment is the same as that in Embodiment 1, except that the object to be detected is a human image; lensless digital holography is used in the infrared range to successfully image smoke and flames; it is worth noting that digital holography using continuous lasers can record targets of dynamic human body size.

[0065] Studies have shown that digital infrared holography can easily detect moving people through smoke and flames. This invention, by numerically processing the acquired holograms, can improve image quality a posteriori, enabling the imaging technology to penetrate fires and potentially offering a completely new opportunity to save human lives.

[0066] Example 3

[0067] In modern warfare, smoke camouflage can confuse and deceive the enemy, improving battlefield survivability, and is therefore highly favored by armies around the world. In recent years, armies around the world have attached great importance to the research, development and production of smoke camouflage equipment, and have widely absorbed and applied new technologies, new materials and new processes to promote the development of smoke camouflage equipment and improve its tactical and technical performance. The method and device described in this invention can be used for reconnaissance of enemy targets behind smoke in war, and can conduct rapid and accurate reconnaissance of enemy targets camouflaged by smoke.

[0068] The method described in this embodiment is the same as in Embodiment 2, and the object to be tested is also a human figure. However, this human figure should be a dynamic human figure, with a relatively higher range of motion and speed compared to Embodiment 2. Scientific research has found that holographic stereoscopic imaging technology has been used to obtain dynamic displays of living people by using pulsed laser sources and photorefractive polymers as recording media. In this invention, continuous lasers and simple infrared sensors are used to acquire holograms of human body size. These holograms can be digitally reconstructed for accurate real-time display. Similarly, by numerically processing the acquired holograms, this invention can improve image quality a posteriori, enabling the imaging technology to penetrate smoke, enhance obscured areas, and determine the enemy's location. Combining this invention with drones in the future could provide a completely new direction for military reconnaissance.

Claims

1. An adaptive shadow enhancement method of a reconstructed image of a smoke-penetrated infrared digital hologram, characterized by, Specifically comprising the following steps: (1) using bilateral filtering to remove speckle noise of original infrared digital holographic reconstruction image, and combining with binarization to segment foreground and background; (2) using Sobel operator to extract edge information of the binarized image, then performing inflation and filling operation on the edge extraction image, further combining with the maximum connected region algorithm, marking the contour of the morphologically segmented target image according to the maximum connected region, and segmenting out the target object; (3) using Sobel operator to extract edge information of the target object, then performing inflation and filling processing; using the maximum connected region algorithm to extract the maximum connected domain, and segmenting out the shadow area; (4) removing the shadow according to the pixel illumination ratio of each region of the image, removing the shadow by calculating the distribution of the shadow in the bright area, and obtaining the image after adaptive shadow enhancement of the reconstruction image; In step (2), the maximum connected region algorithm is used to extract the maximum connected region and segment the target object. The steps of the maximum connected region algorithm are as follows: first, a pixel point is randomly selected, and it is determined whether the pixel point is located in the connected region; then, the 8-neighborhood of the point is searched, and if the points in the 8-neighborhood are also located in the connected region, the points are defined as the next random pixel; then, the above operation is repeated until no new random pixel point is generated, and then one maximum connected region classification is completed; finally, after all connected region classification markers are completed, the number of each class is counted, and the connected region with the largest number is the maximum area region, and the maximum area region is defined as A1, and the target object f is segmented o (x,y) can be calculated by equation (18): f o (x,y) = f(x,y) * A1 (18); In the formula, f(x, y) is an input image; The shadow region segmentation method in step (3) is similar to the target object segmentation method, the difference is that the segmentation of the shadow region is performed on the target object, and the target object is not binarized; the maximum connected region algorithm is also used to extract the maximum connected region, which is marked as A2, and the shadow region f s (x,y) is f s (x,y) = f0(x,y) (Al - Al n A2) (19); Bright regions f in the target object b (x, y) is f b (x,y) = f o (x,y) (A1∩A2) (20); In step (4), the shadow is removed according to the pixel illumination ratio of each region of the image, and the shadow is removed by calculating the distribution of the shadow in the bright area, and the specific formula is as follows: In the formula E(f S (x,y)), E(f) b (x,y) represents the mean of the shaded and bright areas, and SD(f) represents the mean of the shaded and bright areas. S (x,y)), SD(f b (x,y) represents the variance of the shaded and bright regions, f′ S (x,y) represents the shadow area after brightness adjustment, f s (x,y) Shaded area before brightness adjustment.

2. The method of claim 1, wherein the method is applied to reconstructing a smoke- obscured infrared digital hologram. The bilateral filter formula for removing speckle noise of the original infrared digital holographic reconstruction image is as follows: ω (x, y, k, l) = d (x, y, k, l) r (x, y, k, l) (4) ; where f(x, y) is the input image, g(x, y) is the output image, f(k, l) is the pixel point in the filter neighborhood, (x, y) and (k, l) are the pixel point coordinates, ω(x, y, k, l) is the weight coefficient, is the product of the domain kernel d(x, y, k, l) and the value kernel r(x, y, k, l), σ d is the global variance, and σ r is the local variance.

3. The method of claim 1, wherein the method is applied to a smoke- obscured infrared digital holographic reconstructed image. In step (1), the method for combining binarization to segment foreground and background is as follows: using the maximum inter-class variance method to perform image binarization processing, after image binarization segmentation according to the threshold value obtained by the Otsu method, the inter-class variance of the foreground and background image is maximum, thereby segmenting the foreground and background and eliminating the influence of the shadow part on the target object extraction; The calculation formula is as follows: ω0= N0 / (M x N) (5) ; ω1= N1 / (M x N) (6) ; N0+ N1= M x N (7) ; ω0+ ω1= 1 (8) ; μ= ω0 x μ0+ ω1 x μ1 (9) ; σ 2 = ω0(μ0- μ) 2 + ω1(μ1- μ) 2 (12) where N0 is the number of pixels of the foreground, N1 is the number of pixels of the background, MxN is the total number of pixels, ω0 is the proportion of the foreground pixels, ω1 is the proportion of the background pixels, μ0 is the variance of the foreground, μ1 is the variance of the background, T represents the binary threshold, L = 255, μ is the inter-class variance, and p i is the ratio of the total pixels occupied by the pixel point i, σ 2 is the maximum inter-class variance, n i is the number of pixel points with the pixel value i; and the image binary threshold Thr is calculated by the following formula: Let the binarized image be g2(x, y), which can be obtained by the following formula 4. The adaptive shadow enhancement method for infrared digital holographic reconstructed images through smoke according to claim 1, characterized in that: It is to extract edge information of the binarized image using Sobel operator, and for the binarized image g2(x, y), the calculation formula of Sobel operator is as follows: g3(x, y) = |g2(x, y) * G x |+|g2(x, y) * G y | (17) where G x is a horizontal direction convolution template, G y is a vertical direction convolution template, and g3(x, y) is an obtained edge extraction image.

5. The apparatus for use in the method of any one of claims 1 to 4, characterized in that: It comprises He-Ne laser (1), infrared laser (2), full reflection mirror (3), first beam splitter (4), first beam expander (5), first collimating lens (6), second beam splitter (7), second beam expander (8), second collimating lens (9), object to be measured (10), attenuating sheet (11), smoke generating device (12), CCD image sensor (13), computer (14); the light beam is output from the infrared laser (2), is divided into horizontal reference light path and vertical object light path with the light intensity ratio of 1:1 at the first beam splitter (4), the vertical object light is reflected by the full reflection mirror (3), is irradiated on the object to be measured through the first beam expander (5), the first collimating lens (6) and the smoke generating device (12), the horizontal reference light is interfered with the object light at the second beam splitter (7) after passing through the attenuating sheet (11), the second beam expander (8) and the second collimating lens (9), is recorded by the CCD image sensor (13), is input into the computer (14), and finally the adaptive shadow enhancement image after the smoke infrared digital holographic reconstruction is obtained through the above algorithm processing.

6. The apparatus of claim 5, wherein: The infrared laser (2) is MIL-III.

Images