An infrared dim small target detection method based on space-frequency joint self-adaptation

Abstract

The application provides an infrared weak small target detection method based on space-frequency joint self-adaption, and belongs to the field of infrared weak small target detection application. The application firstly extracts high-frequency residual features through multi-scale Gaussian pyramid and improved wavelet decomposition, and generates a space-frequency fusion saliency map in combination with local contrast enhancement. Secondly, a gradient-guided dynamic threshold segmentation strategy is designed, and a local gradient amplitude is used to adaptively adjust a threshold parameter. The threshold segmentation is converted into a binary image, and a connected region analysis is performed to estimate a target size. Through the prior information of the size, the size of a structure element of dilation and corrosion is designed, and an improved adaptive Top-Hat transformation is used to realize accurate detection of the target. The application can effectively improve the accuracy of infrared weak small target detection, has strong robustness, and can adapt to changes of different target sizes and background conditions. The application can be applied to target detection and identification in the fields of military reconnaissance, security monitoring and other fields.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of infrared dim small target detection, and more particularly relates to a self-adaptive Top-Hat infrared dim small target detection method based on space-frequency combination. BACKGROUND

[0002] Infrared detection technology plays an irreplaceable role in military and civilian fields such as missile early warning, forest fire prevention, and border monitoring due to its all-weather working ability and strong anti-interference. As the core link of the infrared system, the performance of the detection algorithm of the dim small target (such as a drone) directly determines the detection accuracy and response speed of the entire equipment. Due to the imaging mechanism, the target at a long distance usually appears as a diffuse spot with a gray level mutation in the infrared image, lacking structured features such as texture and shape. After superimposing the detector noise, atmospheric turbulence interference and other factors, the signal-to-clutter ratio (SCR) of the target is often lower than 3 dB, and it is easy to be submerged in noise in a complex ground background (such as clouds, sea waves, and urban heat sources), leading to the dual dilemma of frequent false alarms and high miss detection rate for traditional algorithms. This characteristic makes the infrared dim small target detection a difficult problem that has long plagued the academic community.

[0003] In traditional target detection methods, the detection method based on spatial filtering mainly realizes clutter suppression by constructing a background estimation model. Typical algorithms such as the Max-Median filter use median calculation in a sliding window to eliminate salt and pepper noise, and the Top-Hat transform separates the background and the target through morphological closing and opening operations. Although this method has advantages such as high computational efficiency and simple deployment, it is difficult for a fixed-size structural element to adapt to the dynamic changes of scene characteristics when facing a non-uniform background (such as the vortex structure of the cloud edge), resulting in serious background suppression residual. To break through this limitation, researchers proposed an improved scheme using a ring-shaped structural element, which significantly improved the detection performance through the collaborative optimization of the inner and outer structures. However, the morphological method has strong dependence on the size and shape of the structural element, and when the target size exceeds the preset range, it is easy to cause target morphological distortion or insufficient background penetration. In actual applications, the size information of the target may be unknown, which limits the application of the Top-Hat method. In addition, the detection method based on frequency domain analysis has also attracted attention. This method extracts the frequency energy features of the target through Fourier transform, wavelet decomposition and other means, and realizes separation by using the frequency difference between the target and the background. However, this method still causes significant residual noise in the complex background (such as cloud texture), which restricts the further improvement of the detection accuracy. SUMMARY

[0004] To address the aforementioned technical problems, this invention proposes an adaptive Top-Hat infrared target detection method based on space-frequency fusion. This method extracts high-frequency residual features through multi-scale Gaussian pyramids and improved wavelet decomposition, combines this with local contrast enhancement to generate a space-frequency fusion saliency map, obtains prior information about the approximate size of the target based on the saliency map, and adaptively sets the size of the improved Top-Hat structuring element according to the target prior information, thereby effectively suppressing background and noise.

[0005] To achieve the above objectives, this invention designs an adaptive Top-Hat infrared weak target detection method based on space-frequency joint detection, comprising:

[0006] The input infrared image I(x,y) is read and converted into a grayscale image G(x,y). Then, the spatial saliency of the grayscale image G(x,y) is calculated based on local contrast to obtain the spatial saliency map R. s On the other hand, the frequency domain saliency of the grayscale image G(x,y) is calculated based on multi-scale wavelet decomposition, resulting in the frequency domain saliency map R. w .

[0007] The obtained spatial saliency map R s Frequency domain saliency plot R w Weighted fusion is performed to achieve spatial-frequency saliency fusion, and the saliency map S after fusion is analyzed. f Gaussian filtering is used to eliminate isolated noise points, making the target area more coherent.

[0008] Calculate and statistically analyze the saliency plot S. f The gradient magnitude ▽S is used to guide the dynamic threshold for segmentation of the saliency map.

[0009] Calculate the equivalent radius r of the largest suspected target in the binary image B(x,y) after thresholding. e According to the equivalent radius r e Set the size of the structuring element for expansion and erosion operations.

[0010] Finally, a morphological Tophat operation is performed on the input image I(x,y) to further refine the target location.

[0011] Furthermore, the spatial saliency calculation based on local contrast includes:

[0012] A 9×9 sliding window is used, which can be divided into nine 3×3 sub-blocks, such as... Figure 2 As shown, the image G(x,y) is traversed from left to right and from top to bottom. During the traversal, the image region corresponding to the sub-block is denoted as I. i Let i = 0, 1, 2, ..., 8. First, calculate the mean of the central sub-region image, denoted as μ.o Then the pixel values of the remaining 8 neighborhood sub-regions are sorted respectively, and the median value (denoted as m i ) is taken, and the absolute deviation of all pixels in each sub-region from the median value (denoted as e i j Finally, the absolute deviation values of all pixels in the 8 neighborhoods are taken as the median value M, and the processing process is shown in the following formula.

[0013]

[0014] Where median() represents taking the median value; denotes the gray value of the jth pixel point in the middle sub-region; denotes the gray value of the jth pixel point in the sub-region I i .

[0015] Then the local contrast C L (x,y) of the image pixel (x,y) corresponding to the center point of the 9x9 window is calculated according to the following formula:

[0016]

[0017] Where ε is a very small constant to avoid division by zero. The introduction of the median value can reduce the interference of abnormal values (such as impulse noise) on the estimation of the background brightness. The larger the LC (Local Contrast) value, the more obvious the difference between the target region and the surrounding background, and the target is determined; if the LC value tends to 0, it is determined as background.

[0018] According to the above process, the local contrast of each pixel is obtained by processing all pixels of G(x,y), and the local contrast is taken as the spatial saliency value to obtain the spatial saliency map (denoted as R s ).

[0019] Further, the frequency domain saliency calculation based on multi-scale wavelet decomposition includes:

[0020] A Gaussian pyramid is constructed, and the gray-scale image G(x,y) is down-sampled by 3 layers to realize multi-scale feature extraction. Haar wavelet transform is performed on each layer to extract the horizontal-vertical detail component where k ∈ {0,1,2}, corresponding to 3 different pyramid layers, and the frequency domain residual R w (x,y) of the image pixel (x,y) is calculated according to the following formula, and the residual value is taken as the frequency domain saliency to obtain the frequency domain saliency map R w :

[0021]

[0022] wherein, is the mean value of the k-th layer sub-band coefficients, and the square operation is used to enhance the contrast between weak targets and background. k is the k-th layer weight coefficient, and the sum of all layer weight coefficients is 1, and higher layer is given higher weight to suppress the noise of the bottom layer.

[0023] Further, the spatial saliency R s and the frequency saliency R w are fused by weighting to obtain a spatial-frequency saliency map S f :

[0024] S f = a · R w + (1-a) · R s

[0025] wherein a is a weight coefficient to balance the frequency and spatial saliency information, and 0

[0026] Further, the saliency map is segmented according to the gradient-guided dynamic threshold, and the calculation formula of the threshold T is:

[0027] T(x, y) = μ f + k x σ f

[0028] wherein μ f and σ f represent the mean value and standard deviation of the saliency map S f respectively; k is a dynamic coefficient which is dynamically adjusted according to the gradient , and the calculation formula of k is:

[0029]

[0030] wherein, is the mean value of the gradient, is the maximum value of the gradient, and β is a scaling factor which is generally set as 1 by default, and can be adjusted appropriately when the scene shows extremely low signal-to-clutter ratio.

[0031] Further, in the structure element scale setting step, firstly, the connected regions of the binary image B(x, y) are analyzed, and the width w and height h of the circumscribed rectangle of the maximum connected region are calculated, the rectangular area is converted into an equivalent radius r e according to the assumption that the target is approximately circular, and the structure element size of the dilation and erosion operation is set according to the radius.

[0032] The structure element B io of the dilation operation is composed of a square structure element B o and a circular structure element B i , and B i is inlaid in the center of B o , and Bi radius equal to r e , B o radius r e long 2 pixel unit length; the structure element of the erosion operation is composed of a circular structure element B b , B b radius r e short 2 pixel unit length.

[0033] Overall, the above technical solutions conceived by the present application have the following beneficial effects:

[0034] (1) The method of the present application uses a Gaussian pyramid to perform multi-scale decomposition on an infrared image, extracts high-frequency residual features of each scale in combination with an improved Haar wavelet transform, breaks through the limitations of traditional linear residuals, preserves the multi-scale characteristics of the target, and enhances the saliency of small targets;

[0035] (2) The method of the present application performs weighted fusion of the frequency domain saliency based on wavelet residuals and the spatial domain saliency based on spatial local contrast, generates a comprehensive saliency map through normalization processing and Gaussian filtering, and the method combines the frequency domain noise resistance and spatial domain detail preservation ability, and the target enhancement effect is obviously better than that of a single domain method;

[0036] (3) The method of the present application dynamically adjusts the threshold parameter according to the local gradient amplitude of the saliency map, automatically reduces the threshold strictness when the background complexity is high, and improves the detection sensitivity when the complexity is low, compared with the traditional fixed threshold method, which can effectively reduce the missed detection rate and false alarm rate;

[0037] (4) The method of the present application adaptively adjusts the size of the structure element of the dilation and erosion operation according to the size of the suspected target, adapts to the detection needs of targets of different sizes, effectively suppresses background noise and clutter interference, and further improves the detection accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 : The flow chart of the adaptive Top-Hat infrared dim and small target detection method based on space-frequency combination provided by the embodiment of the present application;

[0039] Figure 2 : The sliding window schematic diagram provided by the embodiment of the present application;

[0040] Figure 3 : The Gaussian pyramid structure schematic diagram provided by the embodiment of the present application;

[0041] Figure 4 : The structure element diagram provided by the embodiment of the present application;

[0042] Figure 5The structural element size schematic diagram provided by the embodiment of the present application. DETAILED DESCRIPTION

[0043] The above description is only a summary of the technical solutions of the present application. In order to make the purpose, technical solutions and advantages of the present application more clear and obvious, the present application will be described in detail below with reference to the drawings and the preferred examples of the present application, and the present application will be described in detail Figure 1 . It should be understood that the specific examples described herein are only used to explain the present application, and are not used to limit the present application.

[0044] An adaptive Top-Hat infrared dim small target detection method based on space-frequency joint is designed in the embodiment, and the following is a specific embodiment:

[0045] Reference Figures 1-5 , the embodiment is divided into three stages; in the first stage, the saliency of the dim small target is enhanced and the background noise is suppressed through joint analysis of space and frequency domain features; in the second stage, the gradient guided segmentation saliency map is used to preliminarily locate the suspected target and estimate its size, so as to provide prior information for subsequent morphological operation; in the third stage, based on the prior information of the target size, morphological Top-Hat transformation is carried out to accurately locate the dim small target and suppress the background residue. The specific steps are as follows:

[0046] (1) read the input infrared image I(x, y) and convert it into a gray image G(x, y), (x, y) being the image pixel coordinates.

[0047] (2) construct a space domain saliency map. A sliding window with a size of 9x9 is adopted, and the window is divided into 9 3x3 sub-blocks, as shown in Figure 2 , the image G(x, y) is traversed from left to right and from top to bottom. In the traversal process, the image region corresponding to the sub-block is denoted as I i , i = 0, 1, 2, …, 8. First, the mean value of the center sub-region image is calculated, denoted as μ o ; then the pixel values of the remaining 8 neighborhood sub-regions are sorted, and the median value (denoted as m i ) is taken, and the absolute deviation (denoted as ) of each pixel in each sub-region from the median value is calculated. Finally, the median value M of the absolute deviation values of all pixels in the 8 neighborhoods is obtained.

[0048]

[0049]

[0050] Wherein, median() represents taking the median value; represents the gray value of the jth pixel point in the middle sub-region; represents the gray value of the jth pixel point in the middle sub-region;i The gray value of the jth pixel point is calculated, and the LC of the 9*9 center point is calculated according to the following formula:

[0051]

[0052] Wherein, epsilon is a very small constant, to avoid the denominator being 0. The introduction of the median can reduce the interference of abnormal values (such as impulse noise) on the estimation of the background brightness, and the larger the LC value is, the more obvious the difference between the target region and the surrounding background is, and the target is determined; if the LC value tends to 0, it is determined as the background.

[0053] According to the above process, all pixels of G(x, y) are processed to obtain a spatial saliency map, denoted as R s .

[0054] (3) Constructing a frequency domain saliency map. A Gaussian pyramid is established, and the gray scale image G(x, y) is down-sampled by 3 layers to realize multi-scale feature extraction. The Gaussian pyramid down-sampling generates 3 layers of images with different resolutions: the original image, 1 / 2 size, 1 / 4 size, to retain the background information at different scales, and the schematic diagram of the pyramid structure is shown in Figure 3 .

[0055] Performing Haar wavelet transform on each layer, the image is decomposed into four subbands: low-frequency approximation (AD), horizontal detail (HD), vertical detail (VD), and diagonal detail (DD), and only the horizontal-vertical detail component is retained, denoted as

[0056]

[0057] Wherein, k is in {0, 1, 2}, corresponding to 3 different pyramid layers.

[0058] According to the extracted component results, the frequency domain residual error is calculated, and the residual error map is taken as the frequency domain saliency map R w :

[0059]

[0060] Wherein, is the mean value of the kth layer subband coefficient, and the square operation is used to enhance the contrast between the weak target and the background. Lambda k is the weight coefficient of the kth layer, and the sum of the weight coefficients of each layer is 1, and higher weight is given to the higher layer to suppress the noise at the bottom layer.

[0061] (5) Fusion of spatial and frequency saliency maps. The frequency domain saliency map R w and the spatial saliency map R s are normalized, and the spatial and frequency saliency maps are weighted and fused to obtain the saliency map S f:

[0062] S f =α·R w +(1-α)·R s

[0063] Where α is the weighting coefficient to balance the significance information in the frequency domain and the spatial domain, and 0 < α < 1.

[0064] A 3×3 Gaussian filter is applied to the fused saliency map to eliminate isolated noise points and make the target region more coherent. The gradient magnitude of the saliency map is calculated and statistically analyzed. To obtain its mean and maximum values, specifically, the saliency map is calculated using the Sobel operator. Horizontal gradient G x and vertical gradient G y :

[0065]

[0066] (6) Saliency Map Segmentation. The saliency map is segmented based on a gradient-guided dynamic threshold. The equation for calculating the dynamic threshold T is:

[0067] T(x,y)=μ f +k×σ f

[0068] Where μ f and σ f These represent the saliency maps S. f The mean and standard deviation, where k is the dynamic coefficient, based on the gradient. The equation for calculating k is as follows:

[0069]

[0070] in, The gradient average, α is the maximum gradient value, and α is the scaling factor, which is usually set to 1 by default. When the scene exhibits an extremely low signal-to-noise ratio, it can be adjusted appropriately.

[0071] Based on the threshold T, the saliency map S f Binarization segmentation is performed to obtain a binary image B(x,y).

[0072] (6) Equivalent radius estimation. Perform connected component analysis on the binary image B(x,y), count the number of pixels of all suspected targets, calculate the width w and height h of its largest bounding rectangle, and assume that the target is approximately circular, convert the area of ​​the rectangle into an equivalent radius r. e .

[0073]

[0074] (7) Set the structure element of dilation and erosion operation. The structure element B io of dilation operation is composed of square structure element B o and circular structure element B i , B i is inlaid in the center of B o , B i radius is equal to r e , B o radius is 2 units longer than r e ; the structure element of erosion operation is composed of circular structure element B b , because the circular structure element cannot traverse the image edge, it is completed to an circumscribed rectangle, B b radius is 2 units shorter than r e ;

[0075] According to the radius, set the size of the structure element of dilation and erosion operation, the structure element of dilation and erosion is set as shown in Figure 4 , and the size setting diagram is shown in Figure 5 .

[0076] (8) Target detection. Perform top-hat morphological operation on the input image I(x, y), and the processing process is as shown in the following formula:

[0077]

[0078] Wherein, and Θ respectively represent dilation and erosion, considering that when the dilation operation is performed with the structure element B io , if the processed area is background, the relationship between it and the adjacent pixels may not be determined, resulting in negative values in the operation process, so the definition of Top-Hat transformation N top-hat is modified as follows:

[0079]

[0080] The above is easily understood by those skilled in the art, and the above is only the preferred embodiment of the present application, and does not limit the present application, any modification, equivalent replacement and improvement within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. An adaptive Top-Hat infrared weak target detection method based on space-frequency joint detection, characterized in that, Includes the following steps: The input infrared image I(x,y) is read and converted into a grayscale image G(x,y). Then, the spatial saliency of the grayscale image G(x,y) is calculated based on local contrast to obtain the spatial saliency map R. s On the other hand, the frequency domain saliency of the grayscale image G(x,y) is calculated based on multi-scale wavelet decomposition, resulting in the frequency domain saliency map R. w ; The obtained spatial saliency map R s Frequency domain saliency plot R w Weighted fusion is performed to achieve spatial-frequency saliency fusion, and the saliency map S after fusion is analyzed. f Perform Gaussian filtering; Calculate and statistically analyze the saliency plot S. f The gradient magnitude ▽S is used to guide the dynamic threshold to segment the saliency map, resulting in a binary image B(x,y). Calculate the equivalent radius r of the largest suspected target in the binary image B(x,y) after thresholding. e According to the equivalent radius r e Set the size of the structuring element for dilation and erosion operations; Finally, based on the size of the structuring element, a morphological Tophat operation is performed on the input image I(x,y) to further determine the target location accurately; The dimensions of the structural element used for setting expansion and erosion operations specifically include: First, the connected regions of the binary image B(x,y) are analyzed, and the width w and height h of the bounding rectangle of the largest connected region are calculated. The area of ​​the rectangle is then converted into an equivalent radius r. e The structuring element size for expansion and erosion operations is set according to the radius. The structuring element B for the expansion operation... io Composed of square structural element B o and circular structural element B i Composition, B i Inlaid in B o The center, B i Radius equals r e B o radius ratio r e The length is 2 pixels; the structuring element of the erosion operation is a circular structuring element B. b Composition: Complete the circular structure into a circumscribed rectangle, B b radius ratio r e Shorter by 2 pixels; The Tophat morphological operation is performed on the input image I(x,y), and the processing procedure is shown in the following equation: ； ； in, Θ and θ represent dilation and erosion, respectively. The definition of the Top-Hat transformation Ntop-hat is as follows:

2. The adaptive Top-Hat infrared weak target detection method based on space-frequency joint method according to claim 1, characterized in that, The spatial saliency calculation based on local contrast specifically includes: A 9×9 sliding window is used, which is divided into nine 3×3 sub-blocks. The image G(x,y) is traversed from left to right and from top to bottom. During the traversal, the image region corresponding to the sub-block is denoted as I. i Let i = 0, 1, 2, ..., 8; first, calculate the mean of the central sub-region image, denoted as μ. o Then, the pixel values ​​of the remaining 8 neighboring sub-regions are sorted, and the median value is taken, denoted as m. i And calculate the absolute deviation of all pixels in each sub-region from the median, denoted as e. i j Finally, the median of the absolute deviation values ​​of all pixels in the 8 neighboring regions is taken to obtain the median absolute deviation M. The processing procedure is shown in the following formula: ； Wherein, median() means taking the median value; This represents the grayscale value of the j-th pixel in the middle sub-region; Subregion I i The gray value of the j-th pixel Then calculate the local contrast C of the image pixel (x,y) corresponding to the center point of the 9×9 window using the following formula. L (x,y): ； Here, ε is a very small constant to avoid the denominator being 0; Following the above process, all pixels of G(x,y) are processed to obtain the local contrast of each pixel. The spatial saliency map R is obtained by using the local contrast as the spatial saliency value. s .

3. The adaptive Top-Hat infrared weak target detection method based on space-frequency joint method according to claim 1, characterized in that, The frequency domain saliency calculation based on multi-scale wavelet decomposition specifically includes: A Gaussian pyramid is constructed to perform three-layer downsampling on the grayscale image G(x,y) to achieve multi-scale feature extraction; a Haar wavelet transform is performed on each layer to extract the horizontal-vertical detail components of the pixel (x,y). Where k∈{0,1,2}, corresponding to 3 different pyramid levels, and the frequency domain residual R of the image pixel (x,y) is calculated according to the following formula. w (x,y) is used, and the residual value is used as the frequency domain significance to obtain the frequency domain significance map R. w : ； in, Let λ be the mean of the coefficients of the k-th sub-band. k is the weight coefficient of the k-th layer, and the sum of the weight coefficients of all layers is 1. Higher layers are assigned higher weights.

4. The adaptive Top-Hat infrared weak target detection method based on space-frequency joint method according to claim 1, characterized in that, Spatial saliency R s and frequency domain significance R w Weighted fusion is performed to obtain the spatial frequency saliency map S. f Specifically, the steps include the following: S f =α·R w +(1-a)·R s Where α is the weighting coefficient, 0 < α < 1.

5. The adaptive Top-Hat infrared weak target detection method based on space-frequency joint method according to claim 1, characterized in that, The saliency map is segmented based on a dynamic threshold guided by the gradient magnitude ▽S. The formula for calculating the dynamic threshold is: T(x,y)=μ f +k×σ f Where T is the dynamic threshold, μ f and σ f These represent the saliency maps S after fusion. f The mean and standard deviation; k is a dynamic coefficient, dynamically adjusted according to the gradient ▽S, and the formula for calculating k is: ； Where mean(▽S) is the average gradient, max(▽S) is the maximum gradient, and β is the scaling factor.

6. The adaptive Top-Hat infrared weak target detection method based on space-frequency joint method according to claim 3, characterized in that, Gaussian pyramid downsampling generates three layers of images at different resolutions: the original image, half-size image, and quarter-size image.

Images