An Infrared Small Target Detection Method Based on Multi-Directional Fusion

Abstract

The present invention relates to an infrared small target detection method based on multi-directional fusion. This method first simultaneously performs facet information modeling from the perspectives of edges and local regions, and conducts a connected component pixel restoration operation on the fused and filtered image to obtain candidate targets. Then, it performs restricted Random Walker segmentation on the candidate targets, designs an adaptive segmentation box to accurately locate background seed points, improve the segmentation accuracy, and distinguish targets from clutter. Finally, we weight the segmentation map and the facet filtering result, and obtain the final small targets through threshold segmentation. The present invention makes full use of the spatial distribution and gray-scale characteristics of small targets in infrared images, effectively compensates for problems such as few pixels occupied by targets in infrared images, indistinguishable textures and shapes, and low signal-to-clutter ratio, greatly improves the integrity of small targets and the multi-scale adaptability of the segmentation algorithm, and at the same time effectively suppresses background clutter of different forms and improves the detection performance of small targets.

Description

Technical Field

[0001] The present invention relates to an infrared small target detection method based on multi-directional fusion, belonging to the fields of computer vision and image processing such as civilian and military reconnaissance. This method well compensates for the defects commonly existing in infrared images, such as few pixels occupied by the target, indistinguishable texture and shape, low signal-to-clutter ratio, etc. Based on the characteristics of the filter maps constructed by facet first-order and second-order kernel fusion, and with the post-processing of connected domain pixel restoration, the integrity of candidate targets is greatly improved, the target enhancement effect is better, and at the same time, it has stronger robustness to multi-scale targets and clutters of different shapes. Background Art

[0002] Infrared small target detection is one of the important research topics in computer vision, and is widely used in fields such as infrared search and tracking systems, precision guidance, air defense, etc. However, due to the long imaging distance and weak target, there are usually few details such as shape, texture, and structure. In addition, the target is usually immersed in a complex background and is easily interfered by high-brightness noise and complex background. Therefore, infrared small target detection is still a difficult and challenging task. By analyzing the characteristics of small targets in infrared images, they have similar structures and intensities at all angles, showing isotropic Gaussian characteristics. Compared with the surrounding background, small targets have a more compact pixel distribution in space and stronger brightness. Therefore, most methods use the target distribution characteristics to construct appropriate filter templates to process infrared images to highlight small targets and improve the signal-to-clutter ratio; a series of detection methods based on contrast design different patches, and use the contrast enhancement map between small targets and the background as the final weight map. However, in these detection methods, there are also adverse factors such as complex background interference, low contrast, parameter optimization, and time consumption.

[0003] The existing processing methods have achieved good detection effects in enhancing the signal-to-noise ratio, jointly applying features such as direction, scale, and spatial distribution. However, the robustness is poor, and it is not suitable for the detection of small and weak infrared targets in complex backgrounds. At the same time, some methods need to manually set parameters according to different scales and situations, and cannot achieve adaptive image processing. Summary of the Invention

[0004] The purpose of the present invention is to overcome the above deficiencies of the prior art and provide an infrared small target detection method based on the spatial distribution and gray characteristics of the target. For this purpose, the present invention adopts the following technical solutions.

[0005] The infrared small and weak target detection method includes the following steps:

[0006] 1. Remove noise and smooth the image. First, we use a 3×3 order statistic filter to remove the singular high-brightness noise in the image, and then use a 2×2 mean filter to smooth the image. The processed image is more suitable for the RandomWalker segmentation algorithm.

[0007] 2. Based on the isotropic Gaussian characteristics of small targets and the characteristics of facet filtered images of each order, we construct a multi-directional fusion facet kernel decomposition image. After obtaining the annular connected domain contours with similar intensity and integrity at each angle, we restore the pixels inside the contours to obtain complete candidate targets. The process is as follows:

[0008]

[0009] Among them, f′ α are the first-order facet filters in four directions constructed by us, where α represents the filtering angle of the filter, α = 0°, 45°, 90°, 135°, and K i (i = 2, 3, 7, 8, 9, 10) represents the first-order polynomial fitting coefficients, which are estimated by the least squares method. The expression is:

[0010]

[0011] Among them, p i is a set of discrete orthogonal polynomials, which are composed of the symmetric neighborhood r, c, R = {-2, -1, 0, 1, 2}, c = {-2, -1, 0, 1, 2}, and f(r, c) is the intensity function representing the gray value, as follows:

[0012]

[0013]

[0014] After constructing the facet filter kernel, we perform the processing of multi-channel image extreme point integration and multi-directional fusion:

[0015] M f (x, y) = {F P (x, y) + |F N (x, y)|} (5)

[0016] M s = I * F (6)

[0017]

[0018] F P (x, y), F N (x, y), M f (x, y) are the first-order fusion images where the maximum extreme point, the minimum extreme point, and the integrated extreme point are located respectively. |·| represents the absolute value operation. M s is the second-order filtered image, F is the second-order filter kernel, M m is the final fusion filtered image, * represents the convolution operation, represents the dot product operation of matrices.

[0019] After multi-directional fusion processing and filtering, we will obtain the contour of the ring-shaped connected domain composed of target edge pixels, and then we perform the operation of restoring the pixels of the connected domain, thereby obtaining the complete candidate target.

[0020] 3. Design the RW adaptive segmentation box to accurately locate the seed points to achieve multi-scale segmentation. The Random Walker adaptive scale segmentation box can automatically distinguish the target area from the background area according to the size of the candidate target, thereby calculating and changing the size of the segmentation box, accurately locating the position of the background seed points, and achieving accurate pixel-level segmentation of multi-scale targets:

[0021]

[0022]

[0023] where v i (i = 1, 2,...I) represents the side length of the segmentation box, i represents the number of segmentation boxes, u j (j = 1, 2,...J) represents the side length of the target area, j is the number of candidate targets, i = j, m represents the number of pixels in the candidate area, and v and u are calculated according to m, NLCD cp and NLCD hg are contrast descriptors designed based on the segmentation probability map and pixel intensity respectively, represents the dot product operation of the matrix, and the target can be represented. We denote the segmentation result map as M NLCD .

[0024] 4. Perform the dot product operation on the multi-directional filtered map M m and the segmentation mapping map M NLCD to further enhance the target and use it as the final weight map to extract small targets:

[0025]

[0026] T = μ + 4σ (11)

[0027] μ and σ are the mean and variance of the image M W respectively.

[0028] The beneficial effects of the present invention are:

[0029] 1. Multi-scale detection: Based on the multi-directional fusion facet model, pixel operation of the connected domain, and adaptive segmentation box, the present invention fully guarantees the integrity and segmentation accuracy of targets of different sizes, and can achieve multi-scale segmentation without manual parameter setting.

[0030] 2. Good detection effect: The present invention not only ensures the integrity of candidate targets, but also has stronger robustness to clutters of different forms. At the same time, during the segmentation process, seed points for accurate background positioning can be obtained for targets of different scales, significantly improving the detection performance of small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1: Flowchart of the infrared small target image detection method of the present invention.

[0032] Figure 2: Results of facet first-order multi-angle fusion and multi-scale filtering.

[0033] Figure 3: Results of facet second-order multi-scale filtering.

[0034] Figure 4: Results of multi-directional fusion image.

[0035] Figure 5: Results of candidate targets for restoring connected domain pixels.

[0036] Figure 6: Background suppression effects of different forms in the fusion image.

[0037] Figure 7: Adaptive segmentation box.

[0038] Figure 8: Connected domain pixel estimation and adaptive segmentation box.

[0039] Figure 9: Final detection results of different background images. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] The flow of the present invention is shown in Figure 1. This method first removes noise and smooths the image using mean and statistical filtering to make the image suitable for subsequent algorithms for segmentation based on pixel intensity. Then, fusion filtering is performed on the image based on facet first-order and second-order kernels to extract the target annular contour and suppress the complex background, restore the pixels within the contour (connected domain), and obtain complete candidate targets. The number of pixels of each candidate target is counted, an adaptive segmentation box is designed, the background seed points are accurately positioned, and the segmentation accuracy is improved. Finally, the weighted filtering result and the segmentation mapping are used to obtain a weight map, and small targets are segmented according to the threshold. The specific implementation process of the technical solution of the present invention will be described below with reference to the accompanying drawings.

[0041] 1. Smooth the image to remove highlight noise

[0042] Since the subsequent Random Walker segmentation algorithm performs two-class segmentation based on pixel intensity and is very sensitive to high-brightness noise, to improve the segmentation performance, we use a 3×3 order statistic filter to remove the singular high-brightness noise in the image. Specifically, the pixels with pixel intensity greater than that of the surrounding 8 pixels are filtered out and replaced with the highest pixel value of the surrounding pixels. Then, a 2×2 mean filter is used to smooth the image, that is, the average value of four pixels is used to replace the original four pixels. The processed image is more suitable for the RW algorithm, and we denote the processed infrared image as I.

[0043] 2. Integration of extreme points by multi-directional facet first-order filtering

[0044] Since RW belongs to a pixel-level segmentation algorithm, to improve the real-time performance of detection, we first use a facet kernel to filter out candidate targets. First, we use first-order facet filtering kernels in four directions of 0°, 45°, 90°, and 135° to decompose the image I to obtain first-order derivative filtered images in four directions, convert small targets into extremely large and small Gaussian-like points with drastic changes, convert negative extreme points (low peaks) into positive extreme points (high peaks), integrate the peaks in each direction, and construct the contour of small targets (as shown in Figure 2):

[0045]

[0046]

[0047] M f (x, y) = {F P (x, y) + |F N (x, y)|} (3)

[0048] F P (x, y), F N (x, y), M f (x, y) are the positive extreme points, negative extreme points, and the first-order fusion images where the negative extreme points are converted to positive and integrated with the original positive extreme points respectively.

[0049] 3. Facet multi-order fusion filtering

[0050] While the facet first-order fusion map forms the edges of small targets, it also introduces clutter edges in different directions. To remove the clutter while retaining the target contour, we use a second-order kernel to enhance the local area of the target and suppress the background. Since the second-order kernel will gradually convert to an edge detection function similar to the first-order kernel as the target size gradually increases (as shown in Figure 3), for targets of different scales, the multi-order fusion map M mAn enhanced target contour will be formed; for the background, kernels of different orders will generate background clutter of different morphologies. Only a small number of pixel points where the clutter intersects will be retained in the fused image (as shown in Figure 6). Moreover, due to the unstable pixel intensity of the clutter, it is difficult to form a circular-like connected domain. Therefore, we can distinguish the target from the clutter.

[0051] 4. Restore Connected Domain Pixels

[0052] To obtain a complete candidate target, we perform threshold processing on the fused image to extract the candidate target contour. μ and σ are the mean and variance of the fused image respectively, and then the pixel values inside the contour are restored (as shown in Figure 5).

[0053] T = μ + 4σ (4)

[0054] 5. Estimate the Number of Candidate Target Pixels and Design an Adaptive Segmentation Box

[0055] To accurately locate the background seed points in the segmentation algorithm, we design an adaptive segmentation box according to the size of the candidate target. The double-box strategy accurately locates the positions of the target and the background. The size of the background is designed to be 3 times that of the target, and the background seed points are placed at the edge pixel positions of the background box (as shown in Figure 7). Then, two-class segmentation is performed to divide the pixels inside the segmentation box into background pixels or target pixels, obtaining a probability map:

[0056]

[0057]

[0058] Among them, v represents the side length of the segmentation box, that is, the side length of the background region, u represents the side length of the target region, and m represents the number of pixels in the candidate region. v and u are calculated according to m.

[0059] 6. Segmentation Result Mapping

[0060] According to the segmentation probability map and the original pixel intensity, two kinds of NLCD mappings are generated to describe small targets. NLCD(ClsP) cp is a contrast descriptor generated according to the segmentation probability map, which is the average probability of the pixels segmented into the first class (excluding the target seed points) divided by the average probability of the pixels segmented into the second class. NLCD(ClsP) hg is a contrast descriptor constructed by pixel intensity, which is the average intensity of the pixels divided into the first class divided by the maximum intensity value of the background pixels. The background pixels are completed by two dilation operations on the pixels divided into the first class. Through the weighting of these two descriptors, we can distinguish the target from the clutter:

[0061]

[0062]

[0063]

[0064] ClsP represents the pixels segmented into the first class, CtlP represents the target seed points, ClsP\CtlP represents the pixels segmented into the first class (excluding the target seed points), R represents the pixels to be segmented, R\ClsP represents the pixels segmented into the second class, D 2 is a disk-shaped morphological structure with a radius of 2 pixels.

[0065] 7. Weight Map and Small Target Extraction

[0066] To further enhance the target, we fuse the multi-directional fusion facet filter map Mm with the segmentation weight map M NLCD to obtain the final weight image:

[0067]

[0068] where represents the dot product operation of matrices.

[0069] The present invention proposes an infrared small target detection method based on multi-directional fusion. Based on the spatial distribution characteristics and gray-scale characteristics of small targets, this method not only realizes adaptive multi-scale target detection, significantly improves the detection rate, but also has a good suppression effect on complex background clutter (as shown in Figure 9).

Claims

1. An infrared small target detection method based on multi-directional fusion, characterized in that, the method comprises the following steps: Step 1: Use order statistic filtering and mean filtering to remove high-brightness noise and smooth the image; Step 2: Construct multi-directional facet kernels, model the annular contour of small targets, and obtain complete candidate targets through pixel restoration; Based on the isotropic Gaussian characteristics of small targets and the characteristics of facet filtered images of each order, construct a multi-directional fusion facet kernel to decompose the image, and obtain annular connected domain contours with similar intensities and completeness at each angle. The process is as follows: where f' α is the constructed first-order facet filter in four directions, where α represents the filtering angle of the filter, α = 0°, 45°, 90°, 135°, K i , where i = 2, 3, 7, 8, 9, 10, represents the first-order polynomial fitting coefficient, estimated by least squares, and the expression is: where pi is a set of discrete orthogonal polynomials, composed of symmetric neighborhoods r and c, R = {-2, -1, 0, 1, 2}, c = {-2, -1, 0, 1, 2}, and f(r, c) is an intensity function representing the gray value, as follows: After constructing the filtering kernel, perform the processing of extreme points of multi-channel images and multi-directional fusion: M f (x, y) = {F P (x, y) + |F N (x, y)|} M s = I * F F P (x, y), F N (x, y), M f (x, y) are the first-order fusion images where the maximum extreme point, the minimum extreme point, and the integrated extreme point are located, M s is the second-order filtered image, F is the second-order filter kernel, M m is the final facet fusion filtered image, * represents the convolution operation, and o represents the dot product operation of the matrix; Step 3: Use an adaptive Random Walker segmentation box to segment multi-scale candidate targets to remove interference clutter; The adaptive Random Walker segmentation box automatically distinguishes the target area and the background area according to the size of the candidate target, thereby calculating and changing the size of the segmentation box, accurately locating the position of the background seed points, and realizing accurate pixel-level segmentation of multi-scale targets: where v i (i = 1, 2,...I) represents the side length of the segmentation box, i represents the number of segmentation boxes, u j (j = 1, 2,...J) represents the side length of the target area, j is the number of candidate targets, i = j, m represents the number of pixels in the candidate area, v and u are calculated according to m, NLCD cp and NLCD hg are contrast descriptors designed based on the segmentation probability map and pixel intensity respectively, o represents the dot product operation of matrices, the target is represented, and the segmentation result map is denoted as M NLCD ; Step 4: Perform a dot product operation on the fusion filtering result and the segmentation result, and separate small targets through a threshold; where M m is the candidate target result graph, M NLCD is the segmentation mapping result graph, and o represents the dot product operation of matrices.

2. The infrared small target detection method based on multi-directional fusion according to claim 1, characterized in that, In step 4, the weighted filtering result M f and the segmentation mapping result M NLCD , as the final weight map M W , extract small targets according to the threshold: o represents the dot product operation of matrices.

Images