A target detection method for sequential infrared images based on equalized structure and texture representation

Abstract

A target detection method for sequential infrared images based on balanced structural texture representation. First, the local intensity information map is constructed by taking advantage of the fact that small targets in infrared images are brighter than the background, fully suppressing the background clutter whose central pixel value is less than the local average pixel and locking the image block where the target may exist. Secondly, after obtaining the image block where the small target may exist, a polar coordinate system is established at the center point of the image block, and the small target is detected by using the local gradient attribute. The small target is detected by the threshold given by the experiment. Finally, the local intensity and local gradient are combined to form a balanced structural texture representation, and a balance factor is added to adjust the proportion of intensity and gradient in the balanced structural texture representation. Compared with other algorithms, this method can detect the location of small targets more efficiently and accurately.

Description

Technical Field

[0001] The invention belongs to the field of infrared small target detection, and in particular relates to a sequence infrared image target detection method based on equalized structural texture representation. Background Art

[0002] With the rapid development of modern military technology, various precision-guided missiles play an important role in modern warfare, and have the ability to accurately strike military targets, aircraft, ships, aircraft and other aircraft carrier combat platforms at any time. Therefore, in order to ensure the safety of the operating platform, it is necessary to effectively detect and track the hit target.

[0003] So far, the methods for infrared small target detection at home and abroad can be divided into two categories: sequential detection methods and single-frame detection methods. Although the existing methods have achieved good results, it is very necessary to take into account the temporal information because a single-frame image lacks sufficient information to detect small targets. The self-regularized weighted (SRWS) model proposed by Zhang et al. (Zhang T, Peng Z, Wu H, et al. Infrared small target detection via self-regularized weighted sparse model[J]. Neurocomputing, 2021, 420:124-148.) uses the overlapping edge information of the detectable background result information to constrain the sparse items and improve the detection accuracy, but it requires certain screening for the detected targets. Zhang et al. proposed a non-convex optimization (NOLC) model based on norm constraint (Zhang T, Wu H, Liu Y, et al. Infrared small target detection based on non-convex optimization with Lp-norm constraint[J]. Remote Sensing, 2019, 11(5): 559.). This method uses the norm to strengthen the sparse term constraint and appropriately scales the low-rank term constraint. The detection effect of this method in a strong noise environment is unsatisfactory. Zhang et al. proposed a spatiotemporal tensor (BTR) model based on edge and corner awareness (Zhang P, Zhang L, Wang X, et al. Edge and Corner Awareness-Based Spatial–Temporal Tensor Model for Infrared Small-Target Detection [J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59 (12): 10708-10724.) for infrared small target detection. This method has problems in enhancing the target. The edge information of the target cannot be effectively retained, resulting in excessive shrinkage of the target. There are many methods based on spatiotemporal tensor models, but in spatiotemporal tensor models, how to quickly detect the position of small targets is still a difficult problem. Summary of the invention

[0004] The purpose of the present invention is to propose a method for target detection in sequential infrared images based on balanced structural texture representation, so as to accurately detect the location of the target in infrared images lacking texture information and structural information. The present invention starts from the latent space in the infrared image, utilizes the local intensity information and local gradient information of the original infrared image, and constructs a balanced structural texture representation to detect the target more accurately. In addition, in order to improve the compatibility of the invention in more scenarios, a variable balance factor is added to the model.

[0005] The technical solution of the present invention mainly includes the following three contents: First, the value of using local intensity information to detect small targets. Because the background intensity is usually small or almost the same, local intensity information can be used to suppress relatively dark or relatively uniform backgrounds. By calculating the average value of the image block, background clutter whose central pixel value is less than the local average pixel can be suppressed. Second, most of the gradient vectors of the target point to the center of the target, and the local gradient attribute means that the gradient direction of the background with strong edges is consistent, so small targets can be detected using the local gradient attribute. Third, a new equalized structural texture representation is proposed in combination with the local intensity map and the local gradient map to reduce the false alarm rate of the model. A new important factor is added to the equalized structural texture representation to measure the relative importance of the intensity map and the gradient map.

[0006] Compared with the prior art, the present invention has the following beneficial effects:

[0007] (1) The present invention is based on the space-time model of infrared images and obtains the tensor model of the original image through space-time.

[0008] (2) The method used in the present invention effectively suppresses background clutter, overcomes the influence of background clutter on the detected target, and constructs a balanced structural texture representation by combining intensity and gradient, which can quickly locate the target position in the original image.

[0009] (3) The present invention adds an adjustable balancing factor in the equalized structural texture representation to effectively balance the relative weights of the intensity map and the gradient map in different scenarios, which makes the method more adaptable and robust. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 The figure is a general flow chart of the method of the present invention.

[0011] Figure 2 It is the original image of the infrared image data set in the embodiment of the present invention.

[0012] Figure 3 This is a target result diagram after detection in an embodiment of the present invention.

[0013] Figure 4 It is a 3D potential energy diagram of the detection result diagram in the embodiment of the present invention.

[0014] Figure 5 The following is a comparative display of the experimental results in the embodiments of the present invention. DETAILED DESCRIPTION

[0015] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0016] like Figure 1 As shown, the present invention proposes an infrared small target detection method based on equalized structural texture representation, which specifically includes the following steps:

[0017] Step 1: Preprocess the original infrared image, including converting the RGB image into a grayscale image, and constructing the sequence of images into a spatiotemporal tensor. For an RGB image, the image can be converted into a grayscale image by using the MATLAB function rgb2gray, and the image information can be effectively retained. For m L×H size sequence infrared images, where L and H are the length and width of the infrared image, respectively, the method for constructing a spatiotemporal tensor is to construct a three-dimensional tensor of L×H×m.

[0018] In the embodiment of the present invention, there are 5 data sets used, some infrared images are as follows: Figure 2 As shown, they are respectively (a): an airplane in cloudy weather, the image size is 128×128, a total of 84 images; (b): a target under a cloudy background, because the target is too small, no shape information is available, the image size is 256×200, a total of 30 images; (c): a ship under the sea and sky background, the image size is 320×240, a total of 120 images; (d): a vehicle in the field, the image size is 384×288, a total of 160 images; (e): an airplane under a cloudy background, the image size is 256×200, a total of 70 images;

[0019] Step 2: Divide the infrared image into slices according to the size of the sliding window n×n, and calculate the local intensity value I of the infrared image.

[0020] For a given size n×n image patch, the average value of its surrounding area is:

[0021]

[0022] Where N m Indicates the number of pixels in the image block. ij represents the specific value of the pixel at the i-th row and j-th column in the image block, where v0 represents the value of the central pixel. By using formula (1), the local intensity value of the image block can be obtained:

[0023]

[0024] Step 3: Establish polar coordinates with the target image block as the center and calculate the gradient value G of the target image. Combine the intensity value I with the gradient value G to form a balanced structural texture representation.

[0025] First, establish polar coordinates at the center of the image block and divide the image block into 4 parts. Each part can be expressed as:

[0026]

[0027] where p i represents each part of the image (i=1,2,3,4). Since the gradient of the small target does not strictly point to the center, a looser constraint is used to formulate the gradient toward the center:

[0028]

[0029] in Indicates that the region p is satisfied i The set of gradients of the constraints, is a gradient element in the set, m and a are its magnitude and direction respectively. Then, in each region, calculate The mean square of:

[0030]

[0031] Where N j (j=1,2,…,N) is a set middle After calculating the gradient values ​​of the four parts, the local gradient value of the image block can be obtained:

[0032]

[0033] Where k is a constant, G max and G min They represent the maximum mean square gradient and the minimum mean square gradient respectively.

[0034] After comprehensive analysis of infrared images, two noteworthy results were found: (1) In areas with high intensity, there are targets and strong edges in the background clutter; (2) The gradient values ​​of weak and irregular targets are generally low. Therefore, a new balanced structural texture representation is proposed by combining the local intensity map and the local gradient map to reduce the false alarm rate of the model.

[0035]

[0036] Among them, IG ij , I ij , G ijRepresent the pixel values ​​in the balanced structure texture representation, intensity map, and gradient map respectively. The weights of the intensity map and gradient map can be adapted to various scenarios by adjusting the balance factor α. Mathematically, when α=1, the gradient map and the intensity map have equal influence; when α>1, the gradient map has a higher weight; and when α<1, the intensity map is more important. In this method, the value of α is adjusted according to the actual situation of the infrared image so as to flexibly cope with various complex infrared scenes.

[0037] Step 4: Input the image obtained in step 1 and the equalized structure texture representation obtained in step 3 into the truncated nuclear norm tensor model, and iterate through the alternating direction multiplier method to obtain the information of the small target detected in the infrared image. For a given sequence of infrared images, the equalized structure texture representation of the image is calculated, which can almost completely highlight the pixels in the target area. At the same time, the low rank and sparsity of the infrared image tensor are used to regard the tensor as a combination of a sparse tensor and a low rank tensor. Through the truncated nuclear norm tensor model, the truncated nuclear norm is used in the model to quickly approximate the rank of the low rank tensor, and the equalized texture structure representation is used to enhance the value of the sparse tensor, and finally the low rank tensor is restored to the background and the sparse tensor is restored to the target.

[0038] This method finally achieves the purpose of infrared small target detection. The experimental results are as follows Figure 3 As shown in Figure 2, this method can effectively detect small targets in the image. The 3D potential energy diagram of the target image is shown in Figure 2. Figure 4 As shown, from the potential energy diagram, one can intuitively feel that this patent has a significant effect on background suppression and target highlighting of infrared images.

[0039] like Figure 5 As shown, in order to verify the advancement of this patent, some advanced algorithms are selected for comparison, mainly including IPI model, BTR model, TIP model, NOLC model, and SRWS model. Figure 5 The results of each algorithm model testing on the same data are shown. At the same time, the parameter indicators are selected respectively. (background suppression factor) and (Signal-to-noise ratio gain) is used to more intuitively judge the effect of the algorithm, as shown in Table 1 and Table 2. By comparison, this method is in a leading position both in background suppression and target prominence.

[0040] Table 1: The results of each method value

[0041] IPI BTR TIP NOLC SRWS OURS a 0.31 2.30 12.75 21.08 7.60 29.04 b 7.12 27.04 15.41 31.34 20.20 25.38 c INF 10.82 9.34 44.98 13.81 48.09 d 4.14 22.89 16.62 27.09 21.23 24.16 e 1.33 4.10 9.16 4.14 2.07 6.94

[0042] Table 2: The results of each method value

[0043] IPI BTR TIP NOLC SRWS OURS a 0.31 9.02 4.24 21.45 7.73 41.76 b 7.12 27.04 6.65 31.34 20.20 25.38 c NAN 12.53 5.28 52.11 16.00 125.14 d 0.34 11.08 7.42 10.56 9.30 14.29 e 1.33 4.10 2.43 4.14 2.13 5.08

[0044] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed by the present invention should be included in the protection scope recorded in the claims.

Claims

1. A method for target detection in sequential infrared images based on equalized structural texture representation, characterized in that: The method comprises the following steps: Step 1: Preprocess the original infrared image, including converting the RGB image into a grayscale image and constructing the sequence image into a spatiotemporal tensor; Step 2: Slice the infrared image according to the size of the sliding window n×n and calculate the local intensity value I of the infrared image; Step 3: Establish polar coordinates with the target image block as the center, calculate the gradient value G of the target image; and combine the intensity value I with the gradient value G to form a balanced structural texture representation; The specific implementation steps of calculating the gradient value G of the infrared image in step 3 are as follows: First, establish a polar coordinate system (r,θ i ), the image block is divided into 4 parts, each part is represented as: where p i Represent each part of the image, i = 1, 2, 3, 4; use a constraint to formulate the gradient towards the center: where φ pi Indicates that the region p is satisfied i The set of gradients of the constraints, is a gradient element in the set, m and a are its magnitude and direction respectively; then, in each region, calculate The mean square G i : Where N j Is a collection middle The number of pixels is j = 1, 2, ..., N. After calculating the gradient values ​​of the four parts, the local gradient value of the image block is obtained: Where k is a constant, G max and G min Represent the maximum mean square gradient and the minimum mean square gradient respectively; Step 4: Input the image obtained in step 1 and the equalized structure texture representation obtained in step 3 into the truncated nuclear norm tensor model, and iterate through the alternating direction multiplication method to obtain the small target information detected in the infrared image.

2. The method for target detection in sequential infrared images based on equalized structure and texture representation according to claim 1, characterized in that: The specific implementation steps of the image preprocessing in step 1 are as follows: for the input RGB image, the image is converted into a grayscale image through the matlab function rgb2gray, and the image information is retained; for m L×H size sequence infrared images, where L and H are the length and width of the infrared image respectively, and m represents the number of frames of the infrared image, a three-dimensional L×H×m space-time tensor is constructed.

3. The method for target detection in sequential infrared images based on equalized structure and texture representation according to claim 1, characterized in that: The specific implementation steps of calculating the local intensity value I of the infrared image in step 2 are as follows: For a given size n×n image patch, the average value of its surrounding area is: Where N m represents the number of pixels in the image block, v ij represents the specific value of the pixel at the i-th row and j-th column in the image block, where v0 represents the value of the central pixel; the local intensity value I of the image block is obtained by formula (1):

4. The method for target detection in sequential infrared images based on equalized structural texture representation according to claim 1, characterized in that: The specific implementation steps of calculating the balanced structural texture representation of the infrared image in step 3 are as follows: By comprehensively analyzing infrared images and combining local intensity maps and local gradient maps, a balanced structural texture representation is proposed to reduce the false alarm rate of the model: Among them, IG ij , I ij , G ij They represent the pixel values ​​in the balanced structural texture representation, intensity map, and gradient map respectively; the weights of the intensity map and gradient map are adapted to various scenarios by adjusting the balance factor α; when α=1, the gradient map and the intensity map have equal influence; when α>1, the gradient map has a higher weight; and when α<1, the intensity map is more important.

5. The method for target detection in sequential infrared images based on equalized structure and texture representation according to claim 1, characterized in that: In step 4, given a sequence of infrared images, the balanced structural texture representation of the image is obtained by calculation to highlight the pixels in the target area. At the same time, the low rank and sparsity of the infrared image tensor are utilized, and the tensor is regarded as a combination of a sparse tensor and a low rank tensor. Through the truncated nuclear norm tensor model, the low rank tensor is restored to the background, and the sparse tensor is restored to the target.

6. The method for target detection in sequential infrared images based on equalized structure and texture representation according to claim 5, characterized in that: In the truncated nuclear norm tensor model, the truncated nuclear norm is used in the model to quickly approximate the rank of the low-rank tensor, and the equalized texture structure representation is used to enhance the value of the sparse tensor, ultimately restoring the low-rank tensor to the background and the sparse tensor to the target.

Images