Infrared dim small target detection method and device based on visual characteristics

Abstract

The application discloses an infrared weak and small target detection method and device based on visual characteristics, and the method comprises the following steps: performing background suppression on an original infrared image by using an improved adaptive side window enhancement filtering method to obtain a corresponding first processed image; removing small edges and clutter interference and realizing target enhancement by using a diagonal local difference product method and a local difference index weight distribution method based on the first processed image to obtain a corresponding second processed image; obtaining the positions of suspected target points in the second processed image by using a threshold segmentation method; and determining the positions of real target points by using a nearest neighbor multi-frame correlation method based on the positions of the suspected target points obtained from continuous multiple original infrared images. The application adopts a multi-scale local contrast algorithm of adaptive enhancement side window filtering, and improves the accuracy of weak and small target position detection by using the continuity of weak and small target moving tracks through inter-frame correlation between continuous frames of images.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of infrared detection and monitoring, and particularly relates to an infrared weak and small target detection method and device based on visual characteristics. BACKGROUND

[0002] With the development of modern military technology, the infrared search and warning system has been more widely applied, and the infrared image weak and small target detection is a key technology of the guidance and warning system. There are a large number of unknown interference sources in the complex ground background, which seriously affects the performance of the infrared weak and small target detection algorithm. The main reason is that there are a large number of strong edges, clutter interference with similar target distribution and randomly distributed noise points in the complex ground background, and in many cases the gray value of the real target is not the highest point in the scene, which leads to the performance of the weak and small target detection algorithm designed for other scenes to be significantly reduced when applied to the complex ground background weak and small target detection. Therefore, the infrared weak and small target detection in the complex ground background has become a very challenging problem, and it is urgent to design a weak and small target detection algorithm that can effectively enhance the target and suppress various interferences in the complex ground background.

[0003] At present, the infrared weak and small target detection algorithm mainly includes the background estimation based method, the target background separation method and the local contrast (human visual) based method.

[0004] The background estimation based method: Yue Fu Chang et al. combined the kmeans clustering method with the median filter, used the median filter to pre-process the original infrared image, and then used the kmeans method to aggregate and classify the same type of pixel points in the pre-processed result. This algorithm has good suppression effect on the background and noise points in the infrared original image. However, the kmeans clustering method has better aggregation effect on the weak and small target with large size, and cannot well process the target with small size, and the more complex the background is, the more seriously it is affected by the strong edge background and target-like interference. Hadhoud et al. used the least mean square to estimate the background based on the Wiener filter, and separated the background from the infrared original image. The advantage of this method is that it can be iterated, and the parameters of the least mean square can be changed during the iteration process. However, the effect of the algorithm depends on the selection of the nested window size, and in the case of high background brightness and serious noise, the detection probability cannot be guaranteed.

[0005] Target background separation method: Gao et al. constructed a block image model and applied it to weak and small target detection, introduced the concept of sparse matrix, established an equivalence relationship between target detection and low-rank sparse matrix recovery optimization, and used principal component analysis method in the solving process. Although the effect of this algorithm is ideal, the algorithm is complex, and it contains the separation of background and the construction of block image model at the same time, resulting in poor real-time performance of the algorithm. Zhang et al. borrowed the local and idea of tensor nuclear norm and proposed a non-convex low-rank constraint method. When suppressing the background, the nuclear norm is jointly weighted, and then the target is extracted. However, when there are similar target interference in the scene, these interference and the target also have sparse structure, and there will be more false alarm points in the detection result.

[0006] Based on the local contrast method: Chen et al. proposed the original local contrast method (LCM) when detecting infrared dim small targets. This algorithm calculates the average value of eight sub-windows in the background area of the nested window, selects the maximum average value as the background estimate, and uses it as the denominator. The average value of the target area is used as the numerator to construct the enhancement factor, which can effectively enhance the significance of the real target. However, the background part is not suppressed during the entire process, so the interference points in the background area are also enhanced during processing, leading to an increase in false alarm rate. Han Jinhui improved the LCM algorithm and designed the strengthened local contrast measure (SLCM) algorithm. The SLCM algorithm is combined with the weight function to propose the weighted strengthened local contrast measure (WSLCM) algorithm based on weight distribution. First, the original infrared image is preprocessed using Gaussian filtering. Then, the background details of the original infrared image are estimated using a nested window, and the background-suppressed image is obtained by differencing. The ratio-type contrast and difference-type contrast calculation methods are combined to calculate the SLCM of the image, thereby suppressing the background and enhancing the target. Then, the weight function is constructed based on the regional intensity level (RIL) difference information to assign different weighting coefficients to the real target and interference information, further realizing target enhancement and background suppression. Although the WSLCM algorithm has good suppression effect on large-area background, background edges, and noise, it cannot distinguish real targets from interference when the image contains interference with similar gray distribution to the real target. Lu Xiaofeng et al. proposed an improved weighted enhanced local contrast measure (IWELCM) algorithm. First, the enhanced local contrast method is designed to expand the gray difference information between the target area and the background area, and to enhance the suspected target points in the original infrared image. Then, based on the gray statistical difference between the target area and the background area, the concept of variance and block gray count (BGC) is introduced, combined with the variance information of the target area, to construct an improved weight function based on BGC, further realizing the enhancement of weak small targets and the suppression of the background. Finally, the weak small targets are extracted by threshold segmentation. WSLCM and IWELCM use local difference and local ratio respectively in the construction of the weight function, one focuses on suppressing the background and the other focuses on enhancing the target. The scope of application of the two algorithms is different, and the effect is also different.For the public patent literature, Chen Yuan Yuan et al. proposed a weighted multi-scale difference joint contrast infrared small target detection method (MRDLCM_RLD for short), which first calculates the multi-scale difference joint local contrast for the original infrared image; Then calculate the reverse local diversity weight function (RLD) for the original infrared image, and further suppress the complex background through the reverse local diversity weight function; Finally, the calculation result based on the reverse local diversity weight function is taken as the weight function of the multi-scale difference joint local contrast, and then the weighted multi-scale difference joint local contrast is calculated. However, when calculating the multi-scale difference joint local contrast, the ratio of the average gray value of the target region to the average gray value of the background region and the difference between the average gray value of the target region and the average gray value of the background region are directly multiplied, which will enhance the target-like clutter interference and noise with higher significance in the scene; When designing the weight function, the difference between the maximum pixel value and the minimum pixel value in the window is used to represent the gray diversity in the window, but this method depends on the size information of the target.

[0007] For the existing WSLCM, IWELCM and MRDLCM_RLD algorithm, the local contrast method is used for target enhancement and background suppression, and finally a weight function is introduced to assign a weighted coefficient to achieve the purpose of filtering clutter interference. WSLCM first uses Gaussian filtering to preprocess the image before calculating the local contrast. Gaussian filtering can smooth some strong edges, but it also has a smoothing effect on the target, which reduces the local difference of the target area and leads to insufficient target intensity in the processing result. In addition, when constructing the weight function, RIL is used to represent the difference between the mean of the first few maximum pixel values in the region and the mean of the pixels in the entire region. Once there is a gray mutation clutter, the RIL value will be very large, which will affect the allocation of the weighted coefficient when constructing the weight function. IWELCM and MRDLCM_RLD are also based on the difference joint local contrast method to directly process the infrared original image without preprocessing. Although it can suppress the background with gentle gray changes, it will enhance the target and cause more interference information in the processing result. In addition, when constructing the weight function, BGC is used to represent the ratio of the mean of the first few maximum pixel values in the region to the mean of the pixels in the entire region. Once there is a gray mutation clutter, the BGC value will be very large, which will enhance the clutter interference. MRDLCM_RLD uses the difference between the maximum and minimum pixel values in the window to represent the gray diversity in the window when designing the weight function RLD. However, this method depends on the size information of the target. If the target size is small, the difference between the maximum and minimum pixel values in the window will be large. If the target size is large, the difference between the maximum and minimum pixel values in the window will be small, and the adaptability is poor. SUMMARY

[0008] In order to solve the above problems in the prior art, the present application provides an infrared dim small target detection method and device based on visual characteristics. The technical problems to be solved by the present application are solved by the following technical solutions:

[0009] In the first aspect, the present application provides an infrared dim small target detection method based on visual characteristics, which comprises:

[0010] The first processing image is obtained by suppressing the background of a frame of original infrared image using an improved adaptive side window enhancement filtering method.

[0011] The second processing image is obtained by removing small edges, clutter interference and realizing target enhancement using the diagonal local difference product method and the local difference index weight allocation method based on the first processing image.

[0012] The threshold segmentation method is used to obtain the positions of the suspected target points in the second processed image;

[0013] Based on the positions of the suspected target points obtained from the continuous multiple frames of original infrared images, the nearest neighbor multi-frame correlation method is used to determine the positions of the real target points.

[0014] In an embodiment of the present application, the improved adaptive side window enhancement filtering method is provided with eight modified window templates, which are obtained by adding peripheral pixels to the traditional side window filtering window templates; the eight modified window templates correspond to the up, down, left, right, upper left, upper right, lower left and lower right directions respectively; the sizes of the eight modified window templates included in the calculation are 5*3, 5*3, 3*5, 3*5, 3*3, 3*3, 3*3 and 3*3 respectively; for each modified window template, the center pixel is the center pixel, the filtering effective pixels of the traditional side window filtering window template adjacent to the outside of the center pixel and matched with the direction of the modified window template are the inner layer pixels, and the multiple pixels matched with the direction of the modified window template and outside the inner layer pixels are the outer layer pixels.

[0015] In an embodiment of the present application, the improved adaptive side window enhancement filtering is used for background suppression on one frame of original infrared image to obtain the corresponding first processed image, which comprises:

[0016] For each to-be-filtered processing pixel in the one frame of original infrared image, the center pixel of the current modified window template is aligned to the to-be-filtered processing pixel;

[0017] For each inner layer pixel position of the current modified window template, the pixel enhancement result of the inner layer pixel position is calculated by using the pixel value corresponding to the inner layer pixel position and the mean value of the multiple pixel values adjacent to the inner layer pixel position in the one frame of original infrared image;

[0018] For the center pixel position of the current modified window template, the pixel enhancement result of the center pixel position is calculated by using the pixel value corresponding to the center pixel position and the mean value of all the pixel values corresponding to the inner layer pixel positions in the one frame of original infrared image;

[0019] The processing result of the to-be-filtered processing pixel under the current modified window template is obtained by performing Gaussian filtering weighting on the pixel enhancement result of the center pixel position and the pixel enhancement results of all the inner layer pixel positions, and then performing difference operation on the enhancement result of the center pixel;

[0020] From the processing results of the to-be-filtered processing pixel under all the modified window templates, the processing result closest to 0 is selected as the final processing result to replace the to-be-filtered processing pixel;

[0021] After the replacement of all the pixels to be filtered in the one frame of original infrared image is sequentially traversed, a corresponding first processed image is obtained.

[0022] In an embodiment of the present application, the diagonal local difference product method is preset with a nested window, which is 15*15 pixels in size, wherein 3*3 pixels are taken as a sub-window, forming a total of 5*5 sub-windows, the centermost sub-window is the center layer, which is defined as the target region, the 8 sub-windows adjacent to the outside of the center layer are the intermediate layer, which is defined as the intermediate layer background region, and the 16 sub-windows outside the intermediate layer are the outermost layer, which is defined as the outermost layer background region.

[0023] In an embodiment of the present application, the first processed image is processed by the diagonal local difference product method and the local difference index weight distribution method to remove small edges, clutter interference and achieve target enhancement, thereby obtaining a corresponding second processed image, which includes:

[0024] For each pixel to be processed in the first processed image, the pixel to be processed is aligned with the center pixel of the center layer sub-window of the nested window;

[0025] The difference local contrast of each intermediate layer sub-window is calculated by using a preset intermediate layer difference local contrast calculation formula, and the difference local contrast of each outermost layer sub-window is calculated by using a preset outermost layer difference local contrast calculation formula;

[0026] The product of the difference local contrasts of each group of intermediate layer sub-windows belonging to the diagonal is calculated, and the minimum value of the products obtained by all groups of intermediate layer sub-windows belonging to the diagonal is determined as the intermediate layer product metric; and the product of the difference local contrasts of each group of outermost layer sub-windows belonging to the diagonal is calculated, and the minimum value of the products obtained by all groups of outermost layer sub-windows belonging to the diagonal is determined as the outer layer product metric;

[0027] The larger value of the intermediate layer product metric and the outer layer product metric is selected as the diagonal local difference product result;

[0028] The weight coefficient corresponding to the diagonal local difference product result is determined by using the local difference index weight distribution method, and the product of the diagonal local difference product result and the corresponding weight coefficient is taken as the updated pixel value to replace the pixel to be processed;

[0029] After the replacement of all the pixels to be filtered in the first processed image is sequentially traversed, a corresponding first processed image is obtained.

[0030] In an embodiment of the present application, the preset intermediate layer difference local contrast calculation formula is:

[0031]

[0032] The preset outermost layer difference local contrast calculation formula is:

[0033]

[0034] Wherein, d i represents the difference local contrast of the i-th intermediate layer sub-window, i=1, 2, …, 8; M0 represents the pixel average value of the center layer sub-window; Bmi represents the i-th intermediate layer sub-window; M Bmi represents the pixel average value of the i-th intermediate layer sub-window; q j represents the difference local contrast of the j-th outermost layer sub-window, j=1, 2, …, 16; Boj represents the j-th outermost layer sub-window; M Boj represents the pixel average value of the j-th outermost layer sub-window.

[0035] In an embodiment of the present application, the weight coefficient corresponding to the diagonal local difference product result is determined by using the local difference index-based weight distribution method, and the formula used is:

[0036]

[0037]

[0038] Wherein, T i represents the i-th pixel in the original infrared image corresponding to the weight function nested window position when the weight function nested window is used based on the local difference index-based weight distribution method; M T represents the average value of the 9 pixels in the weight function nested window; T max represents the maximum value of the 9 pixels in the weight function nested window; W represents the weight coefficient obtained based on the exponential weight function.

[0039] In an embodiment of the present application, the threshold segmentation method includes a mean and standard deviation combined threshold segmentation method.

[0040] In a second aspect, the embodiments of the present application provide an infrared weak small target detection device based on visual characteristics, and the device comprises:

[0041] An improved adaptive side window enhancement filtering processing module is used to perform background suppression on a frame of original infrared image by using an improved adaptive side window enhancement filtering method, so as to obtain a corresponding first processed image.

[0042] A diagonal local difference product-based method and a local difference index weight distribution-based processing module are used to remove small edges, clutter interference and realize target enhancement on the first processed image by using the diagonal local difference product-based method and the local difference index weight distribution-based method, so as to obtain a corresponding second processed image.

[0043] A threshold segmentation module is used to obtain the positions of suspected target points in the second processed image by using a threshold segmentation method.

[0044] A nearest neighbor multi-frame correlation processing module is used to determine the positions of real target points by using a nearest neighbor multi-frame correlation method based on the positions of suspected target points obtained from continuous multiple original infrared images.

[0045] The present application has the following beneficial effects:

[0046] The embodiment of the present application provides an infrared weak small target detection method based on visual characteristics, aiming at the problems that the traditional side window Gaussian background suppression algorithm reduces the size of real targets while suppressing the background, the present application proposes a background suppression algorithm based on side window enhancement filtering, which enhances the to-be-processed pixel points and then implements the Gaussian filtering algorithm, this method not only retains the suppression effect of the traditional method on the complex background and strong edge, but also avoids the deficiency that the size of the real target is too small after processing; the traditional difference joint local contrast method uses the pixel mean values of the center sub-window and the background sub-window, suppresses various background interference through difference operation, does not consider the gray difference on both sides of the small edge in a targeted manner, leading to poor suppression effect on the small edge, the present application proposes a small edge suppression algorithm based on diagonal local difference product, fully considers the gray difference on both sides of the small edge and the center symmetry characteristic of the real target, this method not only can retain the real target, but also can filter out the small edge in a targeted manner; the existing weight function construction method is only related to the gray distribution in the sub-window, leading to that the clutter interference with the internal gray change is regarded as the real target and retained, by using the characteristics that the exponential function is in an increasing state and is sensitive to the gray change, and the gray distribution in the target region of the original image, the present application proposes a local difference exponential weight distribution method, gives the real target a larger weight value, and gives the interference point a smaller weight value, so that the difference between the target and the interference point is larger, thereby distinguishing the target from the interference point. Through simulation comparison with the processing effects of two traditional algorithms and one existing public patent, the single frame processing part of the present application is better than other algorithms in the background suppression and target extraction effects, the local signal-to-noise ratio gain of the single frame image processing of the present application reaches 1.6961 and 1.5769 in two scenes respectively, which are the maximum values in the comparative algorithms, although the background suppression factor is the minimum value in the comparative algorithms in only one scene, but it can be controlled in a small value in the other two scenes, four kinds of comparative algorithms are added into the nearest neighbor association to detect the probability and false alarm rate, the two indexes of the present application are 93.76% and 1.09% respectively, which are better than the comparative algorithms, indicating that the algorithm proposed in the present application has obvious advantages. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 A flowchart of an infrared weak small target detection method based on visual characteristics provided by the embodiment of the present application is shown in the figure.

[0048] Figure 2 A schematic diagram of eight window templates of the traditional side window filtering method is shown in the figure.

[0049] Figure 3 A schematic diagram of eight modified window templates preset by the improved adaptive side window enhancement filtering method provided by the embodiment of the present application is shown in the figure.

[0050] Figure 4 An understanding diagram for processing the pixel to be filtered by using the modified window template in the upward direction according to the embodiment of the present application;

[0051] Figure 5 An understanding diagram for processing the pixel to be filtered by using the modified window template in the left-downward direction according to the embodiment of the present application;

[0052] Figure 6 A diagram of the nested window provided by the improved contrast measurement method based on the difference combination according to the embodiment of the present application;

[0053] Figure 7 A diagram of the weight function nested window according to the embodiment of the present application;

[0054] Figure 8 An implementation process diagram of the infrared dim small target detection method based on the visual characteristics provided by the embodiment of the present application;

[0055] Figure 9 A comparison result of the Gaussian filter background suppression and the side window Gaussian background suppression in the experiment of the embodiment of the present application;

[0056] Figure 10 A local gray scale of the side window Gaussian background suppression in the experiment of the embodiment of the present application;

[0057] Figure 11 A background suppression result before and after the side window filter improvement in the experiment of the embodiment of the present application;

[0058] Figure 12 A fine edge suppression result in the experiment of the embodiment of the present application;

[0059] Figure 13 An HVC result obtained by using the local difference index weight distribution method in the experiment of the embodiment of the present application;

[0060] Figure 14 A threshold segmentation result in the experiment of the embodiment of the present application;

[0061] Figure 15 A comparison of the different single frame type algorithm weightless results of scene 1 in the experiment of the embodiment of the present application;

[0062] Figure 16 A comparison of the different single frame type algorithm weightless results of scene 2 in the experiment of the embodiment of the present application;

[0063] Figure 17 A comparison of the different single frame type algorithm weightless results of scene 3 in the experiment of the embodiment of the present application;

[0064] Figure 18Results comparison after weight distribution of different single-frame type algorithms for scene 1 in the embodiment experiment of the present application;

[0065] Figure 19 Results comparison after weight distribution of different single-frame type algorithms for scene 1 in the embodiment experiment of the present application;

[0066] Figure 20 Results comparison after weight distribution of different single-frame type algorithms for scene 1 in the embodiment experiment of the present application;

[0067] Figure 21 Nearest neighbor inter-frame association results in the embodiment experiment of the present application;

[0068] Figure 22 A structure schematic diagram of an infrared dim small target detection device based on visual characteristics provided by the embodiment of the present application. DETAILED DESCRIPTION

[0069] The present application will be further described in detail below in combination with specific embodiments, but the embodiments of the present application are not limited thereto.

[0070] The human eye is more sensitive to the change of gray value, and when the gray value of an observed object is obviously different from the gray value of the surrounding environment, the target can be quickly distinguished. In an infrared image, the intensity of the target is not the highest in the scene due to the existence of a high-light background, but the intensity of the target in the local neighborhood at the target position is often higher than that of the neighborhood pixels. The local contrast at the target position is higher. Although there may be a large area of flat distribution background with high intensity in the background area, it cannot highlight the obvious local contrast. Therefore, based on the visual characteristics of the human eye, the human eye can quickly and accurately capture the real target in the image. The present application uses this characteristic to construct an infrared dim small target detection algorithm, filters out the high-light background edge in the complex ground background and the class target clutter interference noise points, and solves the problems of false detection and missed detection of traditional algorithms. The proposed scheme is novel, and combined with inter-frame association, the real target can be better extracted and the interference information can be eliminated.

[0071] In a first aspect, the embodiment of the present application provides an infrared dim small target detection method based on visual characteristics, which can be specifically executed by a ground control console or an airborne device and the like, and is used for detecting infrared dim small targets in a complex ground background. As shown in Figure 1 The method can include the following steps:

[0072] S1, a first processed image corresponding to the original infrared image is obtained by using an improved adaptive side window enhancement filtering method to suppress the background of the original infrared image;

[0073] In raw infrared images, the ground background is complex and varied. While directly processing the original infrared image using a method based on ratio difference and local contrast can suppress backgrounds with gradual grayscale changes, it can also enhance background interference with drastic local grayscale changes, resulting in more interference information in the processed image. If Gaussian filtering is used to preprocess the image before calculating local contrast, while Gaussian filtering smooths some strong edges, it also smooths the target, reducing local differences in the target area during local contrast calculation, leading to insufficient target intensity in the processed result. Therefore, this invention first performs complex background suppression on the raw infrared image to reduce interference information in the scene, ensuring targeted processing in subsequent steps.

[0074] To suppress complex backgrounds and strong edges, a side-window Gaussian filter with edge-preserving properties can be used to estimate the slowly changing background and strong edges in the image. Then, the original image is subtracted from the background estimation result to achieve suppression of complex backgrounds and strong edges. Traditional side-window Gaussian filters have eight window templates, corresponding to different directions. The eight window templates of the side-window filtering method are as follows: Figure 2 As shown in (a) to (h), the possible directions corresponding to the background edge are included. Each filter template has two parts with different directions, as shown in the figure. The light-colored part corresponds to invalid pixels, and the dark-colored part is the effective pixel for filtering corresponding to that window template. Therefore, the dimensions of the eight templates included in the calculation are composed of 3*2, 2*3, and 2*2 respectively. In use, Gaussian filtering is performed on each window separately. When the difference between the filtering result of a certain window and the gray value of the original pixel is the smallest, that result is used as the final filtered value.

[0075] Since the result calculated by each template is the filtered value, the final value obtained after comparing it with the original pixel is the Gaussian filtered value of the side window. The difference between the Gaussian filtered value of the side window and the original pixel can achieve background suppression. It is equivalent to using the difference between the values ​​of the center pixel before and after Gaussian filtering of a certain template to obtain the background suppression result of a single template. Then, by comparison, the value closest to 0 is selected as the final background suppression result.

[0076] Because side-window filtering has good adaptability, even when combined with Gaussian filtering for background suppression, it can still preserve small targets, but the brightness and size of the targets will also be reduced to some extent. Therefore, if the size of small targets in the original infrared image is too small, after side-window Gaussian filtering for background suppression, the size of the small targets in the background suppression result image may be even smaller, or even appear as a single pixel. In this case, it is difficult to distinguish them from impulse noise, and they may be filtered out in subsequent detection.

[0077] In order to avoid the target size being reduced, the application improves the side window Gaussian background suppression algorithm, and locally enhances the original pixel points when eight filtering templates are processed respectively, and then performs Gaussian weighting. The following is the improved eight direction templates. The improved adaptive side window enhancement filtering method is provided with eight correction window templates, which are obtained by adding peripheral pixels to the traditional side window filtering window template; the eight correction window templates correspond to the up, down, left, right, upper left, upper right, lower left and lower right directions respectively; the sizes of the eight correction window templates included in the calculation are 5*3, 5*3, 3*5, 3*5, 3*3, 3*3, 3*3 and 3*3 respectively; for each correction window template, the center position is the center pixel, the filtering effective pixels of the traditional side window filtering window template adjacent to the outside of the center pixel and matched with the direction of the correction window template are the inner layer pixels, and the multiple pixels outside the inner layer pixels and matched with the direction of the correction window template are the outer layer pixels. It should be noted that for any correction window template, the center pixel, the inner layer pixel and the outer layer pixel are all filtering effective pixels thereof.

[0078] In order to facilitate understanding, the eight correction window templates of the improved adaptive side window enhancement filtering method are displayed according to the directions of the eight window templates of the side window filtering method, and details are shown in (a)-(h) of FIG. 1. Figure 3 Taking (a) of FIG. 1 as an example, (a) of FIG. 1 is understood by referring to (a) of FIG. 1. Figure 3 In (a) of FIG. 1, the middle 2*3 size area is (a) of FIG. 1. Figure 2 In (a) of FIG. 1, the blue area corresponds to the purple area in (a) of FIG. 1, that is, the filtering effective pixels of the window template in the up direction, except that the centermost purple is changed to yellow as the center pixel, the five blue pixels around the yellow as the inner layer pixels, and the nine pixels outside the inner layer pixels along the center pixel upwards as the outer layer pixels. Figure 3 Figure 2 Taking (e) of FIG. 1 as an example, (e) of FIG. 1 is understood by referring to (e) of FIG. 1. Figure 3 In (e) of FIG. 1, the middle 2*2 size area is (e) of FIG. 1. Figure 2 In (e) of FIG. 1, the blue area corresponds to the purple area in (e) of FIG. 1, that is, the filtering effective pixels of the window template in the up direction, except that the centermost purple is changed to yellow as the center pixel, the five blue pixels around the yellow as the inner layer pixels, and the nine pixels outside the inner layer pixels along the center pixel upwards as the outer layer pixels.

[0079] Taking (e) of FIG. 1 as an example, (e) of FIG. 1 is understood by referring to (e) of FIG. 1. Figure 3 In (e) of FIG. 1, the middle 2*2 size area is (e) of FIG. 1. Figure 2 In (e) of FIG. 1, the blue area corresponds to the purple area in (e) of FIG. 1, that is, the filtering effective pixels of the window template in the up direction, except that the centermost purple is changed to yellow as the center pixel, the five blue pixels around the yellow as the inner layer pixels, and the nine pixels outside the inner layer pixels along the center pixel upwards as the outer layer pixels. Figure 3 Figure 2 Taking (e) of FIG. 1 as an example, (e) of FIG. 1 is understood by referring to (e) of FIG. 1. Figure 3 In (e) of FIG. 1, the middle 2*2 size area is (e) of FIG. 1. Figure 2 ​​The purple area in the (e) figure, that is, the filtered effective pixel of the window template in the left upper direction, is only changed to yellow in the center pixel, three blue pixels around the yellow as the inner layer pixels, and five pixels outside the inner layer pixels as the outer layer pixels along the center pixel to the left upper direction. Figure 2 The corresponding figure is understood in the foregoing description, and will not be described one by one here.

[0080] Based on the eight modified window templates provided in the embodiments of the present application, taking the combination of the Gaussian filter as an example, the improved adaptive side window enhancement filter is used for background suppression on a frame of original infrared image in S1 to obtain a corresponding first processed image, which can include the following steps:

[0081] In S11, the center pixel of the current modified window template is aligned to the to-be-filtered pixel in the frame of original infrared image.

[0082] In the embodiments of the present application, the frame of original infrared image is sequentially traversed from the first pixel, and each pixel in the frame of original infrared image is processed by using the improved adaptive side window enhancement filter, and the pixel currently traversed is taken as the to-be-filtered pixel. In the processing, the eight modified window templates are processed once, and the center pixel of the current modified window template is aligned to the to-be-filtered pixel, that is, the yellow pixel in the foregoing description is aligned to the to-be-filtered pixel, so that the current modified window template is nested on the frame of original infrared image, and the alignment of each pixel is realized. The filter result of the to-be-filtered pixel corresponding to the yellow position is obtained through subsequent filter processing. Figure 3 In the embodiments of the present application, the frame of original infrared image is sequentially traversed from the first pixel, and each pixel in the frame of original infrared image is processed by using the improved adaptive side window enhancement filter, and the pixel currently traversed is taken as the to-be-filtered pixel. In the processing, the eight modified window templates are processed once, and the center pixel of the current modified window template is aligned to the to-be-filtered pixel, that is, the yellow pixel in the foregoing description is aligned to the to-be-filtered pixel, so that the current modified window template is nested on the frame of original infrared image, and the alignment of each pixel is realized. The filter result of the to-be-filtered pixel corresponding to the yellow position is obtained through subsequent filter processing.

[0083] In S12, the average of the inner layer pixel point and a plurality of outer layer pixel points adjacent to the inner layer pixel point in the frame of original infrared image is used to calculate the enhancement result of the inner layer pixel point for each inner layer pixel position of the current modified window template.

[0084] In S13, the average of the center pixel point and a plurality of inner layer pixel points adjacent to the center pixel point in the frame of original infrared image is used to calculate the enhancement result of the center pixel point for the center pixel position of the current modified window template.

[0085] In S14, the processing result of the to-be-filtered pixel under the current modified window template is obtained by performing Gaussian filter weighting on the pixel enhancement result of the center pixel position and the pixel enhancement result of all inner layer pixel positions, and then performing difference operation on the enhancement result of the center pixel.

[0086] To facilitate understanding of the processing procedure of the to-be-filtered pixel using a correction window template, two examples are given below to illustrate the processing procedure of S12-S14.

[0087] (1) Example 1:

[0088] Take the correction window template shown in Figure 3 (a) as an example for illustration. Please refer to Figure 4 for understanding. Figure 4 The pixel positions of the correction window template are numbered in

[0089] For S12, the pixel enhancement result of the original pixel corresponding to the inner-layer pixel with position 1, 2, 3, 4, or 5 in the correction window template shown in Figure 3 (a) is calculated respectively. The calculation formula is as follows:

[0090]

[0091]

[0092]

[0093]

[0094]

[0095] Wherein, L1-L5 are the 1st-5th pixels in the original infrared image of the frame, and the position distribution corresponds to 1-5 in Figure 4 . C1-C5 are the pixel enhancement results of L1-L5 in the original infrared image of the frame respectively.

[0096] It can be seen that the pixel enhancement result C1 of L1 is obtained according to the average result of L1 and the adjacent pixels L6, L7, and L8 around L1. The pixel enhancement result C2 of L2 is obtained according to the average result of L2 and the adjacent pixels L7, L8, and L9 around L2. The pixel enhancement result C3 of L3 is obtained according to the average result of L3 and the adjacent pixels L9, L 10 , and L 12 around L3. The pixel enhancement result C4 of L4 is obtained according to the average result of L4 and the adjacent pixels L 11 , and L 13 around L4. The pixel enhancement result C5 of L5 is obtained according to the average result of L5 and the adjacent pixels L 12 , and L 14 around L5.

[0097] In S13,Figure 3 (a) The correction window template shown in the figure, for the center pixel with position 0, calculates the pixel enhancement result of the center pixel position corresponding to the original infrared image of the frame. The calculation formula is as follows:

[0098]

[0099] Wherein, L0 is the 0th pixel in the original infrared image of the frame, the position corresponds to 0 in the formula (a). C0 is the pixel enhancement result of L0 in the original infrared image of the frame. It can be seen that C0 is obtained according to the average result of L0 and the adjacent pixels L1, L2, L3, L4 and L5 around L0. Figure 4

[0100] In S14, C0 and C1-C5 are subjected to Gaussian filtering and weighting processing, and the original processing result E of the correction window template can be obtained; then the difference operation is performed with the enhancement result of the center pixel, that is, C0 is subtracted from E to obtain the background suppression result corresponding to the correction window template, which is taken as Figure 3 (a) The processing result of the correction window template on the pixel to be filtered is shown in the figure, because Figure 3 (a) The correction window template is the first one, which can be represented by B1. The processing result is shown in the figure.

[0101] In the embodiment of the application, for all correction window templates, the weight of the pixel enhancement result C0 of the center pixel is set to 4; the weight value of the pixel point in the orthogonal direction thereof is 2, that is, the weight of the pixel enhancement result of the pixel directly above, directly below, directly left and directly right of the center pixel is set to 2; the weight value of the pixel point in the diagonal direction is 1, that is, the weight of the pixel enhancement result of the upper left, lower left, upper right and lower right of the center pixel is set to 1. By such setting, the two-dimensional Gaussian distribution of the weak small target brightness decaying from the center to the periphery is met. Then the sum is calculated and divided by the sum of the weight values, and the processing result of the template is obtained, and the background suppression result corresponding to the template is obtained by subtracting C0 from E.

[0102] The above process is expressed by the formula as follows:

[0103]

[0104] ​As can be seen, this template adds a layer of local pixels outside the traditional 3x3 template to locally enhance the pixels within the 3x3 template. For inner pixels 1-5, the average value of the corresponding outer pixels in the orthogonal and diagonal directions of each inner pixel is used to estimate the background pixel value of the inner pixel. The ratio of the inner pixel value to its corresponding background pixel value is used as the enhancement factor for the inner pixel, which is then multiplied by the inner pixel value to achieve the enhancement effect. For inner pixel 0, the estimated background pixel value is represented by the average value of inner pixels 1-5.

[0105] For details on the Gaussian filtering weighting process, please refer to the relevant technical explanations; it will not be described in detail here.

[0106] (2) Example 2:

[0107] by Figure 3 (e) is used as an example of the correction window template for illustration. Please refer to [link / reference]. Figure 5 understand. Figure 5 The position of each pixel in the modified window template is numbered. 0 represents the center pixel, 1-3 are inner layer pixels, and 4-8 are outer layer pixels.

[0108] Regarding S12, Figure 3 For the correction window template shown in (e), the pixel enhancement results of the original pixels corresponding to the inner layer pixel positions 1, 2, and 3 in the original infrared image of that frame are calculated respectively. The calculation formula is as follows:

[0109]

[0110]

[0111]

[0112] L1 to L3 are the third pixels in the original infrared image of this frame, and their positions are distributed accordingly. Figure 5 In the image, 1 to 3. C1 to C3 represent the pixel enhancement results of L1 to L3 in the original infrared image of this frame.

[0113] As can be seen, the pixel enhancement result C1 of L1 is obtained by averaging the results of L1 and its neighboring pixels L4, L5, and L7. The pixel enhancement result C2 of L2 is obtained by averaging the results of L2 and its neighboring pixels L5 and L6. The pixel enhancement result C3 of L3 is obtained by averaging the results of L3 and its neighboring pixels L7 and L8.

[0114] In S13 Figure 3(e) the correction window template shown, the pixel enhancement result of the original pixel corresponding to the center pixel position in the original infrared image of this frame is calculated for the center pixel with position 0. The calculation formula is as follows:

[0115]

[0116] Wherein, L0 is the 0th pixel in the original infrared image of this frame, the position corresponds to 0 in Figure 5 . C0 is the pixel enhancement result of L0 in the original infrared image of this frame. It can be seen that C0 is obtained according to L0 and the average result of the adjacent pixels L1, L2 and L3 around L0.

[0117] In S14, C0 and C1-C3 are subjected to Gaussian filtering and weighting processing. Specifically, the corresponding pixel enhancement results are weighted and summed by using the preset weights of C0 and C1-C3, and then divided by the sum of the weights, so that the original processing result E of the correction window template can be obtained; then, the difference operation is performed with the enhancement result of the center pixel, that is, C0 is subtracted from E to obtain the background suppression result corresponding to the correction window template, which is taken as Figure 3 (e) the processing result of the correction window template shown on the pixel to be filtered and processed, since Figure 3 (e) the correction window template shown is the fifth one, which can be represented as B5 to represent the processing result.

[0118] Referring to the above two examples, it can be understood that the processing result of the pixel to be filtered and processed under each correction window template can be obtained, which are B1, B2, B3, B4, B5, B6, B7 and B8 in turn, corresponding to each correction window template in Figure 3 . The process of obtaining the processing result of the remaining correction window templates is combined with the examples and Figure 3 corresponding figures and will not be described in detail here.

[0119] It should be noted that in the process of using one correction window template to perform the above processing on the current pixel to be filtered and processed in the original infrared image, if the original pixel at some position in the original infrared image does not exist, then the 0 processing is performed. The specific processing process can be understood in combination with the traditional side window filtering method, and will not be described in detail here.

[0120] S15, from the processing results of the pixel to be filtered and processed under all correction window templates, the processing result closest to 0 is selected as the final processing result to replace the pixel to be filtered and processed;

[0121] Specifically, to achieve the suppression of the edge background, and as in the traditional side window filtering algorithm, one value closest to 0 is selected from B1, B2, B3, B4, B5, B6, B7 and B8 as the final processing result of the pixel to be filtered, and the final processing result is used to replace the pixel to be filtered. Then, the next pixel to be filtered in the original infrared image is processed by S12-S15, and the process is repeated.

[0122] S16, after the replacement of all the pixels to be filtered in the original infrared image is completed, a corresponding first processed image is obtained.

[0123] After each pixel in the original infrared image is processed according to the above processing process, the pixels in the original infrared image are replaced by the corresponding final processing result, and thus the original infrared image becomes the first processed image.

[0124] It can be understood that, in the processing of the pixels in the original infrared image, the most suitable correction window template is selected for the current pixel to be filtered, the original pixel is locally enhanced, the Gaussian weighting is performed, and the difference operation is performed between the enhanced result of the center pixel and the current pixel to be filtered, so that the best filtering result of the current pixel to be filtered is obtained. Therefore, the correction window templates used for all the pixels to be filtered are different.

[0125] The improved adaptive side window enhancement filtering method provided by the embodiment of the present application can suppress the complex background and strong edge of the infrared original image. However, in the infrared image, small weak targets with very small size are inevitable. The size of the target in the background suppression result obtained by combining the traditional side window filtering method with the Gaussian filtering method is even smaller. At this time, the gray distribution of the target and the noise point are the same, and both are in the form of a single pixel point. In the subsequent processing process, the target is mistaken for a noise point and is filtered out. If the adaptive side window enhancement filtering method provided by the present application is used, the two-dimensional Gaussian distribution characteristics of the small weak target are retained, which can be obviously distinguished from the single noise point, lays a foundation for the subsequent detection work, and has high saliency. The residual edge information in the background area is less, although the brightness is enhanced, but it is obviously not characterized by two-dimensional Gaussian distribution, and the saliency is greatly different from that of the real target. The noise may be enhanced in the process of first edge preservation and then suppression, but the enhanced noise and the real target are still obviously different in gray distribution, and the saliency of the real target is much greater than that of the noise. In the subsequent process, the diagonal local difference product method is used to further enhance the target and suppress the background.

[0126] S2, removing small edges, clutter interference and realizing target enhancement on the first processed image by using the diagonal local difference product method and the local difference index weight distribution method to obtain a corresponding second processed image;

[0127] As described above, the background suppression and target enhancement are processed by the difference joint local contrast method, so there is no targeted processing of small edges. Since the emphasis is on filtering out when filtering out small edges, this step mainly relies on the difference local contrast in the difference joint local contrast. The pixel mean value in the center sub-window is selected to estimate the target region pixel value, and the maximum mean value in the background sub-window is selected to estimate the background region pixel value. Then, the difference operation is directly performed without the size comparison and diagonal product process. This method cannot directly suppress the edges to 0, and the remaining edges are likely to be enhanced in the subsequent processing, resulting in a high false alarm rate.

[0128] Therefore, the present application proposes to remove small edges based on the diagonal local difference product method. In the target enhancement and small edge and clutter interference filtering stage, a multi-scale nested window is constructed. In view of the one-sided symmetry of the gray scale distribution at the edge, a method based on the symmetric difference local contrast product is first used. The difference local contrast is introduced, and the difference local contrast results in the orthogonal and diagonal directions are multiplied, thereby realizing the filtering of small edges.

[0129] The diagonal local difference product method is provided with a nested window, which is 15*15 pixels in size, wherein 3*3 pixels are used as a sub-window, forming a total of 5*5 sub-windows. The center sub-window is the center layer, which is defined as the target region. The 8 sub-windows adjacent to the outside of the center layer are the intermediate layer, which is defined as the intermediate layer background region. The 16 sub-windows outside the intermediate layer are the outermost layer, which is defined as the outermost layer background region.

[0130] The nested window is shown in Figure 6 The nested window contains 5*5 sub-windows, each sub-window contains 3*3 pixels, that is, the nested window contains 15*15 pixels. The center yellow sub-window marked 0 is the center layer. The 8 blue sub-windows adjacent to the outside of the center layer marked 1-8 are the intermediate layer. The 16 sub-windows outside the intermediate layer marked 1-16 are the outermost layer.

[0131] The nested window provided by the diagonal local difference product method is used as a window template. In S2, the first processed image is removed by using the diagonal local difference product method and the local difference index weight distribution method to remove small edges, clutter interference and realize target enhancement, and a corresponding second processed image is obtained. The steps can include the following steps:

[0132] S21, for each to-be-processed pixel in the first processed image, aligning the to-be-processed pixel to the center pixel of the center layer sub-window of the nested window;

[0133] In the embodiment of the application, for the first processed image, each pixel in the first processed image is sequentially traversed from the first pixel, and the current traversed pixel is taken as a to-be-processed pixel, and the nested window provided by the diagonal local difference product method is used as a window template for processing. When processing, the center pixel of the nested window is first aligned to the to-be-processed pixel, that is, the Figure 6 The center yellow pixel is aligned to the to-be-processed pixel, so that the nested window is nested on the first processed image, and the alignment of each pixel is realized.

[0134] S22, calculating the difference value local contrast of each intermediate layer sub-window by using a preset intermediate layer difference value local contrast calculation formula, and calculating the difference value local contrast of each outermost layer sub-window by using a preset outermost layer difference value local contrast calculation formula;

[0135] The preset intermediate layer difference value local contrast calculation formula is:

[0136]

[0137] The preset outermost layer difference value local contrast calculation formula is:

[0138]

[0139] Wherein, d i represents the difference value local contrast of the i-th intermediate layer sub-window, that is, the intermediate layer difference value contrast in the i-th direction, i = 1, 2, …, 8; q j represents the difference value local contrast of the j-th outermost layer sub-window, that is, the outermost layer difference value contrast in the j-th direction, j = 1, 2, …, 16;

[0140] M0 represents the pixel average value of the center layer sub-window, that is, the average value of 3x3 pixels in the center layer sub-window; and is represented as: Wherein, is the k-th pixel in the center layer sub-window;

[0141] Bmi represents the i-th intermediate layer sub-window; M Bmi represents the pixel average value of the i-th intermediate layer sub-window, that is, the average value of 3x3 pixels in the i-th intermediate layer sub-window; and is represented as: Wherein, is the k-th pixel in the i-th intermediate layer sub-window; i = 1, 2, …, 8;

[0142] Boj represents the jth outermost sub-window; M Boj represents the pixel average of the jth outermost sub-window, i.e., the average of 3x3 pixels within it; denoted as: wherein, is the kth pixel within the jth outermost sub-window; j = 1, 2,..., 16.

[0143] It can be understood that through the above processing, the difference local contrast d1-d8 of the 8 intermediate layer sub-windows and the difference local contrast q1-q16 of the 16 outermost sub-windows can be obtained. 16 .

[0144] In a real scene, although a small target has a quantitative specification within 80 pixels, the specific size of the small target within this range is not determined. The traditional filtering method only calculates a single scale and does not consider the change of the size of the small target. For some algorithms, it is suitable for detecting small targets, and for some algorithms, it is suitable for small targets with slightly larger size. The calculation of two scales used here is to cope with the small target with constantly changing size.

[0145] S23, calculate the product of the difference local contrast of each group of intermediate layer sub-windows belonging to the diagonal, and determine the minimum value of the product obtained by all groups of intermediate layer sub-windows belonging to the diagonal as the intermediate layer product measure; and calculate the product of the difference local contrast of each group of outermost sub-windows belonging to the diagonal, and determine the minimum value of the product obtained by all groups of outermost sub-windows belonging to the diagonal as the outer layer product measure;

[0146] The small target is generally a centrally symmetric shape, similar to a diffuse spot. Therefore, when there is a target distribution in the center layer of the nested window, the difference local contrast d i and q j in each direction of the intermediate layer and the outermost layer are relatively large, at this time the enhancement of the target can be realized; when the nested window is in a flatly distributed background area, the difference local contrast d i and q jare all small; when the nested window is at the edge of the background, the gray values on both sides of the edge are different, and the edge has single-sided symmetry, at this time the center layer contains both high pixel values and low pixel values, and in the two background areas on the diagonal direction of the center layer, one area must have higher pixel values and the other area must have lower pixel values, then the pixel average value of the background area with higher pixel values is greater than the pixel average value in the center layer, and the pixel average value of the background area with lower pixel values is less than the pixel average value in the center layer, at this time, one of the local contrast of a group of difference values on the diagonal direction of the middle layer and one of the local contrast of a group of difference values on the diagonal direction of the outermost layer will be equal to 0, that is, one of d i , d 9-i and q j , q 17-j on the diagonal direction of the outermost layer will be equal to 0, which can suppress the edge. Therefore, the present application adopts diagonal difference value product factor based edge suppression. The minimum value of the product obtained by calculating the difference value local contrast of the middle layer sub-window belonging to the diagonal is obtained to obtain the middle layer product measure; the minimum value of the product obtained by calculating the difference value local contrast of the outermost layer sub-window belonging to the diagonal is obtained to obtain the outer layer product measure, so as to perform edge suppression.

[0147] Specifically, for the difference value local contrast d1-d8 of the middle layer sub-window, d2, d7 belong to a group of diagonals, d4, d5 belong to a group of diagonals, d1, d8 belong to a group of diagonals, and d3, d6 belong to a group of diagonals. Then calculate d2×d7, d4×d5, d1×d8, d3×d6 respectively, and determine the minimum value of the four products as the middle layer product measure.

[0148] The middle layer product measure is calculated and expressed by the formula as:

[0149] D m =min{(d i ×d 9-i ),i=1,2,3,4};

[0150] Wherein, D m is the middle layer significant value, that is, the middle layer product measure.

[0151] For the difference value local contrast q1-q 16 of the outermost layer sub-window, the outer layer product measure is calculated and expressed by the formula as:

[0152] Q o =min{(q j ×q 17-j ),j=1,2,…,8};

[0153] wherein Q o is the outermost layer saliency value, i.e. the outer layer product measure. See Figure 6 It can be understood that the difference local contrast q1-q 16 , q1, q 16 , q2, q 15 , q3, q 14 , q4, q 13 , q5, q 12 , q6, q 11 , q7, q 10 , q8, q9 respectively belong to a diagonal group, then the product of each group is calculated respectively, and the minimum value of these products is determined as the outer layer product measure. The edge suppression based on the diagonal difference product factor can further suppress small edges and complete the scale operation.

[0154] The above process of the embodiment of the application can well play the effect of edge suppression, but there are still some clutter interferences in the background, at this time, the local contrast method is needed to improve the saliency of the real target and suppress the clutter interferences. The saliency values of two scales are calculated simultaneously, which can cope with the detection of small weak targets of different sizes. For small size targets, the saliency value of the result calculated by the middle layer scale is higher, and for large size targets, the saliency value of the result calculated by the outermost layer scale is higher, so the larger value between the two is selected as the final detection result.

[0155] S24, selecting the larger value between the middle layer product measure and the outer layer product measure as the diagonal local difference product result;

[0156] This step can be expressed by the formula as follows:

[0157]

[0158] wherein DQ represents the diagonal local difference product result.

[0159] It should be noted that in the process of using the nested window preset based on the diagonal local difference product method to process the current to-be-processed pixel in the first processed image, if the original pixel at some position in the first processed image does not exist, then the 0 is supplemented, which will not be described in detail.

[0160] S25, using the local difference index weight distribution method to determine the weight coefficient corresponding to the diagonal local difference product result; using the product of the diagonal local difference product result and the corresponding weight coefficient as the updated pixel value to replace the to-be-processed pixel;

[0161] In different scenarios, the intensity of grayscale distribution within the target varies. Therefore, to avoid situations where the weighting function is related to the degree of grayscale change (e.g., RIL and BGC) or the target size (e.g., RLD effect is related to the target size), an exponential weighting method is used for target enhancement and clutter removal, considering that the exponential function is increasing and sensitive to grayscale changes. The grayscale distribution within the target region in the original image is considered, and local difference information is used to characterize the grayscale distribution within the target region, thereby filtering out clutter and enhancing the target. This step achieves clutter removal and target enhancement based on local difference-based exponential weight allocation.

[0162] For larger targets, the previous steps have already demonstrated high saliency. However, for smaller targets, the previous steps still make them difficult to distinguish from scene clutter, and the saliency is not as good as that of larger targets. Therefore, this step mainly focuses on smaller targets. Furthermore, after the previous steps, both the target and the interfering clutter appear as bright spots, and the target has lost its two-dimensional Gaussian distribution characteristics. Therefore, the construction of the exponential weighting function will be based on the original infrared image. For the nested window of the weighting function used in this step, please refer to [link to relevant documentation]. Figure 7 As shown.

[0163] Since this step mainly targets small-sized targets, a 3*3 nested window is selected. Based on the two-dimensional Gaussian distribution of weak targets in the original infrared image, the gray level in the target area decreases from the center to the surrounding area. Therefore, the ratio of the central pixel to the mean of the pixels in its 8-neighborhood is used as the gray level difference information. Since the e-exponent is increasing and is sensitive to gray level changes, and when the exponent part is greater than 0, the result of the e-exponent is greater than 1, this feature is used to combine the e-exponent with the ratio and local contrast, thereby giving the target a higher weight value.

[0164] When using this nested window of the weighting function, first align the center pixel corresponding to the yellow portion with the pixel to be processed in the original infrared image. Then process it according to the following formula:

[0165]

[0166]

[0167] Among them, T i express Figure 7 The weight function nested window position shown corresponds to the i-th pixel in the original infrared image; M T T represents the average value of 9 pixels within the nested window of the weighting function, i.e., the average pixel value within an 8-neighborhood; maxThe maximum value of 9 pixels in the weight function nested window, i.e. the maximum value of pixels in the 8-neighborhood; W represents the weight coefficient obtained based on the exponential weight function; when the weight function nested window slides to the target, the gray scale inside the target region decays to the surroundings, and the pixel changes are relatively sharp, so is much greater than 1, (T max -M T ) is also large, and the obtained value is also greater than 1, at this time the result of W is a large value; when the weight function nested window slides to the background region, since the background is mostly flat distribution, at this time is close to 1, so that is close to 1, at this time the result is close to 0, so that the result of W is also close to 1; when the weight function nested window moves to the clutter interference, compared with the dim target, the clutter interference is less dense, the background window gray scale distribution is relatively discrete due to the clutter interference, and the nested window is only 3*3, at this time although is greater than 1, (T max -M T ) is greater than 0, the obtained value of the target is small, so the result of W is also smaller than the target.

[0168] The weight function W is multiplied by the diagonal local difference product result DQ, i.e. the processing result DQ of the previous step is assigned a weight value, to obtain an updated pixel value to replace the pixel to be processed, so as to realize background suppression and target enhancement.

[0169] The calculation process of the updated pixel value can be expressed by the formula:

[0170] HVC=W*DQ;

[0171] Wherein, HVC represents the updated pixel value.

[0172] S26, after the replacement of all the pixels to be processed in the first processed image is sequentially completed, a corresponding second processed image is obtained.

[0173] For the first processed image, when each pixel is processed according to the above processing process, the pixels in the first processed image are replaced by the final updated pixel value of themselves, so the first processed image becomes the second processed image.

[0174] After adaptive side window enhancement filtering through S1, some small edges and clutter interference are still left in the image. After S2, based on diagonal local difference product method and local difference index weight distribution method, small edges, clutter interference can be removed and target enhancement can be achieved. Although some interference points exist in the result of this step, the significant value of the target is obviously greater than that of most interference points. After threshold segmentation, the interference points can be well filtered out to achieve a better target detection effect. For those interference points with significant value comparable to or greater than the target, they will also be retained after threshold segmentation, which also reflects the limitations of single-frame processing algorithm. Therefore, in the subsequent process, the nearest neighbor correlation will be used to finally filter out the interference points. Therefore, the embodiment of the present application effectively performs target enhancement and background suppression based on the diagonal local difference product method, which helps to improve the accuracy of subsequent target detection of the original infrared image.

[0175] S3, obtaining the position of the suspected target point in the second processed image by using a threshold segmentation method;

[0176] In an optional embodiment, the threshold segmentation method comprises a threshold segmentation method combining mean value and standard deviation.

[0177] Specifically, for the threshold segmentation method, an empirical value can be set as a weight coefficient, and then the mean value and the standard deviation of each pixel in the second processed image are calculated to determine the threshold used by the threshold segmentation method. Specifically, threshold = mean value + weight coefficient x standard deviation. Then, the determined threshold is used to obtain the position of the suspected target point in the second processed image. Specifically, the pixel positions in the second processed image that are higher than the threshold are extracted as the position of the suspected target point. It can be understood that the position of the suspected target point extracted from the second processed image can be more than one. Since the present application is a single-frame algorithm for target detection of the infrared image, due to the limitations of the single-frame algorithm, some interference points that are difficult to filter out will exist in the position of the suspected target point obtained at this time. These interference points have pixel values higher than the threshold, but they are not real targets.

[0178] It can be seen that by threshold segmentation of S3, the real target can be separated from the residual interference points, so as to perform binary processing on the image, and further separate the interference points with large intensity difference from the target itself. The interference lower than the set threshold in a frame of original infrared image can be removed, and the suspected target points obtained include the real target and can also contain the interference higher than the set threshold. The interference higher than the threshold is defined as a false target point. Due to the limitation of the single-frame algorithm, some interference points that are difficult to filter out still exist in the processing result. In the subsequent process, the nearest neighbor multi-frame correlation method is used to remove them from the suspected target points. For the threshold segmentation method combined with the mean and standard deviation, please refer to the related technical understanding, which will not be repeated here.

[0179] S4, based on the positions of the suspected target points obtained from the continuous multiple frames of original infrared images, the position of the real target point is determined by using the nearest neighbor multi-frame correlation method.

[0180] In the sequence images, the relative moving distance of the target in each frame image is short, and there is strong correlation. The interference information contained in the processing result after the single-frame algorithm processing is not fixed and has no correlation. By using this characteristic, the nearest neighbor correlation is performed on the result after the single-frame algorithm processing, the weak and small target is finally detected, the real target is extracted according to the continuity of the real target motion trajectory by using the nearest neighbor correlation method, the false target is filtered out, and the detection probability can be improved.

[0181] As described above, for a frame of original infrared image, the positions of the suspected target points corresponding thereto can be obtained by using S1-S3. Then, for continuous multiple frames of original infrared images, S1-S3 is performed respectively to obtain the positions of the suspected target points corresponding to each frame. In order to facilitate understanding, three continuous frames of original infrared images can be used here.

[0182] According to whether the positions of the suspected target points in adjacent frames are within their respective neighborhoods, it is judged whether the suspected target points in the two frames of images meet the moving rule of the real target. The suspected target is determined according to the two-by-two correlation results between the three frames of images. By this method, the real target can be effectively extracted, and the clutter interference information without moving rule can be filtered out.

[0183] The nearest neighbor correlation technology is mature enough. For the nearest neighbor correlation method, please refer to the related technical understanding, which will not be repeated here.

[0184] Through the above processing, the coordinates of the real target points can be detected by using three continuous frames of original infrared images. Optionally, the detected real target points can be tracked and processed subsequently.

[0185] The implementation process of the method of the embodiment of the application can be referred to Figure 8It is understood that the present application proposes a small and dim targets detection method based on human visual characteristics (HVC for short). The method first filters out complex background, strong edge, small edge and clutter interference in a single frame of complex ground background infrared original image by using different technical means, and enhances the infrared small and dim targets. The real targets are extracted by threshold segmentation operation and inter-frame correlation, so as to realize the infrared small and dim targets detection in complex ground background. The specific algorithm process is shown in Figure 8 The method mainly includes complex background suppression based on side window enhancement filtering, small edge removal based on diagonal local difference product, clutter interference removal based on local difference exponential weight function distribution, threshold segmentation and nearest neighbor correlation. Figure 8 The complex background suppression based on side window enhancement filtering in S1 means that the background suppression is performed on a frame of original infrared image by using an improved adaptive side window enhancement filtering method, to obtain a corresponding first processed image. Figure 8 The small edge removal based on diagonal local difference product in S21-S24, and the target enhancement and clutter interference removal based on local difference exponential weight distribution in S25-S26. Figure 8 The threshold segmentation binarization in S3. Figure 8 The nearest neighbor correlation in S4. Each part is understood in correspondence with the related content in the foregoing, and will not be repeated here.

[0186] This invention provides a method for detecting small infrared targets based on visual characteristics. Addressing the issue that traditional side-window Gaussian background suppression algorithms reduce the size of the real target while suppressing background, this invention proposes a background suppression algorithm based on side-window enhancement filtering. This algorithm enhances the pixels to be processed before applying the Gaussian filtering algorithm. This method not only retains the suppression effect of traditional methods on complex backgrounds and strong edges but also avoids the problem of the real target becoming too small after processing. Traditional methods using the pixel mean of the central sub-window and the background sub-window suppress various background interferences through difference calculations, but do not specifically consider the gray-level differences on both sides of small edges, resulting in poor suppression of small edges. This invention proposes a method based on... The fine edge suppression algorithm based on diagonal local difference product fully considers the gray-level difference on both sides of the fine edge and the central symmetry of the real target. This method can not only preserve the real target, but also selectively filter out fine edges. Addressing the issue that existing weight function construction methods only relate to the gray-level distribution within the sub-window, leading to the retention of clutter interference with drastic gray-level changes as real targets, this invention utilizes the increasing exponential function's sensitivity to gray-level changes and the gray-level distribution within the target area of ​​the original image. It proposes a local difference exponential weight allocation method, assigning a larger weight value to the real target and a smaller weight value to interference points, thus increasing the difference between the target and interference points and distinguishing them. Through simulation comparison with the processing effects of two traditional algorithms and one existing published patent, the single-frame processing part of the present invention outperforms other algorithms in both background suppression and target extraction. The local signal-to-noise ratio gain of the present invention after processing the single-frame image reaches 1.6961 and 1.5769 in two scenarios, respectively, which are the maximum values ​​among the comparison algorithms. Although the background suppression factor is the minimum value among the comparison algorithms in only one scenario, it can be controlled at a small value in the other two scenarios. The nearest neighbor association was added to the four comparison algorithms to statistically analyze the detection probability and false alarm rate. The two indicators of the algorithm of the present invention are 93.76% and 1.09%, respectively, which are better than the comparison algorithms, indicating that the algorithm proposed in the present invention has obvious advantages.

[0187] To verify the effectiveness of the method of the present invention, the following experimental results are used to illustrate it.

[0188] (i) Regarding S1, this step can use an improved adaptive side window enhancement filtering method to suppress the background of a frame of original infrared image, and initially filter out the complex background and strong edge background interference in the image.

[0189] Please see the experimental results. Figure 9 As shown, Figure 9 This is a comparison of Gaussian background suppression using a filter and Gaussian background suppression using a side window in the experiments of this embodiment of the invention.Figure 9 Fig. a is an original infrared image, Figure 9 Figs. b, c are respectively the results of Gaussian filter background suppression and side window Gaussian background suppression. The red frame part represents the target.

[0190] As Figure 9 Fig. b is the result of Gaussian filter background suppression, the target is well preserved, but there is a lot of background interference around it. It can be known that the background suppression effect is poor if only Gaussian filter is used, so the side window filter is combined with the Gaussian filter. Figure 9 Fig. c is the result of Gaussian side window background suppression. It can be seen that the background suppression effect is good after the combination of side window filter and Gaussian filter. However, in the infrared image, there will inevitably be a case that the size of the weak and small target is very small, and the size of the target will be even smaller after background suppression, Figure 10 is the gray scale of the target area, specifically, Figure 10 is the local gray scale of the side window Gaussian background suppression in the embodiment experiment of the application; Figure 10 Fig. a in the embodiment represents the target area of the original image, Figure 10 Fig. b in the embodiment represents the target area after background suppression. As Figure 10 As shown in Fig. a, the pixel points of the target area are not more than 9, and the gray scale values of the pixel points in the neighborhood are particularly low, such as Figure 10 As shown in Fig. b, the pixel points of the target area after background suppression are left with one, at this time, the gray scale distribution of the target is exactly the same as that of the noise point, both of which are distributed in the form of a single pixel point, and will be mistaken for a noise point and filtered out in the subsequent processing process, so the traditional side window filter needs to be improved.

[0191] Figure 11 Figs. a, b, c are respectively the original infrared images of scenes 1, 2 and 3; Figs. d, e, f are respectively the background suppression results of Figs. a, b, c using the side window Gaussian, Figs. g, h, i are respectively the background suppression results of Figs. a, b, c using the improved side window, that is, the processing results of the improved adaptive side window enhancement filter method in S1 of the application. Figure 13 Figs. a, b, c are respectively the original infrared images of scenes 1, 2 and 3; Figs. d, e, f are respectively the background suppression results of Figs. a, b, c using the side window Gaussian, Figs. g, h, i are respectively the background suppression results of Figs. a, b, c using the improved side window, that is, the processing results of the improved adaptive side window enhancement filter method in S1 of the application.

[0192] Figure 11It can be seen from (d) (e) (f) that most of the complex background in the scene is well suppressed after the traditional side window Gaussian filtering, and the strong edges such as roads are also suppressed, but the size of the target in the original image is small, and after processing, the size of the weak and small target is further reduced, and even only a single pixel point, which will affect the subsequent process. Figure 11 (g) (h) (i) are improved side window filtering results, it can be seen that the target is obviously enhanced, the size is not reduced, and the background suppression is also good; but in the process of enhancing the target, the noise and clutter interference are enhanced to some extent, such as Figure 11 (g) (h) (i) shown by the blue box; and some small edges such as grass, trees, rocks and buildings in the scene are not completely suppressed, such as Figure 11 (g) (h) (i) shown by the yellow box. The subsequent process will be processed.

[0193] (II) For the diagonal local difference product method in S2:

[0194] Using the diagonal local difference product method, the small edge of the image after background suppression can be removed.

[0195] The small edge suppression result in the application is shown in Figure 12 The red box is the target, and the blue box is the clutter interference. Figure 12 (a), (b), (c) are the small edge suppression results of scenes 1, 2 and 3 after S1 processing, Figure 11 (a), (b), (c) are Figure 12 (a), (b), (c) are the original infrared images corresponding to (a), (b) and (c).

[0196] From the simulation results Figure 12 It can be seen that compared with Figure 11 (g) (h) (i), this step has high adaptability when processing small edges, and the small edges such as rocks, buildings and trees in the scene are suppressed to a great extent, but this step is mainly based on the difference local contrast for diagonal difference calculation, and focuses on filtering small edges, so it does not enhance the target in the scene, and there are some clutter interferences in the processing results, such as Figure 12 (g) (h) (i) shown by the blue box, their brightness is comparable to that of the target, so even if threshold segmentation is used, it is difficult to distinguish from the target.

[0197] (III) For the local difference index weight distribution method in S2:

[0198] This step uses a method based on local difference index weight distribution to remove residual clutter interference while achieving the effect of target enhancement, thereby obtaining the single-frame image processing result HVC.

[0199] The simulation results of the single-frame image processed by the algorithm of the present application are shown in Figure 13 The red box is the target and the blue box is the interference information. Figure 13 (a), (b) and (c) in the figure are the HVC results obtained for scenes 1, 2 and 3 respectively, Figure 11 (a), (b) and (c) in the figure are Figure 13 (a), (b) and (c) in the figure are the original infrared images corresponding to the above.

[0200] From the simulation results Figure 13 It can be seen that, compared with Figure 12 the interference point in the scene is significantly weakened, Figure 13 the interference point in (b) is even difficult to distinguish with the human eye, Figure 13 the interference point in (c) has a smaller difference in brightness from the target than Figure 12 (b), but the interference points can be well filtered out through threshold segmentation. Even though the target size in scene 3 is larger and the target sizes in scenes 1 and 2 are smaller, the target can still be given a larger weight coefficient through exponential weight distribution. Therefore, by using the exponential weight function, a larger weight value can be given to the real target and a smaller weight value can be given to the interference point, so that the target and the interference point have a large difference in saliency, thereby distinguishing the target from the interference point. However Figure 13 there are still interference points in (a) that need to be purified through threshold segmentation.

[0201] (Four) for S3:

[0202] S3 is threshold segmentation on the single-frame processing result to separate the real target from the residual interference points, thereby performing binaryzation processing on the image. The result is shown in Figure 14 , where Figure 14 is the threshold segmentation result in the embodiment experiment of the present application; the red box is the target and the blue box is the interference information. Figure 14 (a), (b) and (c) in the figure are the threshold segmentation results obtained for scenes 1, 2 and 3 respectively, Figure 11 (a), (b) and (c) in the figure are Figure 14 (a), (b) and (c) in the figure are the original infrared images corresponding to the above.

[0203] After threshold segmentation, the positions of the real targets can be accurately extracted from the three sequences, Figure 14 there are no interference points in (b) and (c), but Figure 14In (a), even if threshold segmentation is used, the interference points cannot be removed, which indicates that the single-frame algorithm has limitations in processing images, and there is no way to remove the clutter interference with high gray scale distribution density and similar to the real target, so the false target needs to be excluded by inter-frame association subsequently.

[0204] The comparison results with other single-frame algorithms are shown in (red box for target, blue box for interference information), in order to reflect the advantages of the algorithm of the application, the existing algorithms WSLCM, IWELCM, MRDLCM_RLD and the algorithm of the application are compared (all threshold segmentation is performed). Figures 15-17

[0205] Since WSLCM, IWELCM and MRDLCM_RLD do not have a separate background suppression module, for the comparison results of background suppression and small edge filtering, the results of the four algorithms before weight distribution are compared.

[0206] Figure 15 In order to compare the original infrared images shown in (a) in the embodiment experiment of the application with the results of different single-frame algorithms without weight; Figure 11 In order to compare the original infrared images shown in (b) in the embodiment experiment of the application with the results of different single-frame algorithms without weight; Figure 16 In order to compare the original infrared images shown in (c) in the embodiment experiment of the application with the results of different single-frame algorithms without weight; Figure 11 In order to compare the original infrared images shown in (c) in the embodiment experiment of the application with the results of different single-frame algorithms without weight; Figure 17 In order to compare the original infrared images shown in (c) in the embodiment experiment of the application with the results of different single-frame algorithms without weight; Figure 11 In (a), (b), (c) and (d), WSLCM, IWELCM, MRDLCM_RLD and HVC are respectively; HVC is the algorithm of the application. Figures 15-17 Because WSLCM performs Gaussian filtering preprocessing before local contrast calculation, the target is smoothed to a certain extent, so after local contrast calculation, the intensity of the clutter interference in the scene is higher than that of the weak small target, as shown in

[0207] In (a), the intensity of the weak small target is lower than that of the two clutter interferences, Figure 16 In (a), the intensity of the weak small target is lower than that of the two clutter interferences, Figure 17 In (a), the intensity of the weak small target is lower than that of the two clutter interferences, Figure 15 (b) and (c) and Figure 17 ​(b) (c) It can be seen that the interference points are obviously more than WSLCM algorithm, but the target saliency is higher than WSLCM, which is consistent with the analysis above. For the HVC algorithm proposed in the application, compared with WSLCM, the saliency feature of the target can be kept, the interference information in the result is greatly reduced compared with IWELCM and MRDLCM_RLD algorithm, and the target can be effectively extracted in the three scenes, while for the other three algorithms, such as Figure 15 (a) Figure 16 (b) (c), the target is lost, and only interference information is left. The algorithm in the present application uses the diagonal local difference product based on the gray difference on both sides of the edge to filter out the fine edge, as shown in Figure 15 (d), 16 (d), 17 (d), only clutter interference is left in the figure, and fine edges are easily filtered out, which is consistent with the analysis above. After comparison, the effect of HVC is better than the other three algorithms in the complex background suppression and fine edge filtering stage.

[0208] Figures 18-20 is the comparison of the results of the four algorithms after weight allocation and threshold segmentation. Figure 18 is the comparison of the results of the four algorithms after weight allocation and threshold segmentation. Figure 11 (a) is the original infrared image shown in the experiment of the embodiment of the application after different single-frame type algorithms are subjected to weight allocation and threshold segmentation. Figure 19 is the comparison of the results of the four algorithms after weight allocation and threshold segmentation. Figure 11 (b) is the original infrared image shown in the experiment of the embodiment of the application after different single-frame type algorithms are subjected to weight allocation and threshold segmentation. Figure 20 is the comparison of the results of the four algorithms after weight allocation and threshold segmentation. Figure 11 (c) is the original infrared image shown in the experiment of the embodiment of the application after different single-frame type algorithms are subjected to weight allocation and threshold segmentation; wherein, (a), (b), (c), (d) are WSLCM, IWELCM, MRDLCM_RLD, and HVC respectively; HVC is the algorithm of the application.

[0209] The weight function RIL in WSLCM is characterized by the difference between the mean value of the first few maximum pixel values in the region and the mean value of the pixels in the entire region, and once the clutter with gray mutation occurs, the RIL value will be very large, as shown in Figure 19 (a) and Figure 20 (a) can be seen that the interference points in the blue box correspond to the clutter with gray mutation in the scenes Figure 11 (b) (c). In Figure 11(a) Scene 1, the influence of the road edge around the small target, leading to a smaller difference in the surrounding gray scale, a smaller RIL value is given, and after processing, it is directly lost, which is consistent with the analysis above. The weight function BGC of IWELCM is to use the ratio of the mean of the first few maximum pixel values in the region to the mean of the pixels in the entire region to represent. The weighting coefficient obtained by this ratio method is often small. If the gray scale change of the target in the scene is not particularly dramatic, or the size is too small so that a few pixels have little effect on the mean, BGC cannot get a larger value. In Figure 11 (b) Scene 2, the size of the small target is too small, and a few pixel values have little effect on the mean when calculating the BGC, so the weighting coefficient assigned at this time is small, and in Figure 19 (b) the target is lost. As Figure 18 (b) and Figure 20 (b) can be seen, the interference points in the original scene correspond to clutter interference with dramatic changes in gray scale, and are all given larger weight values and are retained, which is consistent with the analysis above. The weight function RLD of MRDLCM_RLD uses the difference between the maximum and minimum pixel values in the window to represent the gray scale diversity in the window, but this method depends on the size information of the target. In Figure 11 (b) Scene 2, the real target size is very small, in principle, a larger weighting coefficient should be assigned, but according to Figure 19 (c) can be seen, the target has been lost, because the target has been lost before the weighting coefficient is assigned. As Figure 16 (c) can be seen, because Figure 11 (a) Scene 1, there is a small size interference point, so it is also given a larger coefficient, and in Figure 18 (c) is retained, Figure 20 (c) the target is extracted, without interference information, which is consistent with the analysis above. The present application uses an exponential function that is sensitive to gray scale changes in an increasing state to construct the weight function, and combines the gray scale difference with the local contrast to calculate the weighting coefficient, thereby filtering out clutter and enhancing the target, as Figure 18 (d), 19(d), 20(d) can detect the target, but there is still an interference point in 18(d), which cannot be removed even after processing the interference classification in the image, so it also reflects the limitations of single-frame algorithms, which cannot better distinguish real targets from clutter interference in specific scenes.

[0210] (Five) for S4:

[0211] The recent inter-frame association result is shown in Figure 21 , the red box is the target, and the blue box is the interference information. Figure 21In the diagram, (a), (b), and (c) represent images 1, 2, and 3, respectively, which are three consecutive frames of raw infrared images input by S4; (d), (e), and (f) represent the single-frame results of images 1, 2, and 3, from which the location of the suspected target point can be obtained; (g), (h), and (i) represent the association results of images 1, 2, and 3, namely the association results of the first and second frames of raw infrared images, the association results of the second and third frames of raw infrared images, and the association results of the first and third frames of raw infrared images.

[0212] As can be seen from the original infrared image, the target movement distance between adjacent frames is particularly small. It is this tiny change that leads to the limitations of the single-frame processing algorithm, resulting in different processing results for each frame. After image 2 is associated with the nearest neighbor of the previous and next frames, it can effectively filter out the interference information in the scene. The same is true for image 3.

[0213] The following is a quantitative analysis of existing traditional algorithms and the present invention.

[0214] LSNR out LSNR represents the target local signal-to-noise ratio after algorithm processing. in LSNR is the target local signal-to-noise ratio before algorithm processing. out With LSNR in Compared to the local signal-to-noise ratio gain (ILSNR), the background suppression factor (BSF) is obtained by comparing the local information entropy after algorithm processing with that before algorithm processing. A larger ILSNR indicates a better target enhancement effect, while a smaller BSF indicates a better background suppression effect. The results are shown in Tables 1 to 3 below. All four algorithms are single-frame algorithms.

[0215] Table 1. Targeting Figure 11 (a) Local signal-to-noise ratio gain and background suppression factor after processing with different algorithms

[0216] Algorithm LSNR in ]] LSNR out ]] ILSNR BSF WSLCM 2.9549 6.5102 2.2032 0.0775 IWELCM 2.9549 6.0959 2.0630 0.1977 MRDLCM_RLD 2.9549 7.0940 2.4008 0.2011 HVC 2.9549 6.8766 2.3272 0.0802

[0217] Table 2. Targeting Figure 11 (b) Local signal-to-noise ratio gain and background suppression factor after processing with different algorithms

[0218] Algorithm LSNR in ]]> LSNR out ]] ILSNR BSF WSLCM 4.0129 6.1228 1.5258 0.3792 IWELCM 4.0129 6.3561 1.5839 0.3304 MRDLCM_RLD 4.0129 6.1299 1.5275 0.5699 HVC 4.0129 6.8064 1.6961 0.3594

[0219] Table 3. Targeting Figure 11 (c) Local signal-to-noise ratio gain and background suppression factor after processing with different algorithms

[0220]

[0221]

[0222] As shown in Tables 1-3, the SNR gain of the HVC single-frame algorithm is the maximum in Tables 2 and 3, and although it is not the maximum in Table 1, the difference between the maximum and the HVC SNR gain is only 0.0736. Therefore, the HVC algorithm has obvious advantages in terms of SNR gain. In terms of background suppression factor, the BSF of the WSLCM is always at a low level, but the SNR gain of the WSLCM is also at a low level, and even the lowest in Tables 2 and 3. The BSF of the IWELCM algorithm is the minimum in Table 2, and the corresponding SNR gain is the second largest in this table. However, in other tables, these two data do not have advantages. The BSF of the MRDLCM_RLD algorithm is always at a large value, and it does not have an advantage in terms of background suppression factor. The SNR gain is the maximum in Table 1. Although the HVC algorithm has the minimum BSF in Table 3, the BSF of the HVC algorithm is at a small level in Tables 1 and 2, and is very close to the minimum. In Tables 2 and 3, the SNR gain of the HVC algorithm is the maximum, and in Table 1, it is the second largest. Through analysis, it can be known that the HVC algorithm has good objective parameters in terms of target reservation and background suppression.

[0223] The original image sequence has a total of 2900 frames, and the number of real targets originally present is 2900. After processing the image sequence by different algorithms, the number of real targets, false targets and missed targets after processing is calculated, and then the detection probability and false alarm rate are calculated. The results are shown in Table 2 below. The four algorithms all include nearest neighbor correlation processing.

[0224] Table 4. Detection probability and false alarm rate after processing by different algorithms

[0225]

[0226] As shown in Table 4, after the nearest neighbor correlation of the four algorithms, although the false alarm rate of the WSLCM algorithm is 1.81%, the detection probability is less than 90%, which indicates that the WSLCM algorithm tends to suppress background interference. The detection probability of the HVC algorithm is 93.76%, and the false alarm rate is only 1.09%. Although the detection probabilities of the IWELCM and MRDLCM_RLD algorithms are close to 90%, the false alarm rates are more than 3, which indicates that too many interferences are left while extracting targets. The detection probability of the HVC algorithm is 93.76%, and the false alarm rate is only 0.0109, which is obviously better than the other three algorithms.

[0227] In summary, the method of the embodiment of the present application has better weak and small target detection performance on multiple sequences than other several comparative algorithms. For infrared images with relatively strong signal-to-noise ratio, good results can be obtained after processing by most algorithms, and the target can be detected and extracted to the greatest extent in single-frame algorithm processing. Due to the limitation of single-frame algorithm, individual sequence images are also affected by interference objects in addition to the target, and the target-like interference objects cannot be well filtered out. In the multi-frame algorithm, the nearest neighbor correlation is used to solve the problem. The method is relatively simple to implement, can ensure effective target extraction and false target elimination, and provides guarantee for detection probability.

[0228] In a second aspect, corresponding to the method embodiment, the embodiment of the present application also provides an infrared weak and small target detection device based on visual characteristics, as shown in the figure, which comprises: Figure 22

[0229] An improved adaptive side window enhancement filtering processing module is used to suppress the background of a frame of original infrared image by using an improved adaptive side window enhancement filtering method to obtain a corresponding first processed image.

[0230] A diagonal local difference product-based method and a local difference index weight distribution-based processing module are used to remove small edges, clutter interference and realize target enhancement by using the diagonal local difference product-based method and the local difference index weight distribution-based method on the first processed image to obtain a corresponding second processed image.

[0231] A threshold segmentation module is used to obtain the position of the suspected target point in the second processed image by using a threshold segmentation method.

[0232] A nearest neighbor multi-frame correlation processing module is used to determine the position of the real target point by using a nearest neighbor multi-frame correlation method based on the position of the suspected target point obtained from continuous multiple frames of original infrared images.

[0233] For the specific processing process of each module of the device, please refer to the related content of the first aspect, which will not be repeated here.

[0234] The above only describes the preferred embodiments of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application is included in the protection scope of the present application.​

Claims

1. A method for detecting weak infrared targets based on visual characteristics, characterized in that, The method includes: An improved adaptive side-window enhancement filtering method is used to suppress the background of a frame of original infrared image to obtain the corresponding first processed image; The first processed image is processed by using a diagonal local difference product method and a local difference exponential weighting method to remove fine edges and clutter interference and achieve target enhancement, resulting in the corresponding second processed image. The location of the suspected target point in the second processed image is obtained using a threshold segmentation method; Based on the location of suspected target points obtained from multiple consecutive frames of raw infrared images, the location of the real target points is determined using the nearest neighbor multi-frame association method. The improved adaptive side-window enhancement filtering method pre-sets eight modified window templates, which are obtained by adding outer pixels to the traditional side-window filtering window templates; the eight modified window templates correspond to the top, bottom, left, right, upper left, upper right, lower left, and lower right directions, respectively; the size of the eight modified window templates included in the calculation is 5. 3, 5 3, 3 5, 3 5, 3 3, 3 3, 3 3, 3 3; For each correction window template, the center position is the center pixel, the effective filtering pixels of the traditional side window filtering window template that are adjacent to the center pixel and match the direction of the correction window template are the inner layer pixels, and the multiple pixels outside the inner layer pixels that match the direction of the correction window template are the outer layer pixels. The step of performing background suppression on a frame of original infrared image using an improved adaptive side-window enhancement filter to obtain the corresponding first processed image includes: For each pixel to be filtered in the original infrared image frame, align the center pixel of the current correction window template with the pixel to be filtered. For each inner pixel position of the current correction window template, the pixel enhancement result of the inner pixel position is calculated using the pixel value corresponding to the inner pixel position and the average value of multiple pixel values ​​adjacent to the inner pixel position in the original infrared image of the frame. For the center pixel position of the current correction window template, the pixel enhancement result of the center pixel position is calculated by using the pixel value corresponding to the center pixel position and the average value of all pixel values ​​corresponding to the inner pixel position in the original infrared image of the frame. By performing Gaussian filtering weighted summation on the pixel enhancement results at the center pixel position and the pixel enhancement results at all inner pixel positions, and then performing a difference operation on the enhancement results of the center pixel, the processing result of the pixel to be filtered is obtained under the current correction window template. From the processing results of the pixel to be filtered under all the correction window templates, select the processing result closest to 0 and use it as the final processing result to replace the pixel to be filtered. After sequentially traversing and replacing all pixels to be filtered in the original infrared image frame, the corresponding first processed image is obtained; The method based on diagonal local difference product has a nested window of 15. 15 pixels in size, of which 3 3 pixels is used as a child window, forming a total of 5 There are 5 sub-windows. The center sub-window is the center layer and is defined as the target area. The 8 sub-windows adjacent to the center layer are the middle layer and are defined as the middle layer background area. The 16 sub-windows outside the middle layer are the outermost layer and are defined as the outermost layer background area. The first processed image is processed using a diagonal local difference product method and a local difference exponential weighting method to remove fine edges and clutter interference and enhance the target, resulting in a corresponding second processed image, including: For each pixel to be processed in the first processed image, align the pixel to be processed with the center pixel of the center layer sub-window of the nested window; The local contrast of the difference between intermediate layers is calculated using a preset formula for calculating the local contrast of the difference between intermediate layers; and the local contrast of the difference between the difference between outermost layers is calculated using a preset formula for calculating the local contrast of the difference between outermost layers. Calculate the product of the difference in local contrast of each group of intermediate layer sub-windows belonging to the diagonal, and determine the minimum value among the products obtained by all groups of intermediate layer sub-windows belonging to the diagonal as the intermediate layer product metric; and calculate the product of the difference in local contrast of each group of outermost layer sub-windows belonging to the diagonal, and determine the minimum value among the products obtained by all groups of outermost layer sub-windows belonging to the diagonal as the outer layer product metric. The larger value between the intermediate layer product metric and the outer layer product metric is selected as the diagonal local difference product result; The weight coefficients corresponding to the diagonal local difference product result are determined using the local difference index weight allocation method; the product of the diagonal local difference product result and the corresponding weight coefficient is used as the update pixel value to replace the pixel to be processed. After all the pixels to be processed in the first processed image are traversed and replaced in sequence, the corresponding second processed image is obtained.

2. The infrared weak target detection method based on visual characteristics according to claim 1, characterized in that, The preset formula for calculating the local contrast of the intermediate layer difference is: ； The preset formula for calculating the local contrast of the outermost layer difference is: ； in, Indicates the first Local contrast of the difference between intermediate layer sub-windows ; This represents the average pixel value of the central layer sub-window; Indicates the first A middle-layer sub-window; Indicates the first The average pixel value of each intermediate layer sub-window; Indicates the first The difference in local contrast between the outermost child windows. ; Indicates the first The outermost child window; Indicates the first The average pixel value of the outermost sub-window.

3. The infrared weak target detection method based on visual characteristics according to claim 2, characterized in that, The formula used to determine the weight coefficients corresponding to the diagonal local difference product result using the local difference index-based weight allocation method is as follows: ， ； ； in, This indicates that when using a nested window of weighting functions provided by the local difference index weighting method, the position of the nested window of the weighting function corresponds to the position of the first element in the original infrared image. 1 pixel; This represents the average value of 9 pixels within the nested window of the weight function; This represents the maximum value of 9 pixels within the nested window of the weight function; This represents the weight coefficients obtained from the exponential-based weighting function.

4. The infrared weak target detection method based on visual characteristics according to claim 1, characterized in that, The threshold segmentation method includes a threshold segmentation method that combines the mean and standard deviation.

5. An infrared weak target detection device based on visual characteristics, characterized in that, include: An improved adaptive side-window enhancement filtering module is used to suppress the background of a frame of original infrared image using an improved adaptive side-window enhancement filtering method to obtain the corresponding first processed image. The processing module based on the diagonal local difference product method and the local difference exponential weight allocation method is used to remove fine edges and clutter interference and enhance the target in the first processed image by applying the diagonal local difference product method and the local difference exponential weight allocation method to obtain the corresponding second processed image. The threshold segmentation module is used to obtain the location of the suspected target point in the second processed image using a threshold segmentation method. The nearest neighbor multi-frame association processing module is used to determine the location of the real target point based on the location of the suspected target point obtained from multiple consecutive frames of raw infrared images, using the nearest neighbor multi-frame association method. The improved adaptive side-window enhancement filtering method pre-sets eight modified window templates, which are obtained by adding outer pixels to the traditional side-window filtering window templates; the eight modified window templates correspond to the top, bottom, left, right, upper left, upper right, lower left, and lower right directions, respectively; the size of the eight modified window templates included in the calculation is 5. 3, 5 3, 3 5, 3 5, 3 3, 3 3, 3 3, 3 3; For each correction window template, the center position is the center pixel, the effective filtering pixels of the traditional side window filtering window template that are adjacent to the center pixel and match the direction of the correction window template are the inner layer pixels, and the multiple pixels outside the inner layer pixels that match the direction of the correction window template are the outer layer pixels. The improved adaptive side-window enhancement filtering module is specifically used for: For each pixel to be filtered in the original infrared image frame, align the center pixel of the current correction window template with the pixel to be filtered. For each inner pixel position of the current correction window template, the pixel enhancement result of the inner pixel position is calculated using the pixel value corresponding to the inner pixel position and the average value of multiple pixel values ​​adjacent to the inner pixel position in the original infrared image of the frame. For the center pixel position of the current correction window template, the pixel enhancement result of the center pixel position is calculated by using the pixel value corresponding to the center pixel position and the average value of all pixel values ​​corresponding to the inner pixel position in the original infrared image of the frame. By performing Gaussian filtering weighted summation on the pixel enhancement results at the center pixel position and the pixel enhancement results at all inner pixel positions, and then performing a difference operation on the enhancement results of the center pixel, the processing result of the pixel to be filtered is obtained under the current correction window template. From the processing results of the pixel to be filtered under all the correction window templates, select the processing result closest to 0 and use it as the final processing result to replace the pixel to be filtered. After sequentially traversing and replacing all pixels to be filtered in the original infrared image frame, the corresponding first processed image is obtained; The method based on diagonal local difference product has a nested window of 15. 15 pixels in size, of which 3 3 pixels is used as a child window, forming a total of 5 There are 5 sub-windows. The center sub-window is the center layer and is defined as the target area. The 8 sub-windows adjacent to the center layer are the middle layer and are defined as the middle layer background area. The 16 sub-windows outside the middle layer are the outermost layer and are defined as the outermost layer background area. The module for processing based on the diagonal local difference product method and the local difference exponential weight allocation is specifically used for: For each pixel to be processed in the first processed image, align the pixel to be processed with the center pixel of the center layer sub-window of the nested window; The local contrast of the difference between intermediate layers is calculated using a preset formula for calculating the local contrast of the difference between intermediate layers; and the local contrast of the difference between the difference between outermost layers is calculated using a preset formula for calculating the local contrast of the difference between outermost layers. Calculate the product of the difference in local contrast of each group of intermediate layer sub-windows belonging to the diagonal, and determine the minimum value among the products obtained by all groups of intermediate layer sub-windows belonging to the diagonal as the intermediate layer product metric; and calculate the product of the difference in local contrast of each group of outermost layer sub-windows belonging to the diagonal, and determine the minimum value among the products obtained by all groups of outermost layer sub-windows belonging to the diagonal as the outer layer product metric. The larger value between the intermediate layer product metric and the outer layer product metric is selected as the diagonal local difference product result; The weight coefficients corresponding to the diagonal local difference product result are determined using the local difference index weight allocation method; the product of the diagonal local difference product result and the corresponding weight coefficient is used as the update pixel value to replace the pixel to be processed. After all the pixels to be processed in the first processed image are traversed and replaced in sequence, the corresponding second processed image is obtained.

Images