Method and device for detecting aerial infrared moving target combined with space-time information

Abstract

The embodiment of the present disclosure discloses an aerial infrared moving target detection method and device combined with space-time information, which comprises: performing prior weight filtering pretreatment on an original infrared sequence image to obtain a pretreated sequence image; the original infrared sequence image is an infrared image collected for an aerial moving target; a local variance feature map of the sequence image is established based on variance information of a mean value of a target region and a mean value of a background region in the pretreated sequence image, and a target potential region is obtained; a target feature map is obtained by performing space domain weighting and time domain weighting on the target potential region; and a moving target is detected based on the target feature map. The technical scheme can improve the accuracy of target detection.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of image technology, in particular to an aerial infrared moving target detection method and device combining space-time information. BACKGROUND

[0002] Infrared search and tracking systems have a wide range of applications in remote sensing and aerospace fields, and are mostly used for detecting and tracking targets with high radiation characteristics. In the infrared search and tracking system, detecting an infrared weak moving target with unknown position and speed under low signal-to-noise ratio is a key link. However, due to the relatively long distance of infrared imaging, poor performance of infrared detectors, and other reasons, the target image is mostly a small point target, lacking obvious texture and geometric structure information, and the image is easily submerged by a complex background also having high radiation characteristics. Moreover, the observation results of the infrared search and tracking system are usually a series of multi-frame sequence images, that is, the target and the background have the possibility of simultaneous motion. These reasons all make it very difficult to detect the infrared weak moving target. Therefore, the detection of the aerial infrared weak moving target is a very challenging task.

[0003] Generally, the detection methods for targets in different situations mainly include the following three categories: (1) a method based on the background consistency assumption; (2) a method based on the extraction of saliency features of the human visual system (HVS); and (3) a method based on sparse and low-rank matrix recovery. Among them, the method based on the background consistency assumption has the shortest detection time, but has weak generalization ability for scenes. The method based on low-rank sparsity is a method of decomposing an infrared image into a low-rank background component and a sparse target component under the assumption of background low-rank and target sparsity, but this method needs to set many hyperparameters and has weak adaptability to scenes. On the other hand, when the time-domain information of multiple frames is used, the time cost of the detection method is high. The method based on HVS is inspired by the human eye visual system, which calculates the contrast of each pixel using a sliding window to obtain a local contrast saliency map of the entire image, and obtains a target image through certain calculation indicators. Due to the short running time, simple calculation principle, and relatively strong detection ability, the method based on HVS has been widely studied. SUMMARY

[0004] The present disclosure provides an aerial infrared moving target detection method and device combining space-time information.

[0005] In a first aspect, an aerial infrared moving target detection method combining space-time information is provided in the embodiments of the present disclosure, which comprises:

[0006] performing prior weight filtering preprocessing on the original infrared sequence image to obtain a preprocessed sequence image; the original infrared sequence image is an infrared image collected for an aerial moving target;

[0007] establishing a local variance feature map of the sequence image based on variance information of a mean value of a target region and a mean value of a background region in the preprocessed sequence image to obtain a target potential region;

[0008] performing spatial weighting and temporal weighting on the target potential region to obtain a target feature map;

[0009] detecting a moving target based on the target feature map.

[0010] Further, the prior weight filtering preprocessing on the original infrared sequence image to obtain a preprocessed sequence image comprises:

[0011] performing Gaussian smoothing filtering on the original infrared sequence image to remove low-frequency components;

[0012] performing filtering again on the sequence image from which the low-frequency components are removed by using the prior weight to enhance target information in the sequence image.

[0013] Further, the establishing a local variance feature map of the sequence image based on variance information of a mean value of a target region and a mean value of a background region in the preprocessed sequence image to obtain a target potential region comprises:

[0014] obtaining an initial potential region by using a first result obtained by subtracting a mean value of a largest inner neighborhood region in the background region from a mean value of the target region and a second result obtained by subtracting a mean value of a largest outer neighborhood region in the background region from the mean value of the target region;

[0015] calculating the target potential region based on the initial potential region and a local feature variance map; the local feature variance map is a feature map obtained based on variances of a plurality of local regions after the preprocessed sequence image is divided into the plurality of local regions.

[0016] Further, the performing spatial weighting and temporal weighting on the target potential region to obtain a target feature map comprises:

[0017] calculating local spatial weights for local regions in an inner neighborhood region and an outer neighborhood region of the target region and the background region, respectively;

[0018] determining a target spatial weight as a larger one of a first difference between the local spatial weight corresponding to the target region and a mean value of local spatial weights of the local regions in the inner neighborhood region and a second difference between the local spatial weight corresponding to the target region and a mean value of local spatial weights of the local regions in the outer neighborhood region;

[0019] spatially weight the target potential region using the target spatial weight.

[0020] Further, the local spatial weight is calculated for the local region in the inner neighborhood region and the outer neighborhood region of the target region and the background region respectively, including:

[0021] The first spatial weight is obtained by subtracting the minimum value between the mean value and the median value of the pixel gray scale in the local region of the current region from the mean value of the first k maximum pixel gray scales in the local region of the current region; the current region is the target region, the inner neighborhood region or the outer neighborhood region;

[0022] The second spatial weight is obtained by subtracting the mean value of the last k minimum pixel gray scales in the local region of the current region from the mean value of the first k maximum pixel gray scales in the local region of the current region;

[0023] The local spatial weight of the current region is obtained by multiplying the first spatial weight and the second spatial weight.

[0024] Further, the target potential region is spatially weighted and temporally weighted to obtain a target feature map, including:

[0025] The kurtosis feature of each pixel in the original infrared sequence image in the time domain direction is calculated;

[0026] The target temporal weight corresponding to different positions in the original infrared sequence image is calculated based on the kurtosis feature;

[0027] The target potential region is temporally weighted using the target temporal weight.

[0028] In a second aspect, the present disclosure provides an aerial infrared moving target detection device combining spatial and temporal information, including:

[0029] A preprocessing module is configured to perform prior weight filtering preprocessing on an original infrared sequence image to obtain a preprocessed sequence image; the original infrared sequence image is an infrared image collected for an aerial moving target;

[0030] A potential region extraction module is configured to establish a local variance feature map of the sequence image based on variance information of the mean value of the target region and the mean value of the background region in the preprocessed sequence image, and obtain a target potential region;

[0031] A target feature map determination module is configured to spatially weight and temporally weight the target potential region to obtain a target feature map;

[0032] A target detection module is configured to detect a moving target based on the target feature map.

[0033] The functions can be implemented by hardware, or by corresponding software executed by hardware. The hardware or software includes one or more modules corresponding to the functions described above.

[0034] In one possible design, the apparatus includes a memory and a processor, where the memory is configured to store one or more computer instructions that support the apparatus to perform the corresponding method described above, and the processor is configured to execute the computer instructions stored in the memory. The apparatus can also include a communication interface configured to enable the apparatus to communicate with other devices or communication networks.

[0035] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory, where the processor executes the computer program to implement the method of any of the aspects described above.

[0036] In a fourth aspect, a computer-readable storage medium is provided, which is configured to store computer instructions for the apparatus of any of the aspects described above, where the computer instructions are executed by a processor to implement the method of any of the aspects described above.

[0037] In a fifth aspect, a computer program product is provided, which includes computer instructions, where the computer instructions are executed by a processor to implement the method of any of the aspects described above.

[0038] The technical solutions provided by the embodiments of the present disclosure can include the following beneficial effects:

[0039] The present disclosure provides an aerial infrared dim and small moving target detection method combined with space-time information, aiming to solve the problem of aerial infrared dim and small target detection in the case of complex background and target movement. Based on the Gaussian smoothing filter to remove low-frequency components, the present disclosure extracts the prior weight information of the original image to filter the image again to enhance the target. Then, the difference between the target mean and the background mean is calculated, and the local contrast feature map of the image is established by combining the variance information to obtain the target potential area. On the other hand, the present disclosure improves the calculation method of RIL to weight the target potential area in the spatial domain, and improves the contrast between the target and the background in the spatial domain. At the same time, the present disclosure calculates the kurtosis feature of the image time domain signal to mine the potential area of the target from the time domain information. Finally, through the combination of spatial and temporal features, the background is suppressed and the target is enhanced, thereby realizing the target detection task. It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0040] Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description when read in conjunction with the accompanying drawings. In the drawings:

[0041] Figure 1 A schematic diagram showing a three-layer window partition effect of an image in the prior art;

[0042] Figure 2 A schematic diagram showing a process of obtaining a target image with suppressed background by weighting a potential position of the target using a region intensity map in the prior art;

[0043] Figure 3 A flowchart showing a method of detecting an aerial infrared moving target in combination with space-time information according to an embodiment of the present disclosure;

[0044] Figure 4 A schematic diagram showing a structure of a Gaussian filter according to an embodiment of the present disclosure;

[0045] Figures 5(a)-5(d) A schematic diagram showing an effect of a local region at different moments when a target is in different regions according to an embodiment of the present disclosure;

[0046] Figures 6(a)-6(b) A schematic diagram showing an image and a time-domain intensity curve of a target pixel in the image according to an embodiment of the present disclosure;

[0047] Figure 7 A flowchart showing a method of detecting an aerial infrared moving target in combination with space-time information according to an embodiment of the present disclosure;

[0048] Figures 8(a)-8(c) A comparative schematic diagram of feature maps according to an embodiment of the present disclosure;

[0049] Figures 9(a)-9(c) A comparative schematic diagram of target detection results according to an embodiment of the present disclosure and the prior art;

[0050] Figure 10 A schematic diagram showing a structure of an electronic device suitable for implementing a method of detecting an aerial infrared moving target in combination with space-time information according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0051] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so as to be easily implemented by those skilled in the art. Also, portions irrelevant to the description of the exemplary embodiments are omitted in the accompanying drawings for the sake of clarity.

[0052] In the present disclosure, it should be understood that terms such as “include” or “have” are intended to indicate that there are features, numbers, steps, actions, components, parts or combinations thereof disclosed in the specification, and do not exclude the possibility that one or more other features, numbers, steps, actions, components, parts or combinations thereof exist or are added.

[0053] In addition, it needs to be further explained that the embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0054] The Weighted Three-Layer Window Local Contrast (WTLLCM) method proposed in the prior art has great improvement in both accuracy and robustness. The method improves the two-layer window filter to a three-layer window, and adds an isolation window in the inner neighborhood of the background region between the target region (Target) and the outer neighborhood (background window) of the background region, i.e. the inner neighborhood region, as shown in the figure. The main process of the method is to first smooth the image by a Gaussian filter kernel to remove the low-frequency components belonging to the background in the image. Then, the local contrast feature map is obtained by calculating the difference between the target region mean value and the neighborhood mean value of the filtered image using the weighted three-layer sliding window, and the possible target location is obtained from the local contrast feature map. At the same time, the intensity of the target potential position is calculated by calculating the region intensity (Region Intensity Level, RIL). The target image with increased target and suppressed background is obtained by weighting the target potential position by the region intensity map, and the flow of the method is shown in the figure. Figure 1 Figure 2

[0055] However, the above-mentioned prior art only uses the spatial domain information of a single frame image, and most of the detection problems are relatively simple scenes with relatively high contrast between the target and the background. However, when the imaging background is complex and non-static, and the contrast of the target is low, the detection result still needs to be improved.

[0056] However, the prior art has the following defects:

[0057] (1) The prior art performs Gaussian filtering on the image to filter out low-frequency components to some extent, removing the background in the image, but the enhancement effect on the target is not great, and in a complex scene, the high-frequency components of the background are difficult to remove and the intensity of the target may be weakened.

[0058] (2) The prior art extracts the local contrast feature map of the image by calculating the difference between the target region mean value and the background region maximum value, which enhances the target and suppresses the background, but some noise with strong radiation is also highlighted.

[0059] ​​(3) The prior art is oriented to a single frame image, only uses spatial domain information, does not use time domain information in a sequence image, ignores the motion characteristics of a target, and when spatial domain features are difficult to distinguish a background from a target, the detection effect still needs to be improved.

[0060] (4) The calculation method of RIL proposed by the prior art can only have a better effect when a background is simple and a target is relatively high in intensity, and when the background becomes complex and the target is very weak, the effect still needs to be improved. On the other hand, because the size of the target is not fixed, the calculation method of a fixed filter window size cannot meet the diversity of the target in practical application, and it is difficult to completely retain the target.

[0061] The present disclosure reweights a local contrast graph by calculating a local variance through three windows and improves the calculation method of region intensity, and simultaneously combines time domain information of a sequence image, and proposes a new infrared weak moving target detection method in the air combining spatial domain and time domain features.

[0062] The present disclosure proposes an infrared weak moving target detection method in the air combining spatial and time information, and aims to solve the problem of infrared weak target detection in the air in the case of a complex background and target motion. The present disclosure filters out low-frequency components on the basis of Gaussian smoothing, filters the original image again to enhance the target by extracting prior weight information of the original image, then calculates the difference between the target mean value and the background mean value, and simultaneously combines variance information to establish a local contrast feature graph of the image to obtain a target potential region. On the other hand, the present disclosure improves the calculation method of RIL to weight the target potential region in the spatial domain, and improves the contrast between the target and the background in the spatial domain. Meanwhile, the present disclosure calculates the kurtosis feature of the time domain signal of the image to mine the potential region of the target from the time domain information. Finally, through the combination of spatial and time domain features, the background is suppressed and the target is enhanced, thereby realizing the target detection task.

[0063] The details of the embodiments of the present disclosure will be described in detail below through specific embodiments.

[0064] Figure 3 A flowchart of an infrared moving target detection method in the air combining spatial and time information according to an embodiment of the present disclosure is shown. As shown in the figure, the infrared moving target detection method in the air combining spatial and time information includes the following steps: Figure 3

[0065] In step S301, the original infrared sequence image is subjected to prior weight filtering pretreatment to obtain a pretreated sequence image; the original infrared sequence image is an infrared image collected for an aerial moving target;

[0066] ​In step S302, a local variance feature map of the sequence image is established based on variance information of a mean value of a target region and a mean value of a background region in the preprocessed sequence image, and a target potential region is obtained;

[0067] In step S303, spatial weighting and time domain weighting are performed on the target potential region, and a target feature map is obtained.

[0068] In step S304, a moving target is detected based on the target feature map.

[0069] In some embodiments, the step S301, that is, the step of performing prior weight filtering preprocessing on the original infrared sequence image to obtain a preprocessed sequence image, comprises:

[0070] Gaussian smoothing is performed on the original infrared sequence image to filter out low-frequency components.

[0071] The sequence image from which the low-frequency components are filtered out is filtered again using a prior weight to enhance target information in the sequence image.

[0072] In some embodiments, the step S302, that is, the step of establishing a local variance feature map of the sequence image based on variance information of a mean value of a target region and a mean value of a background region in the preprocessed sequence image to obtain a target potential region, comprises:

[0073] An initial potential region is obtained by subtracting a first result obtained by subtracting a mean value of a largest inner neighborhood region in the background region from a mean value of the target region and a second result obtained by subtracting a mean value of a largest outer neighborhood region in the background region from the mean value of the target region.

[0074] A target potential region is calculated based on the initial potential region and a local feature variance map; the local feature variance map is a feature map obtained based on variances of a plurality of local regions after the preprocessed image is divided into the plurality of local regions.

[0075] In some embodiments, the step S303, that is, the step of performing spatial weighting and time domain weighting on the target potential region to obtain a target feature map, comprises:

[0076] Local spatial weights are calculated for local regions in an inner neighborhood region and an outer neighborhood region of the target region and the background region, respectively.

[0077] A larger value of a first difference value between the local spatial weight corresponding to the target region and a local spatial weight mean value of the local regions in the inner neighborhood region and a second difference value between the local spatial weight corresponding to the target region and a local spatial weight mean value of the local regions in the outer neighborhood region is determined as a target spatial weight.

[0078] The target potential region is spatially weighted using the target spatial weight.

[0079] In some embodiments, local spatial weights are calculated for local regions within the inner and outer neighborhood regions of the target region and the background region, respectively, including:

[0080] The first spatial weight is obtained by subtracting the minimum value between the mean and median of pixel gray levels in the local regions of the current region from the mean of the k largest pixel gray levels in the local regions of the current region; the current region can be the target region, the inner neighborhood region, or the outer neighborhood region.

[0081] The second spatial weight is obtained by subtracting the mean of the last k smallest pixel gray levels in the local region of the current region from the mean of the mean of the first k largest pixel gray levels in the local region of the current region.

[0082] The local spatial weight of the current region is obtained by multiplying the first spatial weight and the second spatial weight.

[0083] In some embodiments, step S303, namely the step of performing spatial and temporal weighting on the potential target region to obtain a target feature map, includes:

[0084] Calculate the kurtosis feature of each pixel in the original infrared sequence image in the temporal domain;

[0085] Calculate the target temporal weights corresponding to different positions in the original infrared sequence image based on the kurtosis feature;

[0086] The target potential region is weighted in the time domain using the target time domain weight.

[0087] It should be noted that, in some embodiments, after the target spatial weight and target temporal weight are calculated, the target potential region is spatially weighted and temporally weighted using the target spatial region weight and target temporal weight. That is, the target feature map is obtained by calculating the product of the target potential region, the target spatial weight, and the target temporal weight.

[0088] The following describes the specific details of this embodiment through one implementation method.

[0089] (1) Image preprocessing

[0090] First, each frame of the original infrared sequence image is used... Figure 4 The Gaussian filter shown smooths and enhances the target. Small infrared targets typically have a symmetrical center and diffuse outwards, with almost all local gradients pointing towards their center. Furthermore, the target's grayscale value is generally higher than the nearby background. Therefore, a two-dimensional Gaussian function can be used to simulate small targets. The filtered image I... gauswhich can be expressed as formula (1) shown:

[0091]

[0092] wherein K G represents a Gaussian filter, I is an original image, (x, y) represents a pixel coordinate, and (l, k) represents a sliding window;

[0093] In addition, the structure tensor of each pixel position in the image contains two eigenvalues λ1, λ2.

[0094] The intensity indicator C of the corner point in the image can be expressed as:

[0095]

[0096] Meanwhile, the edge intensity indicator is expressed by E = λ1- λ2, and thus the local prior weight filter of the present disclosure is established as:

[0097]

[0098] wherein m1, m2 are different weights corresponding to the corner point information and the edge information. When m2 is set to 1 and m1 is set to be greater than 1, the corner point information is enhanced. When m1 is set to 1 and m2 is set to be greater than 1, the edge information is enhanced. In the present disclosure, m1 is set to 1.8 and m2 is set to 1. m_1 and m_2 are set based on experience, and m1>1, m2=1 is sufficient. Thus, the result I tidu is expressed as formula (4) shown:

[0099] I tidu = H⊙I gaus (4)

[0100] wherein ⊙ is a Hadamard product.

[0101] The prior weight filtering technology used in this step can effectively enhance the corner point pixels in the image, i.e. the pixels that may be the target, while suppressing the interference in the background such as strong edges. Compared with the technology of simply using Gaussian filtering for preprocessing, this step further improves the contrast between the target and the background.

[0102] (2) Target potential region extraction

[0103] The moving target in the infrared image generally has a higher radiation characteristic, so that its gray value is generally higher than that of the surrounding background neighborhood. Therefore, when a target exists in a certain region, the variance of the region will be larger than that of a uniform background region without a target. On the other hand, three windows are extracted from the preprocessed image, i.e. a target region window, an inner neighborhood region window in the background region, and an outer neighborhood region window, and the target region is calculated with the background region such asFigure 1 the gray difference TIB of the inner neighborhood region IB1-IB8 in the target region, and the target region and the outer neighborhood region OB1-OB16 in the target region as shown in FIG. 2B can effectively enhance the target and suppress the background, so the present disclosure first calculates the local contrast feature map filtered based on the three windows, i.e., the target region window, the outer neighborhood region window and the inner neighborhood region window as shown in FIG. 2C: Figure 1 Figure 1

[0104]

[0105]

[0106] wherein M T , M IBi , M OBi are the mean values of the target region, the i-th inner neighborhood region and the i-th outer neighborhood region, respectively. However, in the face of actual complex detection situations, the size of the target is diverse, so the present disclosure changes the three-window filter to a multi-scale three-window filter, thereby adapting to targets of different sizes. TIB is the local contrast feature map between the target region window and the inner neighborhood region window, and TOB is the local contrast feature map between the target region window and the outer neighborhood region window. TIB and TOB can obtain the initial potential region through the product operation (i.e., the operation).

[0107] Although the multi-scale three-window calculation method can effectively improve the contrast of the target, when facing the corner or edge of the strong radiation region, the local contrast calculated by this method can also be high, so that some interference in the background is also enhanced. Considering the influence of the variance difference between the target region and the background region, therefore, the present disclosure utilizes this difference to calculate the local variance feature map VAR T of the image by combining the multi-scale three-layer window, and obtains the multi-scale three-layer window by assigning different scales to the three-layer window, and obtains the local variance feature map VAR Figure 1 T by calculating the variance of each small local region (the preprocessed image can be divided into multiple local regions according to the need of actual application) in the 25 small local regions as shown in FIG. 2D, so as to further enhance the target region, thereby obtaining the region of interest ROI of the image, i.e., the potential region of the target:

[0108] ROI=TIB⊙TOB⊙VAR T (7)

[0109] (3) Spatial Weighting

[0110] ​​​This disclosure addresses the problem of infrared weak moving target detection in complex scenes by proposing a robust RIL (Robust Region Intensity Level) calculation method. The extracted regions of interest are spatially weighted by intensity. The proposed RRIL calculation method is shown in Equation 8-10.

[0111] ARIL i =M k (i)-min(mean(i),median(i)) (8)

[0112] BRIL i =M k (i)-M kmin (i) (9)

[0113] RIL i =ARIL i *BRIL i (10)

[0114] Among them, M k (i) represents Figure 1 The mean of the top k largest pixels in the i-th local region is given by M. Here, i represents each small local region within the three-layer window. The target window contains only one region: i = 1...8 in the inner neighborhood window and i = 1...16 in the outer neighborhood window. mean(i) represents the mean of the pixels in the i-th local region, median(i) represents the median of the pixels in the i-th local region, and M... kmin (i) represents the mean of the last k smallest pixels in the i-th local region.

[0115] Formulas (8) and (9) simultaneously measure the region's RIL to ensure that the target region has a larger RIL while the background region has a smaller RIL. Formulas (8) and (9) simultaneously calculate the region complexity to ensure that the calculated region complexity in the target region is larger, meaning the target region's RIL is larger than the background region's RIL. Assuming the local region size is 3*3, there are four possible scenarios for the target region:

[0116] In the target area, only the target is bright, and the 3*3 area is dark except for the target location. Therefore, both ARIL and BRIL are large, and the target's RIL must be large, as shown in Figure 5(a). 255 is the target's gray level, the gray level of the non-target dark location is 10, and the gray level of the bright location is 210.

[0117] The target is bright in the target area, and the area outside the target location is mostly dark with few bright areas, as shown in Figure 5(b). Thus, the BRIL is a relatively large value. In ARIL, since the median is much smaller than the mean, the existence of the median will make the ARIL calculated to be a relatively large value, so the RIL of the target is also large.

[0118] The target is bright in the target area, and there are more bright and fewer dark areas in the 3*3 area except for the target location, as shown in Figure 5(c). Although ARIL decreases, BRIL is large, so the RIL of the target is also large.

[0119] In the target area, only the target is lit, and in the 3*3 area, all areas except the target are lit, as shown in Figure 5(d). So, although ARIL and BRIL are relatively smaller, the RIL of non-target areas in the uniformly bright area is smaller than the RIL of the target, so they can also be distinguished.

[0120] Finally, the result can be obtained by calculating the difference between the target and the RIL of the inner and outer neighboring regions.

[0121] The final weighted weight RRIL for the spatial domain is:

[0122] RRIL = max(RILT - mean(RIL) OBi ), RILT-mean(RIL) IBi (11)

[0123] Where RILT is the RIL of the target region, RIL OBi RIL IBi The RILs are calculated for the i-th region of the outer and inner neighborhoods, respectively.

[0124] (4) Time-domain weighting

[0125] In infrared images, the target is generally brighter than the surrounding background. When a small infrared target moves, the position coordinates of the target in a given frame are recorded. In the time-domain signal of that coordinate position formed by the current frame and several frames before and after, the image grayscale changes from dark to bright and then from bright to dark, as shown below. Figures 6(a)-6(b) As shown, the target location is marked with a box. Through the above analysis, it can be seen that the temporal intensity distribution curve formed by the target motion is nearly Gaussian. Therefore, different locations can be weighted by calculating the kurtosis characteristics of the temporal signals formed at different coordinate positions in the infrared sequence image. The kurtosis is calculated as shown in formula (12). When the temporal grayscale curve is closer to a Gaussian distribution, the probability of the target appearing at that location is higher, and a larger weight is assigned. When it deviates further from a Gaussian distribution, the location is more likely to be a background pixel, and a smaller weight is assigned. Therefore, the weight TD in the temporal part... weight The calculation method is shown in formula (13):

[0126]

[0127] TD weight = Kurt(img) (13)

[0128] Kurt(·) represents the calculated kurtosis size, E is the mean, μ is the mean of the pixels in the image, σ is the standard deviation of the pixels in the image, and img is the original infrared image sequence.

[0129] (5) Target detection

[0130] Combining the spatial and temporal information, the final target feature map T is formed as:

[0131] T = ROI ⊙ ERIL ⊙ TD weight (14)

[0132] The final detection result can be obtained through threshold segmentation. The overall process of the technical scheme of the disclosure is shown in Figure 7 . The detection result is shown in Figures 8(a)-8(c) . Among them, Fig. 8(a) is the original infrared image, Fig. 8(b) is the feature map obtained by only calculating the mean difference, and Fig. 8(c) is the feature map obtained by combining the mean difference with the variance information.

[0133] The beneficial effects of the disclosure are:

[0134] (1) The traditional preprocessed image obtained by calculating the gradient through the Gaussian filter can filter out the low-frequency background, but the target is also affected to some extent. Therefore, the prior weight information of the image is extracted in the image preprocessing stage, and the corner point information is enhanced in a weighted manner. The target in the preprocessed image obtained by combining the prior weight information and the smoothing filter information is more accurately highlighted, and the background suppression is stronger.

[0135] (2) In the extraction of the region of interest, when the local contrast feature map is calculated only by the simple mean difference, some irregular noises such as point noise in the background will also be detected, which will cause a large number of false alarms in the subsequent detection. In addition, a single scale three-window filter is difficult to meet the size complexity of the actual target. Therefore, the disclosure calculates the local region variance by using a multi-scale three-layer window, so as to perform weighted processing on the feature image obtained by calculating the mean difference by relying on the multi-scale three-layer window filter, and then obtain the region of interest, thereby solving the problem that the edges of some highlight regions in the background may also be enhanced. Figures 9(a)-9(c) The beneficial effects of the disclosure combining the multi-scale three-layer window and the variance feature are illustrated, wherein Fig. 9(a) is an original image, Fig. 9(b) is the detection result of the prior art, and Fig. 9(c) is the detection result of the disclosure.

[0136] (3) By improving the calculation method of RIL, the disclosure proposes that the RRIL performs spatial part weighting on the extracted ROI, solving the problem that the previous RIL also enhances the strong edge and strong noise in the background.

[0137] (4) By calculating the kurtosis feature of the time domain signal distribution through sequence information, the time domain is weighted, effectively utilizing the motion characteristics of the target, and solving the problem that a single spatial feature may be difficult to distinguish noise from the target.

[0138] The following is an embodiment of the device of the disclosure, which can be used to execute the method embodiment of the disclosure.

[0139] The aerial infrared moving target detection device combining space-time information according to an embodiment of the disclosure can be realized as part or all of an electronic device through software, hardware, or a combination of both. The aerial infrared moving target detection device combining space-time information includes:

[0140] The preprocessing module is configured to perform prior weight filtering preprocessing on the original infrared sequence image to obtain a preprocessed sequence image; the original infrared sequence image is an infrared image collected for an aerial moving target;

[0141] The potential region extraction module is configured to establish a local variance feature map of the sequence image based on variance information of a mean value of a target region and a mean value of a background region in the preprocessed sequence image, and obtain a target potential region;

[0142] The target feature map determination module is configured to perform spatial weighting and time domain weighting on the target potential region to obtain a target feature map;

[0143] The target detection module is configured to detect a moving target based on the target feature map.

[0144] The aerial infrared moving target detection device combining space-time information in the embodiment corresponds to the aerial infrared moving target detection method combining space-time information in the foregoing, and specific details can be referred to the description of the aerial infrared moving target detection method combining space-time information in the foregoing. Here, no further description is given.

[0145] Figure 10 It is a structural schematic diagram of an electronic device suitable for realizing the aerial infrared moving target detection device method combining space-time information according to an embodiment of the disclosure.

[0146] As Figure 10As shown, the electronic device 1000 includes a processing unit 1001, which can be implemented as a CPU, a GPU, a FPGA, a NPU, or the like. The processing unit 1001 can perform various processes in the embodiments of any of the methods of the present disclosure according to a program stored in a read only memory (ROM) 1002 or a program loaded into a random access memory (RAM) 1003 from the storage section 1008. Various programs and data required for the operation of the electronic device 1000 are also stored in the RAM 1003. The processing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0147] Connected to the I / O interface 1005 are an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as necessary. A removable recording medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 1010 as necessary, so that a computer program read therefrom is installed into the storage section 1008 as necessary.

[0148] In particular, according to embodiments of the present disclosure, the above with reference to any of the methods in the embodiments of the present disclosure can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for executing any of the methods in the embodiments of the present disclosure. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 1009, and / or installed from the removable recording medium 1011.

[0149] The flow and block diagrams in the drawings represent possible architectural, functional, and operational architectures of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block can represent a module, a segment, or a portion of code that comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0150] The units or modules described in the embodiments of the present disclosure can be implemented by software, or by hardware. The described units or modules can also be arranged in a processor, and the name of the unit or module does not constitute a limitation on the unit or module itself in some cases.

[0151] As another aspect, the present disclosure also provides a computer readable storage medium, which can be the computer readable storage medium included in the apparatus described in the above embodiments, or can exist separately from the apparatus and not be assembled into the apparatus. The computer readable storage medium stores one or more programs for execution by one or more processors to perform the method described in the present disclosure.

[0152] The above description is merely preferred embodiments of the present disclosure and a description of principles of applied technologies. It should be understood by those skilled in the art that the scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and also includes other technical solutions formed by the combinations of the above technical features or equivalent features without departing from the inventive concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features disclosed in the present disclosure (but not limited to) having similar functions.

Claims

1. A method for detecting aerial infrared moving targets by combining spatiotemporal information, characterized in that, include: The original infrared sequence images are preprocessed by prior weight filtering to obtain preprocessed sequence images; the original infrared sequence images are infrared images acquired for aerial moving targets. Based on the variance information of the mean of the target region and the mean of the background region in the preprocessed sequence image, a local variance feature map of the sequence image is established to obtain the potential target region; Spatial and temporal weighting are applied to the potential target region to obtain a target feature map; The moving target is detected based on the target feature map; Calculate the kurtosis feature of each pixel in the original infrared sequence image in the temporal domain; Calculate the target temporal weights corresponding to different positions in the original infrared sequence image based on the kurtosis feature; Calculate local spatial weights for local regions within the inner and outer neighboring regions of the target region and the background region, respectively; The larger of the following values ​​is determined as the target spatial weight: the first difference between the local spatial weight corresponding to the target region and the average local spatial weight of the local regions in the inner neighboring region and the second difference between the average local spatial weight of the local regions in the outer neighboring region. The target potential region is spatially weighted using the target spatial weight.

2. The method according to claim 1, characterized in that, The original infrared sequence image is preprocessed by prior weight filtering to obtain a preprocessed sequence image, including: The original infrared sequence image is then Gaussian smoothed to remove low-frequency components; The sequence images after low-frequency components have been filtered out are filtered again using prior weights to enhance the target information in the sequence images.

3. The method according to claim 1 or 2, characterized in that, Based on the variance information of the mean of the target region and the mean of the background region in the preprocessed image sequence, a local variance feature map of the image sequence is established to obtain the potential target region, including: The initial potential region is obtained by subtracting the mean of the largest inner neighbor region in the background region from the mean of the target region, and by subtracting the mean of the largest outer neighbor region in the background region from the mean of the target region. The target potential region is calculated based on the initial potential region and the local feature variance map; the local feature variance map is a feature map obtained based on the variance of the multiple local regions after dividing the preprocessed image into multiple local regions.

4. The method according to claim 1, characterized in that, Local spatial weights are calculated for local regions within the inner and outer neighborhoods of the target and background regions, respectively, including: The first spatial weight is obtained by subtracting the minimum value between the mean and median of pixel gray levels in the local regions of the current region from the mean of the k largest pixel gray levels in the local regions of the current region; the current region can be the target region, the inner neighborhood region, or the outer neighborhood region. The second spatial weight is obtained by subtracting the mean of the last k smallest pixel gray levels in the local region of the current region from the mean of the mean of the first k largest pixel gray levels in the local region of the current region. The local spatial weight of the current region is obtained by multiplying the first spatial weight and the second spatial weight.

5. A detection device applied to the aerial infrared moving target detection method combining spatiotemporal information as described in claim 1, characterized in that, include: The preprocessing module is configured to perform prior weighted filtering preprocessing on the original infrared sequence image to obtain the preprocessed sequence image; The original infrared sequence images are infrared images acquired for aerial moving targets; The latent region extraction module is configured to establish a local variance feature map of the sequence image based on the variance information of the mean of the target region and the mean of the background region in the preprocessed sequence image, thereby obtaining the target latent region; The target feature map determination module is configured to perform spatial and temporal weighting on the potential target region to obtain a target feature map. The target detection module is configured to detect moving targets based on the target feature map.

6. An electronic device, characterized in that, The invention includes a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the method according to any one of claims 1-4.

7. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by a processor, the computer instructions implement the method described in any one of claims 1-4.

8. A computer program product comprising computer instructions, characterized in that, When executed by a processor, the computer instructions implement the method described in any one of claims 1-4.

Images