Method, device, equipment and medium for detecting small infrared targets under ground-to-air background

Abstract

The present invention provides a method, device, equipment and medium for detecting infrared weak small targets under ground-air background, wherein the method comprises: obtaining an infrared detection image to be processed; performing target extraction on the infrared detection image to obtain an infrared target image; performing ground-air background estimation on the infrared detection image to obtain a ground-air background image; extracting the edge of the ground-air background image to obtain a ground-air background edge image; performing a first differential calculation between the infrared target image and the ground-air background edge image to obtain an infrared target image with the ground-air background edge removed; and performing target detection based on the infrared target image with the ground-air background edge removed. This scheme can eliminate the influence of edge noise on the detection result by extracting the ground-air background edge and then performing a differential calculation to obtain an infrared target image with the ground-air background edge removed, thereby improving the detection probability of infrared weak small targets under ground-air background.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular to a method, device, equipment and medium for detecting infrared dim small targets under a ground-to-air background. Background Art

[0002] At present, anti-UAV detection in ground-to-air areas is an important topic in the fields of military air defense and civil security. With its high-resolution imaging and 360° all-round detection, the infrared search system has become an indispensable target detection method for security monitoring in key areas. The ground-to-air background is the most common background in anti-UAV detection application scenarios, so it is necessary to study the infrared weak target detection algorithm under the ground-to-air background.

[0003] The traditional infrared small target detection method is a pre-tracking detection algorithm that processes a single frame image. This method can be applied to embedded platforms that require real-time response. However, the traditional infrared small target detection method has a high detection probability when the background is relatively simple, but a low detection probability when the ground-to-air background is relatively complex with edge noise. Summary of the invention

[0004] Based on the problem that traditional infrared small target detection methods have low detection probability under a ground-to-air background with relatively complex edge noise, the embodiments of the present invention provide a method, device, equipment and medium for detecting infrared small targets under a ground-to-air background, which can improve the detection probability of detecting infrared small targets under a ground-to-air background.

[0005] In a first aspect, an embodiment of the present invention provides a method for detecting infrared dim small targets under a ground-to-air background, comprising:

[0006] Acquire the infrared detection image to be processed;

[0007] Performing target extraction on the infrared detection image to obtain an infrared target image;

[0008] Performing ground-air background estimation on the infrared detection image to obtain a ground-air background image;

[0009] Extracting the edge of the ground-air background image to obtain a ground-air background edge image;

[0010] Performing a first difference calculation between the infrared target image and the ground-space background edge image to obtain an infrared target image with the ground-space background edge removed;

[0011] Target detection is performed based on the infrared target image with the ground-air background edge removed.

[0012] Preferably, after performing the first difference calculation between the infrared target image and the ground-space background edge image and before obtaining the infrared target image with the ground-space background edge removed, the method further comprises:

[0013] Performing edge extraction on the infrared detection image to obtain an infrared edge image; the edge accuracy in the infrared edge image is greater than the edge accuracy in the ground-air background edge image;

[0014] A second difference calculation is performed between the infrared edge image and the infrared target image after the first difference calculation to obtain the infrared target image with the ground-air background edge removed.

[0015] Preferably, edge extraction is performed on the infrared detection image, including:

[0016] Performing Gaussian filtering on the infrared detection image;

[0017] Calculate the gradient amplitude and gradient direction of each pixel in the infrared detection image after Gaussian filtering;

[0018] Performing non-maximum suppression on the infrared detection image after Gaussian filtering according to the gradient amplitude and gradient direction of each pixel point to filter out non-edge pixels;

[0019] According to two preset pixel thresholds, edge lines are determined from the infrared detection image from which non-edge pixels are filtered out, thereby obtaining the infrared edge image.

[0020] Preferably, determining the edge line from the infrared detection image from which non-edge pixels are filtered out according to two preset pixel thresholds comprises:

[0021] Partitioning the pixels in the infrared detection image after filtering out non-edge pixels;

[0022] For each partition, the following steps are performed: connecting the pixel points in the partition whose gradient amplitude is greater than the first pixel threshold, and determining whether the edge line formed after the connection is closed; if not, for the endpoints of the unclosed edge line, determining the target pixel points whose gradient amplitude is greater than the second pixel threshold among the adjacent pixel points of the endpoints, and connecting the target pixel points with the endpoints until the formed edge line is closed;

[0023] The first pixel threshold is greater than the second pixel threshold.

[0024] Preferably, after obtaining the infrared target image with the ground-air background edge removed, the method further comprises:

[0025] Calculating the pixel value of each pixel in the infrared target image with the ground-air background edge removed;

[0026] Pixel points whose pixel values ​​meet preset conditions are screened out, and the infrared target image with the ground-sky background edge removed after the pixel points are screened out is used to perform the target detection.

[0027] Preferably, the step of filtering out pixel points whose pixel values ​​meet a preset condition includes:

[0028] The product of the maximum pixel value and the preset ratio is determined as a comparison threshold;

[0029] Pixel points whose pixel values ​​are less than the comparison threshold are screened out.

[0030] Preferably,

[0031] Utilizing morphological top-hat transformation to perform target extraction on the infrared detection image;

[0032] and / or,

[0033] Using a median filter operator of a set scale to estimate the ground-air background of the infrared detection image; the set scale is the minimum scale covering the target scale;

[0034] and / or,

[0035] The edge extraction of the ground-sky background image is performed by using Laplace filtering.

[0036] In a second aspect, an embodiment of the present invention further provides an infrared weak small target detection device under a ground-to-air background, comprising:

[0037] An image acquisition unit, used for acquiring an infrared detection image to be processed;

[0038] A target extraction unit, used for performing target extraction on the infrared detection image to obtain an infrared target image;

[0039] A background estimation unit, used for performing ground-air background estimation on the infrared detection image to obtain a ground-air background image;

[0040] A background edge extraction unit, used to extract the edge of the ground-air background image to obtain a ground-air background edge image;

[0041] A first difference calculation unit, used for performing a first difference calculation on the infrared target image and the ground-air background edge image to obtain an infrared target image with the ground-air background edge removed;

[0042] The target detection unit is used to perform target detection based on the infrared target image with the ground-air background edge removed.

[0043] In a third aspect, an embodiment of the present invention further provides a computing device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any embodiment of this specification is implemented.

[0044] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute the method described in any embodiment of this specification.

[0045] The embodiment of the present invention provides a method, device, equipment and medium for detecting infrared weak small targets under ground-air background, which extracts the target from the infrared detection image to obtain the infrared target image, and at the same time estimates the ground-air background of the infrared detection image, extracts the edge of the ground-air background, and obtains the ground-air background edge image with the target removed, so that the obtained infrared target image and the ground-air background edge image can be subjected to a first differential calculation to obtain the infrared target image with the ground-air background edge removed, and finally the target is detected based on the infrared target image with the ground-air background edge removed. It can be seen that this scheme eliminates the influence of edge noise on the detection result by extracting the ground-air background edge, and then performing a differential calculation to obtain the infrared target image with the ground-air background edge removed, thereby improving the detection probability of infrared weak small targets under ground-air background. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 This is a flow chart of a method for detecting infrared weak small targets under ground-to-air background provided by one embodiment of the present invention;

[0048] Figure 2 This is a flow chart of another method for detecting infrared weak small targets under ground-to-air background provided by one embodiment of the present invention;

[0049] Figure 3 is a hardware architecture diagram of a computing device provided by an embodiment of the present invention;

[0050] Figure 4 This is a structural diagram of an infrared weak small target detection device under ground-to-air background provided by an embodiment of the present invention;

[0051] Figure 5 This is a structural diagram of another infrared weak small target detection device under ground-to-air background provided by an embodiment of the present invention;

[0052] Figure 6 It is a structural diagram of another infrared weak small target detection device under ground-to-air background provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0054] As mentioned above, the traditional infrared weak target detection method has a high detection probability when the background is relatively simple, but in the ground-air background with complex edge noise, these edge noises will greatly affect the detection of weak targets, resulting in a low detection probability. Therefore, if it is necessary to detect infrared weak targets in a complex background, it is necessary to first consider removing these edge noises. It can be considered to extract the infrared target and ground-air background edges from the infrared detection image respectively, and then perform a differential calculation between the extracted infrared target image and the ground-air background edge image, so that the ground-air background edge existing in the infrared target image can be removed, and the infrared target image with the ground-air background edge removed can be obtained, so as to achieve the purpose of removing the ground-air background edge noise, thereby improving the detection probability of infrared weak target detection under the ground-air background.

[0055] The specific implementation of the above concept is described below.

[0056] Please refer to Figure 1 The embodiment of the present invention provides a method for detecting small infrared targets under ground-to-air background, the method comprising:

[0057] Step 100: Acquire an infrared detection image to be processed.

[0058] Step 102: extracting a target from the infrared detection image to obtain an infrared target image.

[0059] Step 104, performing ground-air background estimation on the infrared detection image to obtain a ground-air background image.

[0060] Step 106: extract the edge of the ground-air background image to obtain a ground-air background edge image.

[0061] Step 108: Perform a first difference calculation between the infrared target image and the ground-space background edge image to obtain an infrared target image with the ground-space background edge removed.

[0062] Step 110: Target detection is performed based on the infrared target image with the ground-air background edge removed.

[0063] In the embodiment of the present invention, the infrared target image is obtained by performing target extraction on the infrared detection image, and the ground-air background is estimated on the infrared detection image at the same time, and the edge of the ground-air background is extracted to obtain the ground-air background edge image with the target removed, so that the obtained infrared target image and the ground-air background edge image can be subjected to a first differential calculation to obtain the infrared target image with the ground-air background edge removed, and finally target detection is performed based on the infrared target image with the ground-air background edge removed. It can be seen that this scheme eliminates the influence of edge noise on the detection result by extracting the ground-air background edge and then performing differential calculation to obtain the infrared target image with the ground-air background edge removed, thereby improving the detection probability of infrared weak small target detection under the ground-air background.

[0064] Described below Figure 1 How the various steps are performed.

[0065] First, with respect to step 100 , an infrared detection image to be processed is acquired.

[0066] Since all objects in nature can radiate infrared rays, infrared detection images obtained using infrared detection instruments are more adaptable to conditions of insufficient light intensity and poor contrast at night than visible light images. Therefore, infrared detection is an indispensable target detection method for security monitoring in key areas.

[0067] In the embodiment of the present invention, the ground-to-air background includes not only obvious edge noises such as the horizon and the boundary between buildings and the sky, but also fine edge noises caused by factors such as cloud drift, tree shaking, and background noise clutter. In order to detect weak infrared targets in the ground-to-air background by eliminating edge noise, it is first necessary to obtain a single frame of infrared detection image to be processed.

[0068] Then, with respect to step 102, target extraction is performed on the infrared detection image to obtain an infrared target image.

[0069] In an embodiment of the present invention, the morphological top-hat transform is used to extract the target from the infrared detection image. Since the size of the infrared weak target in the infrared detection image is generally 2*2 pixels, a 2*2 structural element is selected to process the infrared detection image to obtain an infrared target image. Although the algorithm can extract the infrared target from the infrared detection image, it will retain more ground-to-sky background edges and noise points, so the boundary also needs to be suppressed.

[0070] Next, with respect to step 104, the ground-air background is estimated for the infrared detection image to obtain a ground-air background image.

[0071] In the embodiment of the present invention, the ground-air background estimation is performed on the infrared detection image using a median filter operator of a set scale, and the set scale is the minimum scale covering the target scale. Since the maximum size of a small infrared target is 4*4 pixels and does not exceed 5*5 pixels, a 5*5 median filter is selected to estimate the ground-air background of the infrared detection image to obtain a ground-air background image with the infrared target filtered out.

[0072] Next, with respect to step 106, the edge of the ground-air background image is extracted to obtain a ground-air background edge image.

[0073] In the embodiment of the present invention, since the infrared target has been filtered out in the ground-air background image obtained in step 104, the ground-air background edge image can be directly obtained by directly performing edge extraction on the ground-air background image obtained in step 104 using Laplace transform.

[0074] Next, with respect to step 108, a first difference calculation is performed between the infrared target image and the ground-space background edge image to obtain an infrared target image with the ground-space background edge removed.

[0075] In the embodiment of the present invention, the infrared target image obtained in step 102 and the ground-air background edge image obtained in step 106 are subjected to a first differential calculation. Since the ground-air background edge extracted by Laplace transform has a low precision, that is, the edge line is coarse, the infrared target image obtained after the first differential calculation still has a fine ground-air background edge. If target detection is performed directly, it is easy to cause a high false alarm rate and a low detection probability.

[0076] In one embodiment of the present invention, after performing a first difference calculation between the infrared target image and the ground-air background edge image and before obtaining the infrared target image with the ground-air background edge removed, a second difference calculation may be performed through the following steps S1-S2:

[0077] S1, extracting edges from the infrared detection image to obtain an infrared edge image; the edge accuracy in the infrared edge image is greater than the edge accuracy in the ground-air background edge image.

[0078] Since the infrared target image obtained after the first difference calculation still has fine ground-air background edges, it is necessary to suppress the remaining fine ground-air background edges for a second time. In order to achieve this goal, it is first necessary to perform a more accurate edge extraction on the infrared detection image, and perform a second difference calculation on the infrared target image obtained after the first difference calculation and the more accurate edge extraction image to remove the fine ground-air background edges.

[0079] It should be noted that, in the embodiment of the present invention, the infrared edge image is used to extract the edge of the infrared detection image, rather than the infrared target image obtained after the first differential calculation. This is because the infrared edge image is used to remove the fine edges of the ground and sky background in the infrared target image obtained after the first differential calculation through the second differential calculation. Therefore, if the edge extraction is directly performed on the infrared detection image, the obtained ground and sky background edge information will be more complete, and the fine edges of the ground and sky background in the infrared target image can be removed to a greater extent.

[0080] In addition, since the edge accuracy in the ground-to-air background edge image obtained by using Laplace transform is low and the extracted ground-to-air background edge line type is coarser, the infrared target image after the first differential calculation still needs to remove fine edges. Therefore, in this step, a more accurate edge extraction method is performed on the infrared detection image, which is capable of refining the ground-to-air background edge, that is, the extracted edge accuracy is higher, so the edge accuracy in the infrared edge image is greater than the edge accuracy in the ground-to-air background edge image.

[0081] In the embodiments of the present invention, please refer to Figure 2 , performing edge extraction with higher precision on the infrared detection image as shown in steps 200-204:

[0082] Step 200: Perform Gaussian filtering on the infrared detection image.

[0083] Step 202, calculating the gradient amplitude and gradient direction of each pixel in the infrared detection image after Gaussian filtering.

[0084] Step 204 , performing non-maximum suppression on the infrared detection image after Gaussian filtering according to the gradient amplitude and gradient direction of each pixel point, so as to filter out non-edge pixels.

[0085] In an embodiment of the present invention, first, based on the gradient amplitude of each pixel point, it is determined whether each pixel point in the infrared detection image after Gaussian filtering is a pixel point with an eight-value neighborhood maximum value within an eight-pixel neighborhood centered on the pixel point. If so, the pixel point with the eight-value neighborhood maximum value is retained; if not, it is determined to be a non-edge pixel point.

[0086] For each eight-value neighborhood maximum pixel point, the following are performed: according to the gradient direction of the eight-value neighborhood maximum pixel point, the intersection point with the eight neighborhood pixels in the gradient direction is determined; then, according to the gradient amplitudes of the two pixels closest to each intersection point among the eight neighborhood pixels, the gradient amplitude of each intersection point is interpolated; it is determined whether the gradient amplitude of the eight-value neighborhood maximum pixel point is greater than the gradient amplitude of each intersection point; if so, the eight-value neighborhood maximum pixel point is retained; if not, the eight-value neighborhood maximum pixel point is determined as a non-edge pixel point.

[0087] After determining whether all eight-value neighborhood maximum pixel points are non-edge pixels, the gradient amplitudes of all non-edge pixels are set to 0 to filter out non-edge pixels.

[0088] Step 206 , determining edge lines from the infrared detection image after filtering out non-edge pixels based on two preset pixel thresholds, to obtain the infrared edge image.

[0089] In this step, first, the pixel points in the infrared detection image from which non-edge pixels are filtered out are partitioned; for each partition, the following steps are performed: connecting the pixel points in the partition whose gradient amplitude is greater than the first pixel threshold, and determining whether the edge line formed after the connection is closed; if not, for the endpoints of the unclosed edge line, determining the target pixel points whose gradient amplitude is greater than the second pixel threshold among the adjacent pixel points of the endpoints, and connecting the target pixel points with the endpoints until the formed edge line is closed; the first pixel threshold is greater than the second pixel threshold.

[0090] In an embodiment of the present invention, the position information of all eight-value neighborhood maximum value pixels retained in the infrared detection image after filtering out non-edge pixels is first determined, and then all eight-value neighborhood maximum value pixels are partitioned according to the position information of the target and the ground and sky background in the original infrared detection image and experience.

[0091] For each partition, the following steps are performed: connect all eight-value neighborhood maximum value pixels in the partition whose gradient amplitude is greater than the first pixel threshold, and determine whether the edge line formed after the connection is closed; if not, for the endpoint of the unclosed edge line, determine whether there is a target pixel with a gradient amplitude greater than the second pixel threshold and less than the first pixel threshold among its eight neighborhood pixel points; if so, connect the target pixel point with the endpoint until the formed edge line is closed to obtain an infrared edge image.

[0092] S2, performing a second difference calculation on the infrared edge image and the infrared target image after the first difference calculation, to obtain the infrared target image with the ground-sky background edge removed.

[0093] In step S2, a second difference calculation is performed on the infrared target image after the first difference calculation and the infrared edge image obtained by the more accurate edge extraction in step S1 to obtain an infrared target image with the ground-air background edge removed, thereby achieving the purpose of secondary suppression of the remaining ground-air background fine edges.

[0094] In addition, after obtaining the infrared target image with the ground-air background edge removed, it is also necessary to perform adaptive threshold segmentation on the infrared target image with the ground-air background edge removed. The specific operation includes the following steps:

[0095] H1, calculate the pixel value of each pixel in the infrared target image after removing the ground-sky background edge.

[0096] H2, filter out the pixel points whose pixel values ​​meet the preset conditions, and use the infrared target image after the pixel points are filtered out and the edge of the ground-sky background is removed to perform the target detection.

[0097] In step H2, the pixel points whose pixel values ​​meet the preset conditions are screened out, including: determining the product of the maximum pixel value and the preset ratio as a comparison threshold; and screening out the pixel points whose pixel values ​​are less than the comparison threshold.

[0098] For example, if the pixel values ​​in the first 40% need to be retained, then the preset ratio is 60%. If the maximum pixel value of each pixel of the infrared target image without the ground-sky background edge is 100, the product of the maximum pixel value and the preset ratio is determined as the comparison threshold, then the calculated comparison threshold is 60, and the pixel values ​​less than 60 are set to 0, and the pixel values ​​between 60 and 100 are retained. The infrared target image without the ground-sky background edge with the pixel values ​​between 60 and 100 retained is subjected to the following target detection.

[0099] Finally, with respect to step 110, target detection is performed based on the infrared target image from which the ground-sky background edge is removed.

[0100] In one embodiment of the present invention, in order to perform target detection on an infrared target image with the ground-air background edge removed, at least the following two methods can be used:

[0101] Method 1: The final detection target is determined based on the motion characteristics of each target in the infrared target image with the ground-sky background edge removed in the time series infrared detection image.

[0102] Method 2: using a pre-trained classification model to classify the targets contained in the infrared target image with the ground-air background edge removed, to obtain the final detection target.

[0103] The above two methods are described below respectively.

[0104] First, method 1 is described.

[0105] In this method one, an embodiment of the present invention may specifically include: marking the target positions corresponding to each target in the infrared detection image and the infrared target image with the ground-sky background edge removed, and determining each target in the infrared detection image; and judging which is the final detection target based on the motion characteristics of each target in the time series infrared detection image.

[0106] For example, if the target is stationary in the time series infrared detection image, then the target is determined to be the ground-air background. If the target is moving in a relatively slow position in the time series infrared detection image, which is consistent with the motion characteristics of the real target, then the target can be determined to be the final detection target.

[0107] In the second mode, the embodiment of the present invention may specifically include the following steps N1-N3:

[0108] N1, training classification model.

[0109] In one embodiment of the present invention, the training method of the classification model may specifically include the following steps M1-M2:

[0110] M1, obtain a number of positive sample images and a number of negative sample images; the positive sample images are images containing infrared targets; the negative sample images are images not containing infrared targets; the sizes of the sample images are the same.

[0111] In the embodiment of the present invention, the main function of the classification model is to judge each target, whether it is an infrared target or a background edge, so it is essentially a binary classification network. When the classification model is trained in advance, the input image is divided into two categories: infrared target and background edge. Several images containing infrared targets are obtained as positive sample images, and several images not containing infrared targets are obtained as negative sample images.

[0112] In addition, since the infrared weak target has no obvious texture features and contour features, and the infrared target pixel accounts for a smaller number of pixels than the entire infrared target image without the ground-sky background edge, when using the classification model for classification, only the area containing each target needs to be locally judged. In order to improve the detection speed and accuracy, and to be larger than the target size, in the embodiment of the present invention, the size of the positive sample image and the negative sample image is set to 13*13 pixels.

[0113] In the embodiment of the present invention, since the diversity of positive sample images and negative sample images will affect the classification performance of the classification network when training the classification network, in addition to the real target images in the actual infrared images, many images of infrared targets as simulated targets are added to the positive sample images to increase the diversity of the training images. The simulated target images in the positive sample images are constructed according to the following formula:

[0114]

[0115] Wherein, α is the maximum grayscale value in the positive sample image, (x0, y0) is the position coordinate of the center of the simulation target, I(x, y) is the grayscale value of the pixel at the position (x, y) in the positive sample image, σ xWith σ y is a parameter that controls the horizontal and vertical dispersion of the simulation target, σ x With σ y The value of is within the set value range, which is used to control the size of the simulated target in the constructed positive sample image to be no larger than the set size.

[0116] For example, when σ x With σ y If the value of is too small, the single pixel value may be too high. Since the size of small infrared targets generally does not exceed 5*5, when σ x With σ y When the value of is too large, the size of the simulated target will exceed the real one, causing the simulated target to lose its authenticity. Both of these situations will affect the detection results. Therefore, x With σ y The value range of is controlled in [0.5,2].

[0117] In the embodiment of the present invention, since the training data of the combination of simulated target images and real target images is conducive to improving the generalization ability of the classification network, 3500 simulated target images are constructed according to the above method, and 3500 real target images in the actual infrared images are intercepted, and these 7000 images are used as positive sample images of the classification network. The negative sample images are infrared background images without targets randomly intercepted from the actual infrared images. Since the ratio of positive and negative sample images is too large, it is easy to cause the classification results of the trained classification model to deviate, so 7000 negative sample images are also intercepted.

[0118] In one embodiment of the present invention, the centers of the simulated target and the real target in the positive sample image are both located at the center of the corresponding positive sample image, which can save the design of the regression network for the target position, greatly reduce the complexity and calculation cost of the network structure, and improve the detection speed. In addition, placing the center of the target at the center of the positive sample image can make the classification model pay more attention to the characteristics of the pixels in the center area of ​​the sample image during the training process, so that when classifying the image to be classified, the characteristics of the pixels in the center area of ​​the image to be classified can also be used for classification.

[0119] M2, using the plurality of positive sample images and the plurality of negative sample images to train a convolutional neural network to obtain the classification model; the last convolution layer of the convolutional neural network is a 1×1 convolution kernel.

[0120] In the embodiment of the present invention, the training set and the test set are determined from the positive sample images and the negative sample images determined in step H1, and 10% of the images are randomly selected from the positive sample images as the test set, and the test set contains both real target images and simulated target images. The 1400 positive sample images extracted are the test set, and the remaining 12600 positive sample images and negative sample images are the training set.

[0121] In the embodiment of the present invention, compared with the traditional method that requires artificial design of features, then extracts features based on a single frame image, and finally determines the target category by artificial parameter adjustment, the local receptive field and weight sharing mechanism of the convolutional neural network have achieved great success in the field of image processing. The convolutional neural network has a powerful feature extraction capability and can automatically extract the features of the target image and then classify it.

[0122] However, since the size of infrared targets is too small, only local judgment is needed for the area containing each target. Therefore, the size of positive sample images and negative sample images is set to 13*13 pixels. The commonly used convolutional neural network structure is not applicable. The commonly used structure has many network layers, and the input images are all whole images. Direct use will cause a lot of computing resources and time waste. Therefore, a lightweight convolutional neural network architecture can be redesigned to perform classification and detection of infrared weak targets.

[0123] In the design of the network structure, you can try a variety of network structures and use the same method for training. The convergence of the loss function on the training set and the detection probability P on the test set can be used to evaluate the convergence of the loss function on the training set and the detection probability P on the test set. d And the false alarm probability F a To evaluate the effectiveness of these networks in classifying infrared weak targets.

[0124] In an embodiment of the present invention, the selected convolutional neural network includes three convolutional layers and two fully connected layers, the convolution kernels of the first two convolutional layers are 3*3, and the convolution kernel size of the last convolutional layer is 1*1.

[0125] The established convolutional neural network is trained using the determined training set. During training, the positive and negative sample images in the training set are processed using normalization. The batch size is set to 72, and the order of the positive and negative sample images is disrupted during training. The cross entropy error is used as the loss function during training. The calculation formula for the cross entropy error is as follows:

[0126]

[0127] Among them, y k represents the output of the convolutional neural network, t kRepresents the correct solution label. The optimizer selects the Adam algorithm. Using this optimization algorithm can make training faster and more effective, accelerate the convergence of the network, and reduce training time. The learning rate is set to 0.00003, divided into 7 epochs for training, and evaluated every 50 steps. Finally, the one with the smallest loss function and the highest detection probability is used as the final classification model.

[0128] N2, for each target contained in the infrared target image with the ground-sky background edge removed, respectively intercept a partial image of the infrared target image with the ground-sky background edge removed, so that each target is located in the corresponding partial image.

[0129] In the embodiment of the present invention, if a 256*256 pixel infrared target image with the ground-sky background edge removed is directly segmented into several local images with each pixel as the center for classification judgment, a large amount of computing resources and time will be wasted. In addition, since the classification model uses positive sample images and negative sample images of the same size during the training process, in order to reduce the influence of images of different sizes on the classification results, when segmenting to obtain the local images, the size of the segmented local images is the same as the size of the sample images used when training the classification model.

[0130] Further, according to step N1, in the positive sample image used by the classification model during the training process, the center position of the target is located at the center position of the positive sample image. It can be understood that the classification model pays more attention to the characteristics of the pixel points in the central area of ​​the input image. Therefore, in order to improve the accuracy of the classification result, when the corresponding local image is segmented for each target, the center position of the target is located at the center position of the corresponding local image. In addition, in the infrared target image with the ground-sky background edge removed, there may be a situation where multiple targets are close to each other. Therefore, when the corresponding local image is segmented for each target, there may be other targets in the edge area of ​​the local image. By locating the center position of the target at the center position of the corresponding local image, the classification model can pay more attention to the characteristics of the pixel points in the central area of ​​the local image, thereby reducing the influence of the targets in the edge area on the classification result.

[0131] When segmenting the local image, the position information of each target in the infrared target image with the ground-air background edge removed obtained in step 108 can be used to intercept a local image of the complete target of 13*13 pixels with each target as the center area.

[0132] N3, input each local image into the classification model respectively, and determine the final detection target according to the output of the classification model.

[0133] Each captured local image is input into the trained classification model respectively, and the final detection target is obtained according to the output result of the classification model.

[0134] Finally, the target position corresponding to the final detected target in the infrared detection image and the infrared target image with the ground-sky background edge removed is marked to realize the detection of infrared weak targets.

[0135] It can be seen that, whether by the first method or the second method, target detection can be completed based on the infrared target image with the ground-air background edge removed, and finally the detection of infrared weak targets can be achieved.

[0136] like Figure 3 , Figure 4 As shown, the embodiment of the present invention provides an infrared weak small target detection device under ground-air background. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. From the hardware level, Figure 3 As shown, it is a hardware architecture diagram of a computing device where an infrared weak small target detection device under ground-air background provided by an embodiment of the present invention is located, except Figure 3 In addition to the processor, memory, network interface, and non-volatile memory shown in the figure, the computing device in which the device is located in the embodiment may also generally include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, Figure 4 As shown, as a device in a logical sense, the CPU of the computing device in which it is located reads the corresponding computer program in the non-volatile memory into the memory and runs it. This embodiment provides an infrared weak small target detection device under ground-to-air background, including:

[0137] An image acquisition unit 401 is used to acquire an infrared detection image to be processed;

[0138] A target extraction unit 402 is used to extract a target from the infrared detection image to obtain an infrared target image;

[0139] The background estimation unit 403 is used to perform ground-air background estimation on the infrared detection image to obtain a ground-air background image;

[0140] The background edge extraction unit 404 is used to extract the edge of the ground-air background image to obtain a ground-air background edge image;

[0141] A first difference calculation unit 405 is used to perform a first difference calculation on the infrared target image and the ground-space background edge image to obtain an infrared target image with the ground-space background edge removed;

[0142] The target detection unit 406 is used to perform target detection based on the infrared target image with the ground-sky background edge removed.

[0143] In one embodiment of the present invention, the first difference calculation unit 405, after performing the first difference calculation between the infrared target image and the ground-space background edge image, before obtaining the infrared target image with the ground-space background edge removed, Figure 5 As shown in the structure diagram of another infrared weak small target detection device under ground-to-air background provided by an embodiment of the present invention, it also includes a second difference calculation unit 407.

[0144] In one embodiment of the present invention, the second differential calculation unit 407 is specifically used to perform edge extraction on the infrared detection image to obtain an infrared edge image; the edge accuracy in the infrared edge image is greater than the edge accuracy in the ground-sky background edge image; the second differential calculation unit 407 is also specifically used to perform a second differential calculation on the infrared edge image and the infrared target image after the first differential calculation to obtain the infrared target image with the ground-sky background edge removed.

[0145] In one embodiment of the present invention, the second differential calculation unit 407, when executing edge extraction on the infrared detection image, is specifically used to perform Gaussian filtering on the infrared detection image; calculate the gradient amplitude and gradient direction of each pixel in the infrared detection image after Gaussian filtering; perform non-maximum suppression on the infrared detection image after Gaussian filtering according to the gradient amplitude and gradient direction of each pixel to filter out non-edge pixels; and determine the edge line from the infrared detection image after filtering out the non-edge pixels according to two pre-set pixel thresholds to obtain the infrared edge image.

[0146] In one embodiment of the present invention, the second differential calculation unit 407, when executing the method of determining the edge line from the infrared detection image after filtering out non-edge pixels according to the two pre-set pixel thresholds, is also used to partition the pixel points in the infrared detection image after filtering out non-edge pixels; for each partition, the following is performed: connecting the pixel points in the partition whose gradient amplitude is greater than the first pixel threshold, and determining whether the edge line formed after the connection is closed; if not, for the endpoints of the unclosed edge line, determining the target pixel point whose gradient amplitude is greater than the second pixel threshold among the adjacent pixel points of the endpoint, and connecting the target pixel point with the endpoint until the formed edge line is closed; the first pixel threshold is greater than the second pixel threshold.

[0147] In one embodiment of the present invention, the first difference calculation unit 405, after executing to obtain the infrared target image with the ground-air background edge removed, Figure 6 As shown in the structure diagram of another infrared weak small target detection device under ground-to-air background provided by an embodiment of the present invention, it also includes an adaptive threshold segmentation unit 408.

[0148] In one embodiment of the present invention, the adaptive threshold segmentation unit 408 is specifically used to calculate the pixel value of each pixel in the infrared target image with the ground-sky background edge removed; filter out the pixel points whose pixel values ​​meet the preset conditions, and use the infrared target image with the ground-sky background edge removed after the pixel points are filtered out to perform the target detection.

[0149] In one embodiment of the present invention, the adaptive threshold segmentation unit 408, when performing the screening of pixel points whose pixel values ​​meet preset conditions, is specifically used to determine the product of the maximum pixel value and the preset ratio as the comparison threshold; and screen out pixel points whose pixel values ​​are less than the comparison threshold.

[0150] In one embodiment of the present invention, the target extraction unit 402 is specifically used to perform target extraction on the infrared detection image using morphological top-hat transform; and / or, the background estimation unit 403 is specifically used to perform ground-to-air background estimation on the infrared detection image using a median filter operator of a set scale; the set scale is the minimum scale covering the target scale; and / or, the background edge extraction unit 404 is specifically used to perform edge extraction of the ground-to-air background image using Laplace filtering.

[0151] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on a device for detecting infrared small targets under a ground-to-air background. In other embodiments of the present invention, a device for detecting infrared small targets under a ground-to-air background may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0152] The information interaction, execution process and other contents between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For the specific contents, please refer to the description in the embodiment of the method of the present invention, and no further description is given here.

[0153] An embodiment of the present invention further provides a computing device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, a method for detecting weak infrared small targets under a ground-to-air background in any embodiment of the present invention is implemented.

[0154] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes a method for detecting infrared weak small targets under a ground-to-air background in any embodiment of the present invention.

[0155] Specifically, a system or device equipped with a storage medium can be provided, on which software program code that implements the functions of any of the above-mentioned embodiments is stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program code stored in the storage medium.

[0156] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0157] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0158] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0159] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0160] It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical factors in the process, method, article or device including the elements.

[0161] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting small infrared targets under ground-to-air background, characterized in that: include: Acquire the infrared detection image to be processed; Performing target extraction on the infrared detection image to obtain an infrared target image; Performing ground-air background estimation on the infrared detection image to obtain a ground-air background image; Extracting the edge of the ground-air background image to obtain a ground-air background edge image; Performing a first difference calculation between the infrared target image and the ground-space background edge image to obtain an infrared target image with the ground-space background edge removed; Performing target detection based on the infrared target image with the ground-air background edge removed; After performing the first difference calculation between the infrared target image and the ground-space background edge image and before obtaining the infrared target image with the ground-space background edge removed, the method further includes: Performing edge extraction on the infrared detection image to obtain an infrared edge image; the edge accuracy in the infrared edge image is greater than the edge accuracy in the ground-air background edge image; A second difference calculation is performed between the infrared edge image and the infrared target image after the first difference calculation to obtain the infrared target image with the ground-air background edge removed.

2. The method according to claim 1, characterized in that Performing edge extraction on the infrared detection image includes: Performing Gaussian filtering on the infrared detection image; Calculate the gradient amplitude and gradient direction of each pixel in the infrared detection image after Gaussian filtering; Performing non-maximum suppression on the infrared detection image after Gaussian filtering according to the gradient amplitude and gradient direction of each pixel point to filter out non-edge pixels; According to two preset pixel thresholds, edge lines are determined from the infrared detection image from which non-edge pixels are filtered out, thereby obtaining the infrared edge image.

3. The method according to claim 2, characterized in that The step of determining the edge line from the infrared detection image after filtering out non-edge pixels according to two preset pixel thresholds includes: Partitioning the pixels in the infrared detection image after filtering out non-edge pixels; For each partition, the following steps are performed: connecting the pixel points in the partition whose gradient amplitude is greater than the first pixel threshold, and determining whether the edge line formed after the connection is closed; if not, for the endpoints of the unclosed edge line, determining the target pixel points whose gradient amplitude is greater than the second pixel threshold among the adjacent pixel points of the endpoints, and connecting the target pixel points with the endpoints until the formed edge line is closed; The first pixel threshold is greater than the second pixel threshold.

4. The method according to claim 1, characterized in that After obtaining the infrared target image with the ground-air background edge removed, the method further includes: Calculating the pixel value of each pixel in the infrared target image with the ground-air background edge removed; Pixel points whose pixel values ​​meet preset conditions are screened out, and the infrared target image with the ground-sky background edge removed after the pixel points are screened out is used to perform the target detection.

5. The method according to claim 4, characterized in that The step of filtering out pixel points whose pixel values ​​meet a preset condition includes: The product of the maximum pixel value and the preset ratio is determined as a comparison threshold; Pixel points whose pixel values ​​are less than the comparison threshold are screened out.

6. The method according to any one of claims 1 to 5, characterized in that: Utilizing morphological top-hat transformation to perform target extraction on the infrared detection image; and / or, Using a median filter operator of a set scale to estimate the ground-air background of the infrared detection image; the set scale is the minimum scale covering the target scale; and / or, The edge extraction of the ground-sky background image is performed by using Laplace filtering.

7. An infrared small target detection device under ground-air background, used to implement the method as described in any one of claims 1 to 6, characterized in that: include: An image acquisition unit, used for acquiring an infrared detection image to be processed; A target extraction unit, used for performing target extraction on the infrared detection image to obtain an infrared target image; A background estimation unit, used for performing ground-air background estimation on the infrared detection image to obtain a ground-air background image; A background edge extraction unit, used to extract the edge of the ground-air background image to obtain a ground-air background edge image; A first difference calculation unit, used for performing a first difference calculation on the infrared target image and the ground-air background edge image to obtain an infrared target image with the ground-air background edge removed; The target detection unit is used to perform target detection based on the infrared target image with the ground-air background edge removed.

8. A computing device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 6.

Images