Infrared weak target detection method, device, equipment and medium under complex background

Abstract

The present invention provides a method, apparatus, device, and storage medium for detecting small infrared targets in complex backgrounds. The method comprises: obtaining an infrared detection image to be processed; performing local contrast assignment processing on the infrared detection image to obtain an infrared target salient image; performing background edge extraction on the infrared detection image to obtain a background edge image; performing a differential calculation between the infrared target salient image and the background edge image to obtain an infrared target salient image with the background edges removed; and using a pre-trained classification model to classify the targets contained in the infrared target salient image with the background edges removed to obtain the final detected target. This solution can improve the detection accuracy of small infrared targets in complex backgrounds.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular to a method, device, equipment and medium for detecting infrared dim small targets under complex backgrounds. Background Art

[0002] Currently, counter-drone detection in low-altitude areas is a critical issue in both military air defense and civilian security. Infrared search systems, with their high-resolution imaging and 360° all-round detection, have become an indispensable target detection tool for security monitoring in key areas. Traditional infrared small target detection methods rely on pre-tracking algorithms that process single-frame images and are applicable to embedded platforms that require real-time response. However, these methods offer high accuracy when background complexity is minimal or the target signal-to-noise ratio is high. However, in complex backgrounds, these methods exhibit lower accuracy. Summary of the Invention

[0003] Based on the problem that traditional infrared dim small target detection methods have low detection accuracy in complex backgrounds, the embodiments of the present invention provide a method, device, equipment and medium for detecting infrared dim small targets in complex backgrounds, which can improve the detection accuracy of infrared dim small targets in complex backgrounds.

[0004] In a first aspect, an embodiment of the present invention provides a method for detecting small infrared targets in a complex background, comprising:

[0005] Acquire the infrared detection image to be processed;

[0006] Performing local contrast assignment processing on the infrared detection image to obtain an infrared target salient image;

[0007] Performing background edge extraction on the infrared detection image to obtain a background edge image;

[0008] performing a differential calculation on the infrared target salient image and the background edge image to obtain the infrared target salient image with the background edge removed;

[0009] The pre-trained classification model is used to classify the targets contained in the infrared target salient image with the background edge removed to obtain the final detection target.

[0010] Preferably, the performing local contrast assignment processing on the infrared detection image includes:

[0011] The infrared detection image is traversed pixel by pixel using a sliding window of a set size. For each sliding window traversed, the following operations are performed:

[0012] Determine a pixel value of a central pixel of the sliding window; the central pixel is a single pixel located at the center of the sliding window;

[0013] Determine the pixel mean of the pixels in the sliding window except the central pixel;

[0014] An assigned pixel value is determined according to the pixel value of the central pixel point and the pixel mean, and the assigned pixel value is used to replace the pixel value of the central pixel point.

[0015] Preferably, determining the assigned pixel value according to the pixel value of the central pixel point and the pixel mean value includes:

[0016] The assigned pixel value is calculated using the following formula:

[0017] C s =(I s -m i (s) ) 2

[0018] Among them, m i (s) is the pixel mean of each sliding window except the central pixel, I s is the pixel value of the central pixel in each sliding window, C s Assign pixel values ​​to each sliding window.

[0019] Preferably, extracting the background edge of the infrared detection image to obtain the background edge image includes:

[0020] performing target extraction on the infrared detection image to obtain an infrared target image;

[0021] Background edge estimation is performed on the infrared target image to obtain a background edge image after the target is removed.

[0022] Preferably, the training method of the classification model includes:

[0023] Acquire a plurality of positive sample images and a plurality of negative sample images; the positive sample images are images containing infrared targets; the negative sample images are images not containing infrared targets; and the sample images have the same size;

[0024] The convolutional neural network is trained using the plurality of positive sample images and the plurality of negative sample images to obtain the classification model; the last convolution layer of the convolutional neural network is a 1×1 convolution kernel.

[0025] Preferably, the infrared target contained in the positive sample image is a simulated target;

[0026] The obtaining of a plurality of positive sample images includes: constructing a positive sample image according to the following formula:

[0027]

[0028] Wherein, α is the maximum grayscale value in the positive sample image, (x0, y0) is the position coordinate of the center of the simulation target, I(x, y) is the grayscale value of the pixel at the position (x, y) in the positive sample image, σ x and σ y σ is a parameter that controls the horizontal and vertical dispersion of the simulation target. x and σ y The value of is within the set value range, which is used to control the size of the simulated target in the constructed positive sample image to be no larger than the set size;

[0029] and / or,

[0030] The center of the infrared target in the positive sample image is located at the center position of the corresponding positive sample image;

[0031] and / or,

[0032] The classification model includes three convolutional layers and two fully connected layers.

[0033] Preferably, the classifying the targets contained in the infrared target salient image with the background edge removed by using a pre-trained classification model includes:

[0034] For each target included in the infrared target salient image with background edges removed, respectively intercepting a partial image of the infrared target salient image with background edges removed, so that each target is located in a corresponding partial image;

[0035] Each local image is input into the classification model respectively, and the final detection target is determined according to the output of the classification model.

[0036] In a second aspect, an embodiment of the present invention further provides an infrared small target detection device under a complex background, comprising:

[0037] An infrared image acquisition unit, used to acquire an infrared detection image to be processed;

[0038] A local contrast amplitude unit is used to perform local contrast assignment processing on the infrared detection image to obtain an infrared target salient image;

[0039] A background edge extraction unit, configured to extract the background edge of the infrared detection image to obtain a background edge image;

[0040] a difference calculation unit, configured to perform a difference calculation on the infrared target salient image and the background edge image to obtain the infrared target salient image with the background edge removed;

[0041] The target classification unit is used to classify the targets contained in the infrared target salient image with the background edge removed using a pre-trained classification model to obtain the final detection target.

[0042] In a third aspect, an embodiment of the present invention further provides a computing device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any embodiment of this specification is implemented.

[0043] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute the method described in any embodiment of this specification.

[0044] The embodiments of the present invention provide a method, apparatus, device, and medium for detecting infrared small targets under complex backgrounds. By performing local contrast assignment processing on an infrared detection image, an infrared target salient image is obtained. Since background edges exist in the infrared target salient image obtained at this time, background edge extraction is performed on the infrared detection image to obtain a background edge image. A differential calculation is then performed between the infrared target salient image and the infrared target salient image, thereby removing the background edges in the infrared target salient image. A pre-trained classification model is then used to classify the targets contained in the infrared target salient image with the background edges removed, thereby quickly obtaining the final detected target based on the classification results. This solution first removes the interference of edge noise under complex backgrounds, greatly improving the detection accuracy of infrared small targets. Furthermore, the classification model utilizes its own powerful feature extraction capabilities to avoid the situation where there are too many suspected target points, thereby achieving detection of infrared small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 This is a flow chart of a method for detecting infrared dim small targets in a complex background provided by one embodiment of the present invention;

[0047] Figure 2 is a flow chart of a local contrast assignment method provided by one embodiment of the present invention;

[0048] Figure 3 This is a hardware architecture diagram of a computing device provided by one embodiment of the present invention;

[0049] Figure 4 This is a structural diagram of an infrared dim small target detection device under complex background provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0051] As previously mentioned, traditional infrared small target detection methods achieve high detection accuracy when background complexity is low or the target signal-to-noise ratio is high. However, in complex backgrounds, these methods suffer from low detection accuracy. This is because complex backgrounds contain a high amount of edge noise, which significantly affects the detection of infrared small targets. Therefore, if infrared small target detection is required in complex backgrounds, it is first necessary to remove this edge noise. One approach is to extract the infrared target and background edges from the infrared detection image separately, then perform a differential calculation between the extracted infrared target image and the background edge image. This removes the background edges from the infrared target image and reduces the interference of edge noise on the infrared target image. Furthermore, traditional infrared small target detection methods use a pre-tracking detection algorithm that processes single-frame images. While this ensures real-time performance, the detected results may still contain suspected targets. Subsequently, the motion characteristics of suspected targets in the image sequence must be used to determine whether they are real targets. Therefore, it is considered to use a pre-trained classification model to classify the targets contained in the infrared target salient image after removing the background edges, directly obtaining the final detected targets. Therefore, this scheme not only improves the detection accuracy of infrared dim targets under complex backgrounds, but also improves the detection speed.

[0052] The specific implementation of the above concept is described below.

[0053] Please refer to Figure 1 The embodiment of the present invention provides a method for detecting infrared dim small targets under complex backgrounds, the method comprising:

[0054] Step 100: Acquire an infrared detection image to be processed.

[0055] Step 102: Perform local contrast assignment processing on the infrared detection image to obtain an infrared target salient image.

[0056] Step 104 : extracting background edges from the infrared detection image to obtain a background edge image.

[0057] Step 106 : performing a difference calculation between the infrared target salient image and the background edge image to obtain the infrared target salient image with the background edge removed.

[0058] Step 108 , using a pre-trained classification model to classify the targets contained in the infrared target salient image with the background edge removed, to obtain the final detection target.

[0059] In an embodiment of the present invention, an infrared target salient image is obtained by performing local contrast assignment processing on the infrared detection image, and background edge extraction is performed on the infrared detection image to obtain a background edge image. Then, a difference calculation is performed between the obtained infrared target salient image and the infrared target salient image to obtain an infrared target salient image with the background edge removed. Then, a pre-trained classification model is used to classify the targets contained in the infrared target salient image with the background edge removed to directly obtain the final detection target. It can be seen that this scheme first removes the interference of edge noise in a complex background, greatly improves the detection accuracy of infrared weak targets, and utilizes the powerful feature extraction capability of the classification model itself to avoid the situation where there are many suspected target points, and directly realizes the detection of infrared weak targets.

[0060] Described below Figure 1 How to perform the steps shown.

[0061] First, with respect to step 100 , an infrared detection image to be processed is acquired.

[0062] Infrared imaging technology is a promising new high-tech. In nature, all objects can radiate infrared rays. Therefore, using an infrared detector to measure the infrared difference between the target itself and the complex background can produce an infrared detection image formed by different thermal infrared rays.

[0063] In the embodiments of the present invention, the practical application scenario of the infrared detection system is mainly to detect small and dim targets at a distance in a complex background containing a large amount of edge noise. Such targets are almost all small point-like targets in the infrared detection image, showing only a few pixels and lacking shape and texture information. Moreover, the intensity of such targets is significantly reduced when they are attenuated by the atmosphere or obscured by clouds and ground objects, which greatly affects the detection accuracy of small and dim infrared targets. Therefore, in order to detect the position information of small and dim infrared targets using an image detection algorithm, it is first necessary to obtain a single frame of the infrared detection image to be processed.

[0064] Then, with respect to step 102, local contrast assignment processing is performed on the infrared detection image to obtain an infrared target salient image.

[0065] In the embodiment of the present invention, the infrared detection image is subjected to local contrast assignment processing. First, a sliding window of a set size is used to traverse the infrared detection image pixel by pixel. Please refer to Figure 2 For each sliding window traversed, the following steps 200-204 are executed:

[0066] Step 200: Determine the pixel value of the central pixel point of the sliding window; the central pixel point is a single pixel point located at the center position of the sliding window.

[0067] In the embodiment of the present invention, the size of the sliding window can be determined based on the size of the small target. Preferably, the sliding window size is the minimum size that can cover the small target. Since the size of a small infrared target generally does not exceed 5*5 pixels, and the number of pixels occupied by a small infrared target is very small, a sliding window size of 5*5 pixels is selected, and the center position of the sliding window is a single pixel. The pixel value of the center pixel of the sliding window is determined.

[0068] Step 202: Determine the pixel mean of the pixels in the sliding window except the central pixel.

[0069] For example, since the size of each sliding window is 5*5 pixels, it is necessary to calculate the pixel mean of the other 24 adjacent pixels in the sliding window except the central pixel.

[0070] Step 204 : determining an assigned pixel value according to the pixel value of the central pixel and the pixel mean, and replacing the pixel value of the central pixel with the assigned pixel value.

[0071] In the embodiment of the present invention, in step 204, the assigned pixel value may be calculated using at least the following formula:

[0072] C s =(I s -m i (s) ) 2

[0073] Among them, m i (s) is the pixel mean of each sliding window except the central pixel, I s is the pixel value of the central pixel in each sliding window, C s Assign pixel values ​​to each sliding window.

[0074] Continuing with the example in step 202, the pixel value of the center pixel point of the sliding window is subtracted from the pixel mean of the other 24 adjacent pixel points in the sliding window except the center pixel point, and then the square of the difference is taken as the assigned pixel value of the sliding window, and the assigned pixel value is used to replace the pixel value of the center pixel point.

[0075] In an embodiment of the present invention, a sliding window with a set size of 5*5 is used to traverse the infrared detection image pixel by pixel, and the contrast assignment of each traversed sliding window is implemented through steps 200-204. When the sliding window completes the pixel-by-pixel traversal of the entire infrared detection image, the local contrast assignment of the infrared detection image is completed.

[0076] The above formula is used to calculate the assigned pixel value for the center pixel in each sliding window. This is because the difference between the center pixel's value and the average value of its neighboring pixels represents the contrast between the center pixel and the neighboring pixels surrounding it in the sliding window. When the contrast levels differ, squaring the difference can increase the gap between smaller and larger contrasts, resulting in a noticeable difference in the image after local contrast assignment. Therefore, using the above formula to calculate the assigned pixel value can make infrared targets in the image more prominent.

[0077] It should be noted that a variety of processing methods can be used to perform local contrast assignment processing on the infrared detection image. In one embodiment of the present invention, at least the local contrast measurement method (LCM) can be used on the infrared detection image. By utilizing local information in the spatial domain and dividing the sliding window into blocks, the contrast difference between the obtained sub-blocks is used as the basis for local contrast assignment. The specific steps are as follows.

[0078] For example, for each traversed sliding window, the sliding window size is 9*9 pixels. The sliding window is divided into panels, and the number of pixels in each sub-block is 3*3, so the number of regions is 3*3. First, the maximum pixel value of the central sub-block of the sliding window is obtained by comparing the pixel values ​​of each single pixel in the central sub-block of the sliding window; then the pixel mean of each of the other 8 sub-blocks except the central sub-block is calculated; finally, the assigned pixel value is calculated according to the following formula:

[0079]

[0080] Among them, L s is the maximum pixel value of the central sub-block of the sliding window, the subscript s is the number of the sliding window, C s is the assigned pixel value of the sliding window. In order to minimize the assigned pixel value, m i is the maximum pixel mean of the other 8 sub-blocks except the central sub-block in the sliding window.

[0081] The calculated assigned pixel value replaces the pixel value of the central pixel in the central sub-block. This method is used to traverse the entire infrared detection image pixel by pixel from left to right and from top to bottom to complete the local contrast assignment of the infrared detection image.

[0082] For these two local contrast assignment processing methods, the method in steps 200-204 is an improvement on the application of the local contrast measurement method (LCM) in the detection of infrared dim small targets under complex backgrounds. By changing the central sub-block of the sliding window from a pixel block to a single pixel point, the "block effect" existing in the local contrast measurement method (LCM) is eliminated, making it more sensitive to small-sized targets while filtering out a large number of coarse edges to improve the detection accuracy of the position information of infrared dim small targets.

[0083] In addition, in an embodiment of the present invention, after performing local contrast assignment processing on the infrared detection image, before obtaining the infrared target salient image, it is necessary to perform threshold segmentation on the image after local contrast assignment, and use the image after threshold segmentation as the infrared target salient image. The specific steps are: calculating the pixel value of each pixel point in the image after local contrast assignment; screening out the pixel points whose pixel values ​​meet the preset conditions, and using the image after local contrast assignment after the pixel points are screened out as the infrared target salient image.

[0084] In an embodiment of the present invention, the method for filtering out pixel points whose pixel values ​​meet a preset condition is: determining the product of the maximum pixel value and a preset ratio as a comparison threshold; and filtering out pixel points whose pixel values ​​are smaller than the comparison threshold.

[0085] For example, if you need to retain pixels with values ​​in the top 40%, then the preset ratio is 60%. If the maximum pixel value in the image after local contrast assignment is 100, the calculated comparison threshold is 60. Pixels with values ​​less than 60 are set to 0, and pixels with values ​​between 60 and 100 are retained.

[0086] In the embodiment of the present invention, the infrared detection image is processed with local contrast assignment through steps 200-204, and then segmented by threshold value to obtain an infrared target salient image including the target and background edges with pixel values ​​in the top 40%.

[0087] Next, with respect to step 104 , background edge extraction is performed on the infrared detection image to obtain a background edge image.

[0088] In one embodiment of the present invention, background edge extraction of an infrared detection image can be performed in at least two ways:

[0089] Method 1: First, use the morphological top-hat transform to extract the target from the infrared detection image, and then use the median filter to estimate the background edge.

[0090] Method 2: First, use median filtering to complete background edge estimation, and then use Laplace filtering to extract background edges.

[0091] The above two methods are described below respectively.

[0092] First, method 1 is described.

[0093] In the first approach, the embodiment of the present invention may specifically include the following steps S1-S2:

[0094] S1, performing target extraction on the infrared detection image to obtain an infrared target image.

[0095] In step S1, since the size of small targets is generally 2*2 and 3*3, a 3*3 convolution kernel is selected to perform a morphological top-hat transformation on the infrared detection image to obtain an infrared target image containing small targets and some local mutation edges.

[0096] S2, performing background edge estimation on the infrared target image to obtain a background edge image after removing the target.

[0097] In step S2, in order to perform subtraction to obtain a significant infrared target image with the background edge removed, it is necessary to estimate the background edge of the infrared target image and remove the target, and the target size does not exceed 5*5 pixels. Therefore, a median filter with a window of 5*5 is used on the infrared target image to obtain a background edge image after removing the target.

[0098] The above describes the first method, and the second method will be described next.

[0099] In the second approach, the embodiment of the present invention may specifically include:

[0100] First, a 5*5 median filter operator is used to perform background estimation on the infrared detection image to obtain a background image after the target is removed; then a Laplacian filter is used to extract the background edge of the background image after the target is removed to obtain a background edge image after the target is removed.

[0101] Since the infrared target salient image obtained in step 102 is also subjected to threshold segmentation after the local contrast assignment processing, the pixels below the threshold are set to 0, and only strong edges with relatively high pixel values ​​are retained. In order to eliminate the influence of weak edges with pixel values ​​below the threshold on the differential calculation results during the subsequent differential calculation, whether the background edge estimation of the infrared detection image is completed by method one or method two, it is necessary to use the same threshold segmentation method as the threshold segmentation performed on the image after the local contrast assignment to perform threshold segmentation on the background edge image after the target is removed, and the image obtained after the threshold segmentation is used as the background edge image.

[0102] Next, for step 106 , a difference calculation is performed between the infrared target salient image and the background edge image to obtain the infrared target salient image with the background edge removed.

[0103] In an embodiment of the present invention, since the background edge image is the target to be removed and only contains the background edges with pixel values ​​in the top 40%, the infrared target salient image containing the target and the background edges with pixel values ​​in the top 40% is used to perform differential calculation with the background edge image. For a complex background, a certain amount of false alarm points still exist in the infrared target salient image obtained by removing the background edges. Therefore, these targets can be judged and processed with the help of a classification network, and the position information of the real target can be finally obtained.

[0104] Finally, for step 108, the pre-trained classification model is used to classify the targets contained in the infrared target salient image with the background edge removed to obtain the final detection target.

[0105] In one embodiment of the present invention, the training method of the classification model may specifically include the following steps H1-H2:

[0106] H1, obtain several positive sample images and several negative sample images; the positive sample images are images containing infrared targets; the negative sample images are images not containing infrared targets; the sizes of the sample images are the same.

[0107] In this embodiment of the present invention, the classification model primarily determines whether each target is an infrared target or a background edge. Therefore, it is essentially a binary classification network. During pre-training, the classification model is trained to classify input images into two categories: infrared targets and background edges. Several images containing infrared targets are obtained as positive sample images, and several images not containing infrared targets are obtained as negative sample images.

[0108] Furthermore, since small infrared targets lack distinct texture and outline features, and their pixels are relatively small compared to the total number of pixels in a significant infrared target image (excluding background edges), the classification model only requires local judgment of the region containing each target. To improve detection speed and accuracy, and to ensure a size larger than the target, in this embodiment, both the positive and negative sample images are set to 13*13 pixels.

[0109] In the embodiment of the present invention, since the diversity of positive and negative sample images will affect the classification performance of the classification network when training the classification network, in addition to the real target images in the actual infrared images, many images of simulated infrared targets are added to the positive sample images to increase the diversity of the training images. The simulated target images in the positive sample images are constructed according to the following formula:

[0110]

[0111] Wherein, α is the maximum grayscale value in the positive sample image, (x0, y0) is the position coordinate of the center of the simulation target, I(x, y) is the grayscale value of the pixel at the position (x, y) in the positive sample image, σ x and σ y σ is a parameter that controls the horizontal and vertical dispersion of the simulation target. x and σ y The value of is within the set value range, which is used to control the size of the simulated target in the constructed positive sample image to be no larger than the set size.

[0112] For example, when σ x and σ y If the value of σ is too small, the value of a single pixel will be too high. Since the size of a small infrared target is generally no more than 5*5, when σ x and σ y When the value of is too large, the size of the simulated target will exceed the real one, causing the simulated target to lose its authenticity. Both of these situations will affect the detection results. Therefore, x and σ y The value range of is controlled in [0.5,2].

[0113] In this embodiment of the present invention, since using training data that combines simulated target images with real target images helps the classification network improve its generalization ability, 3,500 simulated target images were constructed according to the above method. Also, 3,500 real target images were captured from actual infrared images. These 7,000 images served as positive sample images for the classification network. Negative sample images consisted of infrared background images without targets randomly captured from actual infrared images. Since the disparity between the positive and negative sample images is too large, it can easily lead to biased classification results in the trained classification model. Therefore, 7,000 negative sample images were also captured.

[0114] In one embodiment of the present invention, the centers of both the simulated and real targets in the positive sample image are located at the center of the corresponding positive sample image. This eliminates the need for a regression network design specific to the target location, significantly reducing the complexity and computational cost of the network structure and improving detection speed. Furthermore, placing the target center at the center of the positive sample image allows the classification model to focus more on the features of the pixels within the central region of the sample image during training. This allows the classification model to utilize the features of the pixels within the central region of the image to be classified when classifying the image to be classified.

[0115] H2, using the plurality of positive sample images and the plurality of negative sample images to train a convolutional neural network to obtain the classification model; the last convolution layer of the convolutional neural network is a 1×1 convolution kernel.

[0116] In this embodiment of the present invention, a training set and a test set are determined from the positive and negative images determined in step H1. A random 10% of the positive images are selected as the test set, which contains both real and simulated target images. The 1,400 positive images selected form the test set, while the remaining 12,600 positive and negative images form the training set.

[0117] In this embodiment of the present invention, compared to traditional methods that require manually designed features, then extract features based on a single image frame, and finally manually adjust parameters to determine the target category, the local receptive field and weight sharing mechanism of convolutional neural networks have achieved remarkable success in the field of image processing. Convolutional neural networks have powerful feature extraction capabilities and can automatically extract features of target images and then classify them.

[0118] However, due to the small size of infrared targets, only local judgments are required for the area containing each target. Therefore, the size of both positive and negative sample images is set to 13*13 pixels. This makes the commonly used convolutional neural network architecture inapplicable. Commonly used architectures have many network layers and the input image is the entire image. Direct use of this architecture would result in a significant waste of computing resources and time. Therefore, a lightweight convolutional neural network architecture can be redesigned to perform classification and detection of small infrared targets.

[0119] In the design of the network structure, four designs were tried. From Net1 to Net4, the number of network parameters and complexity among them decreased in sequence. Among them, Net1, Net2, and Net3 all include four convolutional layers and two fully connected layers. The convolution kernel size of the first three convolutional layers of Net1, Net2, and Net3 is 3*3. Net4 includes three convolutional layers and two fully connected layers. The convolution kernel size of the first two convolutional layers is also 3*3. All four networks introduce a 1×1 convolution kernel in the last convolution layer. Introducing this type of convolution kernel can significantly reduce the number of network parameters and improve computational speed.

[0120] All four networks are trained using the same method, and the convergence of the loss function on the training set and the detection probability P on the test set are used to evaluate the convergence of the loss function on the training set and the detection probability P on the test set. d And the false alarm probability F a To evaluate the effectiveness of these four networks in classifying infrared dim targets. a And the false alarm probability F a Calculated using the following formula:

[0121]

[0122]

[0123] Among them, N t Indicates the number of true targets in the test set, N d Indicates the number of real targets detected. N a Indicates the number of false alarm targets detected, and N indicates the total number of targets in the test set. The comparison of the effects of the four networks is shown in the following table:

[0124]

[0125] According to the above table, we can get the convergence of the loss function of Net1 on the training set and the detection probability P on the test set. d And the false alarm probability F aBoth are the largest, but compared with Net4, the order of magnitude of Net1 parameters is 7 times that of Net4, and the time consumed for detection is about 2 times that of Net4. Although it has a slightly better detection probability, its false alarm probability is also the highest. Taking into account the mismatch between the number of network layers and training parameters, that is, the more network layers, the more parameters are required, which will lead to overfitting of the network and ultimately a sharp drop in the detection probability, such as Net3, which damages the detection performance of the network. Therefore, after comprehensive comparison of the four network structures, Net4, which performs well in all aspects, was finally selected as the final structure of the convolutional neural network.

[0126] Next, it is necessary to train a convolutional neural network with a Net4 structure. The convolutional neural network consists of three convolutional layers and two fully connected layers. The convolution kernels of the first two convolutional layers are 3*3, and the convolution kernel size of the last convolutional layer is 1×1.

[0127] The established convolutional neural network is trained using the determined training set. During training, the positive and negative sample images in the training set are processed using normalization. The batch size is set to 72, and the order of the positive and negative sample images is disrupted during training. The cross entropy error is used as the loss function during training. The calculation formula for the cross entropy error is as follows:

[0128]

[0129] Among them, y k represents the output of the convolutional neural network, t k Represents a correct label. The Adam algorithm was selected as the optimizer. This optimization algorithm allows for faster and more efficient training, accelerating network convergence and reducing training time. The learning rate was set to 0.00003, and training was divided into seven epochs, with evaluations every 50 steps. The model with the minimum loss function and the highest detection probability was selected as the final classification model.

[0130] Before using the classification model trained in the above-mentioned training method to classify the targets contained in the infrared target salient image with the background edges removed obtained in step 106, the following steps need to be performed: for each target contained in the infrared target salient image with the background edges removed, a local image of the infrared target salient image with the background edges removed is captured so that each target is located in the corresponding local image; each local image is input into the classification model, and the final detection target is determined based on the output of the classification model.

[0131] In the embodiments of the present invention, if a 256*256 pixel infrared target salient image is directly segmented into several local images centered at each pixel for classification and judgment, a large amount of computing resources and time will be wasted. In addition, since the classification model uses positive and negative sample images of the same size during training, in order to reduce the impact of images of different sizes on the classification results, when segmenting the local images, the size of the segmented local images is the same as the size of the sample images used when training the classification model.

[0132] Furthermore, according to step H1, the center position of the target in the positive sample image used by the classification model during training is located at the center position of the positive sample image. It can be understood that the classification model pays more attention to the characteristics of the pixels in the central area of ​​the input image. Therefore, in order to improve the accuracy of the classification results, when the corresponding local image is segmented for each target, the center position of the target is located at the center position of the corresponding local image. In addition, in the infrared target salient image after removing the background edge, there may be multiple targets that are close to each other. Therefore, when the corresponding local image is segmented for each target, other targets may exist in the edge area of ​​the local image. By locating the center position of the target at the center position of the corresponding local image, the classification model can pay more attention to the characteristics of the pixels in the central area of ​​the local image, thereby reducing the impact of targets in the edge area on the classification results.

[0133] When segmenting the local image, the position information of each target in the infrared target salient image (with the background edges removed) obtained in step 106 can be used to capture a 13*13 pixel local image of the complete target, with each target as the center area. Each captured local image is input into the trained classification model, and the final detected target is obtained based on the output of the classification model.

[0134] Finally, the target position corresponding to the final detected target in the infrared detection image and the infrared target salient image with the background edge removed is marked to achieve the detection of infrared weak targets.

[0135] like Figure 3 、 Figure 4 As shown, the embodiment of the present invention provides an infrared dim small target detection device under complex background. The device embodiment can be implemented by software, hardware or a combination of software and hardware. From the hardware level, Figure 3 As shown in FIG. 1 , a hardware architecture diagram of a computing device where an infrared weak target detection device under a complex background is provided by an embodiment of the present invention is located, except for Figure 3In addition to the processor, memory, network interface, and non-volatile memory shown, the computing device in the embodiment may also include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, Figure 4 As shown, as a device in a logical sense, the CPU of the computing device in which it is located reads the corresponding computer program in the non-volatile memory into the internal memory and runs it. This embodiment provides an infrared weak target detection device under complex background, including:

[0136] Infrared image acquisition unit 401, used to acquire infrared detection images to be processed;

[0137] A local contrast amplitude unit 402 is used to perform local contrast assignment processing on the infrared detection image to obtain an infrared target salient image;

[0138] A background edge extraction unit 403 is used to extract the background edge of the infrared detection image to obtain a background edge image;

[0139] A difference calculation unit 404 is configured to perform a difference calculation on the infrared target salient image and the background edge image to obtain the infrared target salient image with the background edge removed;

[0140] The target classification unit 405 is configured to classify the targets contained in the infrared target salient image with the background edge removed using a pre-trained classification model to obtain a final detection target.

[0141] In one embodiment of the present invention, the local contrast amplitude unit 302, when performing local contrast assignment processing on the infrared detection image, is specifically used to traverse the infrared detection image pixel by pixel using a sliding window of a set size, and for each traversed sliding window, performs the following: determining the pixel value of the center pixel point of the sliding window; the center pixel point is a single pixel point located at the center position of the sliding window; determining the pixel mean of the pixels in the sliding window other than the center pixel point; determining the assigned pixel value based on the pixel value of the center pixel point and the pixel mean, and replacing the pixel value of the center pixel point with the assigned pixel value.

[0142] In one embodiment of the present invention, the local contrast amplitude unit 402, when determining the assigned pixel value based on the pixel value of the central pixel and the pixel mean, is specifically configured to calculate the assigned pixel value using the following formula:

[0143] C s =(I s -m i (s) ) 2

[0144] Among them, m i (s) is the pixel mean of each sliding window except the central pixel, I s is the pixel value of the central pixel in each sliding window, C s Assign pixel values ​​to each sliding window.

[0145] In one embodiment of the present invention, the background edge extraction unit 403, when performing background edge extraction on the infrared detection image to obtain a background edge image, is specifically used to perform target extraction on the infrared detection image to obtain an infrared target image; and perform background edge estimation on the infrared target image to obtain a background edge image after removing the target.

[0146] In one embodiment of the present invention, the target classification unit 405, when performing classification model training, is specifically used to obtain a plurality of positive sample images and a plurality of negative sample images; the positive sample images are images containing infrared targets; the negative sample images are images not containing infrared targets; the sample images have the same size; a convolutional neural network is trained using the plurality of positive sample images and the plurality of negative sample images to obtain the classification model; the last convolution layer of the convolutional neural network is a 1×1 convolution kernel.

[0147] In one embodiment of the present invention, when the target classification unit 405 performs classification model training, the infrared target contained in the positive sample image is a simulated target; when performing the step of obtaining a plurality of positive sample images, the positive sample image is constructed specifically according to the following formula:

[0148]

[0149] Wherein, α is the maximum grayscale value in the positive sample image, (x0, y0) is the position coordinate of the center of the simulation target, I(x, y) is the grayscale value of the pixel at the position (x, y) in the positive sample image, σ x and σ y σ is a parameter that controls the horizontal and vertical dispersion of the simulation target. x and σ y The value of is within the set value range, which is used to control the size of the simulated target in the constructed positive sample image to be no larger than the set size; and / or, the center of the infrared target in the positive sample image is located at the center position of the corresponding positive sample image; and / or, the classification model includes three convolutional layers and two fully connected layers.

[0150] In one embodiment of the present invention, the target classification unit 405, when executing the classification of each target contained in the infrared target salient image with the background edges removed using a pre-trained classification model, is specifically used to, for each target contained in the infrared target salient image with the background edges removed, intercept a local image of the infrared target salient image with the background edges removed, so that each target is located in the corresponding local image; input each local image into the classification model respectively, and determine the final detection target according to the output of the classification model.

[0151] It should be understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on a device for detecting small infrared targets under complex backgrounds. In other embodiments of the present invention, a device for detecting small infrared targets under complex backgrounds may include more or fewer components than illustrated, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0152] The information interaction, execution process, etc. between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For specific contents, please refer to the description in the embodiment of the method of the present invention and will not be repeated here.

[0153] An embodiment of the present invention further provides a computing device including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, a method for detecting infrared dim small targets under a complex background according to any embodiment of the present invention is implemented.

[0154] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor executes a method for detecting infrared dim small targets in a complex background according to any embodiment of the present invention.

[0155] Specifically, a system or device equipped with a storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program codes stored in the storage medium.

[0156] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute part of the present invention.

[0157] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0158] In addition, it should be clear that the functions of any of the above embodiments can be achieved not only by executing the program code read by the computer, but also by enabling the operating system operating on the computer to complete part or all of the actual operations based on the instructions of the program code.

[0159] In addition, it can be understood that the program code read from the storage medium is written into a memory provided in an expansion board inserted into the computer or into a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above embodiments.

[0160] It should be noted that, in this article, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical factors in the process, method, article or device comprising the elements.

[0161] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk, etc. Various media that can store program codes.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for detecting infrared small targets under complex backgrounds, characterized in that: include: Acquire the infrared detection image to be processed; Performing local contrast assignment processing on the infrared detection image to obtain an infrared target salient image; Performing background edge extraction on the infrared detection image to obtain a background edge image; performing a differential calculation on the infrared target salient image and the background edge image to obtain the infrared target salient image with the background edge removed; Using a pre-trained classification model to classify each target contained in the infrared target salient image with the background edge removed, to obtain a final detection target; The classification model is trained using a plurality of positive sample images and a plurality of negative sample images; The performing local contrast assignment processing on the infrared detection image includes: The infrared detection image is traversed pixel by pixel using a sliding window of a set size. For each sliding window traversed, the following operations are performed: Determine a pixel value of a central pixel of the sliding window; the central pixel is a single pixel located at the center of the sliding window; Determine the pixel mean of the pixels in the sliding window except the central pixel; Determine an assigned pixel value according to the pixel value of the central pixel and the pixel mean, and replace the pixel value of the central pixel with the assigned pixel value; The step of determining the assigned pixel value according to the pixel value of the central pixel point and the pixel mean value includes: The assigned pixel value is calculated using the following formula: C s =(I s -m i (s) ) 2 Among them, m i (s) is the pixel mean of each sliding window except the central pixel, I s is the pixel value of the central pixel in each sliding window, C s Assign pixel values ​​for each sliding window; The infrared target contained in the positive sample image is a simulated target; The obtaining of a plurality of positive sample images includes: constructing a positive sample image according to the following formula: Wherein, α is the maximum grayscale value in the positive sample image, (x0, y0) is the position coordinate of the center of the simulation target, I(x, y) is the grayscale value of the pixel at the position (x, y) in the positive sample image, σ x and σ y σ is a parameter that controls the horizontal and vertical dispersion of the simulation target. x and σ y The value of is within the set value range, which is used to control the size of the simulated target in the constructed positive sample image to be no larger than the set size; x and σ y The setting range of is [0.5,2].

2. The method according to claim 1, characterized in that The step of extracting background edges from the infrared detection image to obtain a background edge image includes: performing target extraction on the infrared detection image to obtain an infrared target image; Background edge estimation is performed on the infrared target image to obtain a background edge image after the target is removed.

3. The method according to claim 1, characterized in that The training method of the classification model includes: Acquire a plurality of positive sample images and a plurality of negative sample images; the positive sample images are images containing infrared targets; the negative sample images are images not containing infrared targets; and the sample images have the same size; The convolutional neural network is trained using the plurality of positive sample images and the plurality of negative sample images to obtain the classification model; the last convolution layer of the convolutional neural network is a 1×1 convolution kernel.

4. The method according to claim 3, characterized in that The center of the infrared target in the positive sample image is located at the center position of the corresponding positive sample image; and / or, The classification model includes three convolutional layers and two fully connected layers.

5. The method according to any one of claims 1 to 4, characterized in that: The method of using a pre-trained classification model to classify each target contained in the infrared target salient image after removing the background edge comprises: For each target included in the infrared target salient image with background edges removed, respectively intercepting a partial image of the infrared target salient image with background edges removed, so that each target is located in a corresponding partial image; Each local image is input into the classification model respectively, and the final detection target is determined according to the output of the classification model.

6. A device for detecting small infrared targets under complex backgrounds, for implementing the method according to any one of claims 1 to 5, characterized in that: include: An infrared image acquisition unit, used to acquire an infrared detection image to be processed; A local contrast amplitude unit is used to perform local contrast assignment processing on the infrared detection image to obtain an infrared target salient image; A background edge extraction unit, configured to extract the background edge of the infrared detection image to obtain a background edge image; a difference calculation unit, configured to perform a difference calculation on the infrared target salient image and the background edge image to obtain the infrared target salient image with the background edge removed; The target classification unit is used to classify the targets contained in the infrared target salient image with the background edge removed using a pre-trained classification model to obtain the final detection target.

7. A computing device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 5.

Images