An infrared dim small target detection method, device and storage medium

Abstract

The present application relates to a kind of infrared weak target detection method, equipment and storage medium, the method comprises: constructing three-layer sliding window based on the sub-window structure of oversampling to the infrared image collected is traversed pixel by pixel;Wherein, the three-layer sliding window based on the sub-window structure of oversampling includes the center sub-window of innermost layer, middle layer area and outer layer area;The local contrast feature of center sub-window and middle layer sub-window, center sub-window and outer layer sub-window of each pixel corresponding sliding window in infrared image is calculated respectively;The local contrast feature of center sub-window and middle layer sub-window is multiplied with the local contrast feature of outer layer sub-window, and the final local contrast saliency map is obtained;The local contrast saliency map obtained is normalized, and threshold is set to carry out target segmentation and obtain final target.Compared with prior art, the present application has the advantages of high accuracy and good real-time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of infrared target detection, and in particular to an infrared weak small target detection method, device and storage medium. BACKGROUND

[0002] The infrared small target detection algorithm is one of the core components of the infrared detection system, and its performance is crucial to the overall performance of the detection system.

[0003] In recent years, the infrared small target detection algorithm based on local contrast has attracted widespread attention from many researchers due to its good detection performance and real-time performance. However, this kind of algorithm still has the following main shortcomings:

[0004] 1) In order to achieve accurate detection of small targets of different sizes, the algorithm needs to introduce multi-scale operation and maximum pooling operation, and the overall repetition of the algorithm is too high.

[0005] 2) The sliding window structure used by the algorithm usually only uses 4 or 8 non-overlapping image sub-windows adjacent to each other when calculating the contrast feature, as shown in the accompanying drawings. Figure 2 When the target or highlight interference is in the middle of the adjacent image sub-windows, the accuracy of the local contrast feature calculated will be greatly reduced.

[0006] 3) When calculating the contrast feature through the sliding window structure, the algorithm complexity is proportional to the number of image sub-windows used. For example, if 8 adjacent windows are used to calculate the contrast feature, the gray mean difference of the center sub-window T and the surrounding 8 adjacent sub-windows needs to be calculated 8 times, and the algorithm time complexity is high.

[0007] Therefore, there is an urgent need to design an infrared weak small target detection method with high accuracy and good real-time performance. SUMMARY

[0008] The present application provides an infrared weak small target detection method, device and storage medium with high accuracy and good real-time performance to overcome the defects of the prior art.

[0009] The object of the present application can be achieved by the following technical solutions:

[0010] According to a first aspect of the present application, an infrared weak small target detection method is provided, which comprises:

[0011] A three-layer sliding window based on an oversampling sub-window structure is constructed to perform pixel-by-pixel traversal on the collected infrared image; wherein the three-layer sliding window based on the oversampling sub-window structure comprises a center sub-window in the innermost layer, a middle layer region and an outer layer region;

[0012] The local contrast features of the center sub-window and the middle layer sub-window and the local contrast features of the center sub-window and the outer layer sub-window corresponding to each pixel in the infrared image are calculated respectively;

[0013] The local contrast features of the center sub-window and the middle layer sub-window are multiplied by the local contrast features of the outer layer sub-window to obtain a final local contrast saliency map;

[0014] The obtained local contrast saliency map is normalized, a segmentation threshold is set, and target segmentation is performed to obtain a final target.

[0015] Preferably, the three-layer sliding window based on the oversampling sub-window structure comprises a center sub-window in the innermost layer, a middle layer region and an outer layer region, specifically: the center sub-window has a size of 3x3; the middle layer region has a size of 9x9 and is divided into 24 sub-windows with a size of 3x3, and adjacent sub-windows have a step distance of 1 pixel; and the outer layer region has a size of 15x15 and is divided into 48 sub-windows with a size of 3x3, and adjacent sub-windows have a step distance of 1 pixel.

[0016] Preferably, the local contrast features of the center sub-window and the middle layer sub-window and the local contrast features of the center sub-window and the outer layer sub-window corresponding to each pixel in the infrared image are calculated respectively by a morphological dilation simplification strategy.

[0017] Preferably, the local contrast features of the center sub-window and the middle layer sub-window corresponding to each pixel in the infrared image are calculated by a morphological dilation simplification strategy, specifically:

[0018] The local contrast features D M (x,y) of the center sub-window and the middle layer sub-window corresponding to each pixel in the infrared image are calculated by a morphological dilation simplification strategy, and the expression is

[0019]

[0020] wherein g(x,y) is the result of the input infrared image f(x,y) processed by a mean filter, (x,y) is the current pixel coordinate of the input infrared image, is a morphological dilation operation, B W (x,y) is a structure element used in the dilation operation, which determines the operation range of the dilation.

[0021] Preferably, the local contrast features D O (x,y) of the center sub-window and the outer layer sub-window corresponding to each pixel in the infrared image are calculated by a morphological dilation simplification strategy, and the expression is

[0022]

[0023] Wherein, g(x, y) is the result of the input infrared image f(x, y) after mean filter processing, (x, y) is the current pixel coordinate of the input infrared image, is a gray scale dilation operation, B O (x, y) is a structure element used in the dilation operation, which determines the operation range of the dilation.

[0024] Preferably, the multiplication of the local contrast feature of the intermediate layer sub-window and the local contrast feature of the outer layer sub-window obtains the final local contrast saliency map, and the expression is as follows:

[0025] C(x, y) = D M (x, y) 2 D O (x, y)

[0026] In the formula, D M (x, y), D O (x, y) are respectively the local contrast features of the center sub-window and the intermediate layer sub-window and the center sub-window and the outer layer sub-window of each pixel corresponding sliding window in the infrared image.

[0027] According to the second aspect of the present application, an electronic device is provided, comprising a memory and a processor, the memory has a computer program stored thereon, and the processor realizes the method of any one of the aspects when executing the program.

[0028] According to the third aspect of the present application, a computer readable storage medium is provided, which has a computer program stored thereon, and the program realizes the method of any one of the aspects when executed by a processor.

[0029] Compared with the prior art, the present application has the following beneficial effects:

[0030] 1) The method of the present application introduces a three-layer sliding window structure based on oversampling sub-windows, the local contrast feature of the intermediate layer sub-window can be used to determine the local contrast feature of a small target, and the local contrast feature of the outer layer sub-window can be used to determine the local contrast feature of a large target; thus, only a single scale of local contrast feature needs to be calculated to realize the detection of small targets of all sizes, which simplifies the algorithm process and improves the real-time performance of the algorithm.

[0031] 2) The sliding window used in the traditional local contrast algorithm usually only uses adjacent non-overlapping image sub-windows when calculating the contrast feature. When a target or a highlight interference is in the middle of adjacent image sub-windows, the accuracy of the calculated local contrast feature cannot be guaranteed; however, the three-layer sliding window structure based on oversampling sub-windows designed in the present application can fully utilize the contrast information of the local background in all directions near the target, which greatly improves the accuracy of the obtained contrast feature.

[0032] 3) Traditional local contrast algorithm, when calculating the contrast feature through the sliding window structure, the algorithm complexity is proportional to the number of image sub-windows used; the invention can calculate the local contrast in any number of directions through a single inflation operation by introducing a morphological inflation simplification strategy, and the real-time performance is greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 The method flowchart of the present application is shown in the figure;

[0034] Figure 2 It is a traditional sliding window structure containing 8 sub-windows;

[0035] Figure 3 It is an operation schematic diagram of the method of the present application;

[0036] Figure 4 It is a three-layer sliding window structure based on oversampling sub-windows proposed in the method of the present application;

[0037] Figure 5 It is an infrared image used for testing in the present example;

[0038] Figure 6 It is Figure 5 The result after processing by the method of the present application. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0040] EMBODIMENT

[0041] As shown in the figure, the present embodiment gives an infrared dim small target detection method, which comprises: Figure 1

[0042] A three-layer sliding window based on oversampling sub-window structure is constructed to perform pixel-by-pixel traversal on the collected infrared image; wherein the three-layer sliding window based on oversampling sub-window structure comprises a center sub-window in the innermost layer, a middle layer region and an outer layer region;

[0043] The local contrast features of the center sub-window and the middle layer sub-window, and the center sub-window and the outer layer sub-window of the sliding window corresponding to each pixel in the infrared image are calculated respectively;

[0044] ​The local contrast features of the center sub-window and the intermediate layer sub-window are multiplied with the local contrast features of the outer layer sub-window to obtain a final local contrast saliency map, expressed as:

[0045] C(x,y) = D M (x,y) 2 D O (x,y)

[0046] In the formula, D M (x,y) and D O (x,y) are the local contrast features of the center sub-window and the intermediate layer sub-window, and the center sub-window and the outer layer sub-window of each pixel in the infrared image corresponding to the sliding window, respectively.

[0047] The obtained local contrast saliency map is normalized, a segmentation threshold is set, and a final target is obtained through target segmentation.

[0048] The electronic device includes a central processing unit (CPU) that can perform various appropriate actions and processes in accordance with computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). Various programs and data required for device operation can also be stored in the RAM. The CPU, the ROM, and the RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0049] A plurality of components in the device are connected to the I / O interface, including an input unit such as a keyboard, a mouse, and the like, an output unit such as various types of displays, a speaker, and the like, a storage unit such as a magnetic disk, an optical disk, and the like, and a communication unit such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit allows the device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0050] The processing unit performs the various methods and processes described above. For example, in some embodiments, the methods can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed on the device via the ROM and / or the communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, the CPU can be configured to perform the methods by any other appropriate means (e.g., by means of firmware).

[0051] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload programmable logic devices (CPLDs), and so on.

[0052] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0053] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0054] Example 2

[0055] This embodiment focuses on an infrared image with a resolution of 640×480 (attached). Figure 5 This paper presents an infrared method for detecting weak targets and determining their locations, including:

[0056] First, a three-layer sliding window based on oversampling sub-windows is used to traverse the image pixel by pixel from left to right and from top to bottom, as shown in the attached diagram. Figure 3 The specific structure of the three-layer sliding window used is shown in the attached figure. Figure 4 As shown, the central sub-window C is used to capture energy in the central region of the target. The area surrounding the central sub-window is the intermediate layer region, which can be divided into 24 intermediate layer sub-windows, denoted as M. i, i = 1, 2…24. The outermost region of the sliding window is the outer region, which can be divided into 48 outer sub-windows, denoted as O i , i = 1, 2…48. The overall size of the three-layer sliding window is 15x15, and the size of all sub-windows is 3x3.

[0057] Then, the local contrast feature D M of each pixel corresponding to the middle layer sub-window of the sliding window is calculated in turn

[0058] D M = min(L M ) (1)

[0059] wherein L M is the gray scale difference between the center sub-window and the 24 middle layer sub-windows of the sliding window corresponding to each pixel

[0060]

[0061] m c represents the average gray scale of all pixels of the center sub-window C, and m Mi represents the average gray scale of all pixels of the sub-window M i . As can be seen from the above formula, 24 subtraction operations and one minimum value operation are required to calculate the gray scale difference of each pixel, and the calculation amount is large. Therefore, the calculation is optimized by an inflation simplification strategy.

[0062] First, the calculation method of L M , formula (2), is converted to the following method:

[0063] min i (m c -m Mi ) = m c -max i (m Mi ) (3)

[0064] For a gray scale image, the inflation operation of the image can be equivalent to the operation of obtaining the maximum value of the image gray scale within a certain range. That is

[0065]

[0066] wherein f(x, y) is the gray scale value of the current pixel coordinate of the input image, (x, y) is the current pixel coordinate of the input image, is the gray scale inflation operation, and B(x, y) is the structure element used for the inflation operation, which determines the operation range D B of the inflation.

[0067] Therefore, the operation of maximum value of the mean value of 24 sub-windows in formula (3) can be replaced by an expansion operation, specifically replaced by the following:

[0068]

[0069] In formula (5), g(x, y) is the result of the input image f(x, y) processed by a 3x3 mean filter, (x, y) is the current pixel coordinate of the input image, is a gray-scale dilation operation, B W (x, y) is a structure element used by the dilation operation, which determines the operation range of the dilation and has a size of 7x7. Its value is as follows:

[0070]

[0071] From the above, this method only needs to perform a mean filtering, a morphological dilation operation and an image subtraction operation on the original image to obtain the local contrast features of the intermediate layer of all pixels in the entire image, and the algorithm efficiency is greatly improved.

[0072] Then, the outer layer local contrast feature D O of each pixel corresponding sliding window is calculated in turn according to the same method.

[0073] D O = min(L O ) (6)

[0074] wherein L O is the gray level difference between the center sub-window of each pixel corresponding sliding window and the 48 intermediate layer sub-windows, as formula (7)

[0075]

[0076] m c represents the gray level mean value of all pixels of the center sub-window C, and m Oi represents the gray level mean value of all pixels of the sub-window O i . From the above formula, it can be seen that 48 times of subtraction operation and one minimum value operation are required to calculate the gray level difference of each pixel. Therefore, it can also be optimized by the expansion simplification strategy. Specifically, it is replaced by formula (8):

[0077]

[0078] wherein g(x, y) is the result of the input image f(x, y) processed by a 3x3 mean filter, (x, y) is the current pixel coordinate of the input image, is a gray-scale dilation operation, B O(x, y) is a structure element used in the dilation operation, which determines the operation range of the dilation, and has a size of 13x13. The value is as follows:

[0079]

[0080] Then, the local contrast feature D M (x, y) of the middle layer sub-window is multiplied by the local contrast feature D O (x, y) of the outer layer sub-window to obtain the final local contrast feature map C(x, y), as shown in equation (9).

[0081] C(x, y) = D M (x, y) 2 D O (x, y) (9)

[0082] Finally, the obtained local contrast feature map (i.e., the attached Figure 6 ) is normalized, a segmentation threshold is set, and the target is segmented to obtain the final target to be detected.

[0083] The other settings in this embodiment are the same as those in Embodiment 1.

[0084] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An infrared dim small target detection method, characterized in that, The method comprises: Constructing a three-layer sliding window based on an over-sampling sub-window structure to perform pixel-by-pixel traversal on the collected infrared image; wherein the three-layer sliding window based on the over-sampling sub-window structure comprises a center sub-window in an innermost layer, a middle layer region, and an outer layer region; Respectively calculating local contrast features of the center sub-window and the middle layer sub-window and the center sub-window and the outer layer sub-window of the sliding window corresponding to each pixel in the infrared image; Multiplying the local contrast features of the center sub-window and the middle layer sub-window and the local contrast features of the outer layer sub-window to obtain a final local contrast saliency map; Performing normalization processing on the obtained local contrast saliency map, setting a segmentation threshold, and performing target segmentation to obtain a final target; Respectively calculating local contrast features of the center sub-window and the middle layer sub-window and the center sub-window and the outer layer sub-window of the sliding window corresponding to each pixel in the infrared image through a morphological dilation simplification strategy; Respectively calculating local contrast features of the center sub-window and the middle layer sub-window and the center sub-window and the outer layer sub-window of the sliding window corresponding to each pixel in the infrared image through a morphological dilation simplification strategy; Local contrast features of a center sub-window of the sliding window and the middle layer sub-window corresponding to each pixel in the infrared image D M ( x , y ), expressed as ， wherein, g ( x , y ) are the results of the input infrared image f ( x , y ) after mean filter processing, ( x , y ) are the current pixel coordinates of the input infrared image, ⨁ is a gray scale dilation operation, B W ( x , y ) are the structure elements used by the dilation operation, which determines the operation range of the dilation; The local contrast features of the center sub-window and the outer sub-window of the sliding window corresponding to each pixel in the infrared image are calculated by a morphological dilation simplification strategy D O ( x , y ), the expression is: ， wherein, g x y are the results of the input infrared image f x y after the mean filter processing, x y are the current pixel coordinates of the input infrared image, ⨁ is a gray scale dilation operation, B O x y is a structure element used by the dilation operation, which determines the operation range of the dilation.​​​​​​​ 2. The method of claim 1, wherein, The three-layer sliding window based on the over-sampling sub-window structure comprises a center sub-window in an innermost layer, a middle layer region, and an outer layer region, and specifically: the center sub-window has a size of 3x3; the middle layer region has a size of 9x9 and is divided into 24 sub-windows with a size of 3x3, and adjacent sub-windows have a spacing step of 1 pixel; and the outer layer region has a size of 15x15 and is divided into 48 sub-windows with a size of 3x3, and adjacent sub-windows have a spacing step of 1 pixel.

3. The method of claim 1, wherein, The multiplication of the local contrast features of the middle layer sub-window and the local contrast features of the outer layer sub-window to obtain the final local contrast saliency map is expressed as: ， In the formula, D M ( x , y )、 D O ( x , y ) are the local contrast features of the center sub-window and the middle layer sub-window, the center sub-window and the outer layer sub-window of the sliding window corresponding to each pixel in the infrared image, respectively.

4. An electronic device comprising a memory and a processor, said memory having stored thereon a computer program, characterized in that, The processor executes the program to implement the method of any one of claims 1-3.

5. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1-3.

Images