Infrared small target detection method, device, equipment, medium and program product

Abstract

The invention provides an infrared small target detection method and device, equipment, a medium and a program product, and the method comprises the steps: carrying out the time domain enhancement of a current frame image according to the change of a current frame and a continuous reference frame in a time domain, obtaining a self-adaptive threshold value, segmenting the current frame image through the self-adaptive threshold value, and recognizing an infrared small target. Small target enhancement is carried out by introducing time domain information, false alarm caused by random noise influence is avoided, background and interferents can be inhibited, and infrared small target detection accuracy is improved. In addition, the method has little dependence on parameter adjustment and is high in robustness and adaptability.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, device, equipment, medium and program product for detecting small infrared targets. Background Art

[0002] With the continuous development and progress of science and technology and manufacturing level, the low-altitude aircraft industry has been booming, and it has also brought huge safety risks. For example, in the military field, small drones flying at low altitudes are widely used in reconnaissance, surveillance and attack missions on modern battlefields. Their low flight altitude, small size and small radar reflection area pose a major challenge to traditional defense systems; stealth targets such as cruise missiles and gliding ammunition flying at low altitudes can evade radar detection and have strong penetration capabilities. In the civilian field, drones are currently highly popular in logistics, agriculture, photography and other fields, and the risks of illegal flights and intrusions into key areas (such as airports, government buildings, etc.) are gradually increasing. Therefore, it is necessary to effectively detect and regulate such low-altitude aircraft. The current detection methods mainly include visible light detection, radar detection, acoustic detection, radio detection, laser detection and infrared detection.

[0003] Infrared detection is based on the thermal radiation characteristics of objects. Any object with a temperature above absolute zero will emit infrared radiation. Through infrared imaging equipment, the difference in thermal radiation between the target and the background can be captured to identify the target. Because it does not rely on visible light, it can still work effectively at night and in bad weather conditions (such as smoke and mist), and has strong anti-interference ability. As a result, infrared small target detection has a wide range of applications in military fields such as missile early warning, stealth target detection, battlefield surveillance, and civilian fields such as "black flying" drone detection and natural disaster detection.

[0004] The single-frame infrared small target detection method is a technology that relies only on single-frame image information for target detection. It separates the target from the background by analyzing the grayscale, texture or contrast characteristics in the spatial domain. However, since small infrared targets usually have low signal-to-noise ratio and weak contrast, their detection tasks face significant challenges when facing complex backgrounds and random noise.

[0005] The shortcomings of the single-frame method are mainly reflected in the following aspects:

[0006] First, the single-frame method is easily affected by complex background interference. In natural scenes, infrared images may contain clutter such as clouds and ground thermal radiation that are similar to the target characteristics. These background noises may be mistaken for targets, resulting in a high false alarm rate. At the same time, the single-frame method has limited detection capabilities for weak targets under low signal-to-noise ratio conditions. When the target signal is weak, its characteristics are easily masked by background noise, resulting in frequent missed detections.

[0007] Secondly, small infrared targets usually have continuity of motion, but single-frame detection only uses image information at a single moment, thus ignoring the trajectory and dynamic changes of the target between consecutive frames. Such limitations make the single-frame method unable to effectively distinguish between real targets with motion characteristics and strong edge interference from random noise, false targets or static background.

[0008] In addition, the single-frame method relies heavily on parameter adjustment, and often requires manual adjustment of parameters in different scenarios, lacking robustness and adaptability. Summary of the invention

[0009] The present application proposes a method, device, equipment, medium and program product for detecting small infrared targets, which can solve the problem of low accuracy of detecting small infrared targets in a single frame.

[0010] In order to achieve the above objectives, this application adopts the following technical solutions:

[0011] In a first aspect, a method for detecting a small infrared target is provided, the method comprising:

[0012] Obtaining a current frame image and a plurality of reference frame images that are continuous with the current frame in the time domain;

[0013] According to the change in the time domain between the current frame image and the reference frame image and the image spatial domain information of the current frame, the current frame image is enhanced in the time and space domains to obtain an adaptive threshold for segmenting the current frame image; and

[0014] The current frame image is segmented using the adaptive threshold to identify the small infrared target.

[0015] Based on the above technical scheme, the current frame image is enhanced in the time domain according to the changes between the current frame and the continuous reference frames in the time domain, and an adaptive threshold is obtained. The current frame image is segmented using the adaptive threshold to identify small infrared targets. In this way, by introducing time domain information to enhance small targets, false alarms caused by random noise are avoided, and the background and interference can be suppressed, thereby improving the accuracy of infrared small target detection. In addition, the above method has less dependence on parameter adjustment and has high robustness and adaptability.

[0016] In a possible design manner of the first aspect, according to a change between the current frame image and the reference frame image in the time domain, performing time domain enhancement on the current frame image specifically includes:

[0017] Based on the grayscale value change between the current frame image and the reference frame image, performing a first time domain enhancement on the current frame image; and / or

[0018] Based on the feature changes between the current frame image and the reference frame image that meets the limited number requirements, the current frame image is subjected to a second time domain enhancement.

[0019] In a possible design manner of the first aspect, based on a grayscale value change between the current frame image and the reference frame image, performing a first time domain enhancement on the current frame image is specifically:

[0020]

[0021] Among them, the image sequence used in the time domain enhancement scheme is (f t-Pnum ,…,f t-1 ,f t ,f t+1 ,…,f t+Pnum ), represents the first enhanced image in the time domain, exp() represents the exponential function, abs() represents the absolute value function, and max() represents the maximum value function.

[0022] Based on the above technical solution, while acquiring the target time domain enhancement information, the above calculation formula is used to solve the problem of detection position dispersion caused by the displacement of small targets in adjacent frame images.

[0023] In a possible design of the first aspect, while performing a first enhancement in the time domain on the current frame image, the current frame image is enhanced in the spatial domain.

[0024] In a possible design manner of the first aspect, based on a feature change between the current frame image and the reference frame image that meets a limited number of requirements, performing a second temporal enhancement on the current frame image specifically includes:

[0025] Extracting features from the structure tensor corresponding to the current frame image and the reference frame image that meets the limited number requirement;

[0026] Based on the features, obtaining a background suppression map based on the structure tensor; and

[0027] Performing front-to-back difference processing on the background suppression map corresponding to the current frame image and the reference frame image to obtain a first enhancement factor.

[0028] In a possible design manner of the first aspect, based on a feature change between the current frame image and the reference frame image that meets a limited number of requirements, performing a second temporal enhancement on the current frame image further includes:

[0029] The current frame image is traversed using a Gaussian filter to obtain a second enhancement factor.

[0030] The second enhancement factor is combined with the first enhancement factor as a joint enhancement factor for time-domain second enhancement.

[0031] In a second aspect, an infrared small target detection device is provided, the device comprising:

[0032] An acquisition unit, used to obtain a current frame image and a plurality of reference frame images that are continuous with the current frame in the time domain;

[0033] an enhancement unit, configured to enhance the current frame image in the time domain according to a change between the current frame image and the reference frame image in the time domain, so as to obtain an adaptive threshold for segmenting the current frame image; and

[0034] The segmentation unit is used to segment the current frame image with the adaptive threshold to identify the small infrared target.

[0035] In a third aspect, an electronic device is provided, comprising: a processor, and a memory coupled to the processor, the memory being used to store a computer program; the processor being used to execute the computer program stored in the memory, so that the electronic device performs the infrared small target detection method as any possible implementation method in the first aspect.

[0036] In a fourth aspect, a computer-readable storage medium is provided, comprising a computer program or instructions, which, when executed on a computer, enables the computer to execute the infrared small target detection method of any possible implementation of the first aspect.

[0037] In a fifth aspect, a computer program product is provided, comprising: a computer program or instructions, which, when executed on a computer, enables the computer to execute the infrared small target detection method of any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0039] Figure 1 It is a flow chart of the algorithm provided in the embodiment of the present application;

[0040] Figure 2 It is a schematic diagram of a ring structure provided in an embodiment of the present application;

[0041] Figure 3 It is a schematic diagram of an improved small target enhancement filtering operator provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0045] like Figure 1 As shown, an embodiment of the present application provides a method for detecting small infrared targets, which mainly includes two parts: annular local contrast calculation based on spatiotemporal enhancement, and spatiotemporal weighting scheme design based on target structure and motion continuity.

[0046] 1. Annular local contrast based on spatiotemporal enhancement

[0047] Considering that using only a single frame image for small target enhancement is easily affected by random noise and thus leads to false alarms, this scheme further utilizes the time domain information of the image sequence to enhance the small target. The specific steps are as follows: (1) perform spatial domain small target enhancement on the current frame image; (2) use the previous and next multiple frames of images to perform temporal domain small target enhancement; (3) combine the spatial and temporal enhancement results to obtain the final enhanced image.

[0048] Small infrared targets have the following Gaussian-like distribution characteristics:

[0049]

[0050] Considering the highly symmetrical nature of the distribution, constructing a ring window to calculate the single-frame local contrast of small targets can achieve good detection results.

[0051] Among them, (x 0 ,y 0) represents the center pixel coordinate of the target, (x, y) is the image pixel coordinate, α is the coefficient, reflecting the radiation intensity of the current target, and σ represents the variance of the two-dimensional Gaussian distribution.

[0052] like Figure 2 The window is a ring structure, where Ω C is the central area, its size is 3*3, is the outer annular background area, and its expression is:

[0053]

[0054] Among them, (x, y) is the central pixel coordinate of the local structure, (p, q) is the pixel coordinate of the ring area, k represents the serial number of the peripheral ring area, and L represents the total number of rings. Represents the Euclidean distance between pixels.

[0055] Let the image at time t be f t , for the airspace enhancement scheme, the specific operations are:

[0056]

[0057] in, Represents image f t Spatial domain enhancement of small target images, Represents the image f t When traversing, the structure window corresponds to the central area Ω C The nth largest pixel value, express The weighting factor of N represents the number of pixels used.

[0058] Assume that the image sequence used in the time domain enhancement scheme is (f t-Pnum ,…,f t-1 ,f t ,f t+1 ,…,f t+Pnum ), where Pnum represents the number of image frames before and after the current frame taken in the calculation. For a pixel point time domain sequence {(x t-Pnum ,y t-Pnum ),…,(x t ,y t ),…(x t+Pnum ,y t+Pnum)}, if the sequence is a target pixel sequence, due to the strong change of grayscale value, the sequence will have a peak with a large time domain span in the time domain; if the sequence is a background pixel sequence, due to the slow change of grayscale value in the time domain, the sequence has small fluctuations; if the sequence is a background pixel sequence, but there is a little random noise, the sequence will have a peak similar to an impulse response; if the sequence is a highlight interference pixel sequence, since such noise is often caused by internal noise or bad pixels of the sensor, there is basically no grayscale fluctuation. Based on this, in order to highlight small targets and suppress the small target diffusion effect caused by the current pixel being a background pixel and the target pixel in the previous and next frames in the sequence, the following time domain enhancement scheme is constructed:

[0059]

[0060] in, It represents the time domain enhanced image, exp() represents the exponential function, abs() represents the absolute value function, and max() represents the maximum value function.

[0061] Get the final spatiotemporal enhanced small target image for:

[0062]

[0063] After obtaining the spatiotemporal enhancement image of the target, the background image needs to be estimated. The maximum gray value is selected as the background estimation value of the jth (0<j<L) ring window of the current structure window, which is recorded as In this way, the background estimation values ​​of L ring windows are calculated in sequence to obtain the sequence The final background estimate is obtained as follows:

[0064]

[0065] Finally, according to the obtained background estimation image and small target enhanced image, the annular local contrast based on spatiotemporal enhancement is calculated:

[0066]

[0067] Where ⊙ represents the Hadamard product of the matrix.

[0068] 2. Spatiotemporal weighting scheme based on target structure and motion continuity

[0069] The annular local contrast based on spatiotemporal enhancement can well enhance small targets in the image, but it has poor suppression ability for interference objects in the ground background that are very similar to the target (such as boulders and man-made buildings in the grass background under the scorching sun). However, such interference objects are static objects and there is basically no strong grayscale value change in the imaging sequence. Based on this, this scheme designs a spatiotemporal weighting factor based on the target structure and motion continuity. The specific implementation steps of this scheme are: (1) background and clutter suppression based on image characteristics; (2) calculation of weighting factors for small moving targets based on the improved multi-frame difference method.

[0070] In the background and clutter suppression based on image characteristics, considering that the structure tensor of the image can better describe the image region type according to its eigenvalue, this scheme uses it to enhance small targets and suppress background and clutter at the same time. The structure tensor (ST) is a 2*2 matrix constructed by calculating the four second-order derivatives of each pixel, which is defined as:

[0071] Among them, g x ,g y are the gradients of the current pixel in the x and y directions, G σ represents the Gaussian blur function, σ is the standard deviation of the Gaussian kernel, T 11 =g x g x T ,T 12 =g x g y T ,T 21 =g y g x T ,T 22 =g y g y T , then the two eigenvalues ​​of the structure tensor matrix are:

[0072] Then, for the image f t For , its background suppression map based on the structure tensor is:

[0073]

[0074] The use of structure tensor can effectively suppress most of the background in the image, but its ability to suppress some strong edges and corners is still a little bit improved. Based on this, this scheme designs a small target highlighting scheme based on image block gradient and Gaussian filtering. Considering the Gaussian-like distribution of small targets in the image, Gaussian filtering and calculating the multi-directional gradient of the image can well highlight the small targets in the image. Secondly, considering the existence of random noise and highlight noise, the use of traditional pixel-based gradient methods may introduce unnecessary random errors. Therefore, this scheme uses a multi-directional image block gradient method, combined with a Gaussian filter template, to design Figure 3 Based on this filtering template, the image is traversed to obtain the current image frame f t Small target enhancement results:

[0075] T t stren =imfilter(f t ,se) (11)

[0076] Among them, imfilter() represents the filter function, se is Figure 3 The rightmost filter structure element is a 5*5 structure element operator. The values ​​of the first row are -1, -1, 0, -1, -1; the values ​​of the second row are -1, 0, 3, 0, -1; the values ​​of the third row are 0, 3, 8, 3, 0; the values ​​of the fourth row are -1, 0, 3, 0, -1; and the values ​​of the fifth row are -1, -1, 0, -1, -1.

[0077] Based on the above two steps, we get the image f t Enhanced image after background and clutter suppression t stren for:

[0078]

[0079] According to the above method, the image sequence {f 1 ,…,f j ,…,f Num}(Num is the length of the image sequence) corresponding to the enhanced image sequence Based on the enhanced image sequence, the multi-frame difference method can be used to obtain the information of small moving targets in the image. Since small targets occupy fewer pixels and most shooting scenes are shot at a long distance, the movement between frames may not be large. Therefore, directly using the traditional frame difference method may cause the target to be lost when processing some frames. Therefore, this scheme specifically designs a five-frame difference algorithm suitable for small target detection.

[0080] For the image f to be processed at time t t stren, take the two frames of images before and after for difference processing, specifically:

[0081]

[0082] According to the differential results obtained above, calculate the spatiotemporal weighting factor W of the target t for

[0083]

[0084] Among them, normal() represents the normalization function.

[0085] Finally, the annular local contrast based on spatiotemporal prior weighting is obtained as:

[0086] WSTRLCM t =W t ⊙STRLCM t (15)

[0087] According to WSTRLCM t The adaptive segmentation threshold can be obtained as:

[0088] Th=βmax(WSTRLCM t )+(1-β)mean(WSTRLCM t ) (16)

[0089] Where β is a constant between 0 and 1, max( ) represents the maximum pixel value in the image, and mean( ) represents the mean pixel value in the image.

[0090] Finally, according to formula (14) and formula (15), the final detection map is obtained as follows:

[0091]

[0092] Among them, seg BW ( ) represents a binary segmentation function.

[0093] The present application also provides an infrared small target detection device, the device comprising:

[0094] An acquisition unit, used to obtain a current frame image and a plurality of reference frame images that are continuous with the current frame in the time domain;

[0095] an enhancement unit, configured to enhance the current frame image in the time domain according to a change between the current frame image and the reference frame image in the time domain, so as to obtain an adaptive threshold for segmenting the current frame image; and

[0096] The segmentation unit is used to segment the current frame image with the adaptive threshold to identify the small infrared target.

[0097] An embodiment of the present application also provides an electronic device, comprising: a processor, and a memory coupled to the processor, wherein the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory, so that the electronic device executes a method as described in any one of the above embodiments.

[0098] The electronic device may be a computing device such as a desktop computer, a notebook, a palmtop computer, a cloud server, etc. The electronic device may include, but is not limited to, a processor and a memory.

[0099] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, and various interfaces and lines are used to connect various parts of the entire device.

[0100] The memory may be used to store the computer program, and the processor implements various functions of the electronic device by running or executing the computer program stored in the memory and calling the data stored in the memory.

[0101] The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the mobile phone, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0102] The embodiment of the present application also provides a storage medium, the storage medium is a computer-readable storage medium, the computer program is stored in the computer-readable storage medium, and the computer program, when executed by the processor, can implement the steps of each of the above-mentioned method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0103] An embodiment of the present application further provides a computer program product, including: a computer program or instructions, which, when executed on a computer, enables the computer to execute any of the above-mentioned possible implementation methods.

[0104] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications are also considered to be within the scope of protection of the present application.

Claims

1. A method for detecting small infrared targets, characterized in that: The method comprises: Obtaining a current frame image and a plurality of reference frame images that are continuous with the current frame in the time domain; According to the change in the time domain between the current frame image and the reference frame image and the image spatial domain information of the current frame, the current frame image is enhanced in the time and space domains to obtain an adaptive threshold for segmenting the current frame image; and The current frame image is segmented using the adaptive threshold to identify the small infrared target.

2. The method according to claim 1, characterized in that According to the change between the current frame image and the reference frame image in the time domain, the current frame image is enhanced in the time domain, specifically comprising: Based on the grayscale value change between the current frame image and the reference frame image, performing a first time domain enhancement on the current frame image; and / or Based on the feature changes between the current frame image and the reference frame image that meets the limited number requirements, the current frame image is subjected to a second time domain enhancement.

3. The method according to claim 2, characterized in that Based on the grayscale value change between the current frame image and the reference frame image, the current frame image is first enhanced in the time domain, specifically: Among them, the image sequence used in the time domain enhancement scheme is (f t-Pnum ,…,f t-1 ,f t ,f t+1 ,…,f t+Pnum ), represents the first enhanced image in the time domain, exp() represents the exponential function, abs() represents the absolute value function, and max() represents the maximum value function.

4. The method according to claim 3, characterized in that While performing a first time domain enhancement on the current frame image, the current frame image is also enhanced in the space domain.

5. The method according to claim 2, characterized in that Based on the feature changes between the current frame image and the reference frame image that meets the limited number requirement, performing a second time domain enhancement on the current frame image specifically includes: Extracting features from the structure tensor corresponding to the current frame image and the reference frame image that meets the limited number requirement; Based on the features, obtaining a background suppression map based on the structure tensor; and Performing front-to-back difference processing on the background suppression map corresponding to the current frame image and the reference frame image to obtain a first enhancement factor.

6. The method according to claim 5, characterized in that Based on the feature changes between the current frame image and the reference frame image that meets the limited number requirement, performing a second time domain enhancement on the current frame image, further comprising: The current frame image is traversed using a Gaussian filter to obtain a second enhancement factor. The second enhancement factor is combined with the first enhancement factor as a joint enhancement factor for time-domain second enhancement.

7. An infrared small target detection device, characterized in that: The device comprises: An acquisition unit, used to obtain a current frame image and a plurality of reference frame images that are continuous with the current frame in the time domain; an enhancement unit, configured to enhance the current frame image in the time domain according to a change between the current frame image and the reference frame image in the time domain, so as to obtain an adaptive threshold for segmenting the current frame image; and The segmentation unit is used to segment the current frame image with the adaptive threshold to identify the small infrared target.

8. An electronic device, characterized in that: The electronic device comprises: a processor, and a memory coupled to the processor, The memory is used to store a computer program; and The processor is configured to execute the computer program stored in the memory, so that the electronic device executes the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a computer program or instructions. When the computer program or instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 8.

10. A computer program product, characterized in that The computer program product comprises: a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to perform the method according to any one of claims 1 to 8.

Images