Infrared image weak target enhancement and real-time tracking system for complex background

Abstract

The present application discloses a system for enhancing and real-time tracking of weak targets in infrared images with complex backgrounds, which relates to the field of image processing technology. The present application first obtains the original infrared image through the infrared imaging device of the acquisition module. Secondly, the determination module determines a query template adapted to the target according to its morphology on the infrared image to improve the efficiency of whole-plate query on the infrared image. Then, the enhancement module uses the determined query template to query the original infrared image. During the query process, the pixel values ​​of the infrared image under the template are transformed to enhance the weak targets and highlight the feature information they carry. Finally, the image is filtered by the filtering module to eliminate the noise missed when suppressing the complex background, thereby improving the detection signal-to-noise ratio of the weak targets, significantly improving their detectability, and realizing the effective extraction of feature information of the weak targets on the infrared image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a system for enhancing and real-time tracking of small targets in infrared images with complex backgrounds. Background Art

[0002] Infrared imaging technology holds a promising future. Electromagnetic waves longer than 0.78 microns, located outside the red region of the visible light spectrum, are called infrared rays, also known as infrared radiation. These waves range from 0.78 to 1000 microns in wavelength, with the wavelength between 0.78 and 2.0 microns considered near-infrared and the wavelength between 2.0 and 1000 microns considered thermal infrared. All objects in nature radiate infrared radiation. Therefore, using a detector to measure the infrared radiation difference between a target and its background can produce an infrared image formed by varying thermal infrared radiation.

[0003] Small targets captured on infrared images are difficult to identify because their pixels account for a low proportion of the pixels in the entire image. In addition, their low signal-to-noise ratio makes them easily obscured by complex background noise, increasing the difficulty of extracting effective feature information. Summary of the Invention

[0004] The main purpose of this application is to provide a system for enhancing and real-time tracking of small targets in infrared images with complex backgrounds, aiming to solve the problem in the prior art that small targets in infrared images are difficult to extract effectively.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present application are as follows:

[0006] In a first aspect, an embodiment of the present application provides an infrared image small target enhancement system for complex backgrounds, comprising:

[0007] An acquisition module is used to obtain original infrared images based on infrared imaging equipment;

[0008] a determination module, for determining the target shape of the query template according to the morphology of the target features on the original infrared image;

[0009] An enhancement module is used to transform the pixel values ​​of the original infrared image based on the query template to enhance the target features and obtain an enhanced infrared image;

[0010] The filtering module is used to filter the enhanced infrared image to obtain the target infrared image.

[0011] In a possible implementation of the first aspect, the enhancement system further includes:

[0012] A strategy module is used to transform the pixel values ​​of the original infrared image based on the query template to enhance the target features and obtain the enhanced infrared image;

[0013] The relative brightness of the background and target features of the original infrared image is judged, and the enhancement strategy is determined based on the judgment result.

[0014] In a possible implementation of the first aspect, the enhancement module includes:

[0015] The enhancement unit is used to transform the pixel values ​​of the original infrared image based on the query template according to the enhancement strategy to enhance the target features and obtain an enhanced infrared image; wherein the enhancement strategy includes a first enhancement strategy and a second enhancement strategy, the first enhancement strategy corresponds to the judgment result that the brightness of the background of the original infrared image is less than the brightness of the target features, and the second enhancement strategy corresponds to the judgment result that the brightness of the background of the original infrared image is greater than the brightness of the target features.

[0016] In a possible implementation of the first aspect, the enhancement unit includes:

[0017] A first enhancement unit is configured to perform a first matching query on the original infrared image using the query template as a sliding window according to a first enhancement strategy;

[0018] During the first matching query, the pixel values ​​of the area of ​​the original infrared image covered by the sliding window are replaced with the minimum pixel value in the sliding window area to obtain a first infrared image;

[0019] Performing a second matching query on the first infrared image using the query template as a sliding window;

[0020] During the second matching query, the pixel values ​​of the area of ​​the first infrared image covered by the sliding window are replaced with the maximum value of the pixels in the area of ​​the sliding window to obtain a second infrared image;

[0021] The second infrared image is subtracted from the original infrared image to enhance the target features and obtain an enhanced infrared image.

[0022] In a possible implementation of the first aspect, the enhancement unit includes:

[0023] A second enhancement unit is configured to perform a third matching query on the original infrared image using the query template as a sliding window according to the second enhancement strategy;

[0024] During the third matching query, the pixel values ​​of the area of ​​the original infrared image covered by the sliding window are replaced with the maximum value of the pixels in the area of ​​the sliding window to obtain a third infrared image.

[0025] performing a fourth matching query on the third infrared image using the query template as a sliding window;

[0026] In the fourth matching query process, the pixel values ​​of the area of ​​the third infrared image covered by the sliding window are replaced with the minimum pixel value in the area of ​​the sliding window to obtain a fourth infrared image;

[0027] The original infrared image is subtracted from the fourth infrared image to enhance the target features and obtain an enhanced infrared image.

[0028] In a possible implementation of the first aspect, the enhancement system further includes:

[0029] The fusion module is used to transform the pixel values ​​of the original infrared image based on the query template to enhance the target features and obtain an enhanced infrared image;

[0030] Perform weighted fusion on the enhanced infrared images obtained based on query templates of different scales to obtain a fused enhanced infrared image;

[0031] Perform filtering on the enhanced infrared image to obtain the target infrared image, including:

[0032] The fused enhanced infrared image is filtered to obtain the target infrared image.

[0033] In a possible implementation of the first aspect, the determining module includes:

[0034] a first determining unit, configured to determine that the target shape of the query template is a circle according to the shape of the target feature in the original infrared image being a circle;

[0035] a second determining unit, configured to determine that the target shape of the query template is a cross according to the shape of the target feature on the original infrared image being a line;

[0036] The third determining unit is configured to determine that the target shape of the query template is a rectangle according to the shape of the target feature on the original infrared image being a block.

[0037] In a possible implementation of the first aspect, the filtering module includes:

[0038] A filtering unit, used for performing Fourier transform on the enhanced infrared image to obtain a spectrum diagram;

[0039] Performing filtering on the spectrum graph to obtain a filtered image;

[0040] Perform inverse Fourier transform on the filtered image to obtain the target infrared image.

[0041] In a second aspect, an embodiment of the present application provides a real-time tracking system for infrared images of small targets in complex backgrounds, comprising:

[0042] A recognition module for performing target detection on infrared images and identifying and tracking targets; wherein the infrared images are obtained using the infrared image weak target enhancement system for complex backgrounds provided by any one of the first aspects above;

[0043] The tracking module predicts the trajectory of the tracking target based on the infrared images of consecutive frames, and tracks the target according to the predicted trajectory.

[0044] In a possible implementation of the second aspect, the real-time tracking system further includes:

[0045] The stabilization module is used after detecting the target and identifying and tracking the target in the infrared image;

[0046] Binarize the tracking target identified by the infrared images of the continuous frames to obtain a binary identification image;

[0047] The binary recognition images of consecutive frames are superimposed to obtain a heat map;

[0048] The tracking target is highlighted according to the heat map, and the trajectory is extracted based on the highlighted tracking target to obtain the movement trajectory;

[0049] The target is identified by continuous infrared image frames and its trajectory is predicted, and the target is tracked according to the predicted trajectory, including:

[0050] The trajectory is predicted based on the moving trajectory, and the target is tracked based on the predicted trajectory.

[0051] Compared with the prior art, the present invention has the following advantages:

[0052] An embodiment of the present application proposes an infrared image weak target enhancement and real-time tracking system for complex backgrounds. The enhancement system includes an acquisition module, a determination module, an enhancement module and a filtering module, wherein the acquisition module is used to obtain an original infrared image based on an infrared imaging device; the determination module is used to determine the target shape of a query template based on the morphology of the target features on the original infrared image; the enhancement module is used to perform pixel value transformation on the original infrared image based on the query template to enhance the target features and obtain an enhanced infrared image; and the filtering module is used to perform filtering processing on the enhanced infrared image to obtain a target infrared image. This application first obtains the original infrared image through the infrared imaging device of the acquisition module. Secondly, the determination module determines the query template adapted to the target according to the morphology of the target on the infrared image to improve the efficiency of whole-plate query of the infrared image. Then, the enhancement module uses the determined query template to query the original infrared image. During the query process, the pixel values ​​of the infrared image under the template are transformed to enhance the weak targets and highlight the feature information they carry. Finally, the image is filtered by the filtering module to eliminate the noise missed when suppressing the complex background, thereby improving the detection signal-to-noise ratio of the weak targets, significantly improving their detectability, and realizing the effective extraction of feature information of weak targets on the infrared image. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of a module for a system for enhancing small targets in infrared images with complex backgrounds, provided in an embodiment of the present application;

[0054] Figure 2 A schematic diagram of a module of a real-time tracking system for infrared images of small targets in complex backgrounds provided by an embodiment of the present application;

[0055] Figure 3 A schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of the present application;

[0056] Figure 4 A flowchart of a method for enhancing small targets in infrared images with complex backgrounds provided by an embodiment of the present application;

[0057] Figure 5 A flowchart of a method for real-time tracking of small targets in infrared images with complex backgrounds provided by an embodiment of the present application;

[0058] Markings in the figure: 101 - processor, 102 - communication bus, 103 - network interface, 104 - user interface, 105 - memory. DETAILED DESCRIPTION

[0059] It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present application and are not intended to limit the present application. Those skilled in the art should understand that the division of the various modules in the embodiments is merely a division of logical functions, and in actual applications, all or part of the modules may be integrated into one or more actual carriers, and these modules may be implemented entirely in the form of software called by a processing unit, entirely in the form of hardware, or in a combination of software and hardware.

[0060] Refer to the attached Figure 1 The embodiment of the present application provides an infrared image small target enhancement system for complex backgrounds, comprising:

[0061] An acquisition module is used to obtain original infrared images based on infrared imaging equipment;

[0062] a determination module, for determining the target shape of the query template according to the morphology of the target features on the original infrared image;

[0063] An enhancement module is used to transform the pixel values ​​of the original infrared image based on the query template to enhance the target features and obtain an enhanced infrared image;

[0064] The filtering module is used to filter the enhanced infrared image to obtain the target infrared image.

[0065] In this embodiment, the original infrared image is first acquired by the infrared imaging device of the acquisition module. Secondly, the determination module determines the query template adapted to the target on the infrared image according to its morphology to improve the efficiency of whole-plate query on the infrared image. Then, the enhancement module uses the determined query template to query the original infrared image. During the query process, the pixel values ​​of the infrared image under the template are transformed to enhance the weak targets and highlight the feature information they carry. Finally, the image is filtered by the filtering module to eliminate the noise missed when suppressing the complex background, thereby improving the detection signal-to-noise ratio of the weak targets, significantly improving their detectability, and realizing the effective extraction of feature information of the weak targets on the infrared image.

[0066] In the specific implementation process, infrared imaging equipment such as infrared cameras and infrared imagers capture infrared images through infrared imaging equipment, which are recorded as original infrared images. These images carry small targets with small size, little difference from the background, and weak signals, such as people, cars, drones, etc. The features of the small targets are recorded as target features. Considering that they are not easy to detect directly, the enhancement system of the present application is used to enhance them. In order to improve the efficiency of the enhancement method, a query template with a shape that matches the morphology of the target features is selected. Specifically, the determination module includes:

[0067] a first determining unit, configured to determine that the target shape of the query template is a circle according to the shape of the target feature in the original infrared image being a circle;

[0068] a second determining unit, configured to determine that the target shape of the query template is a cross according to the shape of the target feature on the original infrared image being a line;

[0069] The third determining unit is configured to determine that the target shape of the query template is a rectangle according to the shape of the target feature on the original infrared image being a block.

[0070] Target features are categorized into three types: circular, linear, and block. Blocks are similar to circular shapes, differing in that circular edges have smooth transitions, while block edges are mostly linear. Accordingly, a circular query template can evenly suppress background in all directions while preserving point-shaped targets. A cross-shaped query template is more likely to preserve linear structures, but may miss point-shaped targets. In this case, a circular query template can be used after processing with the cross-shaped query template to recheck them. The third option, a rectangular query template, has a shape that better matches the target features and offers faster processing speed.

[0071] The query template is used to query the entire infrared image, and the pixel values ​​of the area covered by the template are adjusted during the query process to achieve pixel value transformation of the original infrared image. After the transformation, the target features are enhanced. The enhanced infrared image is called the enhanced infrared image. Complex backgrounds such as background textures or low-frequency features such as clouds and ground, after the enhancement process highlights the target features, there may be omissions or new noise added during the processing on the complex background. Filtering is performed to improve the quality of the enhanced image, further suppress the complex background, and obtain the target infrared image. Specifically, the filtering module includes:

[0072] A filtering unit, used for performing Fourier transform on the enhanced infrared image to obtain a spectrum diagram;

[0073] Performing filtering on the spectrum graph to obtain a filtered image;

[0074] Perform inverse Fourier transform on the filtered image to obtain the target infrared image.

[0075] In the specific implementation process, the image is processed in the frequency domain, and the image is subjected to FTT, i.e. Fourier transform, to obtain a spectrum diagram, and a filter is used to suppress the low-frequency background. For example, under the first enhancement strategy, a high-pass filter is selected for filtering processing, and finally an inverse FTT, i.e. inverse Fourier transform, is performed to restore the spatial domain image to obtain the target infrared image, further suppress the low-frequency background, and retain the high-frequency target components.

[0076] In one embodiment, the enhancement system further comprises:

[0077] A strategy module is used to transform the pixel values ​​of the original infrared image based on the query template to enhance the target features and obtain the enhanced infrared image;

[0078] The relative brightness of the background and target features of the original infrared image is judged, and the enhancement strategy is determined based on the judgment result.

[0079] In the specific implementation process, there are two states of target features and background in the infrared image: one is that the weak target appears dim because its brightness is lower than the background, and the other is that its brightness is higher than the background and appears bright. Different enhancement strategies are formulated for different states, that is, the enhancement strategy includes a first enhancement strategy and a second enhancement strategy. The first enhancement strategy corresponds to the judgment result that the brightness of the background of the original infrared image is lower than the brightness of the target feature, and the second enhancement strategy corresponds to the judgment result that the brightness of the background of the original infrared image is higher than the brightness of the target feature. The enhancement strategy is used to guide the process operation of the enhancement module, that is, the enhancement module includes:

[0080] The enhancement unit is configured to transform the pixel values ​​of the original infrared image based on the query template according to the enhancement strategy to enhance the target features and obtain an enhanced infrared image; wherein the enhancement strategy includes a first enhancement strategy and a second enhancement strategy, the first enhancement strategy corresponding to the judgment result that the brightness of the background of the original infrared image is less than the brightness of the target features, and the second enhancement strategy corresponding to the judgment result that the brightness of the background of the original infrared image is greater than the brightness of the target features. It should be noted that the enhanced image provided in the embodiment of the present application uses image operations, which can grayscale the original infrared image to improve the computational efficiency when performing enhancement according to the enhancement strategy.

[0081] Specifically, under the first enhancement strategy, the enhancement unit includes:

[0082] A first enhancement unit is configured to perform a first matching query on the original infrared image using the query template as a sliding window according to a first enhancement strategy;

[0083] During the first matching query, the pixel values ​​of the area of ​​the original infrared image covered by the sliding window are replaced with the minimum pixel value in the sliding window area to obtain a first infrared image;

[0084] Performing a second matching query on the first infrared image using the query template as a sliding window;

[0085] During the second matching query, the pixel values ​​of the area of ​​the first infrared image covered by the sliding window are replaced with the maximum value of the pixels in the area of ​​the sliding window to obtain a second infrared image;

[0086] The second infrared image is subtracted from the original infrared image to enhance the target features and obtain an enhanced infrared image.

[0087] In the specific implementation process, the first enhancement strategy is defined as ,in, I is the original infrared image, B For the query template, The first operation is as follows: when the query template is used to perform the first matching query, the query template slides over the image, and each pixel value is replaced by the minimum value in the neighborhood, that is, the area covered by the query template. In this way, isolated bright spots, that is, weak targets, are eliminated and the background is smoothed. The image after the first matching query is recorded as the first infrared image, and then the above matching query operation is performed again with the query template as the sliding window. The difference is that during this matching, each pixel value is replaced by the maximum value in the neighborhood to restore the background structure, and the weak targets cannot be restored because they are eliminated. The image after the second matching query is recorded as the second infrared image. Finally, the original infrared image and the second infrared image are used to perform image subtraction, so that the weak targets lost in the above process are retained, that is, the high-frequency components are retained, and the low-frequency components of the background are suppressed, so that the weak targets are enhanced and the target features are strengthened and displayed. The final enhanced image is recorded as the enhanced infrared image.

[0088] Under the second enhancement strategy, the enhancement unit includes:

[0089] A second enhancement unit is configured to perform a third matching query on the original infrared image using the query template as a sliding window according to the second enhancement strategy;

[0090] During the third matching query, the pixel values ​​of the area of ​​the original infrared image covered by the sliding window are replaced with the maximum value of the pixels in the area of ​​the sliding window to obtain a third infrared image.

[0091] performing a fourth matching query on the third infrared image using the query template as a sliding window;

[0092] In the fourth matching query process, the pixel values ​​of the area of ​​the third infrared image covered by the sliding window are replaced with the minimum pixel value in the area of ​​the sliding window to obtain a fourth infrared image;

[0093] The original infrared image is subtracted from the fourth infrared image to enhance the target features and obtain an enhanced infrared image.

[0094] In the specific implementation process, the second enhancement strategy is defined as ,in, I is the original infrared image, B For the query template, The second operation is similar to the first operation, except that the nodes selected for the maximum and minimum values ​​are swapped. That is, when the query template is used to perform the third matching query, the query template slides across the image, and each pixel value is replaced by the maximum value in the neighborhood, that is, the area covered by the query template. The image after the third matching query is recorded as the third infrared image. Then, the above matching query operation is performed again using the query template as a sliding window. The difference is that during this matching, each pixel value is replaced by the minimum value in the neighborhood. In this way, dark areas smaller than the query template are enhanced, and smooth or bright areas in the background are suppressed to fill the dark areas inside the target. The image after the fourth matching query is recorded as the fourth infrared image. Finally, the fourth infrared image is subtracted from the original infrared image to retain the dark areas filled in the above process. That is, the weak targets with high-frequency components are retained, while the low-frequency background components are suppressed, so that the weak targets are enhanced and the target features are displayed in a strengthened manner. The final enhanced image is recorded as the enhanced infrared image.

[0095] In one embodiment, the enhancement system further comprises:

[0096] The fusion module is used to transform the pixel values ​​of the original infrared image based on the query template to enhance the target features and obtain an enhanced infrared image;

[0097] Perform weighted fusion on the enhanced infrared images obtained based on query templates of different scales to obtain a fused enhanced infrared image;

[0098] Perform filtering on the enhanced infrared image to obtain the target infrared image, including:

[0099] The fused enhanced infrared image is filtered to obtain the target infrared image.

[0100] In the specific implementation process, the size of the query template directly affects the enhancement effect. If the size is too large, missed detection or lost targets may occur. If the size is too small, noise may not be suppressed. The actual size of weak targets may change with distance and resolution. Therefore, query templates of different sizes can be used for enhancement. For example, when using a rectangular query template, query templates of 3*3, 5*5, and 7*7 sizes can be used. When using a circular query template, circular templates with radii of 3, 5, and 7 can be used. Query templates of different scales are used to enhance infrared images respectively. After processing, multiple enhanced infrared images are weighted fused. Background noise is suppressed on average at multiple scales, and the real target is retained due to continuous enhancement. After fusion, a wider target size range is covered, which improves detection robustness and improves stability while retaining sensitivity.

[0101] Refer to the attached Figure 2Based on the same inventive concept as in the aforementioned embodiment, the present embodiment further provides a real-time tracking system for infrared images of small targets in complex backgrounds, comprising:

[0102] A recognition module for detecting targets in infrared images and identifying and tracking targets; wherein the infrared images are obtained using the infrared image weak target enhancement system for complex backgrounds provided by an embodiment of the present application;

[0103] The tracking module predicts the trajectory of the tracking target based on the infrared images of consecutive frames, and tracks the target according to the predicted trajectory.

[0104] During the specific implementation process, the beneficial effects of this embodiment can refer to the effects of the aforementioned enhancement system. In this embodiment, the infrared image obtained by the enhancement system provided by the embodiment of the present application can be identified more completely and accurately because the weak targets captured thereon are enhanced. After being extracted by the target detection algorithm, the trajectory prediction of the target identified in the continuous frames is combined to achieve real-time tracking of the target.

[0105] In one embodiment, the real-time tracking system further comprises:

[0106] The stabilization module is used after detecting the target and identifying and tracking the target in the infrared image;

[0107] Binarize the tracking target identified by the infrared images of the continuous frames to obtain a binary identification image;

[0108] The binary recognition images of consecutive frames are superimposed to obtain a heat map;

[0109] The tracking target is highlighted according to the heat map, and the trajectory is extracted based on the highlighted tracking target to obtain the movement trajectory;

[0110] The target is identified by continuous infrared image frames and its trajectory is predicted, and the target is tracked according to the predicted trajectory, including:

[0111] The trajectory is predicted based on the moving trajectory, and the target is tracked based on the predicted trajectory.

[0112] During implementation, to improve the stability and noise resistance of small target tracking, a stabilization module was set up to process images by fusing multi-frame detection results in the temporal dimension. The acquired multi-frame images were first binarized to obtain the target detection results. The detection results of multiple consecutive frames were then weighted and superimposed to generate a heat map. Since the true target appears continuously in consecutive frames, while noise is random and transient, the highlighted expression is the true target, which is the target to be tracked, while transient noise is suppressed. Based on the more realistic and accurate expression of the target, the trajectory of the target's movement is obtained, and finally, trajectory prediction is performed based on this trajectory, achieving more accurate real-time tracking.

[0113] Refer to the attached Figure 3 , attached Figure 3 This is a schematic diagram of the structure of an electronic device of the hardware operating environment involved in the embodiment of the present application. The electronic device may include: a processor 101, such as a central processing unit (CPU), a communication bus 102, a user interface 104, a network interface 103, and a memory 105. Among them, the communication bus 102 is used to realize the connection and communication between these components. The user interface 104 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and optionally the user interface 104 may also include a standard wired interface and a wireless interface. The network interface 103 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 105 may optionally be a storage device independent of the aforementioned processor 101. The memory 105 may be a high-speed random access memory (RAM) memory, or a stable non-volatile memory (NVM), such as at least one disk storage. The processor 101 may be a general-purpose processor, including a central processing unit, a network processor, etc., or may be a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0114] Those skilled in the art will understand that Figure 3 The structure shown in the figure does not constitute a limitation to the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0115] As attached Figure 3 As shown, the memory 105 as a storage medium may include an operating system, a network communication module, a user interface module, and an enhancement system / real-time tracking system.

[0116] In the attached Figure 3 In the electronic device shown, the network interface 103 is mainly used for data communication with the network server; the user interface 104 is mainly used for data interaction with the user; the processor 101 and the memory 105 in this application can be set in the electronic device, and the electronic device calls the enhancement system / real-time tracking system stored in the memory 105 through the processor 101 and executes the enhancement method / real-time tracking method provided in the embodiment of the present application.

[0117] Based on the hardware devices of the aforementioned embodiments and the same inventive concept as in the aforementioned embodiments, the embodiments of the present application further provide a method for enhancing weak targets in infrared images with complex backgrounds, as shown in the attached Figure 4 As shown, the following steps are included:

[0118] S10: Obtaining an original infrared image according to the infrared imaging device;

[0119] S20: determining the target shape of the query template according to the morphology of the target feature on the original infrared image;

[0120] S30: Based on the query template, pixel values ​​of the original infrared image are transformed to enhance target features and obtain an enhanced infrared image;

[0121] S40: Filtering the enhanced infrared image to obtain a target infrared image.

[0122] Based on the hardware devices of the aforementioned embodiments and the same inventive concept as in the aforementioned embodiments, the embodiments of the present application further provide a method for real-time tracking of small targets in infrared images with complex backgrounds, as shown in the attached Figure 5 As shown, the following steps are included:

[0123] S50: Target detection is performed on the infrared image to identify and track the target; wherein the infrared image is obtained using the infrared image weak target enhancement system for complex background provided by an embodiment of the present application;

[0124] S60: Predicting the trajectory of the tracking target identified based on the infrared images of the consecutive frames, and tracking the target based on the predicted trajectory.

[0125] It should be noted that the steps in the enhancement method / real-time tracking method in this embodiment correspond one-to-one to the modules in the enhancement system / real-time tracking system in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the aforementioned implementation and will not be repeated here.

[0126] Based on the same inventive concept as in the aforementioned embodiments, an embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, it implements a method for enhancing weak targets in infrared images with complex backgrounds / a method for real-time tracking weak targets in infrared images with complex backgrounds as provided in the embodiments of the present application.

[0127] Based on the same inventive concept as in the aforementioned embodiments, an embodiment of the present application also provides an electronic device, including a processor and a memory, wherein the memory is used to store a computer program; the processor is used to load and execute the computer program so that the electronic device can perform a method for enhancing weak targets in infrared images with complex backgrounds / a method for real-time tracking weak targets in infrared images with complex backgrounds as provided in an embodiment of the present application.

[0128] In some embodiments, the computer-readable storage medium may be a memory device such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface mount memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. The computer may be various computing devices including smart terminals and servers.

[0129] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0130] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0131] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0132] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0133] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0134] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a multimedia terminal device (which can be a mobile phone, a computer, a television receiver, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0135] In summary, the embodiment of the present application provides an infrared image weak target enhancement and real-time tracking system for complex backgrounds, and the enhancement system includes an acquisition module, a determination module, an enhancement module and a filtering module, wherein the acquisition module is used to obtain the original infrared image based on the infrared imaging device; the determination module is used to determine the target shape of the query template based on the morphology of the target features on the original infrared image; the enhancement module is used to perform pixel value transformation on the original infrared image based on the query template to enhance the target features and obtain an enhanced infrared image; the filtering module is used to filter the enhanced infrared image to obtain a target infrared image. This application first obtains the original infrared image through the infrared imaging device of the acquisition module. Secondly, the determination module determines the query template adapted to the target according to the morphology of the target on the infrared image to improve the efficiency of whole-plate query of the infrared image. Then, the enhancement module uses the determined query template to query the original infrared image. During the query process, the pixel values ​​of the infrared image under the template are transformed to enhance the weak targets and highlight the feature information they carry. Finally, the image is filtered by the filtering module to eliminate the noise missed when suppressing the complex background, thereby improving the detection signal-to-noise ratio of the weak targets, significantly improving their detectability, and realizing the effective extraction of feature information of weak targets on the infrared image.

[0136] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A system for enhancing small targets in infrared images with complex backgrounds, characterized in that: include: An acquisition module is used to obtain original infrared images based on infrared imaging equipment; a determination module, configured to determine a target shape of a query template according to the morphology of target features on the original infrared image; an enhancement module, configured to perform pixel value transformation on the original infrared image based on the query template to enhance the target features and obtain an enhanced infrared image; A filtering module, configured to filter the enhanced infrared image to obtain a target infrared image; The enhancement system further comprises: A strategy module is used for performing pixel value transformation on the original infrared image based on the query template to enhance the target features and obtain an enhanced infrared image; Determining the relative brightness between the background of the original infrared image and the target feature, and determining an enhancement strategy based on the determination result; The enhancement module includes: an enhancement unit, configured to perform pixel value transformation on the original infrared image based on the query template according to the enhancement strategy, so as to enhance the target feature and obtain an enhanced infrared image; wherein the enhancement strategy includes a first enhancement strategy and a second enhancement strategy, the first enhancement strategy corresponding to a judgment result that the brightness of the background of the original infrared image is less than the brightness of the target feature, and the second enhancement strategy corresponding to a judgment result that the brightness of the background of the original infrared image is greater than the brightness of the target feature; The enhancement unit comprises: a first enhancement unit, configured to perform a first matching query on the original infrared image using the query template as a sliding window according to the first enhancement strategy; During the first matching query, pixel values ​​of the area of ​​the original infrared image covered by the sliding window are replaced with the minimum pixel value in the area of ​​the sliding window to obtain a first infrared image; Performing a second matching query on the first infrared image using the query template as a sliding window; During the second matching query, the pixel values ​​of the area of ​​the first infrared image covered by the sliding window are replaced with the maximum value of the pixels in the area of ​​the sliding window to obtain a second infrared image; the original infrared image; The enhancement unit comprises: a second enhancement unit, configured to perform a third matching query on the original infrared image using the query template as a sliding window according to the second enhancement strategy; During the third matching query, the pixel values ​​of the area of ​​the original infrared image covered by the sliding window are replaced with the maximum value of the pixels in the area of ​​the sliding window to obtain a third infrared image; performing a fourth matching query on the third infrared image using the query template as a sliding window; During the fourth matching query, pixel values ​​of the area of ​​the third infrared image covered by the sliding window are replaced with the minimum pixel value in the area of ​​the sliding window to obtain a fourth infrared image; The original infrared image is subtracted from the fourth infrared image to enhance the target feature and obtain an enhanced infrared image.

2. The infrared image small target enhancement system for complex background according to claim 1 is characterized in that: The enhancement system further comprises: A fusion module is used for performing pixel value transformation on the original infrared image based on the query template to enhance the target features and obtain an enhanced infrared image; performing weighted fusion on the enhanced infrared images obtained based on the query templates of different scales to obtain a fused enhanced infrared image; The filtering process on the enhanced infrared image to obtain the target infrared image includes: The fused enhanced infrared image is filtered to obtain a target infrared image.

3. The infrared image small target enhancement system for complex background according to claim 1 is characterized in that: The determination module includes: a first determining unit, configured to determine that the target shape of the query template is a circle according to the shape of the target feature in the original infrared image being a circle; a second determining unit, configured to determine that the target shape of the query template is a cross according to the shape of the target feature on the original infrared image being a line; The third determining unit is configured to determine that the target shape of the query template is a rectangle according to the shape of the target feature on the original infrared image being a block.

4. The infrared image small target enhancement system for complex background according to claim 1 is characterized in that: The filtering module includes: a filtering unit, configured to perform Fourier transform on the enhanced infrared image to obtain a spectrum graph; Performing filtering on the spectrum graph to obtain a filtered image; Perform inverse Fourier transform on the filtered image to obtain a target infrared image.

5. A real-time tracking system for infrared images of small targets in complex backgrounds, characterized by: include: an identification module for performing target detection on an infrared image and identifying and tracking a target; wherein the infrared image is obtained using the infrared image weak target enhancement system for complex backgrounds according to any one of claims 1 to 4; The tracking module predicts the trajectory of the tracking target identified according to the infrared images of consecutive frames, and tracks the target according to the predicted trajectory.

6. The real-time tracking system for small infrared target in complex background according to claim 5, characterized in that: The real-time tracking system also includes: A stabilization module is used after the target detection is performed on the infrared image and the target is identified and tracked; Binarizing the tracking target identified by the infrared images of consecutive frames to obtain a binary identification image; Superimposing the binary recognition images of consecutive frames to obtain a heat map; Highlighting the tracking target according to the heat map, and extracting a trajectory of the highlighted tracking target to obtain a movement trajectory; Predicting the trajectory of the tracking target identified according to the infrared images of consecutive frames, and tracking the target according to the predicted trajectory, includes: A trajectory prediction is performed according to the movement trajectory, and a target is tracked according to the predicted trajectory.

Images