Multi-frame accumulated infrared dim moving target enhancement method, system, device and terminal

Abstract

The application belongs to the technical field of image processing, and discloses a multi-frame accumulation infrared dim moving target enhancement method, system, device and terminal, multi-frame accumulation is carried out based on target coarse positioning, energy collection and maximum value pooling are carried out in a single frame; target coarse positioning is obtained by combining multi-frame pre-accumulation and maximum value projection, and a motion vector is estimated; the optimal solution is obtained by traversal to obtain a target real trajectory, so that the target can accumulate energy along the correct trajectory, and target enhancement is realized. The application realizes effective enhancement of an infrared dim moving target through multi-frame accumulation based on target coarse positioning, solves the problem of decreased detection capability caused by long detection distance and target weakness, improves image signal-to-noise ratio, and provides convenience for subsequent processing such as detection and tracking; the spatial moving target enhancement algorithm for a low signal-to-noise ratio image is suitable for energy accumulation of a dim moving target in a deep space background in an infrared image, enhances target signals, suppresses noise interference, and thus improves signal-to-noise ratio.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a method, system, device and terminal for enhancing infrared dark and weak moving targets by multi-frame accumulation. Background Technology

[0002] As human exploration of space increases, so too do space threats. To detect and mitigate dangers as early as possible, the use of space-based infrared detection systems for the detection and localization of space targets is crucial. The continuous advancement of detection technology has led to greater focus on the detection and identification of small point targets at long distances under low signal-to-noise ratio conditions. However, limitations in imaging principles and manufacturing processes result in inherent shortcomings in infrared images, such as low contrast and weak texture information. Furthermore, in space-based infrared target detection applications, long-distance, high-speed moving targets typically appear as small targets or point targets in images, occupying only a few pixels and exhibiting weak intensity. The target's movement also causes variations in its image across multiple frames, such as a trailing effect. On the other hand, the target exists within a certain background, including interference from stars, other satellites, and space debris. Additionally, noise within the infrared detection system and the vacuum environment can affect imaging quality, making the grayscale difference between the target and the background unstable and easily obscured by background noise. These factors greatly increase the difficulty of target detection, and even cause many detection and recognition algorithms to fail at low signal-to-noise ratios. Therefore, enhancing the target signal using effective methods before processing becomes the key to solving this problem.

[0003] Traditional infrared target enhancement methods are mainly divided into three categories: spatial domain-based enhancement algorithms, transform domain-based enhancement algorithms, and time domain-based enhancement algorithms.

[0004] Spatial domain-based enhancement methods directly process pixels in the image, such as histogram equalization, linear gray-level transformation, two-dimensional least mean square filtering, maximum mean filtering, maximum median filtering, Butterworth high-pass filtering, and morphological filtering algorithms. Transform domain-based enhancement methods convert the infrared image to the transform domain, obtaining a spectrum containing signals of different frequencies. Filtering operations are then performed on the spectrum, such as traditional frequency domain filtering like ideal high-pass filtering, Butterworth high-pass filtering, and Gaussian high-pass filtering, as well as wavelet transforms and their extended transforms—ridge transform, curve transform, and contour transform algorithms. In general, these methods are simple to implement, but their enhancement effects are limited. Furthermore, improper processing can cause target points to be mistakenly identified as background or noise and removed, resulting in the loss of the target. This not only makes subsequent processing difficult but also increases the complexity of the process.

[0005] Because infrared targets with weak radiation energy are often affected by various background and noise factors in a single frame image, their signal energy fluctuates. Time-domain enhancement methods fully utilize the target's information in the time dimension to avoid such interference. The main time-domain methods are inter-frame differencing and inter-frame accumulation. Inter-frame differencing utilizes the target's motion characteristics to perform a difference operation on two images. In the differencing image, the portion with unchanged grayscale is subtracted, achieving target enhancement. However, multi-frame registration is usually required before differencing, which is difficult when the target's position is unknown. Inter-frame accumulation accumulates the target's energy in adjacent frames, but directly using frame averaging time-domain filtering or various iterative averaging processes can cause blurring and loss of moving targets. The dynamic range (DP) algorithm is a classic energy accumulation algorithm, but it is prone to getting trapped in local optima during computation and is not suitable for low signal-to-noise ratio situations. Besides DP, algorithms based on target motion models have been proposed, but they often require global search, resulting in excessive computation and sensitivity to noise interference.

[0006] In summary, current methods for target enhancement using single-dimensional information all have drawbacks due to the limited information obtained. Single-frame processing is prone to target loss, while multi-frame processing involves too much computation and is difficult to implement, thus limiting their enhancement effects. Therefore, there is an urgent need to design a new multi-frame cumulative infrared dark and weak moving target enhancement method and system.

[0007] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0008] (1) In traditional infrared target enhancement methods, the enhancement effect of spatial infrared target enhancement methods is limited. Furthermore, if the processing is not done properly, the target point may be mistaken for background or noise and removed, resulting in the loss of the target. In subsequent processing, it is not only difficult to recover the target, but it also increases the processing difficulty.

[0009] (2) In traditional time-domain-based infrared target enhancement methods, multiple frames of images usually need to be registered before inter-frame differential processing. However, when the target position is unknown, inter-frame registration is difficult to achieve. Directly using frame averaging time-domain filtering or various iterative averaging processes will cause blurring and loss of moving targets.

[0010] (3) In the traditional inter-frame accumulation method, the DP algorithm is prone to getting trapped in local optima during the calculation process. In addition to the DP algorithm, the algorithm based on the target motion model often requires global search, which is too computationally intensive and difficult to implement. It is also very sensitive to noise interference and is not suitable for low signal-to-noise ratio situations. Summary of the Invention

[0011] To address the problems existing in the prior art, the present invention provides a method, system, device, and terminal for enhancing infrared targets in dark and weak motion using multi-frame cumulative infrared technology, and particularly relates to a method, system, device, and terminal for enhancing infrared targets in dark and weak motion based on coarse target localization.

[0012] This invention is implemented as follows: a multi-frame cumulative infrared target enhancement method for low-light, moving targets, the multi-frame cumulative infrared target enhancement method comprising:

[0013] The target region is magnified and its intensity is enhanced by two filtering processes in a single frame. Based on this, the target coarse localization is obtained by combining multi-frame pre-accumulation and maximum value projection, and the motion vector is estimated by using spatiotemporal fusion information. The target's true trajectory is obtained by defining an evaluation function and performing optimal matching, so that the target accumulates energy along the correct trajectory, thereby achieving target enhancement.

[0014] Furthermore, the multi-frame cumulative infrared dark and weak moving target enhancement method also includes:

[0015] Perform 3D processing on each pixel in a single image 3-neighborhood filtering is used to harvest energy and perform 5-neighborhood filtering on the image. The algorithm employs a 5-neighborhood maximum filtering method; iterative accumulation of the sequence images over 8 frames to obtain a new image sequence; maximum projection of the new image sequence and coarse target localization through stripe detection; backtracking of the sequence images based on the target's window position to estimate the target's motion direction and velocity; and for the enhanced sequence images of a single frame, traversing the multi-frame accumulation results of all possible combinations of the target's motion direction and velocity, using the signal-to-noise ratio gain as the confidence level to obtain the optimal solution for multi-frame accumulation.

[0016] Furthermore, the multi-frame cumulative infrared dark and weak moving target enhancement method includes the following steps:

[0017] Step 1, perform 3 steps on each frame of the image. Collect energy within a 3-neighborhood and perform 5 The maximum value filtering of the 5-neighborhood enhances and amplifies the intensity and size of the target through spatial domain enhancement, laying the foundation for subsequent processing;

[0018] Step 2: Every 8 frames are iterated and superimposed to obtain K frames of images, and the maximum value of the sequence images is projected to form a rough motion trajectory of the target;

[0019] Step 3: Perform stripe detection to obtain the window where the target's motion trajectory is located, and backtrack to the image obtained in Step 2 to estimate the speed and direction of the target's motion, and combine them into a possible set of motion vectors;

[0020] Step 4: Perform single-frame enhancement on each frame of the original image sequence to obtain the image sequence after removing a certain amount of noise;

[0021] Step 5: Accumulate multiple frames of the sequence image obtained in Step 4 based on the motion vector, traverse all results, take the candidate value that maximizes the evaluation function as the true value, obtain the optimal solution, and achieve the goal of shifting and accumulating along the correct trajectory.

[0022] Furthermore, in step one, each frame of the image is processed in 3... Energy harvesting within a 3-neighborhood includes:

[0023] ;

[0024] in, The image mean was used as the background.

[0025] The 5 Maximum filtering within a 5-neighborhood includes:

[0026] ;

[0027] in, express 5 5 neighborhoods.

[0028] The step two involves iteratively stacking every 8 frames to obtain K-frame images, including:

[0029] ;

[0030] ;

[0031] The number of frames superimposed depends on the target's movement speed.

[0032] The sequence image maximum projection MTI includes:

[0033] ;

[0034] ;

[0035] in, K is the pixel grayscale value, and K is the frame rate.

[0036] Furthermore, step three, which involves stripe detection to obtain the window containing the target, includes:

[0037] (1) Adaptive threshold segmentation

[0038] ;

[0039] in, It's a pixel value, a threshold. ,parameter .

[0040] (2) Connected component detection

[0041] After marking the connected components, traverse them and retain the connected component with the most non-zero pixels.

[0042] ;

[0043] ;

[0044] in, Indicates a connected component. Represents connected components The number of non-zero pixels contained.

[0045] The images obtained in the second backtracking step include:

[0046] Find the maximum value within the window of the first and last frames. If there are multiple maximum values, calculate the centroid position and estimate the magnitude and direction of the target's velocity in the adjacent frames.

[0047] ;

[0048] ;

[0049] ;

[0050] ;

[0051] ;

[0052] right and Round up and down:

[0053] ;

[0054] in, This indicates the target window area obtained. This represents the nth frame of the image; and This indicates the possible velocity values ​​of the target in the row and column directions of adjacent frames.

[0055] Furthermore, step four, which involves performing single-frame enhancement on each frame of the original image sequence, includes:

[0056] ;

[0057] in, It's a pixel value. .

[0058] ;

[0059] in, express 5 5 neighborhoods.

[0060] The evaluation function expression in step five is as follows:

[0061] ;

[0062] in, This indicates the target window area obtained. Represents the pixel value of a point within the region. It represents the standard deviation of pixel values ​​within the region.

[0063] Multi-frame accumulation:

[0064] ;

[0065] ;

[0066]

[0067] ;

[0068] ;

[0069] ;

[0070] Find the optimal solution:

[0071] ;

[0072] in, In the nth frame Pixel value at that location, and It is the result and Any combination of To output the image.

[0073] Another object of the present invention is to provide a multi-frame cumulative infrared low-light target enhancement system applying the aforementioned multi-frame cumulative infrared low-light target enhancement method, the multi-frame cumulative infrared low-light target enhancement system comprising:

[0074] The spatial filtering module is used to perform 3D filtering on each pixel in a single image. 3. Neighborhood operations are used to collect energy and perform 5. Maximum value filtering in a 5-neighborhood;

[0075] The target coarse localization module is used to perform 8-frame iterative accumulation on the sequence of images to obtain a new image sequence; the new image sequence is then projected onto the maximum value, and the target coarse localization is obtained through stripe detection.

[0076] The sequence image backtracking module is used to backtrack the sequence images based on the target's window position and estimate the target's motion direction and speed.

[0077] The multi-frame accumulation module is used to accumulate the multi-frame results of all possible motion directions and velocity combinations of the target in the sequence image after single-frame enhancement, and obtain the optimal solution of multi-frame accumulation by using the signal-to-noise ratio gain as the confidence level.

[0078] Another object of the present invention is to provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, the computer program being executed by the processor causing the processor to perform the multi-frame cumulative infrared dark and weak moving target enhancement method.

[0079] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the multi-frame cumulative infrared dark and weak moving target enhancement method.

[0080] Another objective of this invention is to provide an information data processing terminal for implementing the multi-frame cumulative infrared dark and weak moving target enhancement system.

[0081] Based on the above technical solutions and the technical problems solved, please analyze the advantages and positive effects of the technical solution to be protected by this invention from the following aspects:

[0082] First, addressing the technical problems existing in the prior art and the difficulty in solving them, this paper closely analyzes, in conjunction with the technical solution to be protected by this invention and the results and data obtained during the research and development process, how the technical solution of this invention solves the technical problems, and the inventive technical effects brought about by solving these problems. The specific description is as follows:

[0083] This invention provides a multi-frame accumulation infrared target enhancement method for faint, moving targets in low-light conditions, addressing the problems of target loss in single-frame processing and high computational complexity in multi-frame processing. Utilizing spatiotemporal fusion information to achieve target enhancement represents a novel approach. Therefore, the infrared target enhancement algorithm based on coarse target localization and multi-frame accumulation proposed in this invention is of significant importance in the field of infrared target enhancement. This invention provides a spatiotemporal fusion infrared target enhancement method for faint, moving targets in low-light conditions, its main feature being multi-frame accumulation based on coarse target localization. This method performs energy harvesting and maximum pooling in a single frame, then combines multi-frame pre-accumulation and maximum projection to obtain coarse target localization and estimate the motion vector; by traversing and selecting the optimal solution, the true target trajectory is obtained, allowing the target to accumulate energy along the correct trajectory, ultimately achieving effective target enhancement.

[0084] Second, considering the technical solution as a whole or from a product perspective, the technical effects and advantages of the technical solution to be protected by this invention are specifically described as follows:

[0085] The method of this invention effectively enhances the performance of faint, moving targets in infrared images by accumulating multiple frames based on coarse target localization. This solves the problem of decreased detection capability caused by long detection distances and the weakness of the target, improves the image signal-to-noise ratio, and facilitates subsequent processing such as detection and tracking. The spatial moving target enhancement algorithm provided by this invention for low signal-to-noise ratio images is suitable for accumulating energy in faint, moving targets against a deep-space background in infrared images, enhancing the target signal, suppressing noise interference, and thus improving the signal-to-noise ratio.

[0086] Compared with the prior art, the present invention also has the following beneficial effects:

[0087] 1) This invention combines spatial and temporal information to coarsely locate targets under low signal-to-noise ratio conditions under passive detection, thus achieving stable pre-detection.

[0088] 2) Based on 1), this invention traverses the trajectory and accumulates multiple frames to determine the precise trajectory of the target motion, thus realizing the effective accumulation of target energy on the motion path.

[0089] 3) This invention uses a spatiotemporal fusion method, which is less likely to cause target loss and avoids global search, greatly reducing the amount of computation.

[0090] Third, as supplementary evidence of the inventive step of the claims of this invention, it is also reflected in the following important aspects:

[0091] (1) The expected benefits and commercial value of the technical solution of this invention after transformation are as follows:

[0092] Space-based target surveillance systems can monitor and perceive the situation of space targets in real time, and react promptly to dangerous targets when necessary, making them extremely valuable for military applications. The dim and weakly moving target enhancement technology proposed in this invention will significantly improve the detection capabilities and range of infrared detection systems, providing crucial technical support for the construction of my country's future space target detection system. Furthermore, this technology can also be widely applied in civilian fields such as satellite atmospheric infrared cloud image analysis, space remote sensing, pathological analysis of infrared medical images, geological analysis of ground infrared images taken by aircraft, urban infrared pollution analysis, and maritime search and rescue.

[0093] (2) The technical solution of the present invention solves a technical problem that people have long wanted to solve but have never been able to solve successfully:

[0094] Currently, researchers both domestically and internationally have conducted extensive research on methods for enhancing infrared targets that are dark or weakly moving, proposing a variety of effective algorithms. Traditional methods for selectively amplifying targets mostly utilize single-dimensional information, such as mathematical morphology methods utilizing target grayscale characteristics, filtering methods utilizing target frequency domain features, methods based on human visual characteristics utilizing target shape features, or difference-frame fusion methods utilizing target motion characteristics. While these methods each have their advantages, due to limited information, most algorithms suffer from performance limitations or even failure under low signal-to-noise ratio (SNR) conditions. The multi-frame accumulation method based on coarse target localization proposed in this invention combines spatiotemporal fusion technology, demonstrating excellent performance under low SNR conditions, breaking through current algorithm performance constraints, and providing new possibilities for research in this field. Attached Figure Description

[0095] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0096] Figure 1 This is a flowchart of the multi-frame cumulative infrared dark and weak moving target enhancement method provided in the embodiments of the present invention;

[0097] Figure 2 This is a schematic diagram of the multi-frame accumulation infrared dark and weak moving target enhancement method provided in the embodiments of the present invention;

[0098] Figure 3 This is a schematic diagram of multi-frame accumulation in step eight of the embodiments of the present invention;

[0099] Figure 4 The input test image provided in the embodiment of the present invention; wherein Figure (1) is an infrared simulation image and the real target position is marked; Figure (2) is a three-dimensional view of its grayscale distribution;

[0100] Figure 5 This is a schematic diagram showing the result of single-frame preprocessing of the test image provided in the embodiment of the present invention through steps one and two;

[0101] Figure 6 This is a schematic diagram showing the result of multi-frame preprocessing of the test image provided in the embodiment of the present invention through steps three and four;

[0102] Figure 7 The test image provided in this embodiment of the invention is the result image after processing in step five; wherein Figure (1) is the threshold segmentation result; Figure (2) is the connected component detection result; and Figure (3) is the stripe detection result.

[0103] Figure 8 This is the result image of the original test image provided in the embodiment of the present invention after single-frame enhancement in step seven;

[0104] Figure 9 The image provided in this embodiment of the invention is the result of accumulating 8 frames through step eight; wherein Figure (1) is the result image and the location of the target is marked; Figure (2) is a three-dimensional view of the result. Detailed Implementation

[0105] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0106] To address the problems existing in the prior art, the present invention provides a method, system, device, and terminal for enhancing infrared dark and weak moving targets by multi-frame accumulation. The present invention will be described in detail below with reference to the accompanying drawings.

[0107] I. Explanatory and Illustrative Embodiments. To enable those skilled in the art to fully understand how the present invention is specifically implemented, this section provides an explanatory and illustrative description of the embodiments described in the claims.

[0108] like Figure 1 As shown, the multi-frame cumulative infrared dark and weak moving target enhancement method provided in this embodiment of the invention includes the following steps:

[0109] S101, based on coarse target localization, performs multi-frame accumulation, and collects energy and performs maximum pooling in a single frame;

[0110] S102, combining multi-frame pre-accumulation and maximum value projection, obtains coarse target localization and estimates motion vectors;

[0111] S103 obtains the target's true trajectory by iterating through the optimal solution, enabling the target to accumulate energy along the correct trajectory and thus enhance the target.

[0112] As a preferred embodiment, such as Figure 2 As shown, the multi-frame cumulative infrared dark and weak moving target enhancement method provided in this embodiment of the invention specifically includes the following steps:

[0113] (1) For each frame of image In 3 Collect energy within the 3-neighborhood to obtain .

[0114] (1)

[0115] in, Since the target occupies very few pixels, the average of the image can be used as the background. This represents the pixel value within the neighborhood.

[0116] (2) Based on step (1), perform 5 Maximum value filtering within a 5-neighborhood.

[0117] (2)

[0118] in, express 5 5 neighborhoods.

[0119] (3) The images obtained in step (2) form a new sequence, and are superimposed every 8 frames to obtain K frames of images.

[0120]

[0121] (3)

[0122] The number of frames superimposed depends on the target's movement speed.

[0123] (4) Perform sequence image maximum projection (MTI) based on step (3).

[0124]

[0125] (4)

[0126] in, yes middle The pixel value at that location, where K is the frame number.

[0127] (5) Perform stripe detection to obtain the window where the target is located.

[0128] (a) Adaptive threshold segmentation

[0129] (5)

[0130] Among them, threshold , , ,parameter .

[0131] (b) Connectivity Component Detection

[0132] After marking the connected components, traverse them and retain the connected component with the most non-zero pixels.

[0133]

[0134] (6)

[0135] in, Indicates a connected component. Represents connected components The number of non-zero pixels contained.

[0136] (6) The image obtained by backtracking step (3):

[0137] Find the maximum value within the window of the first and last frames. If there are multiple maximum values, calculate the centroid position and estimate the target's velocity (size and direction) in the adjacent frames.

[0138]

[0139] (7)

[0140]

[0141] (8)

[0142] (9)

[0143] (10)

[0144] in, This represents the target window area obtained in step (5). Let n be the nth frame of the image, and formula (10) represents the... and Round up and down, and This indicates the possible velocity values ​​of the target in the row and column directions of adjacent frames.

[0145] (7) Perform single-frame enhancement on each frame of the original sequence image.

[0146] (a) (11)

[0147] in, Indicates the original image Pixel value at that location, .

[0148] (b) (12)

[0149] in, express 5 5 neighborhoods This represents the standard deviation of pixel values ​​within that neighborhood.

[0150] (8) Based on the velocity, i.e., the motion vector, obtained in step (6), perform multi-frame accumulation on the sequence image obtained in step (7), iterate through all results, and take the candidate value that maximizes the evaluation function as the true value to obtain the optimal solution. The expression of the evaluation function is as follows:

[0151] (13)

[0152] in, This represents the target window area obtained in step (5). Represents the pixel value of a point within the region. It represents the standard deviation of pixel values ​​within the region.

[0153] Multi-frame accumulation:

[0154]

[0155]

[0156]

[0157]

[0158] (14)

[0159] (15)

[0160] Find the optimal solution:

[0161] (16)

[0162] in, In the nth frame Pixel value at that location, and It is obtained from step (6) and Any combination of To output the image.

[0163] Figure 3 This is a schematic diagram of multi-frame accumulation in step eight of the embodiments of the present invention;

[0164] Figure 4 The input test image provided in the embodiment of the present invention; wherein Figure (1) is an infrared simulation image and the real target position is marked; Figure (2) is a three-dimensional view of its grayscale distribution;

[0165] Figure 5 This is a schematic diagram showing the result of single-frame preprocessing of the test image provided in the embodiment of the present invention through steps one and two;

[0166] Figure 6 This is a schematic diagram showing the result of multi-frame preprocessing of the test image provided in the embodiment of the present invention through steps three and four;

[0167] Figure 7 The test image provided in this embodiment of the invention is the result image after processing in step five; wherein Figure (1) is the threshold segmentation result; Figure (2) is the connected component detection result; and Figure (3) is the stripe detection result.

[0168] Figure 8 This is the result image of the original test image provided in the embodiment of the present invention after single-frame enhancement in step seven;

[0169] Figure 9 The image provided in this embodiment of the invention is the result of accumulating 8 frames through step eight; wherein Figure (1) is the result image and the location of the target is marked; Figure (2) is a three-dimensional view of the result.

[0170] The multi-frame cumulative infrared dark and weak moving target enhancement system provided in this embodiment of the invention includes:

[0171] The spatial filtering module is used to perform 3D filtering on each pixel in a single image. 3-neighborhood, operations to achieve energy collection, and 5-dimensional image processing. Maximum value filtering in a 5-neighborhood;

[0172] The target coarse localization module is used to perform 8-frame iterative accumulation on the sequence of images to obtain a new image sequence; the new image sequence is then projected onto the maximum value, and the target coarse localization is obtained through stripe detection.

[0173] The sequence image backtracking module is used to backtrack the sequence images based on the target's window position to estimate the target's direction of motion and velocity;

[0174] The multi-frame accumulation module is used to accumulate the multi-frame results of all possible motion directions and velocity combinations of the target in the sequence image after single-frame enhancement, and obtain the optimal solution of multi-frame accumulation by using the signal-to-noise ratio gain as the confidence level.

[0175] II. Application Examples. To demonstrate the inventiveness and technical value of the technical solution of this invention, this section provides application examples of the technical solution of the claims on specific products or related technologies.

[0176] 1. By analyzing typical detection distances, the flight trajectories of space targets, detector resolution, and background and noise characteristics under space-based detection conditions, infrared images of actual scenarios were simulated. Considering the diffusion phenomenon caused by the optical system on point targets, experiments were conducted under different target sizes, signal-to-noise ratios, and target movement speeds. All experiments achieved excellent enhancement effects, significantly improving the signal-to-noise ratio. Experiments revealed that the multi-frame accumulation algorithm based on coarse target localization proposed in this invention is applicable to different conditions, including point target sizes greater than 2*2, signal-to-noise ratios greater than 2, and target movement speeds within 1.

[0177] 2. A long-wave infrared camera, a blackbody, a target plate, and a two-dimensional turntable were used to simulate the infrared detection scenario of a space target. The target intensity was simulated by adjusting the temperature of the blackbody; the size of the point target was simulated by adjusting the size of the holes on the target plate; and the target motion was simulated by adjusting the speed of the turntable. Actual infrared images captured under different conditions were used as test images, and non-uniformity correction was applied. The corrected images were then used to verify the multi-frame accumulation method based on coarse target localization proposed in this invention. Experimental results show that the algorithm proposed in this invention can effectively enhance the target, suppress a large amount of background and noise, remove some interference from detector blind elements, and exhibits stability under different conditions.

[0178] 3. Based on the algorithm of this invention, a user interface was designed, realizing two main functions: simulation and enhancement. The simulation module allows for the output, display, and storage of infrared image sequences by setting parameters including target energy concentration, motion speed, image signal-to-noise ratio, image size, and turntable tracking stability. The enhancement module can perform multi-frame cumulative target enhancement based on coarse target localization within a specified image sequence by selecting a file, outputting the resulting image and simultaneously plotting the enhanced signal-to-noise ratio curve.

[0179] III. Evidence of the Relevant Effects of the Embodiments. The embodiments of the present invention have achieved some positive effects during research and development or use, and indeed possess significant advantages compared to existing technologies. The following description, in conjunction with data, charts, and other materials from the experimental process, illustrates these advantages.

[0180] For space targets, due to the long detection distance, the target size is smaller than the sensor's smallest resolution unit, resulting in a speckled image, which is generally considered a point target. Furthermore, influenced by the spaceborne optoelectronic imaging system, point target imaging exhibits diffusion, represented by the point spread function. In contrast, the deep space background is a relatively pure 4K cold background, and cosmic background radiation can be ignored. The noise that significantly affects the target in infrared images is random noise, which can generally be considered to approximately follow a Gaussian distribution. Therefore, the test images used in this invention are mainly obtained from infrared simulation images based on the above analysis.

[0181] Simulation environment: Matlab2020b.

[0182] Test image: Infrared simulation image, size 256 256.

[0183] Target information: Point target, with some diffusion; target size is determined by energy concentration.

[0184] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0185] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for enhancing the performance of low-light, moving targets using multi-frame cumulative infrared technology, characterized in that: The multi-frame cumulative infrared dark and weak moving target enhancement method includes: Based on the coarse target localization, multi-frame accumulation is performed, and energy harvesting and maximum pooling are performed in a single frame. The coarse target localization is obtained by combining multi-frame pre-accumulation and maximum projection, and the motion vector is estimated. The true target trajectory is obtained by traversing and taking the optimal solution, so that the target accumulates energy along the correct trajectory, thereby achieving target augmentation. The multi-frame cumulative infrared dark and weak moving target enhancement method also includes: For each pixel in a single image, a 3×3 neighborhood is created to collect energy, and a 5×5 neighborhood maximum value filter is applied to the image. The sequence of images is iteratively accumulated for 8 frames to obtain a new image sequence. The new image sequence is then subjected to maximum value projection, and the target is coarsely located through stripe detection. Based on the target's window position, the sequence of images is backtracked to estimate the target's motion direction and velocity. For the enhanced sequence of images, the multi-frame accumulation results of all possible combinations of the target's motion direction and velocity are traversed, and the signal-to-noise ratio gain is used as the confidence level to obtain the optimal solution for multi-frame accumulation.

2. The multi-frame cumulative infrared dark and weak moving target enhancement method as described in claim 1, characterized in that, The multi-frame cumulative infrared dark and weak moving target enhancement method includes the following steps: Step 1: Collect energy in a 3×3 neighborhood for each frame of the image and perform maximum filtering in a 5×5 neighborhood. Step 2: Iterate and overlay every 8 frames to obtain K frames of images, and then project the maximum value of the sequence of images. Step 3: Perform stripe detection to obtain the window where the target is located, and then backtrack to the image obtained in Step 2; Step 4: Perform single-frame enhancement on each frame of the original image sequence to obtain the image sequence. Step 5: Accumulate multiple frames of the sequence images obtained in Step 4 based on the motion vectors, traverse all results, take the candidate value that maximizes the evaluation function as the true value, and obtain the optimal solution.

3. The multi-frame cumulative infrared dark and weak moving target enhancement method as described in claim 2, characterized in that, The step one, collecting energy in a 3×3 neighborhood for each frame of image, includes: Where, x background =mean(I(x,y)), and use the image mean as the background; The maximum value filtering within the 5×5 neighborhood includes: f(x,y)=max(I Φ (x,y)); Where Φ represents the 5×5 neighborhood of (x,y); The step two involves iteratively stacking every 8 frames to obtain K-frame images, including: ； The number of frames to be superimposed is determined by the target's motion speed; The sequence image maximum projection MTI includes: z(x,y)=max[r(x,y,t)]-mean{sum[(r(x,y,t)]-max[r(x,y,t)]}; t = 1, ..., K; Where r(x,y,t) is the pixel grayscale value, and K is the frame number.

4. The multi-frame cumulative infrared dark and weak moving target enhancement method as described in claim 2, characterized in that, Step three, which involves stripe detection to obtain the window containing the target, includes: (1) Adaptive threshold segmentation if f(x,y)>T,f(x,y)=1, else f(x,y)=0; Where f(x,y) is the pixel value, the threshold T=μ+k×σ, and the parameter k=2-5; (2) Connected component detection After marking the connected components, traverse them and retain the connected component that contains the most non-zero pixels; in,{ 1, 2, 3,…} represents a connected component. Represents connected components The number of non-zero pixels contained; The images obtained in the second backtracking step include: Find the maximum value within the window of the first and last frames. If there are multiple maximum values, calculate the centroid position and estimate the magnitude and direction of the target's velocity in the adjacent frames. {(x1,y1),(x2,y2),…}=max[frame(1) Ω ]; r1=mean{x n },c1=mean{y n }; {(x1,y1),(x2,y2),…}=max[frame(K) Ω ]; r K =mean{x n },c K =mean{y n }; For v r and v c Round up and down: Where Ω represents the obtained target window region, frame(n) represents the nth frame image; Δr and Δc represent the possible velocity values ​​of the target in the row and column directions of adjacent frames.

5. The multi-frame cumulative infrared dark and weak moving target enhancement method as described in claim 2, characterized in that, Step four, which involves single-frame enhancement of each frame of the original image sequence, includes: =f(x,y)-x background ; Where f(x,y) is the pixel value, x background =mean(I(x,y)); if f(x,y)<2×σ Φ ,f(x,y)=0; Where Φ represents the 5×5 neighborhood of (x,y); The evaluation function expression in step five is as follows: Where Ω represents the obtained target window region, f Ω (x,y) represents the pixel value of a point within the region, σ Ω The standard deviation of pixel values ​​within the region; Multi-frame accumulation: frame' n+1 (x,y)=frame n (x-Δr1,y-Δc1)+frame n+1 (x,y); when x-Δr1,y-Δc1∈size[frame]; … frame' K (x,y)=frame K-1 (x-Δr K-1 ,y-Δc K-1 )+frame K (x,y); when x-Δr K-1 ,y-Δc K-1 ∈size[frame]; result=frame' K ; Find the optimal solution: output=result,when F(result)=max{F(result)}; Among them, frame n (x,y) represents the pixel value at (x,y) in the nth frame, {Δr1,…Δr K-1 } and {Δc1,…Δc K-1 } represents any combination of the obtained Δr and Δc, and output is the output image.

6. A multi-frame cumulative infrared low-light motion target enhancement system applying the multi-frame cumulative infrared low-light motion target enhancement method as described in any one of claims 1 to 5, characterized in that, The multi-frame cumulative infrared dark and weak moving target enhancement system includes: The maximum value filtering module is used to perform 3×3 neighborhood operations on each pixel in a single image to collect energy, and to perform 5×5 neighborhood maximum value filtering on the image. The target coarse localization module is used to perform 8-frame iterative accumulation on the sequence of images to obtain a new image sequence; the new image sequence is then projected onto the maximum value, and the target coarse localization is obtained through stripe detection. The sequence image backtracking module is used to backtrack the sequence images based on the target's window position and estimate the target's motion direction and speed. The multi-frame accumulation module is used to accumulate the multi-frame results of all possible motion directions and velocity combinations of the target in the sequence image after single-frame enhancement, and obtain the optimal solution of multi-frame accumulation by using the signal-to-noise ratio gain as the confidence level.

7. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the multi-frame cumulative infrared dark and weak moving target enhancement method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the multi-frame cumulative infrared dark and weak moving target enhancement method as described in any one of claims 1 to 5.

9. An information data processing terminal, characterized in that, The information data processing terminal includes the multi-frame accumulation infrared dark and weak moving target enhancement system as described in claim 6.

Images