An on-board autonomous closed-loop real-time detection method for dim and weak targets

Abstract

The application discloses a kind of on-orbit autonomous closed-loop real-time detection methods of micro dim target, constructs on-orbit autonomous closed-loop detection framework, from the characteristics of high frame frequency detection payload, the dynamic windowing advantage of high frame frequency detection payload is exerted, target detection result is fed back in real time in closed loop, realizes the integration of load and processing, better improves system efficiency, at the same time, for the small area image of high frame frequency strong clutter, the characteristics of target in time dimension are fully utilized, the problem of low signal-to-noise ratio of target image in strong clutter area is better overcome, and high-reliability target extraction is effectively realized, and for weak clutter area image, the motion and radiation difference characteristics of target and background in space-time dimension are fully utilized to realize effective extraction of target.

Description

Technical Field

[0001] This invention belongs to the field of satellite information processing technology, specifically relating to an on-board autonomous closed-loop real-time detection method for small, faint targets. Background Technology

[0002] For the new generation of large-area array high-frame-rate payloads, due to the large amount of image downlink data and long inter-frame intervals, the traditional real-time image downlink method is not conducive to target detection. In addition, when detecting strong clutter areas, the traditional target detection method has the problem that the traditional threshold segmentation method is difficult to effectively correlate the target trajectory due to the excessive density of candidate points. Summary of the Invention

[0003] In view of this, the present invention provides an on-board autonomous closed-loop real-time detection method for small and faint targets, which realizes the detection of small and faint targets in background regions with different background complexities.

[0004] This invention provides an on-board autonomous closed-loop real-time detection method for small, faint targets, comprising the following steps:

[0005] Spatial processing is performed on the global image acquired by the high frame rate probe payload to obtain candidate points. Then, strong clutter background regions are extracted by clustering the dense regions of candidate points. Finally, the strong clutter background regions are fed back to the high frame rate probe payload.

[0006] The high frame rate detection payload performs a high frame rate windowing operation on the strong clutter background region to obtain a local high frame rate image sequence. The high frame rate image sequence is then used for target detection by temporal variance filtering. Then, candidate points are determined based on threshold segmentation to achieve on-board autonomous real-time closed-loop detection.

[0007] The high frame rate detection payload uses low frame rate imaging to obtain local low frame rate image sequences for areas outside the strong clutter background region. Then, using the information of the current frame and historical frames, a spatiotemporal contrast fusion algorithm is used to perform target detection on the local low frame rate image sequences. Based on threshold segmentation, candidate points are determined to achieve on-board autonomous real-time closed-loop detection.

[0008] Furthermore, the high-frame-rate detection payload performs a high-frame-rate windowing operation on the strong clutter background region to obtain a local high-frame-rate image sequence. Temporal variance filtering is then used to perform target detection on this high-frame-rate image sequence. Candidate points are then determined based on threshold segmentation to achieve onboard autonomous real-time closed-loop detection, including:

[0009] Step 1.1: Using each candidate point as the center point, obtain a neighborhood sliding window image within a range of (2k+1)×(2k+1) centered on the center point. Iterate through the number of candidate points in all neighborhood sliding window images, where k represents the minimum distance between classes.

[0010] Step 1.2: Determine the area with the most candidate points and calculate the density η of candidate points in that area;

[0011] Step 1.3: Compare the density of candidate points η with the density threshold ε. If η > ε, find the four extreme values ​​of the candidate point coordinates in the four directions (up, down, left, right) in the region, and expand the neighborhood image by k pixels in each of the four extreme values ​​to obtain the expanded neighborhood image. Calculate the density of candidate points in the neighborhood image. If η < ε, remove the points in the region from the candidate point set to obtain a new candidate point set.

[0012] Step 1.4: Repeat steps 1.1 to 1.3 until the set of candidate points is empty and the execution ends.

[0013] Furthermore, the step of using the information from the current frame and historical frames to perform target detection on low-frame-rate image sequences in local areas through a spatiotemporal contrast fusion algorithm includes:

[0014] Step 2.1: Perform temporal contrast filtering enhancement on the dark background regions of the current frame and historical frames. The enhancement formula is as follows:

[0015] C(i,j,t)=|I(i,j,t)-m(i,j,t)|

[0016]

[0017] In the formula, C(i,j,t) represents the temporal contrast filtering enhancement result, I(i,j,t) represents the pixel gray value at position (i,j) in the image at time t, m(i,j,t) represents the background estimation of the current frame image, and N... l Increase the length of the filter window to enhance the temporal contrast of the image;

[0018] Step 2.2: Perform spatial contrast filtering enhancement on the dark background region of the current frame. The enhancement formula is as follows:

[0019] C s (i,j)=|I(i,j)-m s (i,j)|

[0020]

[0021] In the formula, C s (i,j) represents the spatial domain filtering enhancement result of the image, I(i,j) represents the gray value of the pixel at position (i,j) in the image, s represents the sum of gray values ​​of the neighborhood window of the removed center pixel, w represents the size of the spatial sliding window, and m represents the value of the pixel. s (i,j) represents the local spatial background estimation result of the image;

[0022] Step 2.3: The image spatiotemporal contrast enhancement result is obtained by multiplying the temporal enhancement result from the first step and the spatial enhancement result from the second step. The enhancement formula is as follows:

[0023]

[0024] In the formula, max i,j (C(i,j,t)) is the maximum value of C(i,j,t), max i,j (C s (i,j) is C s The maximum value of (i,j);

[0025] Step 2.4: Perform target detection and target extraction on the image obtained in Step 2.3.

[0026] Furthermore, the method of using temporal variance filtering to perform target detection on the high frame rate image sequence is as follows:

[0027] The variance characteristics of the temporal profile of each pixel in the statistical sequence image are analyzed. If a pulse appears in the temporal sequence, a target is present. If the temporal gray level changes continuously and slowly, there is no target. Then, the short trajectory of the target passing through the temporal analysis window is extracted by adaptive threshold segmentation.

[0028] Furthermore, the temporal profile variance characteristics of each pixel in the statistical sequence image are obtained by using a temporal variance filtering algorithm to calculate the temporal mean statistical result μ(p) of each pixel in the image. The filtering formula is as follows:

[0029]

[0030]

[0031] In the formula, Var(p) is the statistical result of the temporal variance of each pixel in the image, M and N are the window size of the temporal mean and temporal variance statistics of the pixel, respectively, p is the pixel, and I(p) is the gray value of pixel p.

[0032] Furthermore, the selection of M and N is related to the payload imaging frame rate and the target motion speed.

[0033] Beneficial effects:

[0034] 1. This invention constructs an on-board autonomous closed-loop detection framework. Starting from the characteristics of high-frame-rate detection payloads, it leverages the dynamic windowing advantage of high-frame-rate detection payloads to provide closed-loop real-time feedback of target detection results, realizing the integration of payload and processing, and better improving system efficiency. At the same time, for high-frame-rate images with strong clutter in small regions, it makes full use of the target's temporal characteristics, effectively overcoming the problem of low signal-to-noise ratio in target images in strong clutter regions, and effectively achieving highly reliable target extraction. For images with weak clutter regions, it makes full use of the differences in motion and radiation between the target and the background in the spatiotemporal dimensions to achieve effective target extraction.

[0035] 2. This invention proposes an on-board autonomous high-frame-rate windowing strategy based on candidate point clustering, which can filter different regions of the background and obtain two types of background data, strong background and dark background, based on the filtering results. Subsequently, only the different types of background data need to be analyzed and processed, thus greatly reducing the amount of data to be processed and effectively improving the computational efficiency.

[0036] 3. This invention proposes a target detection algorithm for spatiotemporal contrast fusion sequence images in dark background areas. By utilizing the contrast difference between the target in the current frame and the background in the spatiotemporal neighborhood, it effectively extracts small and weak targets in a smooth background.

[0037] 4. This invention proposes a high frame rate sequence image target detection algorithm, which realizes the effective extraction of targets with inter-frame sub-pixel motion features in complex scenes. The representation form is the short trajectory of the target's motion within a certain time window. The algorithm process is simple and occupies few on-board resources. Attached Figure Description

[0038] Figure 1 This is a schematic flowchart of an on-board autonomous closed-loop real-time detection method for small, faint targets provided by the present invention.

[0039] Figure 2 This is a schematic diagram of a large field-of-view image local windowing strategy based on background complexity in a real-time on-board autonomous closed-loop detection method for small, faint targets provided by the present invention.

[0040] Figure 3-1 This is the first original image.

[0041] Figure 3-2 The image is obtained by extracting the strong clutter region from the first original image using the on-board autonomous closed-loop real-time detection method for small, faint targets provided by this invention.

[0042] Figure 4 This is a flowchart illustrating the spatiotemporal fusion sequence image target detection algorithm for dark background regions in the on-board autonomous closed-loop real-time detection method for small, dark targets provided by the present invention.

[0043] Figure 5-1 This is the second original image.

[0044] Figure 5-2 The image is obtained by performing clutter suppression residual on the second original image using the on-board autonomous closed-loop real-time detection method for small and faint targets provided by this invention.

[0045] Figure 6 This is a flowchart illustrating the algorithm for extracting temporal profile characteristics of high frame rate sequence images in a real-time on-board autonomous closed-loop detection method for small, faint targets provided by the present invention.

[0046] Figure 7 This diagram illustrates the target detection results obtained using the high frame rate sequence image temporal profile feature extraction algorithm in the on-board autonomous closed-loop real-time detection method for small, faint targets provided by this invention. Detailed Implementation

[0047] The following examples illustrate the invention in detail.

[0048] This invention provides an on-board autonomous closed-loop real-time detection method for small and faint targets. The core idea is as follows: First, a local windowing strategy based on background complexity is proposed for large field-of-view images, enabling high-frame-rate imaging of key image regions to obtain more complex scene information and target motion information in a short time. Second, for other dark background regions, an on-board target detection algorithm based on spatiotemporal contrast fusion is proposed, achieving efficient and rapid detection of small and faint targets in the scene. Finally, for complex images in high-frame-rate windowed regions, a target detection algorithm based on temporal variance filtering is proposed, effectively solving the problem of detecting sub-pixel moving targets in high-frame-rate images. This lays the foundation for the development of satellite information processing systems.

[0049] This invention provides an on-board autonomous closed-loop real-time detection method for small, faint targets, the process of which is as follows: Figure 1 As shown, the specific steps include:

[0050] First, the global image acquired by the high frame rate probe payload is processed in the spatial domain to obtain candidate points. Then, strong clutter background regions are extracted by clustering the dense regions of candidate points. Finally, the strong clutter background regions are fed back to the high frame rate probe payload.

[0051] The high frame rate detection payload performs a high frame rate windowing operation on the strong clutter background region to obtain a local high frame rate image sequence. The high frame rate image sequence is then used for target detection by temporal variance filtering. Then, candidate points are determined based on threshold segmentation to achieve on-board autonomous real-time closed-loop detection.

[0052] The high frame rate detection payload uses low frame rate imaging for areas outside the strong clutter background region. Then, it uses information from the current frame and historical frames to perform target detection through a highly reliable spatiotemporal contrast fusion algorithm. Based on threshold segmentation, it determines candidate points and achieves on-board autonomous real-time closed-loop detection.

[0053] This invention leverages the characteristics of high-frame-rate detection payloads, utilizing their dynamic windowing advantages to provide closed-loop real-time feedback of target detection results. This integrated design and optimization of the payload and processing workflow significantly improves system efficiency. Furthermore, for high-frame-rate images in small, cluttered regions, the invention fully utilizes the target's temporal characteristics, monitoring the variance changes of each image pixel. The significant variance changes in target pixels effectively address the low signal-to-noise ratio issue in cluttered regions, thus achieving highly reliable target extraction. For images in weakly cluttered regions, the invention fully utilizes the spatial and temporal differences in motion and radiation between the target and background to achieve effective target extraction.

[0054] To further leverage the impact of high-frame-rate windowing strategies on target detection, this invention proposes a large field-of-view image local windowing strategy based on background complexity. This strategy enables high-frame-rate imaging of key areas of the image, allowing for the acquisition of more complex scene information and target motion information in a short time.

[0055] The local windowing strategy for large field-of-view images based on background complexity established in this invention is as follows: First, the large field-of-view image is processed using a spatial background suppression method to obtain a residual image of the large field-of-view image; then, segmentation is performed using an adaptive threshold to obtain candidate point images; finally, statistical analysis of the candidate points in the windowing region is performed to obtain the density of candidate points, and the windowing size is determined based on the density of candidate points. Therefore, this invention can provide a reference for the integrated design of large-area array payloads and processing.

[0056] The large field-of-view image local windowing strategy based on background complexity mainly processes the fully probed background image with a certain time interval, such as... Figure 2 As shown, within each imaging cycle, spatial filtering is performed on each large frame image to remove noise. After threshold segmentation, a candidate point set for each frame image is obtained. Windowing is applied to dense regions of candidate points, and the window size is determined based on the density of the candidate points. The detailed implementation process is as follows:

[0057] Step 1.1: Using each suspected candidate point as the center point, obtain a neighborhood sliding window image with a range of (2k+1)×(2k+1) centered on the center point. Iterate through the number of candidate points in all neighborhood sliding window images, where k represents the minimum distance between classes.

[0058] Step 1.2: Determine the area with the most candidate points and calculate the density η of candidate points in that area;

[0059] Step 1.3: Compare the density of candidate points η with the threshold ε. If η > ε, find the four extreme values ​​of the candidate point coordinates in the four directions (up, down, left, right) in the region, and expand the region by k pixels in each of the four extreme values ​​to obtain the expanded neighborhood image. Recalculate the density of candidate points in the expanded region. If η < ε, determine that the candidate point set in the region is of one class, remove the points in the region from the candidate point set, and obtain a new candidate point set.

[0060] Step 1.4: Repeat steps 1.1 to 1.3 until the set of candidate points is empty and the execution ends.

[0061] The large field-of-view image local windowing strategy proposed in this invention, based on background complexity, can adaptively adjust the size of the region by determining different k values. Furthermore, by processing candidate points extracted based on pixel spatial distribution, it can reflect the spatial distribution information of pixels, and the computational complexity is low. Specific verification results are as follows: Figure 3-1 and Figure 3-2 As shown in the figure, the present invention can effectively segment the background, extract the strong clutter region, effectively feed back the strong clutter image region to the payload, and then use the high frame rate characteristics of the probe payload to perform high frame rate imaging of the region. Finally, the high frame rate image data is transmitted to the high frame rate image information processing unit.

[0062] Considering the significant differences in physical properties between the target's motion and radiation characteristics and those of natural environments such as the atmosphere and cirrus clouds, the target will form a bright, rapidly moving track against a slowly changing dark background. Therefore, taking advantage of the relatively weak background and relatively small undulations in dark background regions, this invention proposes a spatiotemporal fusion-based image sequence target detection algorithm for dark background regions. The basic principle is to use a spatiotemporal contrast filtering fusion algorithm to enhance the target and suppress the background in the dark background region image for the current frame and (t-1) historical frames. Then, adaptive threshold segmentation is applied to the filtering result to detect weak and small targets in the image. This algorithm fully utilizes the difference between the target's energy and the background, better realizing the detection of small and weak targets. It has the advantages of low computational load and ease of implementation on satellite hardware, and can meet the target detection needs of large-area dark background regions.

[0063] The spatiotemporal fusion sequence image target detection algorithm for dark background regions proposed in this invention has the following process: Figure 4 As shown, it includes the following steps:

[0064] Step 2.1: Perform temporal contrast filtering enhancement on the dark background regions of the current frame and historical frames. The enhancement formula is as follows:

[0065] C(i,j,t)=|I(i,j,t)-m(i,j,t)| (1)

[0066]

[0067] In the formula, I(i,j,t) represents the pixel gray value at position (i,j) in the image at time t, m(i,j,t) represents the background estimation of the current frame image, and N l Indicates the length of the temporal contrast enhancement filter window for the image;

[0068] Step 2.2: Perform spatial contrast filtering enhancement on the dark background region of the current frame image. The enhancement formula is as follows:

[0069] C s (i,j)=|I(i,j)-m s (i,j)| (3)

[0070]

[0071] In the formula, C s (i,j) represents the spatial domain filtering enhancement result of the image, I(i,j) represents the gray value of the pixel at position (i,j) in the image, s represents the sum of the gray values ​​of the neighborhood window of the removed center pixel, w represents the size of the spatial sliding window, and m s (i,j) represents the local spatial background estimation result of the image;

[0072] Step 2.3: The image spatiotemporal contrast filtering enhancement result is obtained by multiplying the temporal enhancement result from the first step and the spatial enhancement result from the second step. The final enhancement formula is as follows:

[0073]

[0074] In the formula, I st This represents the final result of the spatiotemporal contrast filtering and fusion algorithm proposed in this invention for dark background regions in large field-of-view images;

[0075] Step 2.4: Perform target detection and target extraction on the image obtained in Step 2.3.

[0076] This algorithm is designed for high-frame-rate images with strong clutter in small regions. It makes full use of the target's features in the time dimension and monitors the variance change of each image pixel. The target's variance changes significantly after passing through the pixels, which can effectively overcome the problem of low signal-to-noise ratio in the target image in strong clutter regions. It can effectively achieve highly reliable target extraction. The target detection results are shown in Figure 5.

[0077] Since the background clutter energy is strong and the clutter fluctuations are more intense in strong clutter regions, and the background energy is stronger than that of the target, it is difficult to achieve good target detection performance by directly utilizing the spatiotemporal characteristics. Therefore, this invention proposes to use local high-frequency windowing to take advantage of the target's motion characteristics and the significant difference between the target and clutter in the time domain profile, so as to meet the target detection requirements in strong clutter regions.

[0078] The algorithm for extracting temporal profile features of high frame rate sequence images proposed in this invention is as follows: Figure 6 As shown, the mechanism is as follows: taking the temporal grayscale changes of each pixel in the sequence image as the research object, the grayscale of pixels where the target passes will have abrupt changes, that is, pulses will appear in its temporal sequence. Pixels where no target appears, i.e., background pixels, will maintain the characteristic of continuous and gradual grayscale changes. Therefore, fully considering the difference in temporal energy changes between target pixels and background pixels, a method is proposed to distinguish target and background pixels by statistically analyzing the temporal profile variance characteristics of each pixel in the sequence image. Then, through adaptive threshold segmentation, the short trajectory of the target passing through the temporal analysis window is extracted as the final target short trajectory detection result. To this end, this invention proposes a temporal variance filtering algorithm, which has a simple process, significant target enhancement effect, and low on-board resource consumption. The filtering formula is as follows:

[0079]

[0080]

[0081] In the formula, μ(p) is the statistical result of the temporal mean of each pixel in the image, Var(p) is the statistical result of the temporal variance of each pixel in the image, M and N are the window sizes for the statistical results of the temporal mean and temporal variance of the pixel, respectively, p is the pixel, I(p) is the gray value of pixel p, and the selection of M and N is related to the frame rate of the payload imaging and the speed of the target motion.

[0082] Target detection results in high frame rate sequence images are as follows Figure 7 As shown, the results demonstrate the practicality and effectiveness of the algorithm, which solves the problem of inter-frame sub-pixel motion of targets in high frame rate sequence images. At the same time, it makes full use of the temporal difference features between the target and the background, effectively improving the detection capability of small and weak targets in complex scenes.

[0083] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A satellite-based autonomous closed-loop real-time detection method for small, faint targets, characterized in that, Includes the following steps: Spatial processing is performed on the global image acquired by the high frame rate probe payload to obtain candidate points. Then, strong clutter background regions are extracted by clustering the dense regions of candidate points. Finally, the strong clutter background regions are fed back to the high frame rate probe payload. The high frame rate detection payload performs a high frame rate windowing operation on the strong clutter background region to obtain a local high frame rate image sequence. The high frame rate image sequence is then used for target detection by temporal variance filtering. Then, candidate points are determined based on threshold segmentation to achieve on-board autonomous real-time closed-loop detection. The high frame rate detection payload uses low frame rate imaging to obtain local low frame rate image sequences for areas outside the strong clutter background region. Then, using the information of the current frame and historical frames, a spatiotemporal contrast fusion algorithm is used to perform target detection on the local low frame rate image sequences. Based on threshold segmentation, candidate points are determined to achieve on-board autonomous real-time closed-loop detection.

2. The on-board autonomous closed-loop real-time detection method according to claim 1, characterized in that, The high-frame-rate detection payload performs a high-frame-rate windowing operation on the strong clutter background region to obtain a local high-frame-rate image sequence. Temporal variance filtering is used to perform target detection on this high-frame-rate image sequence, and candidate points are determined based on threshold segmentation to achieve on-board autonomous real-time closed-loop detection, including: Step 1.1: Using each candidate point as the center point, obtain a neighborhood sliding window image within a range of (2k+1)×(2k+1) centered on the center point. Iterate through the number of candidate points in all neighborhood sliding window images, where k represents the minimum distance between classes. Step 1.2: Determine the area with the most candidate points and calculate the density η of candidate points in that area; Step 1.3: Compare the density of candidate points η with the density threshold ε. If η > ε, find the four extreme values ​​of the candidate point coordinates in the four directions (up, down, left, right) in the region, and expand the neighborhood image by k pixels in each of the four extreme values ​​to obtain the expanded neighborhood image. Calculate the density of candidate points in the neighborhood image. If η < ε, remove the points in the region from the candidate point set to obtain a new candidate point set. Step 1.4: Repeat steps 1.1 to 1.3 until the set of candidate points is empty and the execution ends.

3. The on-board autonomous closed-loop real-time detection method according to claim 1, characterized in that, The method of using current frame and historical frame information to perform target detection on low frame rate image sequences in local areas through a spatiotemporal contrast fusion algorithm includes: Step 2.1: Perform temporal contrast filtering enhancement on the dark background regions of the current frame and historical frames. The enhancement formula is as follows: C(i,j,t)=|I(i,j,t)-m(i,j,t)| In the formula, C(i,j,t) represents the temporal contrast filtering enhancement result, I(i,j,t) represents the pixel gray value at position (i,j) in the image at time t, m(i,j,t) represents the background estimation of the current frame image, and N... l Increase the length of the filter window to enhance the temporal contrast of the image; Step 2.2: Perform spatial contrast filtering enhancement on the dark background region of the current frame. The enhancement formula is as follows: C s (i,j)=|I(i,j)-m s (i,j)| In the formula, C s (i,j) represents the spatial domain filtering enhancement result of the image, I(i,j) represents the gray value of the pixel at position (i,j) in the image, s represents the sum of gray values ​​of the neighborhood window of the removed center pixel, w represents the size of the spatial sliding window, and m represents the value of the pixel. s (i,j) represents the local spatial background estimation result of the image; Step 2.3: The image spatiotemporal contrast enhancement result is obtained by multiplying the temporal enhancement result from the first step and the spatial enhancement result from the second step. The enhancement formula is as follows: In the formula, max i,j (C(i,j,t)) is the maximum value of C(i,j,t), max i,j (C s (i,j) is C s The maximum value of (i,j); Step 2.4: Perform target detection and target extraction on the image obtained in Step 2.

3.

4. The on-board autonomous closed-loop real-time detection method according to claim 1, characterized in that, The method for target detection of this high frame rate image sequence using temporal variance filtering is as follows: The variance characteristics of the temporal profile of each pixel in the statistical sequence image are analyzed. If a pulse appears in the temporal sequence, a target is present. If the temporal gray level changes continuously and slowly, there is no target. Then, the short trajectory of the target passing through the temporal analysis window is extracted by adaptive threshold segmentation.

5. The on-board autonomous closed-loop real-time detection method according to claim 4, characterized in that, The temporal profile variance characteristics of each pixel in the statistical sequence image are obtained by using a temporal variance filtering algorithm to calculate the temporal mean statistical result μ(p) of each pixel in the image. The filtering formula is as follows: In the formula, Var(p) is the statistical result of the temporal variance of each pixel in the image, M and N are the window size of the temporal mean and temporal variance statistics of the pixel, respectively, p is the pixel, and I(p) is the gray value of pixel p.

6. The on-board autonomous closed-loop real-time detection method according to claim 5, characterized in that, The selection of M and N is related to the payload imaging frame rate and the target's motion speed.

Images