A faint light super-resolution imaging method for moving targets

Abstract

A kind of micro light super-resolution imaging method for moving target, comprising: using the space-based remote sensing micro light area array camera with micro sweep super-resolution imaging mode, ground scene is carried out continuous frame imaging;The acquired continuous frame image is combined, and the down-sampling image with improved signal-to-noise ratio is obtained;The obtained down-sampling image is subjected to point target detection, target positioning and motion information extraction, and target coordinate information is calculated;According to the coordinate corresponding relationship, the target coordinates are mapped to the image of original resolution, and the image slice containing moving target is extracted according to the set neighborhood value;The obtained each frame image slice is used for fine matching of local area, and sub-pixel displacement matrix is obtained;The obtained sub-pixel displacement matrix is used as prior input, and the image reconstruction is carried out on multiple image slices, and the super-resolution imaging result is obtained.

Description

Technical Field

[0001] This invention belongs to the field of optical remote sensing technology and relates to a low-light super-resolution imaging method for moving targets. Background Technology

[0002] Leveraging the unique staring imaging mode of low-light area array cameras, super-resolution imaging can be achieved by combining multi-frame image fusion enhancement technology. The camera mechanism, through closed-loop control, performs two-dimensional sub-pixel-level step-by-step translation of the optical image plane on the imaging focal plane, acquiring multiple frames of area array images with sub-pixel dislocations. Combined with on-board registration and super-resolution reconstruction techniques, the system's capabilities exceed the sampling frequency of area array detectors, achieving super-resolution imaging. During the multi-frame super-resolution reconstruction process using the micro-scanning mechanism, since both the micro-scanning mechanism's oscillation and low-light integration imaging require time, if the image contains a moving target, the inter-frame sub-pixel-level displacement of the moving target itself does not match the micro-scanning displacement value returned by the micro-scanning mechanism and its control loop. Instead, it is the superposition result of the micro-scanning mechanism's displacement and the moving target's motion projected onto the focal plane. If the moving target's speed is low (inter-frame displacement difference within 0.1 pixels), the restoration effect is basically unaffected. However, if the moving target's speed is high, registration failure will occur between two frames in the area where the moving target is located. This causes errors in local areas of the image in the multi-frame super-resolution method, thus rendering the camera's super-resolution design capability ineffective when performing imaging and recognition of moving targets. Therefore, an adaptive super-resolution imaging method must be proposed for scenarios with moving targets. Summary of the Invention

[0003] The technical problem solved by this invention is to overcome the shortcomings of the prior art and propose a low-light super-resolution imaging method for moving targets, which solves the problem that existing low-light area array super-resolution imaging cannot effectively reconstruct moving targets when there are moving targets in the image frame.

[0004] The technical solution of this invention is: a low-light super-resolution imaging method for moving targets, comprising:

[0005] A space-based remote sensing low-light array camera with micro-scan super-resolution imaging mode is used to perform continuous frame imaging of ground scenes.

[0006] Pixel merging is performed on the acquired consecutive frame images to obtain a downsampled image with improved signal-to-noise ratio;

[0007] Point target detection, target localization, and motion information extraction are performed on the obtained downsampled image to calculate the target coordinate information; the target coordinates are mapped onto the original resolution image according to the coordinate correspondence, and image slices containing moving targets are extracted according to the set neighborhood values;

[0008] The obtained image slices from each frame are used to perform fine matching of local regions to obtain the sub-pixel displacement matrix;

[0009] Using the obtained sub-pixel displacement matrix as prior input, image reconstruction is performed on multi-frame image slices to obtain super-resolution imaging results.

[0010] The continuous frame imaging of ground objects includes: the space-based remote sensing low-light array camera with micro-scan super-resolution imaging mode is mounted on a satellite platform with step staring function, and performs multi-frame staring low-light imaging of ground targets within a continuous integration time.

[0011] The step of performing pixel merging on the acquired consecutive frame images to obtain a downsampled image with improved signal-to-noise ratio includes: performing pixel merging and downsampling operation on each acquired low-light area array image containing a moving target according to the rule of weighted averaging of adjacent n×n pixels to obtain a downsampled image with improved signal-to-noise ratio; where n is an integer between 1 and 8 and is proportional to the target size.

[0012] The process of performing point target detection, target localization, and motion information extraction on the obtained downsampled image to calculate target coordinate information; mapping the target coordinates onto the original resolution image according to the coordinate correspondence, and extracting image slices containing moving targets based on a set neighborhood value, includes: performing moving target detection on a sequence of frames of the downsampled image and returning target coordinate values; mapping the returned coordinate values ​​onto the original resolution image to obtain the target center coordinate value; setting a slice parameter N to extract image slices containing moving targets centered on the target center coordinate value according to N×N pixel neighborhood values, and storing the image slice sequence in a temporary storage; wherein the slice parameter N is set to be greater than the target envelope size.

[0013] Target coordinate information can also be calculated by accessing infrared point target detection results data synchronized with external time.

[0014] The access to external time-synchronized infrared point target detection result data includes: equipping another medium / long-wave infrared channel imaging camera with a field of view that includes the space-based remote sensing low-light area array camera; the target is detected, located, and positioned in the infrared channel, and after spatiotemporal matching, the coordinates are transformed to the image plane coordinate system of the space-based remote sensing low-light area array camera to obtain infrared point target detection result data.

[0015] When using infrared point target detection result data synchronized with an external source, extracting image slices containing moving targets includes: equipping another mid / long-wave infrared channel imaging camera with a field of view encompassing the space-based remote sensing low-light array camera; detecting, locating, and positioning the target in the infrared channel; transforming the coordinates to the image plane coordinate system of the space-based remote sensing low-light array camera after spatiotemporal matching; obtaining the target center coordinates by mapping the returned coordinates onto the original resolution image; setting the slice parameter N to extract image slices containing moving targets centered on the target center coordinates according to N×N pixel neighborhood values; storing the image slice sequence in a temporary storage; the slice parameter N is set to be greater than the target envelope size.

[0016] The step of performing fine matching of local regions using the obtained image slices to obtain the sub-pixel displacement matrix includes: for each frame F in the image slice i Relative to the reference frame F1, i.e., the first frame in time sorting, the image matching algorithm is executed, and the sub-pixel displacement matrix {H} of each frame relative to the reference frame is calculated through matrix operations. i}

[0017] The process of using the obtained sub-pixel displacement matrix as prior input to reconstruct images from multiple frame image slices to obtain super-resolution imaging results includes:

[0018] Each frame of image slice and its transformation matrix {H} relative to the reference frame i Using the data as input, a multi-frame super-resolution reconstruction algorithm is employed to calculate the super-resolution imaging results.

[0019] The multi-frame super-resolution reconstruction algorithm specifically uses the convex set projection algorithm.

[0020] The advantages of this invention compared to the prior art are:

[0021] 1. This invention can take into account both super-resolution reconstruction of dynamic and static targets, and can also achieve parallel processing of multiple targets by means of multi-target parallel short-time tracking. The modular algorithm has strong plasticity and great upgrade potential.

[0022] 2. By performing image matching and fusion on the original frames with low signal-to-noise ratio and low resolution through post-processing, an output image with improved signal-to-noise ratio and resolution is synthesized. By delegating the workload to post-processing, the complexity and cost of implementing the low-light imaging mode are simplified.

[0023] 3. This invention can acquire additional information such as target motion vectors while completing super-resolution reconstruction, thereby improving the accuracy of multimodal target recognition. Attached Figure Description

[0024] Figure 1 Flowchart of the super-resolution algorithm for moving targets.

[0025] Figure 2 This is a schematic diagram illustrating the principle of forward and backward trajectory optical flow prediction.

[0026] Figure 3 This is a schematic diagram of a pyramid-shaped multi-scale space.

[0027] Figure 4 This is a flowchart of the POCS algorithm.

[0028] Figure 5 This is a schematic diagram of the segmentation results for a moving target.

[0029] Figure 6 A schematic diagram showing the before and after super-resolution of a moving target. Detailed Implementation

[0030] This invention addresses the application needs of space-based nighttime low-light imaging and moving target extraction, providing a low-light super-resolution imaging method for moving targets. The camera mechanism, through closed-loop control, performs two-dimensional sub-pixel-level step translation of the optical image plane on the imaging focal plane, acquiring multiple frames of area array images with sub-pixel dislocations. Combined with registration and super-resolution reconstruction techniques, the system's capability exceeds the sampling frequency of area array detectors, achieving super-resolution imaging. During the multi-frame super-resolution reconstruction process using the micro-scanning mechanism, both the movement and imaging of the micro-scanning mechanism require time. If the image contains a moving target of interest, the inter-frame sub-pixel-level displacement of the moving target itself does not conform to the micro-scanning displacement value returned by the micro-scanning mechanism and its control loop. Instead, it is the superposition of the micro-scanning mechanism's displacement and the moving target's own motion projected onto the focal plane.

[0031] This invention, based on the aforementioned micro-scanning super-resolution imaging technology, describes a low-light super-resolution imaging method for moving targets. Specifically, it comprises low-light moving target detection, moving target image slice extraction, moving target image registration, and super-resolution reconstruction. Low-light moving target detection utilizes multimodal methods such as low-resolution point target detection or infrared point target detection to discover, detect, and acquire motion information of moving targets. Real-time motion information is used to extract slices of the moving target's neighborhood image. Sub-pixel-level registration is performed on the moving target's neighborhood slices, and super-resolution reconstruction of the moving target is completed. This method effectively and adaptively solves the problems of multi-frame super-resolution processing technology failure when moving targets are present in the image and insufficient signal-to-noise ratio under low-light conditions, significantly improving the image resolution of the moving target region.

[0032] The specific process of this invention is as follows: Figure 1As shown, an area-array camera equipped with a micro-scanning super-resolution imaging mode performs continuous frame imaging and performs n×n template pixel merging on the continuous frame images to improve the signal-to-noise ratio performance of target detection, or accesses external time-synchronized mid-wave infrared multi-modal point target detection data; it performs point target detection, localization, and motion information extraction and estimation on the continuous frame images, expresses and extracts the vector information of the moving targets, maps the localization coordinate information onto the original resolution image, and extracts image slices containing moving targets according to a set N×N neighborhood value, storing the slice sequence in a temporary storage; for each frame slice, an image fine matching algorithm is performed relative to a reference frame (usually the first frame in time sorting), and the transformation matrix H of each frame relative to the reference frame is calculated through matrix operations. The obtained H matrix is ​​used as input to perform super-resolution reconstruction on multiple frame slices, simultaneously improving the resolution and signal-to-noise ratio performance within the neighborhood of the moving target.

[0033] The downsampled image is subjected to point target detection, target localization, and motion information extraction to calculate target coordinate information. The target coordinates are then mapped onto the original resolution image according to the coordinate correspondence, and image slices containing moving targets are extracted based on a set neighborhood value. This includes: performing moving target detection on a sequence of frames of the downsampled image and returning target coordinate values; mapping the returned coordinate values ​​onto the original resolution image to obtain the target center coordinate value; setting a slice parameter N to extract image slices containing moving targets centered on the target center coordinate value according to an N×N pixel neighborhood value, and storing the image slice sequence in a temporary storage; the slice parameter N is set to be greater than the target envelope size. In the above steps, it is necessary to detect moving targets in a sequence of frames of stationary orbital array images and to have short-term tracking capability. For motion description in the sequence of images, a corresponding motion field can be used to express the three-dimensional motion. Optical flow is defined as the instantaneous velocity field reflected by a grayscale target point when it undergoes specific motion in an image, satisfying the properties of continuity and differentiability in both space and time within the image region. This assumption is usually considered an important condition for related calculations. Given a sequence of images, let (x, y) be a specific pixel in the image, and let I(x, y, t) be the specific value of this pixel at time t. Let u(x, y) and v(x, y) be the components of the optical flow in the x and y directions, respectively, which are very small with respect to time t, i.e., dt. At this time, for the pixel, it has moved from its initial position to the point (x+dx, y+dy), and there exists dx = udt and dy = vdt. We know that there exists constant brightness, that is, for pixels moving along a certain trajectory, they all have the same grayscale value, which is also called constant brightness. At this time, we have:

[0034] I(x+dx,y+dy,t+dt)=I(x,y,t) (1)

[0035] Now, let's perform a Taylor expansion on the above equation.

[0036]

[0037] In the formula, Since dx, dy, and dt are infinitesimal quantities of second order or greater, they can be ignored. Because dx = udt and dy = vdt exist, after transformation, we obtain...

[0038] I x u+I y v+I t =0 (3)

[0039] In the formula, I x I y I t Decibels represent the partial derivatives of a pixel in different directions, corresponding to x, y, and t. The above equation is then called the optical flow constraint equation, which expresses the relationship between optical flow velocity and spatial gradient. There are two unknowns, u and v. Since there is only one equation, a unique solution cannot be obtained. To solve this equation, other corresponding conditions must be added.

[0040] The Lucas-Kanada LK optical flow method assumes that motion vectors are invariant in a small spatial neighborhood. Furthermore, it uses weighted least squares for specific optical flow estimation. This method is chosen because it is relatively convenient for inputting a set of points from the image, making it widely applicable in sparse optical flow fields. This algorithm is also based on certain assumptions, specifically:

[0041] 1) Unchanging brightness. For the target pixel, it undergoes relevant motion in different frames and has the property of invariant appearance. For grayscale images, it is assumed that the brightness remains constant throughout the entire tracking process.

[0042] 2) It has temporal continuity. As the image moves continuously throughout the entire process, its movement is relatively slow relative to time. However, in reality, the former has a small proportion to the image movement, which results in very small movements between adjacent frames.

[0043] 3) Spatial consistency. For the same surface in the same scene, nearby pixels have similar movements, and their projections are also adjacent.

[0044] For very small domains, appropriate changes can be made when estimating motion-related information:

[0045]

[0046] Among them W 2 (X) specifically represents the window weight function. The existence of this function results in a larger weighting ratio between the center and its surroundings. Where X1, X2, ..., X... n These are the n specific points in the set Ω, defined here as V = (u, v). T ΔI(X)=(I x I y ) T The solution to the above equation is then obtained using the least squares method:

[0047] A T W 2 AV = A T W 2 b (5)

[0048] in,

[0049]

[0050] W = diag(W(X1), ..., W(X)) N (7)

[0051]

[0052] V = (A T W 2 A) -1 A T W 2 b (9)

[0053] The reliability of the obtained V estimate is directly affected by A. T W 2 The influence of eigenvalues ​​of A. Assuming eigenvalues ​​exist λ1, λ2, and λ1≥λ2, then τ is used as a hypothetical threshold. When λ1≥λ2≥τ, V can be fully obtained. When λ2=0, the fragmented matrix becomes a singular matrix, making it impossible to solve for its optical flow. When λ1≥τ and λ2<τ, obtaining V information is impossible; only the discovery component of the optical flow is obtained. In the TLD algorithm, the forward and backward optical flow method uniformly divides the initial tracking box according to the current frame I... t Predict the next frame I t+1 The location of the midpoint. Simultaneously, perform reverse prediction, i.e., from I... t+1 The location of the point detected in the middle is predicted to be I.t The original position is used as the reference point. If the displacement deviation between the predicted feature point and the original feature point exceeds a threshold, the original feature point is discarded to maintain the relative stability of the short-term tracker. The principle of forward and backward trajectory optical flow prediction is as follows: Figure 2 As shown.

[0054] If at time k=1, X t+1 and The two overlap. Therefore, for I... t and I t+1 As two consecutive frames, forward prediction can be performed at this point, thus enabling the prediction of X. t Predict, and thus obtain By applying the forward and backward optical flow method, which employs a relative motion field, it exhibits adaptability in extracting moving targets from sequential frame images with sub-pixel micro-scanning and possesses a certain short-term target tracking and prediction capability, thereby improving the execution efficiency of subsequent algorithms.

[0055] Matching multi-frame sequence slices involves: based on the relative displacement estimate obtained from the motion estimation in the above formula, coarsely locating the pixel-level matching position in the fine matching stage; then, using this position as the center, performing sub-pixel interpolation on the neighborhood of the correlation function, reducing computational load while improving matching accuracy to the sub-pixel level; after matching, filtering matching point pairs based on epipolar constraints to remove mismatched points. Combining the invariance of SIFT feature point extraction method to rotation, scaling, and affine transformations, as well as its stability against noise, viewpoint, and illumination changes, a sub-pixel-level inter-frame fine matching method suitable for the research content of this invention and with excellent performance is proposed.

[0056] Image matching algorithms applied to remote sensing imaging need to be stable against noise, viewpoint, and illumination, and invariant to rotation, scaling, and affine transformations. Spatial feature point extraction and representation methods that meet these conditions and offer excellent performance include SIFT (Scale Invariant Feature Transform) and SURF (Speeded Up Robust Feature). Both SIFT and SURF algorithms are based on Gaussian pyramids to find local extrema to determine scale-invariant stable points. In the feature point description stage, both algorithms use the feature point as the center and statistically analyze features in its surrounding neighborhood. SIFT calculates a histogram of gradient magnitudes within a square neighborhood and finds the direction corresponding to the largest magnitude. SURF calculates the Haar wavelet responses along the two coordinate axes of the image space within a circular neighborhood and finds the sector direction with the largest modulus. In terms of performance, SURF runs only one-third as fast as SIFT, but its description accuracy is lower. Considering the application background of this method and weighing resource efficiency, this invention uses SIFT as the theoretical basis for sub-pixel-level feature point extraction and representation, and makes adaptive improvements. Feature point extraction and description consists of the following five steps:

[0057] 1) Constructing scale space

[0058] Since the Gaussian convolution kernel is the only linear kernel that achieves scale transformation, the original image is convolved with a two-dimensional scale-variable Gaussian function.

[0059] L(x,y,σ)=G(x,y,σ)*I(x,y) (10)

[0060]

[0061] In the formula, I(x,y) is the original image; (x,y) represents the coordinates in the image space coordinate system, and σ is the scale coordinate.

[0062] The maxima and minima of the Laplacian function of Gaussian produce the most stable image features compared to other commonly used feature extraction functions, such as Hessian or Harris corner features. To simplify computation, the Difference of Gussian (DOG) operator is typically used to approximate the Laplacian function of Gaussian to construct a pyramid-shaped multi-scale space, such as... Figure 3 As shown.

[0063] D(x,y,σ)=(G(x,y,kσ)-G(x,y,σ))·I(x,y)=L(x,y,kσ)-L(x,y,σ) (12)

[0064] In the formula, k is a constant factor.

[0065] The image pyramid consists of O groups, each with S layers. The image of the next group is obtained by downsampling the image of the previous group. Figure 3 Each layer on the right is formed by subtracting two adjacent Gaussian images.

[0066] 2) Coarse localization of feature points

[0067] Each sampling point is compared with all its neighboring points in the multi-resolution scale space, that is, with its 8 neighboring points at the same scale and 9×2 points corresponding to the adjacent scales above and below, for a total of 26 points, to find the extreme points in the scale space and image space as coarse localization feature points of the image.

[0068] 3) Precise localization of feature points

[0069] To obtain accurate and robust feature points and improve the positioning accuracy to the sub-pixel level, a binary quadratic fitting is performed on the coarse positioning space. The discrete spatial points are interpolated to obtain the information of the continuous spatial extrema points. This method is called sub-pixel interpolation. At the same time, edge points generated by the edge response of the DOG operator are removed.

[0070] 4) Determining the principal direction of feature points

[0071] The image gradient of key points is calculated, and a histogram is used to statistically analyze the gradient and direction of pixels in the neighborhood. The gradient histogram divides the directional range from 0° to 360° into 36 bars. The peak direction of the histogram is used as the principal direction of the feature point.

[0072] 5) Feature point description

[0073] For each feature point, there are three descriptive pieces of information: location, scale, and orientation. The purpose of feature point description is to represent the feature point with a unique feature vector. The descriptor uses gradient information in eight directions calculated within a 4×4 window in the keypoint scale space, resulting in a 4×4×8=128-dimensional vector representation.

[0074] The cosine similarity method of the angle between feature point vectors is used to calculate and match the similarity of the 128-dimensional feature descriptor vectors extracted by the SIFT algorithm. First, for each feature point P0 in the reference frame, the inverse cosine function is used to calculate the similarity of all feature points P in the image to be matched. k Angle with the feature descriptor vector of P0

[0075]

[0076] In the formula and P0 and P respectively k The normalized 128-dimensional feature descriptor vector.

[0077] After calculation, the 128 included angles are sorted. If the ratio of the smallest included angle to the second smallest included angle is less than a certain threshold, then P0 and P1 are determined. k This is a pair of matching feature points. Regarding the threshold selection, 0.4 to 0.6 is generally optimal; the smaller the threshold, the more rigorous the identification of the matching relationship. This step performs fine matching on regions of the same target in different frames, averaging the matching results, achieving an accuracy of 0.1 pixels. Based on the coordinate correspondence of the matching point pairs in the images, the displacement matrix H of the two frames can be directly calculated.

[0078] Super-resolution reconstruction is performed on the matched multi-frame sequence slices, including target slice super-resolution reconstruction based on the convex set projection method. The convex set projection (POCS) algorithm considers a nonlinear degradation model, and the algorithm flow is as follows: Figure 4 As shown, a high-resolution image is reconstructed from multiple low-resolution images. Using the displacement matrix H between the multiple frames as a priori, the rich information contained in the multiple low-resolution images is super-resolution processed to obtain a high-resolution image higher than the camera resolution used in the recorded image. Corresponding to the frequency domain algorithm, the POCS algorithm is a spatial domain high-resolution image reconstruction method. It starts iteratively from a guessed initial image and corrects the image successively according to the simulation error. The unknown image is assumed to be an element in a suitable Hilbert space. Each prior knowledge or constraint about the unknown image restricts the solution of a closed convex set in the Hilbert space. An amplitude boundary constraint (0 to the maximum gray value) is introduced, deriving an iterative formula for solving the unknown image. The super-resolution image is iteratively calculated from the initial estimate, and set theory methods are used to recover the super-resolution image. It effectively utilizes the spatial range observation model while allowing the inclusion of prior information. The image space solved by the SuperResolution Reconstruction method intersects with a set of constraints describing the features of the ideal SuperResolution image, including determinism, energy within the bounds, data fidelity, smoothness, etc., and then forms a simplified solution space.

[0079] The POCS algorithm is an iterative process that, given any point in the super-resolution image space, locates a point that satisfies the set of all convex constraints.

[0080] The convex set of any observed low-resolution image can be written as:

[0081]

[0082] Where 0 ≤ n1, n2 ≤ N-1, k = 1, 2, ..., L

[0083] In this formula, The value can be given by the following formula:

[0084]

[0085] Where δ0 represents the confidence level in the observation, it is set to be equal to cσ. ν Where σ ν The standard noise bias c > 0 is determined by an appropriate confidence boundary. These parameters define the super-resolution image, and this set defines a confidence boundary for a super-resolution image related to low-resolution frames within a certain boundary range.

[0086] The boundary and noise deviation are in a certain proportion.

[0087] For any x i (m1,m2) to Projection on: The definition is as follows:

[0088]

[0089] In the above formula equal:

[0090]

[0091] In addition, limitations such as amplitude can be used to improve the results.

[0092] P based on amplitude constraint A Defined as:

[0093]

[0094] After the projection above is given, the super-resolution image s i Estimation of (m1, m2) It can be obtained through iteration, and the formula is as follows:

[0095]

[0096] j=0,1,…,0≤m1≤M-1, 0≤m2≤M-1.

[0097] Here, T refers to the projection operator, the initial one. This can be achieved by bilinear interpolation on a low-resolution reference frame. Generally, the more iterations, the better the results; additionally, using a wider set of constraints also yields better results. Figure 5 and Figure 6 The images show the results of moving target slicing and the effects before and after image super-resolution reconstruction, respectively.

Claims

1. A method for micro-light super-resolution imaging of a moving target, characterized in that, The application relates to a method for realizing super-resolution imaging of a moving target on the ground. The method comprises the following steps: a space-based remote sensing micro-light area array camera with a micro-sweep super-resolution imaging mode is used to continuously image a ground scene; a pixel combination is performed on the obtained continuous frame images to obtain a down-sampling image with improved signal-to-noise ratio; point target detection, target positioning and motion information extraction are performed on the obtained down-sampling image to calculate target coordinate information; the target coordinates are mapped to the original resolution image according to the coordinate correspondence, and an image slice containing a moving target is extracted according to a set neighborhood value; a sub-pixel displacement matrix is obtained by performing local area precise matching on the obtained image slices; 2. The faint light super-resolution imaging method for moving targets according to claim 1, characterized in that, the obtained sub-pixel displacement matrix is used as prior input to perform image reconstruction on the multiple image slices to obtain a super-resolution imaging result.

3. The faint light super-resolution imaging method for moving target according to claim 1, characterized in that, The continuous frame imaging of the ground scene comprises the following steps:

4. The faint light super-resolution imaging method for moving target according to claim 3, characterized in that, the space-based remote sensing micro-light area array camera with the micro-sweep super-resolution imaging mode is carried on a satellite platform with a step-by-step staring function, and multiple staring micro-light images of the ground target are obtained in a continuous integration time.

5. The faint light super-resolution imaging method for moving target according to claim 1, characterized in that, The pixel combination of the obtained continuous frame images to obtain a down-sampling image with improved signal-to-noise ratio comprises the following steps:

6. The faint light super-resolution imaging method for moving target according to claim 5, characterized in that, a pixel combination and down-sampling operation is performed on each frame of the micro-light area array image containing a moving target according to the rule of averaging the adjacent n*n pixels to obtain a down-sampling image with improved signal-to-noise ratio; n is an integer between 1 and 8, and is proportional to the target size. The point target detection, target positioning and motion information extraction of the obtained down-sampling image to calculate target coordinate information; the target coordinates are mapped to the original resolution image according to the coordinate correspondence, and an image slice containing a moving target is extracted according to a set neighborhood value, which comprises the following steps: motion target detection is performed on the sequence frames of the down-sampling image to return target coordinate values; the target center coordinate values are obtained by mapping the returned coordinate values to the original resolution image; an image slice containing a moving target is extracted according to the N*N pixel neighborhood value with the target center coordinate values as the center, and the image slice sequence is stored in a temporary storage; the setting of the slice parameter N is greater than the target envelope size. The target coordinate information can also be calculated by accessing external time-synchronized infrared point target detection result data. The external time-synchronized infrared point target detection result data comprises the following steps: another mid / long-wave infrared channel imaging camera is equipped, and the field of view range contains the space-based remote sensing micro-light area array camera; the target is detected, detected and positioned in the infrared channel, and the coordinates are converted to the image plane coordinate system of the space-based remote sensing micro-light area array camera after space-time matching to obtain infrared point target detection result data.

7. The faint light super-resolution imaging method for moving target according to claim 6, characterized in that, When the infrared point target detection result data is accessed to external time synchronization, the image slice containing the moving target is extracted, including: equipping another middle / long wave infrared channel imaging camera, the field of view range containing the space-based remote sensing low-light face array camera; the target is detected, detected, positioned in the infrared channel, and the coordinates are converted to the image plane coordinate system of the space-based remote sensing low-light face array camera after space-time domain matching, the infrared point target detection result data; the target center coordinate value is obtained according to the mapping of the returned coordinate value to the original resolution image; the slice parameter N is set to the target center coordinate value as the center, and the image slice containing the moving target is extracted according to the N*N pixel neighborhood value, and the image slice sequence is stored in the temporary storage; the setting of the slice parameter N is greater than the target envelope size.

8. The faint light super-resolution imaging method for moving target according to claim 1, characterized in that, The obtained each frame image slice is used to perform fine matching on a local region to obtain a sub-pixel displacement matrix, including: performing image matching algorithm on each frame F i The image matching algorithm is performed on each frame relative to the reference frame F1, i.e., the first frame in time sequence, and a sub-pixel displacement matrix {H i} of each frame relative to the reference frame is calculated through matrix operation.

9. The faint light super-resolution imaging method for moving target according to claim 8, characterized in that, The obtained sub-pixel displacement matrix is used as a prior input to perform image reconstruction on the multiple image slices to obtain a super-resolution imaging result, including: Taking each image slice and its transformation matrix {H i} relative to the reference frame as input, a multi-frame super-resolution reconstruction algorithm is used to calculate a super-resolution imaging result.

10. The faint light super-resolution imaging method for moving target according to claim 9, characterized in that, The multi-frame super-resolution reconstruction algorithm is specifically selected as a convex set projection algorithm.

Images