A multi-frame fusion night vision image super-resolution enhancement method

Abstract

This invention provides a method for super-resolution enhancement of night vision images through multi-frame fusion, relating to the field of night vision image processing technology. The method includes: continuously acquiring multiple frames of low-resolution night vision images and constructing a frame set to be fused using a sliding window; selecting a reference night vision image from the frame set; performing inter-frame alignment and overlapping region clipping of the frame set based on the reference night vision image; updating the frame set; initializing the projection stage of the frame set; during the projection stage, matching the projection kernel radius and kernel weight reduction ratio from a database; and based on the projection kernel radius and kernel weight reduction ratio, sequentially mapping each frame of night vision images in the frame set to be fused to a target high-resolution pixel matrix. This invention enables the night vision image coverage distribution to become more balanced, reduces checkerboard-like spots and local brightness fluctuations, suppresses high-gain noise and pseudo-textures induced by supplementary lighting saturation, and improves the accuracy and observation stability of super-resolution reconstruction.

Description

Technical Field

[0001] This invention relates to the field of night vision image processing technology, and in particular to a method for super-resolution enhancement of night vision images through multi-frame fusion. Background Technology

[0002] Night vision imaging equipment is widely used in scenarios such as nighttime patrols and security, field observation, emergency search and rescue, and evidence recording. Due to the limitations of low-light environments, night vision images often suffer from high noise levels, insufficient contrast, and weak texture information, making it difficult to clearly present the fine structures and contour details of distant targets. To improve discernibility under low-light conditions, current technologies generally focus on image super-resolution reconstruction. This involves feature extraction, mapping modeling, and reconstruction of low-resolution images to obtain high-resolution outputs containing richer details, thereby improving the effectiveness of observation, identification, and recording.

[0003] For example, the Chinese invention patent CN110111252B discloses a single-image super-resolution method based on a projection matrix. First, by introducing inconsistency constraints between dictionary atoms, a lower-resolution dictionary with stronger expressive power is obtained. Then, for each column of the dictionary, an optimized projection matrix is ​​found. In this matrix, a weight matrix is ​​set to enhance the expressive power of neighboring dictionary atoms for the current image patch and weaken the expressive power of more distant atoms, thus making the dictionary more expressive of the current image patch. Finally, under the constraint of l2 norm regularization of the projection matrix, the correlation between dictionary atoms and feature blocks, as well as the Euclidean distance relationship between feature blocks and their cluster centers, is used to reconstruct high-resolution image features with more detailed information.

[0004] For example, Chinese invention patent CN120563322B discloses a rotation-aware enhanced variable window attention super-resolution method and system, which includes acquiring and preprocessing the original remote sensing image of the target object, segmenting it into several basic windows and extracting feature vectors, using quadrilaterals to predict the basic parameters of the projective transformation matrix based on the extracted feature vectors to obtain the reconstructed projective transformation matrix, extracting the rotation angle difference between any two quadrilaterals as the rotation-aware term, establishing a variable window attention mechanism, embedding the variable window attention mechanism into a SwinIR network to establish the architecture of a high-precision super-resolution model, training the high-precision super-resolution model, inputting real-time remote sensing images into the high-precision super-resolution model, and outputting a high-resolution image containing the target sampling area.

[0005] The aforementioned prior art has the following technical problems: Existing technologies have improved reconstruction quality and geometric consistency from the perspectives of dictionary and projection constraints in single-frame reconstruction, as well as deep model structures based on attention mechanisms. However, in the engineering implementation of multi-frame fusion super-resolution for night vision devices, there are still key issues strongly related to the imaging link and usage conditions. Night vision devices are mostly handheld or lightweight installations, and inter-frame displacement is generally sub-pixel level and exhibits a non-ideal distribution with attitude jitter and image stabilization mechanisms. At the same time, night vision imaging gain is high, and noise and fixed pattern noise are more significant, which may be accompanied by local saturation and reflection anomalies caused by infrared supplementation. This makes it easier to expose uneven sampling coverage and weight accumulation bias during the multi-frame alignment and projection fusion stages.

[0006] Specifically, in the process of mapping multiple frames of low-resolution night vision images to a high-resolution pixel matrix according to the estimated displacement, since the displacement is mostly non-integer pixels and the landing point is a decimal coordinate, if single-point writing or an excessively narrow resampling or projection kernel is used, it is easy for some pixel units in the high-resolution grid to not receive effective contributions for a long time, resulting in insufficient coverage, while other pixel units are repeatedly hit and accumulate excessive contributions, resulting in over-accumulation. If the system does not uniformly normalize the accumulated weight of each high-resolution pixel unit, or does not compensate and degrade areas with insufficient coverage, there will be phenomena of non-conservation of brightness energy and uneven spatial distribution. On the screen, this manifests as checkerboard patterns, speckles, local flickering brightness, dirty flat areas, and enhanced pseudo-texture in detailed areas, which in turn leads to inaccurate resolution adjustment and decreased observation stability. Summary of the Invention

[0007] Therefore, this invention provides a method for super-resolution enhancement of night view images by multi-frame fusion, which can improve the resolution and observation stability of night view images.

[0008] The technical solution of this invention is implemented as follows: This invention provides a method for super-resolution enhancement of night vision images through multi-frame fusion. The method includes: Step 1: Continuously acquiring multiple frames of low-resolution night vision images and constructing a frame set to be fused using a sliding window. A reference night vision image is selected from the frame set to be fused, and inter-frame alignment and overlapping region clipping are performed based on the reference night vision image, thereby updating the frame set to be fused; Step 2: Initializing the projection stage of the frame set to be fused. During the projection stage, the projection kernel radius and kernel weight reduction ratio are matched from the database. Based on the projection kernel radius and kernel weight reduction ratio, each frame of night vision image in the frame set to be fused is sequentially mapped to a target high-resolution pixel matrix. Kernel weights are allocated in descending order of distance within the neighborhood of the projection landing point in the target high-resolution pixel matrix, and an accumulation operation is performed to obtain the accumulation result and update the target high-resolution pixel matrix; Step 3: During the updating of the target high-resolution pixel matrix, the effective contribution times of each high-resolution pixel unit are counted, and partition anomaly analysis is performed. When the anomaly result is that the anomaly ratio coefficient deviates from the anomaly ratio reference range, the projection kernel radius is adaptively increased or decreased based on the anomaly ratio coefficient, thereby enhancing the super-resolution of the night vision images.

[0009] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: (1) This invention continuously acquires multiple frames of low-resolution night view images, constructs a set of frames to be fused by sliding window, selects a reference night view image and uses it as a reference to complete inter-frame alignment and overlapping area clipping, reduces field of view drift caused by hand shake and unifies fusion coordinates; during the initialization projection stage, the projection kernel radius and kernel weight reduction ratio are matched from the database, and the pixels of each frame are mapped to the target high-resolution pixel matrix according to the alignment relationship. The kernel weight is allocated in the neighborhood of the projection landing point according to the distance and the accumulation is performed to alleviate the insufficient coverage caused by single-point writing of small landing points, and suppress the excessive accumulation caused by repeated local hits; during the update of the target high-resolution pixel matrix, the effective contribution times of each high-resolution pixel unit are counted and partition anomaly analysis is performed. When the anomaly ratio coefficient deviates from the anomaly ratio reference range, the projection kernel radius is adaptively increased or decreased to make the coverage distribution tend to be balanced and reduce the weight deviation, thereby reducing checkerboard spots and local brightness fluctuations, suppressing high gain noise and false textures induced by supplementary light saturation, and improving the accuracy and observation stability of super-resolution reconstruction.

[0010] (2) This invention introduces a hierarchical constraint mechanism of frame-level initial weights and pixel-level effective weights during the projection stage. This allows the contribution intensity of each frame to be fused to be limited based on quality differences and alignment complexity before entering the high-resolution pixel matrix mapping. This avoids low-quality frames or frames with insufficient alignment stability from amplifying noise and fixed pattern noise during the accumulation process, and reduces the interference of local saturation anomalies caused by infrared illumination on the fusion results. Furthermore, this invention adaptively matches the projection kernel radius based on the frame-level initial weights during pixel projection, and controls the contribution distribution in the neighborhood of the landing point by the kernel weight reduction ratio. This allows the energy of the landing point with fractional coordinates to spread orderly within the neighborhood, significantly alleviating the problems of insufficient coverage caused by single-point writing and excessive accumulation caused by concentrated hits. At the same time, the synchronous update of the high-resolution contribution accumulation matrix and the high-resolution weight accumulation matrix provides a reliable basis for subsequent consistency normalization, thereby suppressing the checkerboard pattern, speckles, and local dimming phenomena caused by non-conservation of brightness energy and uneven spatial distribution. This reduces dirtiness in flat areas and false texture enhancement in detailed areas, improving the reconstruction accuracy and observation stability of multi-frame super-resolution enhancement in night vision.

[0011] (3) This invention analyzes the effective filling of the target high-resolution pixel matrix online, identifies two types of anomalies—insufficient coverage and excessive hits—based on the effective contribution count of each high-resolution pixel unit, and statistically analyzes the density of abnormal units at the sub-matrix scale to locate abnormal clustering areas. This generates an anomaly ratio coefficient, which is then compared with an anomaly ratio reference interval to achieve adaptive optimization control of the projection kernel radius. Thus, under the non-ideal distribution of sub-pixel displacement caused by handheld shaking and image stabilization in night vision devices, it suppresses coverage gaps caused by single-point writing or an excessively narrow projection kernel, reduces the proportion of pixel units that have not been effectively contributed to the high-resolution grid for a long time, and reduces holes and checkerboard tendencies caused by insufficient coverage. Simultaneously, it limits the risk of excessive accumulation caused by repeated hits of local pixel units, alleviates the uneven spatial distribution of brightness energy, and reduces phenomena such as speckles, local dimming, and false texture enhancement. By using an anomalous scaling factor to drive the increase or decrease of the kernel radius, the projection diffusion range is matched with the contribution distribution under the current working conditions. Even in night vision scenes with strong noise, fixed pattern noise, and supplementary light saturation interference, the credibility of reconstructed details and image consistency can still be maintained, thereby improving the accuracy and observation stability of super-resolution enhancement. Attached Figure Description

[0012] Figure 1 This is a flowchart of a multi-frame fusion night vision image super-resolution enhancement method provided by an embodiment of the present invention; Figure 2 This is a night view image before super-resolution enhancement provided in an embodiment of the present invention; Figure 3 This is a super-resolution enhanced night view image provided in an embodiment of the present invention; Figure 4 This is the channel power test spectrum provided in the embodiments of the present invention. Figure 1 ; Figure 5 This is the channel power test spectrum provided in the embodiments of the present invention. Figure 2 . Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0014] In this application, the term "at least one" means one or more, and the term "multiple" means two or more; for example, multiple devices means two or more devices. "At least two" means two or more. "At least three" means three or more.

[0015] In this application, the terms "first," "second," "third," etc., are used to distinguish identical or similar items with essentially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor does it limit the quantity or order of execution. It should also be understood that although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are merely used to distinguish one element from another.

[0016] Night vision devices typically rely on low-light imaging devices or infrared thermal imaging devices to acquire images of target scenes in low-light or no-light environments. However, due to limitations such as insufficient photon count, sensor pixel size and readout noise, optical system aperture and aberrations, as well as miniaturized lenses and low sampling rates used to meet size and power consumption constraints, their output images often exhibit problems such as limited resolution, loss of detail and texture, significant noise and fixed pattern interference, blurred edges, and insufficient contrast. Based on this, a multi-frame fusion night vision image super-resolution enhancement method is proposed, which significantly improves night vision imaging quality and target readability without increasing optical aperture or hardware pixel cost, meeting the requirements for clarity and stability in scenarios such as night reconnaissance, security patrol, and emergency rescue.

[0017] Taking a handheld low-light night vision device for nighttime inspection as an example, the imaging module of the night vision device is a low-light CMOS sensor with an output resolution of 640×480 and a frame rate of 25fps. The system enables and runs a multi-frame fusion night vision image super-resolution enhancement method on the image processing unit inside the night vision device. Specifically, it can be implemented in the processing link of the main control processor (system-on-a-chip) plus the image signal processor.

[0018] Example 1, as Figure 1The flowchart shown is a method for multi-frame fusion night vision image super-resolution enhancement. The processing flow of this method may include the following steps: Specifically, when the night vision device is working continuously, it continuously acquires multiple frames of low-resolution night vision images and maintains a sliding window. The sliding window represents a buffer structure that retains only the most recent N frames for one fusion. The number of frames N in the window is automatically configured by the device. In this embodiment, N=7.

[0019] When a new frame arrives, the window slides forward and updates, and the system constructs the set of frames to be fused {P1, P2, P3, ..., P7} based on this, where P7 is the latest frame and P1 is the earliest frame in the window. Due to the black level drift and automatic gain change of the sensor under low light, the system first performs brightness scale correction on each night view image in the set of frames to be fused. Brightness scale correction refers to black level offset correction and inter-frame gain normalization.

[0020] Black level offset correction refers to eliminating the base brightness output by the night vision sensor in the absence of light or low light conditions. Specifically, ideally, when the lens is blocked or there is a completely dark area in the field of view, the corresponding pixel should output a pixel grayscale value close to 0. However, due to factors such as sensor dark current, zero-point drift of the readout circuit, and temperature changes, there will be an approximately constant increase in the value of the entire frame, causing the dark area to also exhibit a non-zero grayscale value. This increased reference amount is the black level offset b of that frame. The black level offset can be obtained by directly reading from the light-blocking pixels (black reference pixels) reserved in the night vision module, or by averaging the values ​​in the rows and columns of black borders reserved at the edge of each frame of night vision image. The black level offset correction is completed by subtracting the black level offset of each frame of night vision image from the pixel grayscale value of each pixel in each frame of night vision image.

[0021] Inter-frame gain normalization is used to eliminate the inconsistency in brightness between different frames of the same scene caused by the automatic gain or exposure fluctuations of night vision devices over time. Specifically, a statistical quantity m that can represent the overall brightness scale of the night vision image is first calculated, such as the median of the pixel grayscale values ​​of all pixels in the night vision image. Then, a target brightness scale m_ref is selected, such as the mean of the pixel grayscale values ​​of all pixels in the set of frames to be fused corresponding to the night vision image. Subsequently, all pixels in the night vision image are adjusted by the same proportion: the pixel grayscale value h of each pixel is further transformed into h1 = h × (m_ref / m), where h1 is the changed pixel grayscale value. By using a unified target brightness scale, the night vision image is brightened or darkened as a whole. The above process is performed on each night vision image to align the brightness scale of each night vision image, thereby avoiding artifacts such as brightness jumps, ghosting, or stripes caused by the difference in brightness between frames during multi-frame fusion.

[0022] After completing the above two steps, the frame set to be merged will enter the subsequent quality scoring process under the condition that the black baseline is consistent and the overall brightness scale is consistent.

[0023] The quality score of each night image in the frame set to be fused is analyzed. The quality score is an evaluation index used to characterize the detail quality of night images. The quality score analysis method for a certain night image is as follows: Pixels with a gray value greater than the preset maximum acceptable gray value in the database are marked as saturated pixels. The maximum acceptable gray value represents the maximum allowed gray value of a pixel. Adjacent saturated pixels are grouped into the same saturated connected region. Adjacent means that pixels that satisfy a 4-neighbor or 8-neighbor connection relationship in the pixel grid are considered to be in the same connected region. For each saturated connected region, the number of saturated pixels it contains is counted, and this number is converted into the area of ​​the connected region. The region with the largest area among all saturated connected regions is recorded as the maximum saturated connected region. Its area is called the maximum area of ​​the saturated pixel connected region, which is used to characterize whether there are large areas of overexposure or top-level areas in the frame. The area of ​​the night image in the frame is compared with the maximum area of ​​the saturated pixel connected region. The result is normalized, and the final result is the quality score of the night image in the frame.

[0024] Based on the quality scores of each night view image, the first reduction coefficient corresponding to each night view image is retrieved from the quality score-first reduction coefficient mapping table in the database. The first reduction coefficient represents the proportion of the frame-level initial weight reduction correction. The first reduction coefficient is inversely correlated with the quality score. Thus, under low light conditions, for low-quality frames with high noise, motion blur, local saturation, or unstable exposure, the contribution ratio of these frames in subsequent fusion is actively reduced. This avoids problems such as detail refeedback errors, texture artifact amplification, ghosting, and brightness jumps caused by low-quality frames participating with high weights, thereby improving the stability and reliability of multi-frame fusion results.

[0025] The night view images of each frame in the frame set to be fused are sorted in descending order of quality score; the night view image with the highest quality score is extracted as the reference night view image. By prioritizing the selection of high-quality frames with more detailed information and more controllable noise and blur as the registration benchmark, the inter-frame registration error and sub-pixel displacement estimation deviation can be reduced, and the global alignment drift and edge tearing caused by the degradation of the reference frame can be reduced.

[0026] By using the reference night vision image as a benchmark, the frame alignment and overlapping area clipping of the frame set to be fused can be completed, which can eliminate invalid edge areas and inconsistent coverage areas caused by displacement, and alleviate the phenomena of holes, stretching and ghosting at the fusion boundary. This enables more robust alignment and higher quality super-resolution enhancement output in night vision scenarios where handheld shaking and low light noise coexist.

[0027] The specific process for inter-frame alignment and overlapping region cropping is as follows: Based on the reference night view image, the night view images of each frame in the frame set to be fused, excluding the reference night view image, are determined as the night view images to be fused.

[0028] The displacement information between the night view image to be fused and the reference night view image in each frame is obtained. The displacement information ΔP=(Δx, Δy) is used to characterize the inter-frame translational offset caused by hand shake or slight posture changes. Specifically, it refers to how many pixels the pixel position of the same physical point / texture point in the same scene has changed between the two frames. Δx represents the horizontal coordinate translational offset, and Δy represents the vertical coordinate translational offset. The displacement information can be calculated by feature point matching algorithms, etc. If the coordinates of the tree branch intersection point in the reference night view image are (320, 240), and the coordinates of the same intersection point in the night view image to be fused in the k-th frame are (322, 237), then the displacement information ΔP=(+2, −3) is obtained, which means that the k-th frame is shifted 2 pixels to the right horizontally and 3 pixels downward vertically relative to the reference frame.

[0029] Using the reference night view image coordinate system as the target coordinate system, an alignment relationship is constructed for each frame of the night view image to be fused from the corresponding original coordinates to the reference coordinates in the target coordinate system. The alignment relationship is used to indicate the corresponding position (x′, y′) of any pixel point (x, y) in the night view image to be fused in the reference coordinate system. For example, the relationship can be expressed as: (x′, y′) = (x, y) + (Δx, Δy).

[0030] Based on the alignment relationship between the night view images to be fused in each frame, the night view images to be fused in each frame are aligned to the target coordinate system, so that the same target point in different frames (such as the outline point of a pedestrian's shoulder) falls to the same or approximately the same pixel position after alignment, reducing ghosting and edge tearing during subsequent fusion.

[0031] The alignment time of each frame of night view image to be fused is obtained from the data log. This represents the total time taken for each frame of night view image to complete alignment with the target coordinate system. Based on the alignment time of each frame of night view image to be fused, the second reduction coefficient corresponding to each frame of night view image to be fused is matched from the alignment time-second reduction coefficient mapping table in the database. The second reduction coefficient represents the proportion of the frame-level initial weight reduction correction. The second reduction coefficient is positively correlated with the alignment time, thereby automatically reducing the impact of difficult-to-align frames on the final fusion result.

[0032] After alignment, invalid regions are cropped, retaining only the areas that overlap with the reference night view image. This updates the set of frames to be fused. This cropping operation avoids invalid edge regions being mistakenly treated as valid details during the fusion process, thus preventing the introduction of boundary stripes, stretching, and black edge artifacts. At the same time, it ensures that subsequent super-resolution fusion processes the same field of view intersection in space, improving the stability and consistency of the fusion output.

[0033] After completing inter-frame alignment and overlapping region cropping, the initialization of the frame set to be fused is carried out in the projection stage. The projection stage refers to projecting each frame of low-resolution night view image onto the target high-resolution pixel matrix at a predetermined magnification (e.g., 2×) according to the aligned geometric mapping relationship, and accumulating kernel weights in the neighborhood of the projection landing point of the high-resolution pixel matrix to form the initial high-resolution estimate and statistics for subsequent super-resolution reconstruction.

[0034] For each night view image in the updated set of frames to be fused (including the reference frame and each aligned frame), the system constructs a frame-level initial weight W for that night view image. k Where k represents the number of each night view image frame, and in this embodiment, k is any integer from 1 to 7. The frame-level initial weight is used to characterize the influence of the pixel grayscale value of each pixel in the night view image frame on the final fusion result. The base value of the frame-level initial weight is set to 1, and then the first reduction coefficient α corresponding to each night view image frame is introduced. k With the second decreasing coefficient β k The base values ​​of the initial frame-level weights are adjusted, therefore, the final initial frame-level weights for each night view image are as follows: W k =1×α k ×β k With the above settings, the system can reduce the overall weight of unreliable frames during the projection stage, thereby reducing the risk of misalignment, strong noise or local saturation being amplified in the high-resolution initial estimation, thus mitigating ghosting, stripes and bright spot artifacts.

[0035] After obtaining the frame-level initial weights of each night view image, the frame-level initial weights W of each night view image are... k Mapped to pixel-level effective weights X for each frame of night view image k X k =W k Among them, the pixel-level effective weight is used to characterize the contribution intensity of each pixel in the night view image during the projection stage.

[0036] Initialize the high-resolution contribution accumulation matrix and the high-resolution weight accumulation matrix. The system initializes two accumulation matrices with the same dimensions as the target high-resolution pixel matrix: one is the high-resolution contribution accumulation matrix A, used to accumulate the pixel grayscale values ​​of the updated frame set to be fused during the projection stage; the other is the high-resolution weight accumulation matrix B, used to accumulate the corresponding pixel-level effective weights. The target high-resolution pixel matrix can be denoted as H, and its size is determined by the magnification factor. For example, if 640×480 is magnified by 2× to obtain 1280×960, then the dimensions of matrices A, B, and H are all 1280×960. During initialization, all elements of A, B, and H are set to 0 to facilitate subsequent point-by-point accumulation of contributions for each frame.

[0037] For each frame of night vision image, the system initializes its corresponding projection mapping parameter Π. k The projection mapping parameters are used to map the pixel coordinates in the low-resolution image of this frame to the corresponding positions in the target high-resolution pixel matrix. Since the preceding steps have aligned each frame to the reference coordinate system and completed cropping, Π k This mainly reflects the combined mapping of magnification and subpixel alignment offset: for example, with a magnification of s (such as 2×), the point (x, y) in the high-resolution coordinate system can be represented as (u, v) = (s×x + δu) k , s×y+δv k ), where (δu k ,δv k The subpixel residual offset is obtained by inter-frame alignment. The system can accurately project each pixel of each frame to the correct position of the high-resolution grid, thereby recovering high-frequency details by utilizing the subpixel displacement information between multiple frames.

[0038] The subpixel residual offset term refers to the residual displacement of the night image to be fused relative to the reference night image after inter-frame alignment is completed, which is less than one pixel. It is used to perform subpixel-level position correction on the pixel landing point during projection mapping, so as to improve the super-resolution reconstruction accuracy and reduce ghosting artifacts by utilizing multi-frame micro-displacement information. For example, the displacement of a frame image relative to the reference frame is not exactly an integer number of pixels, but a finer offset such as half a pixel or 0.2 pixels. After inter-frame alignment is completed, there may still be a small offset that has not been completely eliminated. This remaining small offset is called the residual offset, and its magnitude is usually less than 1 pixel, hence the name subpixel. For example, the estimated displacement of a pixel in the k-th frame night image relative to the corresponding pixel in the reference night image is (0.35, -0.20), which means that it has shifted 0.35 pixels to the right in the x-direction and 0.20 pixels down in the y-direction. Even if the image is resampled and aligned to the reference coordinate system, due to interpolation errors, noise, local non-rigidity, etc., there may still be an offset of (0.08, -0.05). Therefore, (0.08, -0.05) is marked as a subpixel residual offset term.

[0039] After the initialization of the projection phase is completed, the projection process is performed on the set of frames to be fused, thus entering the projection phase.

[0040] The night vision images of each frame in the frame set to be fused are projected sequentially according to their timestamps. The timestamps can be output by the camera or generated by the edge processor. For example, for any pixel (a, b) in the k-th frame night vision image, its pixel intensity value T is read. k (a, b), and mapped by projection parameter Π k If we obtain the projection point (u, v) of the target in the high-resolution pixel matrix, then the coordinates of the projection point are (u, v).

[0041] Based on the frame-level initial weight corresponding to the night vision image, the corresponding projection kernel radius is retrieved from the frame-level initial weight-projection kernel radius mapping table in the database. The projection kernel radius is used to limit the effective influence range of the projection landing point coordinates in the target high-resolution pixel matrix. In low-light and high-noise night vision scenarios, the system can use a smaller kernel radius or faster reduction for low-weight (low-quality or difficult-to-align) frames to reduce their diffusion pollution. For high-weight (high-quality) frames, more reasonable kernel coverage is allowed to make full use of their detailed information.

[0042] In the target high-resolution pixel matrix, with the projection point coordinates (u, v) in the k-th frame night view image as the center, the system constructs a neighborhood projection set N in the target high-resolution pixel matrix. k (u, v). The neighborhood projection set consists of the points surrounding the landing point whose distance does not exceed the corresponding projection kernel radius R. kIt consists of high-resolution pixel units that take over the projection contribution of the low-resolution pixel and achieve kernel-weighted diffusion around the landing point.

[0043] The system retrieves the kernel weight reduction ratio ρ of the k-th night view image from the database. k Among them, the kernel weight decrease ratio, used to characterize the degree of attenuation of pixel-level effective weights along the diffusion direction as distance increases, is set and stored by relevant technical personnel for the neighborhood projection set N. k For each high-resolution pixel unit (u', v') in (u, v), calculate the straight-line distance between it and the projection landing point coordinates (u, v). Normalize this straight-line distance and label it as d. When neighboring pixel units do not coincide with the projection landing point, d > 0. Then, based on the distance and the decreasing kernel weight ratio ρ... k Determine the neighborhood projection set N k The kernel weight G for each high-resolution pixel unit (u', v') in (u, v) k (u', ​​v')=W k ×(1 / d)×ρ k When a neighboring pixel unit coincides with the projection landing point, i.e., d equals 0, the system directly sets the kernel weight at that landing point as the pixel-level effective weight X. k .

[0044] The neighborhood projection set and its corresponding kernel weights are used as the projection weight set for the projection landing point coordinates. The high-resolution contribution cumulative matrix and the high-resolution weight cumulative matrix are then updated. Specifically, for each high-resolution pixel unit (u', v') in the projection weight set, the corresponding pixel intensity value T is obtained. k (u', ​​v') and kernel weight G k (u', ​​v'), then A(u', v')←A(u', v')+T k (u',v')×G k (u', ​​v'); B(u', v') ← B (u', v') + G k (u', ​​v'), the above update process is executed sequentially according to the timestamp order of each night view image in the frame set to be merged, and is repeated for all pixels in each frame.

[0045] After updating the high-resolution contribution cumulative matrix and the high-resolution weight cumulative matrix, the target high-resolution pixel matrix is ​​updated. The specific update process is: H(u', v') = A(u', v') / [B(u', v') + ε], where ε is a small positive number to prevent the denominator from being zero.

[0046] This processing mechanism constructs a neighborhood projection set centered on the projection landing point, distributes kernel weights in the neighborhood in decreasing order of distance for contribution accumulation and weight accumulation, and then obtains the target high-resolution pixel value through normalization. This mechanism effectively alleviates problems such as high noise, sampling inconsistencies caused by inter-frame micro-shifts, and holes and invalid pixels at boundaries caused by alignment and resampling in low-light night vision images. By limiting the projection kernel radius and performing kernel-weighted diffusion within the landing point's neighborhood, the contribution of low-resolution pixels can be distributed to the high-resolution grid within a controllable range, reducing holes, rasterized textures, and broken edges caused by sub-pixel landing point discretization. Furthermore, by decreasing kernel weights with distance, the main contributions are concentrated... The influence of objects near the landing point but far from it is naturally suppressed, thereby reducing the probability of alignment errors, local motion, and noise spikes being amplified after spatial diffusion, and reducing ghosting, stripes, and false details. Finally, by accumulating grayscale contributions and weights separately and then normalizing the updates, the contributions of different frames and different pixels in the projection stage can be adaptively fused at the same scale, avoiding brightness jumps and local overexposure or graying caused by inter-frame brightness fluctuations or local weight differences. Thus, without increasing hardware pixels and optical aperture, the stability, edge consistency, and detail readability of the initial super-resolution estimation of night view images are improved, providing a more reliable high-resolution foundation for subsequent iterative reconstruction.

[0047] To further improve the stability and detail consistency of night vision super-resolution images, the system updates the target high-resolution pixel matrix, counts the effective contribution times of each high-resolution pixel unit, and performs partition anomaly analysis to determine whether the current projection kernel radius needs to be adaptively adjusted.

[0048] Among them, the effective contribution count refers to the number of times a certain high-resolution pixel unit is hit by the projection weight set in this projection stage, which is used to characterize the degree to which the pixel unit obtains real information support in the multi-frame projection process.

[0049] A number of high-resolution pixel units whose effective contribution counts are lower than the preset low-count threshold in the database are identified. This indicates that these locations are not adequately covered in multi-frame projection and are prone to holes, broken edges, or missing local details. These are marked as first abnormal units. The low-count threshold represents the minimum number of effective contributions allowed.

[0050] Several high-resolution pixel units whose effective contribution count exceeds a preset high-count threshold are identified. This indicates that these locations are excessively concentrated in multi-frame projection, which may be caused by alignment errors in local superposition, excessive diffusion of the projection kernel, or repeated projection of local textures. This can easily lead to local oversmoothing, brightness accumulation, or enhanced artifacts. These are marked as second abnormal units. The high-count threshold represents the maximum allowed number of effective contributions.

[0051] To avoid unnecessary parameter adjustments triggered by occasional anomalies in a single pixel, the system divides the target high-resolution pixel matrix into several sub-matrices, for example, by block sizes of 32×32 or 64×64, with the specific division size determined by relevant technical personnel.

[0052] In a certain submatrix, the total number of first abnormal units is counted. If the total number of first abnormal units is greater than the preset threshold for the total number of first abnormal units in the database, the submatrix is ​​marked as the first abnormal submatrix. The total number of second abnormal units is counted. If the total number of second abnormal units is greater than the preset threshold for the total number of second abnormal units in the database, the submatrix is ​​marked as the second abnormal submatrix. The first abnormal submatrix reflects insufficient projection coverage and inadequate filling in the region. The threshold for the total number of first abnormal units represents the maximum allowed value for the total number of first abnormal units. The second abnormal submatrix reflects excessive projection coverage and over-concentration in the region. The threshold for the total number of second abnormal units represents the maximum allowed value for the total number of second abnormal units.

[0053] The total number of the first and second abnormal sub-matrices is counted, and the ratio is processed. The result is marked as the abnormality ratio coefficient.

[0054] The submatrix can be simultaneously labeled as both a first and second anomalous submatrix. This is because the first and second anomalous units respectively characterize two different imbalance patterns in the projection filling statistics within the same region. The former corresponds to pixel holes or sparse coverage points with insufficient effective contribution, while the latter corresponds to pixel dense hits or repeated accumulation points with excessive effective contribution. During multi-frame projection in night vision, influenced by factors such as local texture strength, boundary clipping, occlusion / motion, and non-uniformity of alignment residual space, some pixels within the same region may be chronically under-covered, forming sparse holes, while others may be over-accumulated due to pixel clustering or kernel diffusion. This results in a mixed anomaly of insufficient and excessive coverage within the region. By allowing dual labeling of the submatrix, the system can fully preserve the composite anomaly information of the projection statistics in that region, providing a more accurate basis for subsequent adaptive adjustment of the projection kernel radius based on anomaly ratio coefficients or partitioning strategies. This avoids masking anomalous features or biasing the parameter tuning direction due to retaining only a single label.

[0055] It should be explained that, in the same projection phase, the projection kernel radius and kernel weight diffusion strategy have a consistent direction in adjusting the global coverage density: when the projection kernel radius is too small or the kernel weight decays too quickly, the projection contribution of each low-resolution pixel is more concentrated near the landing point, making it difficult to cover the gaps in the high-resolution grid. This results in a generally low number of effective contributions from a large number of high-resolution pixel units, thus exhibiting an abnormal pattern dominated by insufficient coverage. Conversely, when the projection kernel radius is too large or the kernel weight decays too slowly, the contribution of each projection landing point diffuses excessively in the neighborhood, causing a large number of pixel units in the high-resolution grid to be frequently hit and repeatedly accumulated, resulting in an overall high number of effective contributions, thus exhibiting an abnormal pattern dominated by excessive coverage. Since the kernel radius and attenuation parameters have the same spatial influence on the projection coverage and hit probability, and the same kernel configuration is used for the same round of projection to uniformly affect the entire target high-resolution mesh, the anomaly distribution usually shows that one type of anomaly dominates within a single projection statistical cycle. That is, either the overall coverage tends to be insufficient or the overall coverage tends to be excessive. Even if there are slight fluctuations in the local area due to differences in texture richness or edge clipping, it is difficult to form a situation where the two types of anomalies occur simultaneously and in large numbers globally and counteract each other. This makes the unidirectional optimization decision based on the anomaly ratio coefficient interpretable and stable.

[0056] The abnormality ratio coefficient is compared with the abnormality ratio reference range stored in the database, where the abnormality ratio reference range represents the allowable numerical range of the abnormality ratio coefficient.

[0057] If the abnormal scaling factor falls within the abnormal scaling reference range, then there is no need to optimize the projection kernel radius.

[0058] If the abnormal ratio coefficient does not belong to the abnormal ratio reference range, the projection kernel radius needs to be optimized. The specific optimization process is as follows: If the abnormal ratio coefficient is greater than the maximum value of the abnormal ratio reference range, it indicates that the coverage is insufficient and abnormality is dominant. Then, the ratio between the abnormal ratio coefficient and the maximum value of the abnormal ratio reference range is multiplied by the projection kernel radius to increase the projection kernel radius, thereby expanding the effective influence range of the projection point, so as to improve the coverage density of the high-resolution grid and reduce holes and breaks.

[0059] If the anomaly ratio coefficient is less than the minimum value of the anomaly ratio reference interval, it indicates that over-coverage anomalies dominate. In this case, the ratio between the anomaly ratio coefficient and the minimum value of the anomaly ratio reference interval is multiplied by the projection kernel radius to reduce the projection kernel radius. This makes the projection contribution more concentrated near the landing point, thereby suppressing the risks of repeated accumulation, local stacking, and over-smoothing caused by excessive diffusion.

[0060] Through the aforementioned adaptive parameter tuning method based on proportional relationships, the system can dynamically adjust the projection kernel radius according to the projection filling statistics without introducing additional manual intervention, so that the night vision multi-frame super-resolution enhancement can maintain more stable coverage uniformity and detail consistency under different jitter levels and different texture distribution conditions.

[0061] Figure 2 This is a night view image before super-resolution enhancement provided in an embodiment of the present invention. Figure 3 This is a night vision image enhanced by super-resolution according to an embodiment of the present invention. The comparison image shows the imaging differences of the same night vision scene before and after super-resolution enhancement: the main subject is two owls perched on a stake and a branch / rope-like structure, with dense vegetation in the background. Figure 2 compared to, Figure 3 The outline of the owl's head, the edge of its facial disc, and the texture of its feathers are clearer, and the details of the spots around its eyes and body are easier to distinguish. The fiber direction and layer boundaries of the foreground branches / tangle are sharper, the local grayscale transitions are more continuous, and the overall blurriness is reduced, thereby improving the readability of target details and the consistency of the image structure under low-light night vision conditions.

[0062] In Example 2, if the number of valid contributions is counted only once for each hit, low-weight, low-confidence hits will be considered as equally valid as high-weight, high-confidence hits. This could easily lead to the statistical results being artificially inflated by a large number of weak contributions or marginal diffusion contributions, thus misjudging the coverage as sufficient and masking the actual holes, breaks, or local unstable areas.

[0063] Therefore, based on the above, when a high-resolution pixel unit is hit in a certain projection weight set, the system reads the kernel weight corresponding to the hit, accumulates the kernel weight corresponding to the hit of the high-resolution pixel unit, and marks the integer part of the accumulation result as the number of valid contributions.

[0064] By accumulating kernel weights and using their integer parts as the number of valid contributions, the statistics of hit events can be improved from a simple count to a comprehensive measurement of hit confidence and contribution strength. On the one hand, the kernel weights already include factors such as frame-level initial weights and distance reduction. Strong contributions from high-confidence frames near the landing point will significantly increase the accumulated value, while weak diffusion contributions from low-confidence frames or those far from the landing point will only produce a small increment. This avoids the problem of traditional counts of each hit leading to a large number of weak contributions that inflate the statistical results and mask the actual insufficient coverage such as holes and breaks. On the other hand, using the integer part of the accumulated value as the equivalent number of valid contributions can transform continuous weight strength into discrete statistics that are easy to threshold discrimination and partition anomaly analysis. This makes the coverage adequacy assessment of different regions more stable and easier to implement, and enables a more reliable distinction between insufficient and excessive coverage. This provides a more reliable basis for the adaptive adjustment of the projection kernel radius, reduces the risk of artifact diffusion in super-resolution fusion, and improves the consistency of output details.

[0065] The purpose of taking the integer part is to convert the continuous weight support strength into discrete equivalent number, which is convenient for comparison with the preset low number threshold and high number threshold; at the same time, it can suppress the statistical jitter caused by the accumulation of a small number of very small weights and improve the stability of anomaly analysis.

[0066] It should be explained that after completing multi-frame fusion super-resolution enhancement, the night vision device can further transmit the enhanced night vision image to a mobile phone via its integrated 2.4GHz wireless transmission module (Wi-Fi). Specifically, after the night vision device performs frame caching and necessary image encoding and encapsulation on the super-resolution enhancement result, it establishes a connection with a mobile application (such as a mobile phone) via a Wi-Fi link, enabling the mobile phone and the eyepiece to display the real-time image simultaneously on both screens. It also supports saving and sharing photo / video files, thereby meeting the needs for remote preview and portable distribution in scenarios such as night observation, patrol evidence collection, or outdoor recording.

[0067] Figure 4 This is the channel power test spectrum provided in the embodiments of the present invention. Figure 1 , Figure 5 This is the channel power test spectrum provided in the embodiments of the present invention. Figure 2 , Figure 4 and Figure 5 The horizontal axis represents frequency, centered at 2.412 GHz (i.e., Center 2.412 GHz), and the vertical axis represents power amplitude, measured in dBm. The vertical scale indicates power magnitude; a higher curve indicates stronger energy near that frequency. Both graphs are used to evaluate the average conducted output power and spectral distribution of a device transmitting on a specific 2.4 GHz Wi-Fi channel within a specified bandwidth. Figure 4This is a common energy distribution appearance in Direct Sequence Spread Spectrum / Complementary Code Keying (DSSS / CCK); the corresponding Channel Power is 12.83dBm / 20MHz. Figure 5 This is a typical energy distribution pattern of orthogonal frequency division multiplexing (OFDM) transmitted signals; the Channel Power measured in this mode is 14.62 dBm / 20 MHz.

[0068] Figure 4 and Figure 5 All results indicate that the super-resolution enhanced night vision image transmitted to the mobile device remains clear. On one hand, channel power reflects the average conducted output power level of the wireless transmission module at the corresponding frequency and mode. Sufficient and stable output power means that under given transmission distance and obstruction conditions, the receiver can more easily obtain sufficient received signal-to-noise ratio and link margin, thereby reducing the probability of bit errors and retransmissions, and reducing image stuttering and bit rate degradation caused by image quality deterioration. On the other hand, the spectrum curve shows that the transmission energy is mainly concentrated within the specified channel bandwidth, indicating that the transmission spectrum is normal and out-of-band spread is controlled, which can reduce the risk of self-interference and adjacent channel interference, and is conducive to maintaining stable demodulation and effective throughput. On this basis, the transmission of night vision images to the mobile device is usually achieved in the form of encoded and encapsulated digital data streams. Therefore, the data received by the mobile device can maintain pixel-level consistency. The details restored by super-resolution enhancement will not be diluted by analog noise during transmission, thus ensuring that the clarity of the night vision image displayed on the mobile device is consistent with the enhancement result output by the night vision device.

[0069] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0070] The above description is only an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for super-resolution enhancement of night view images through multi-frame fusion, characterized in that, The method includes: Step 1: Continuously acquire multiple frames of low-resolution night vision images and construct a frame set to be fused using a sliding window. Select a reference night vision image from the frame set to be fused and use the reference night vision image as a reference to complete the inter-frame alignment and overlapping area cropping of the frame set to be fused, thereby updating the frame set to be fused. Step 2: Initialize the projection stage of the frame set to be fused. In the projection stage, the projection kernel radius and kernel weight reduction ratio are matched from the database. Based on the projection kernel radius and kernel weight reduction ratio, each night view image in the frame set to be fused is mapped to the target high-resolution pixel matrix in sequence. Kernel weights are assigned in descending order of distance in the neighborhood of the projection landing point in the target high-resolution pixel matrix and the accumulation operation is performed to obtain the accumulation result and update the target high-resolution pixel matrix. Step 3: During the update of the target high-resolution pixel matrix, count the number of effective contributions of each high-resolution pixel unit and perform partition anomaly analysis. When the anomaly result is that the anomaly ratio coefficient deviates from the anomaly ratio reference range, adaptively increase or decrease the projection kernel radius based on the anomaly ratio coefficient to enhance the super-resolution of the night view image.

2. The multi-frame fusion night view image super-resolution enhancement method as described in claim 1, characterized in that, The specific selection process for choosing a reference night view image from the set of frames to be fused is as follows: Continuously acquire multiple frames of low-resolution night vision images; Obtain the number of window frames of the sliding window, and then construct the set of frames to be merged according to the number of window frames of the sliding window; Luminance scale correction is performed on each night view image in the frame set to be fused. The luminance scale correction refers to black level bias correction and inter-frame gain normalization. Analyze the quality scores of each night view image in the frame set to be fused. The quality score is an evaluation index used to characterize the detail quality of the night view image. The night view images in the frame set to be fused are sorted in descending order of quality score; Extract the night view image corresponding to the highest quality score as the reference night view image, and use the reference night view image as the basis to complete the inter-frame alignment and overlapping region cropping of the frame set to be fused. Based on the quality score of each night view image, the first reduction coefficient corresponding to each night view image is matched from the database.

3. The multi-frame fusion night view image super-resolution enhancement method as described in claim 2, characterized in that, The process of aligning the frames to be fused and cropping the overlapping areas based on the reference night view image is as follows: Using the reference night view image as a benchmark, the night view images of each frame in the frame set to be fused, excluding the reference night view image, are determined as the respective night view images to be fused. Obtain the displacement information between the night view image to be fused and the reference night view image in each frame; Using the reference night view image coordinate system as the target coordinate system, an alignment relationship is constructed for each frame of the night view image to be fused from the corresponding original coordinates to the reference coordinates in the target coordinate system. The alignment relationship is used to indicate the corresponding position of any pixel in the night view image to be fused in the reference coordinate system. Based on the alignment relationship between the night view images to be fused in each frame, align the night view images to be fused in each frame to the target coordinate system; Obtain the alignment duration of each frame of the night view image to be fused, and match the second reduction coefficient corresponding to each frame of the night view image to be fused from the database; After alignment, invalid areas are cropped, and areas that overlap with the reference night view image are retained, thereby updating the set of frames to be merged.

4. The multi-frame fusion night view image super-resolution enhancement method as described in claim 1, characterized in that, The initialization process for the projection stage of the frame set to be fused is as follows: For each night view image in the updated set of frames to be fused, construct frame-level initial weights; The frame-level initial weight is determined by the first reduction coefficient and the second reduction coefficient, and is used to limit the contribution intensity of the night view image during the projection stage. After obtaining the frame-level initial weights, the frame-level initial weights are mapped to pixel-level effective weights. The pixel-level effective weights are used to characterize the contribution intensity of the pixel grayscale value of each pixel in the night view image during the projection stage. Initialize the high-resolution contribution accumulation matrix and the high-resolution weight accumulation matrix for projection accumulation. The high-resolution contribution accumulation matrix is ​​used to accumulate the pixel grayscale values ​​of the updated frame set to be fused during the projection stage, and the high-resolution weight accumulation matrix is ​​used to accumulate the corresponding pixel-level effective weights. The matrix dimensions of the high-resolution contribution cumulative matrix and the high-resolution weight cumulative matrix are consistent with the matrix dimensions of the preset target high-resolution pixel matrix, and their initial values ​​are all set to zero. Initialize the corresponding projection mapping parameters for each frame of night view image. The projection mapping parameters are used to map the coordinates of all pixels in each frame of night view image to the corresponding positions in the target high-resolution pixel matrix. After the initialization of the projection phase is completed, projection processing is performed on the set of frames to be fused, thus entering the projection phase.

5. The multi-frame fusion night view image super-resolution enhancement method as described in claim 1, characterized in that, The process of sequentially mapping each night view image in the frame set to be fused to the target high-resolution pixel matrix is ​​as follows: The night view images of each frame in the frame set to be fused are projected sequentially according to the timestamp order. For the pixel coordinates in a certain night view image in the frame set to be fused, the pixel coordinates are mapped to the target high-resolution pixel matrix through the projection mapping parameters corresponding to the night view image, and the projection landing point coordinates are obtained. Based on the frame-level initial weight corresponding to the night view image, the projection kernel radius is matched from the database. The projection kernel radius is used to limit the effective influence range of the projection landing point coordinates in the target high-resolution pixel matrix. Centered on the coordinates of the projection landing point, a projection neighborhood projection set is constructed in the target high-resolution pixel matrix. The neighborhood projection set consists of high-resolution pixel units around the landing point coordinates and within a distance not exceeding the radius of the projection kernel. The kernel weight reduction ratio is matched from the database. The kernel weight reduction ratio is used to characterize the degree of decay of the effective weight at the pixel level as the distance increases along the diffusion direction. For each high-resolution pixel unit in the neighborhood projection set, calculate the distance between the pixel unit and the coordinates of the projection landing point, and determine the corresponding pixel-level effective weight based on the distance and the kernel weight decreasing ratio, and mark it as the kernel weight; The neighborhood projection set and the corresponding kernel weights are used as the projection weight set of the projection landing point coordinates, and the high-resolution contribution cumulative matrix and the high-resolution weight cumulative matrix are updated.

6. The multi-frame fusion night view image super-resolution enhancement method as described in claim 5, characterized in that, The update process for the high-resolution contribution cumulative matrix and the high-resolution weight cumulative matrix is ​​as follows: Obtain the pixel grayscale value of the night view image corresponding to the coordinates of the projection landing point; For each high-resolution pixel unit in the projection weight set, according to the corresponding kernel weight, the product of the pixel gray value and the corresponding kernel weight is added to the position of the high-resolution pixel unit in the high-resolution contribution accumulation matrix to update the high-resolution contribution accumulation matrix. The kernel weights are accumulated to the position of the high-resolution pixel unit in the high-resolution weight accumulation matrix to update the high-resolution weight accumulation matrix; After updating the high-resolution contribution cumulative matrix and the high-resolution weight cumulative matrix, the target high-resolution pixel matrix is ​​updated.

7. The multi-frame fusion night view image super-resolution enhancement method as described in claim 6, characterized in that, The specific update process for the target high-resolution pixel matrix is ​​as follows: The cumulative value of the high-resolution pixel unit in the high-resolution contribution cumulative matrix is ​​divided by the cumulative value of the high-resolution pixel unit in the high-resolution weight cumulative matrix to obtain the pixel gray value of the high-resolution pixel unit in the target high-resolution pixel matrix, thereby updating the target high-resolution pixel matrix. During the update process of the target high-resolution pixel matrix, the effective contribution count of each high-resolution pixel unit is counted and partition anomaly analysis is performed to determine whether the projection kernel radius should be optimized.

8. The multi-frame fusion night view image super-resolution enhancement method as described in claim 7, characterized in that, The method involves counting the effective contribution counts of each high-resolution pixel unit and performing partition anomaly analysis. The specific analysis process is as follows: Analyze the effective filling of the target high-resolution pixel matrix; For each high-resolution pixel unit in the target high-resolution pixel matrix, accumulate the number of effective contributions of each high-resolution pixel unit to be hit by the projection weight set. Based on the number of effective contributions of each high-resolution pixel unit, the effective filling of the target high-resolution pixel matrix is ​​statistically analyzed. Specifically, several high-resolution pixel units whose number of effective contributions is lower than a preset low number threshold are marked as first abnormal units. Several high-resolution pixel units whose effective contribution count exceeds a preset high-count threshold are marked as second abnormal units. Divide the target high-resolution pixel matrix into several sub-matrices; In a certain submatrix, the total number of first abnormal units is counted. If the total number of first abnormal units is greater than a preset threshold for the total number of first abnormal units, the submatrix is ​​marked as a first abnormal submatrix. The total number of second abnormal units is counted. If the total number of second abnormal units is greater than a preset threshold for the total number of second abnormal units, the submatrix is ​​marked as a second abnormal submatrix. Based on the first and second anomaly matrices, determine whether to optimize the projection kernel radius.

9. The multi-frame fusion night view image super-resolution enhancement method as described in claim 8, characterized in that, The specific process for determining whether to optimize the projection kernel radius is as follows: The total number of the first and second abnormal sub-matrices is counted, and the ratio is processed. The result is marked as the abnormality ratio coefficient. Compare the abnormality ratio coefficient with the abnormality ratio reference range; If the abnormal scaling factor belongs to the abnormal scaling reference range, then there is no need to optimize the projection kernel radius; If the abnormal scaling factor does not belong to the abnormal scaling reference range, the projection kernel radius needs to be optimized.

10. The multi-frame fusion night view image super-resolution enhancement method as described in claim 9, characterized in that, The optimization process for the projection kernel radius is as follows: If the anomaly ratio coefficient is greater than the maximum value of the anomaly ratio reference interval, the projection kernel radius is increased based on the proportional relationship between the anomaly ratio coefficient and the maximum value of the anomaly ratio reference interval. If the anomaly ratio coefficient is less than the minimum value of the anomaly ratio reference interval, the projection kernel radius is reduced based on the proportional relationship between the anomaly ratio coefficient and the minimum value of the anomaly ratio reference interval.

Images