A method and system for enhancing imaging of a low-light-level enhanced camera

Abstract

The present invention provides a low-light enhancement camera imaging enhancement method and system. The method comprises the following steps: obtaining a target video shot by a micro-array compound-eye lens camera, performing frame-by-frame splitting processing on the target video to obtain a plurality of target video frames, and pre-processing the corresponding target video frames to obtain a plurality of target images; establishing an initial filter, updating the initial filter to obtain an updated filter, and performing adaptive filtering processing on the target image based on the updated filter to obtain a filtered image; identifying the noise level of the filtered image, and determining a low-light image frame based on the noise level; performing Gaussian filtering processing and detail enhancement on the low-light image frame; and performing interpolation enhancement processing on the initial enhanced image frame to obtain an enhanced video frame. The present invention determines a corresponding low-light image based on the noise level of the image, and performs enhancement processing on the low-light image, thereby significantly reducing the computational complexity and realizing super-resolution enhancement of the low-light image.

Description

Technical Field

[0001] The present invention belongs to the technical field of low-light-level cameras and low-light-level imaging and enhancement processing, and in particular relates to a low-light-level enhancement camera imaging enhancement method and system. Background Art

[0002] Under low-light conditions, when the ambient light is insufficient, the number of photons received by the camera (camcorder) decreases, resulting in reduced image brightness and loss of details. In addition, in harsh and hostile environments (rain, fog, dust, low light, interference, etc.), the pixel value and readout circuit output signal-to-noise ratio (SNR) of the imaging sensor used by the camera / camcorder are low, and the noise becomes significant and disturbs the image content, resulting in image video degradation and quality reduction.

[0003] As for the imaging enhancement methods in the prior art, all pixels of the acquired low-light or degraded images or video frame sequences are processed equally weighted, regardless of their temporal and spatial distribution. This globally unified processing method, which does not differentiate between bright light, weak light, extremely dark areas or weak video frame signals, is also unable to recover image details from extremely dark areas or the boundary between bright light areas and shadows, thereby affecting the image quality. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a low-light-enhanced camera imaging enhancement method and system, which are used to solve the technical problems in the prior art.

[0005] On the one hand, the present invention provides the following technical solution, a low-light enhancement camera imaging enhancement method, comprising:

[0006] Obtain a target video captured by a microarray compound eye lens camera, split the target video frame by frame to obtain a number of target video frames, and pre-process the corresponding target video frames to obtain a number of target images;

[0007] Establishing an initial filter, updating the initial filter to obtain an updated filter, and performing adaptive filtering on the target image based on the updated filter to obtain a filtered image;

[0008] identifying a noise level of the filtered image, and determining a low-light image frame based on the noise level;

[0009] Performing Gaussian filtering and detail enhancement on the low-light image frame to obtain an initial enhanced image frame;

[0010] Performing interpolation enhancement processing on the initial enhanced image frame to obtain an enhanced video frame;

[0011] The step of performing Gaussian filtering and detail enhancement on the low-light image frame to obtain an initial enhanced image frame comprises:

[0012] The low-light image frame Perform Gaussian filtering to obtain a Gaussian filtered image :

[0013] ;

[0014] ;

[0015] In the formula, , Respectively represent the pixels in the low-light image frame and the Gaussian filter image The gray value of is the Gaussian standard deviation, is the Gaussian kernel function, Indicated in pixels The grayscale value of the pixel in the neighborhood centered at is the difference threshold;

[0016] The low-light image frame Perform domain conversion and scale fusion to obtain a scale fused image :

[0017] ;

[0018] In the formula, For scale, The scale is The weight factor under , They represent inverse Fourier transform and Fourier transform respectively. The scale is Gaussian function under ;

[0019] Based on the Gaussian filter image Fusion image with the scale An initial enhanced image frame is determined.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention first obtains a target video captured by a micro-array compound-eye lens camera, performs frame-by-frame splitting processing on the target video to obtain a number of target video frames, and pre-processes the corresponding target video frames to obtain a number of target images; then an initial filter is established, the initial filter is updated to obtain an updated filter, and the target image is adaptively filtered based on the updated filter to obtain a filtered image; then the low-light image frame is Gaussian filtered and detail enhanced to obtain an initial enhanced image frame; finally, the initial enhanced image frame is interpolated and enhanced to obtain an enhanced video frame. The present invention determines the corresponding low-light image based on the noise level of the image, and performs filtering, detail enhancement and interpolation processing on the low-light image, which significantly reduces the computational complexity, realizes super-resolution enhancement of the low-light image, and enhances the contrast of the image and the detail information in the image.

[0021] Preferably, the step of splitting the target video frame by frame to obtain a plurality of target video frames, and preprocessing the corresponding target video frames to obtain a plurality of target images comprises:

[0022] Label each sub-camera in the micro-array compound eye lens camera, and split the target video corresponding to each sub-camera frame by frame to obtain several groups of target video frames;

[0023] Performing median filtering and histogram equalization processing on each group of target video frames in sequence to obtain processed image frames;

[0024] According to the label of each sub-camera, the processed image frames of the corresponding frames are sequentially processed with image registration and image stitching to obtain a number of target images.

[0025] Preferably, the step of updating the initial filter to obtain an updated filter includes:

[0026] Initializing the weight vector of the initial filter and the target image;

[0027] Calculate the error signal of the initial filter :

[0028] ;

[0029] In the formula, represents the expected output signal, Indicates The weight vector after iterations is Indicates The target image signal after iterations;

[0030] Determine the weight update formula:

[0031] ;

[0032] In the formula, Indicates The weight vector after iterations is Represents the learning rate parameter;

[0033] The weight vector of the initial filter is iteratively updated based on the weight update formula until an iteration stop condition is met to obtain an updated filter.

[0034] Preferably, the step of identifying the noise level of the filtered image and determining the low-light image frame based on the noise level comprises:

[0035] Setting a plurality of vertices in each of the filter images and performing graph domain transformation respectively to obtain a plurality of undirected graphs;

[0036] Calculate the Laplacian matrix corresponding to each undirected graph :

[0037] ;

[0038] In the formula, , Respectively represent The degree matrix and adjacency matrix of an undirected graph;

[0039] Based on the Laplace matrix Determine the equation to solve:

[0040] ;

[0041] In the formula, represents the matrix determinant function, Indicates The eigenvalue vector of an undirected graph, represents the identity matrix;

[0042] Solve the eigenvalue vector of each undirected graph based on the solution equation , and arrange the eigenvalues ​​in the eigenvalue vector in descending order to obtain the arrangement vector :

[0043] ;

[0044] In the formula, Indicates The first eigenvalues, Indicates The number of vertices in an undirected graph;

[0045] Determine the largest eigenvalue in each of the permutation vectors :

[0046] ;

[0047] Based on the maximum eigenvalue Determine the low-light image frame.

[0048] Preferably, the maximum eigenvalue based The steps of determining the low-light image frame include:

[0049] like , then the corresponding filtered image is a low-noise image;

[0050] like , then the corresponding filtered image is a medium noise image;

[0051] like , the corresponding filtered image is a high-noise image, and the high-noise image is used as the low-light image frame.

[0052] Preferably, the Gaussian filter image Fusion image with the scale The steps of determining the initial enhanced image frame include:

[0053] For the Gaussian filter image Perform linear transformation enhancement to obtain a linear transformation image :

[0054] ;

[0055] In the formula, , Respectively represent the minimum gray level and maximum gray level in the linear transformation image, Respectively represent the minimum gray level and maximum gray level in the Gaussian filter image, Respectively represent the first partition threshold and the second partition threshold in the Gaussian filter image, Respectively represent the first partition threshold and the second partition threshold in the linear transformation image, is the gray level of the linear transformation image;

[0056] Fusion of images at the scale Perform brightness enhancement processing to obtain a brightness enhanced image :

[0057] ;

[0058] In the formula, Indicated in pixels In the neighborhood of the center, except for the pixel point The average brightness of the remaining pixels, Represents the brightness mean of the scale fusion image;

[0059] The linear transformation image , Brightness Enhanced Image Fusion is performed to obtain the initial enhanced image frame :

[0060] ;

[0061] In the formula, Indicates Gaussian templates, represents the number of Gaussian templates, Represent the first proportional coefficient and the second proportional coefficient respectively.

[0062] Preferably, the step of performing interpolation enhancement processing on the initial enhanced image frame to obtain an enhanced video frame includes:

[0063] Determine the floating point coordinates of the pixel points in the initial enhanced image frame mapped to the target image according to the preset scaling ratio :

[0064] ; ;

[0065] In the formula, represents the pixel coordinates in the low-light image frame, , Respectively represent the width and height of the target image, , Indicates the width and height of the low-light image frame;

[0066] In the target image, the floating point coordinates Determine the neighborhood range for the central pixel and determine the bicubic interpolation function;

[0067] Determine the interpolation weights between the pixels in the neighborhood and the central pixel based on the bicubic interpolation function;

[0068] The interpolation weights of the pixels within the neighborhood are weighted averaged to obtain enhanced pixels, and an enhanced video frame is determined based on the enhanced pixels.

[0069] In a second aspect, the present invention provides the following technical solution: a low-light-enhanced camera imaging enhancement system, the system adopts the low-light-enhanced camera imaging enhancement method as described above, and the system comprises:

[0070] A processing module is used to obtain a target video captured by a microarray compound eye lens camera, split the target video frame by frame to obtain a plurality of target video frames, and pre-process the corresponding target video frames to obtain a plurality of target images;

[0071] A filtering module, used for establishing an initial filter, updating the initial filter to obtain an updated filter, and performing adaptive filtering on the target image based on the updated filter to obtain a filtered image;

[0072] an identification module, configured to identify a noise level of the filtered image and determine a low-light image frame based on the noise level;

[0073] A detail module performs Gaussian filtering and detail enhancement on the low-light image frame to obtain an initial enhanced image frame;

[0074] The enhancement module is used to perform interpolation enhancement processing on the initial enhanced image frame to obtain an enhanced video frame.

[0075] In a third aspect, the present invention provides the following technical solution: a computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned low-light-enhanced camera imaging enhancement method when executing the computer program.

[0076] In a fourth aspect, the present invention provides the following technical solution: a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-mentioned low-light-enhanced camera imaging enhancement method. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0078] Figure 1 A flowchart of a low-light enhancement camera imaging enhancement method provided in Embodiment 1 of the present invention;

[0079] Figure 2 A structural block diagram of a low-light-level enhancement camera imaging enhancement system provided in Embodiment 2 of the present invention;

[0080] Figure 3 A schematic diagram of the hardware structure of a computer provided in another embodiment of the present invention.

[0081] The embodiments of the present invention will be further described below with reference to the accompanying drawings. DETAILED DESCRIPTION

[0082] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the embodiments of the present invention, and should not be construed as limiting the present invention.

[0083] In the description of the embodiments of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0084] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0085] In the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.

[0086] Embodiment 1

[0087] In the first embodiment of the present invention, Figure 1 As shown, a low-light enhancement camera imaging enhancement method includes:

[0088] S1, obtaining a target video captured by a microarray compound eye lens camera, splitting the target video frame by frame to obtain a number of target video frames, and preprocessing the corresponding target video frames to obtain a number of target images;

[0089] Specifically, the microarray compound-eye lens camera includes a number of sub-cameras distributed in an array. Each sub-camera captures an image of a corresponding area, and then the images are stitched together to obtain a complete image.

[0090] Wherein, the step S1 comprises:

[0091] S11, labeling each sub-camera in the micro-array compound-eye lens camera, and splitting the target video corresponding to each sub-camera frame by frame to obtain several groups of target video frames.

[0092] S12, performing median filtering and histogram equalization processing on each group of the target video frames in sequence to obtain processed image frames;

[0093] Specifically, median filtering is a nonlinear filtering technology. The basic principle is to sort the grayscale values ​​of the surrounding pixels of each pixel in the image, and then take the median of the sorting as the grayscale value of the pixel. It can effectively remove fine noise points such as salt and pepper noise in the image while retaining the edge information of the image. At the same time, in a dim light environment, due to insufficient light, the brightness and contrast of the image video are often low, resulting in unclear details and layering of the image. Therefore, histogram equalization is generally used to transform the image histogram so that the grayscale value distribution of the output image is more uniform, which can effectively enhance the brightness and contrast of low-light image videos and improve the visual effect of the image.

[0094] S13, performing image registration and image stitching processing on the processed image frames of the corresponding frames in sequence according to the label of each sub-camera to obtain a plurality of target images;

[0095] Specifically, for each sub-camera, it can form a partial image. By stitching and correcting these partial images, a complete image can be obtained. Image registration determines the relative position and transformation relationship between each ommatidium image through methods such as feature point matching. Through the above relationship, the images taken by the sub-cameras are spatially transformed and stitched to form a complete image. There are overlapping areas in the stitching process. The weighted average algorithm is introduced to process these overlapping areas and smoothly transition the images in the overlapping areas to avoid the appearance of stitching seams.

[0096] S2, establishing an initial filter, updating the initial filter to obtain an updated filter, and performing adaptive filtering on the target image based on the updated filter to obtain a filtered image;

[0097] Specifically, the initial filter is an LMS adaptive filter.

[0098] Wherein, the step S2 comprises:

[0099] S21, initializing the weight vector of the initial filter and the target image.

[0100] S22, calculating the error signal of the initial filter :

[0101] ;

[0102] In the formula, represents the expected output signal, Indicates The weight vector after iterations is Indicates The target image signal after iterations.

[0103] S23. Determine the weight update formula:

[0104] ;

[0105] In the formula, Indicates The weight vector after iterations is Represents the learning rate parameter.

[0106] S24. Iteratively update the weight vector of the initial filter based on the weight update formula until an iteration stop condition is met to obtain an updated filter.

[0107] S3, identifying a noise level of the filtered image, and determining a low-light image frame based on the noise level;

[0108] Specifically, the core of low-light image frame detection is to convert the signal into a graph and then extract its topological features as an estimate of the image noise level. The process is as follows: first, the image video sequence is framed, each frame corresponds to a one-dimensional time series, and then the sequence is converted into a simple undirected graph through normalization, quantization and edge construction; second, the Laplacian matrix of the graph is extracted and eigendecomposed, and its maximum eigenvalue is defined as the topological feature of the graph; finally, according to the size of the maximum eigenvalue, a mapping relationship between it and the noise level is established, where the noise level is determined by the number of vertices in the graph.

[0109] Wherein, the step S3 comprises:

[0110] S31, setting a number of vertices in each of the filtered images and performing graph domain transformation respectively to obtain a number of undirected graphs.

[0111] S32. Calculate the Laplacian matrix corresponding to each undirected graph :

[0112] ;

[0113] In the formula, , Respectively represent The degree matrix and adjacency matrix of an undirected graph.

[0114] S33, based on the Laplace matrix Determine the equation to solve:

[0115] ;

[0116] In the formula, represents the matrix determinant function, Indicates The eigenvalue vector of an undirected graph, Represents the identity matrix.

[0117] S34, solving the eigenvalue vector of each undirected graph based on the solution equation , and arrange the eigenvalues ​​in the eigenvalue vector in descending order to obtain the arrangement vector :

[0118] ;

[0119] In the formula, Indicates The first eigenvalues, Indicates The number of vertices in an undirected graph.

[0120] S35, determining the maximum eigenvalue in each of the permutation vectors :

[0121] .

[0122] S36, based on the maximum eigenvalue determining a low-light image frame;

[0123] Wherein, step S36 comprises:

[0124] S361, if , then the corresponding filtered image is a low-noise image;

[0125] S362, if , then the corresponding filtered image is a medium noise image;

[0126] S363, if , the corresponding filtered image is a high-noise image, and the high-noise image is used as a low-light image frame;

[0127] Specifically, when the image is a low-noise image, it means that the clarity of the frame image is high and no enhancement processing is required. When the image is a medium-noise image, it means that the clarity of the frame image is moderate and can be removed through denoising and other methods in subsequent processes. Therefore, in order to reduce the computational complexity, in this embodiment, only the high-noise image needs to be processed. When the image is a high-noise image, it means that the image has a weak signal and dim light due to external environmental factors. Therefore, it needs to be treated as a low-light image frame, which can also reduce the computational complexity.

[0128] S4, performing Gaussian filtering and detail enhancement on the low-light image frame to obtain an initial enhanced image frame;

[0129] Wherein, the step S4 comprises:

[0130] S41, processing the low-light image frame Perform Gaussian filtering to obtain a Gaussian filtered image :

[0131] ;

[0132] ;

[0133] In the formula, , Respectively represent the pixels in the low-light image frame and the Gaussian filter image The gray value of is the Gaussian standard deviation, is the Gaussian kernel function, Indicated in pixels The grayscale value of the pixel in the neighborhood centered at is the difference threshold;

[0134] Specifically, in this step, by adopting an improved Gaussian filtering process, that is, adjusting the Gaussian kernel function therein, the presence of halo in the image can be avoided.

[0135] S42, processing the low-light image frame Perform domain conversion and scale fusion to obtain a scale fused image :

[0136] ;

[0137] In the formula, For scale, The scale is The weight factor under , They represent inverse Fourier transform and Fourier transform respectively. The scale is Gaussian function under ;

[0138] Specifically, by performing domain transformation and scale fusion on the image, the brightness image can be effectively extracted.

[0139] S43, based on the Gaussian filter image Fusion image with the scale determining an initial enhanced image frame;

[0140] Wherein, the step S43 comprises:

[0141] S431, Gaussian filter image Perform linear transformation enhancement to obtain a linear transformation image :

[0142] ;

[0143] In the formula, , Respectively represent the minimum gray level and maximum gray level in the linear transformation image, Respectively represent the minimum gray level and maximum gray level in the Gaussian filter image, Respectively represent the first partition threshold and the second partition threshold in the Gaussian filter image, Respectively represent the first partition threshold and the second partition threshold in the linear transformation image, is the gray level of the linear transformation image;

[0144] Specifically, performing linear transformation enhancement on the image can effectively enhance the image contrast, and in the actual enhancement process, the enhanced grayscale value interval range can be determined by the first partition threshold and the second partition threshold. If the interval before enhancement is smaller than the interval after enhancement after enhancement, the interval before enhancement needs to be expanded. If the interval before enhancement is equal to the interval after enhancement, it remains unchanged. If the interval before enhancement is larger than the interval after enhancement, the space before enhancement is compressed.

[0145] S432: fusion of the scale image Perform brightness enhancement processing to obtain a brightness enhanced image :

[0146] ;

[0147] In the formula, Indicated in pixels In the neighborhood of the center, except for the pixel point The average brightness of the remaining pixels, Represents the brightness mean of the scale fusion image;

[0148] Specifically, this step can adaptively adjust the enhancement amplitude according to the brightness mean of the pixels within the neighborhood and the global brightness mean.

[0149] S433: transform the linear transformation image , Brightness Enhanced Image Fusion is performed to obtain the initial enhanced image frame :

[0150] ;

[0151] In the formula, Indicates Gaussian templates, represents the number of Gaussian templates, represent the first proportionality coefficient and the second proportionality coefficient respectively;

[0152] Specifically, in this embodiment, , the number of Gaussian templates is 3. After being processed by the Gaussian templates, three detail layer images can be obtained. The three detail layer images are superimposed and averaged, so that the detail information in the image can be fully retained.

[0153] S5, performing interpolation enhancement processing on the initial enhanced image frame to obtain an enhanced video frame;

[0154] Specifically, the low-light image frame referred to in this application specifically refers to a low-light / low-quality / low-light image.

[0155] Wherein, the step S5 comprises:

[0156] S51, determining the floating point coordinates of the pixels in the initial enhanced image frame mapped to the target image according to a preset scaling ratio. :

[0157] ; ;

[0158] In the formula, represents the pixel coordinates in the low-light image frame, , Respectively represent the width and height of the target image, , Indicates the width and height of the low-light image frame.

[0159] S52, in the target image, using floating point coordinates Determine the neighborhood range for the central pixel and determine the bicubic interpolation function;

[0160] Specifically, the bicubic interpolation function can be:

[0161] ;

[0162] in, It can be expressed as a difference weight associated with the distance, Represents the distance between the pixel points in the neighborhood and the central pixel point. is the adjustment factor, which is ;

[0163] S53, determining the interpolation weights between the pixels within the neighborhood and the central pixel based on the bicubic interpolation function;

[0164] Specifically, the corresponding interpolation weight can be determined according to the distance between the pixel points in the neighborhood and the central pixel point.

[0165] S54, performing weighted averaging on the interpolation weights of the pixels within the neighborhood to obtain enhanced pixels, and determining an enhanced video frame based on the enhanced pixels;

[0166] Specifically, the interpolation weights are then weighted averaged to obtain an average weight, and the average weight is multiplied by the center pixel to obtain an enhanced pixel. The above operation is performed on all pixels in the low-light image frame to obtain an enhanced video frame.

[0167] The imaging enhancement method of a low-light enhancement camera provided in the first embodiment of the present invention first obtains a target video shot by a micro-array compound-eye lens camera, splits the target video frame by frame to obtain a plurality of target video frames, and pre-processes the corresponding target video frames to obtain a plurality of target images; then establishes an initial filter, updates the initial filter to obtain an updated filter, and adaptively filters the target image based on the updated filter to obtain a filtered image; then performs Gaussian filtering and detail enhancement on the low-light image frame to obtain an initial enhanced image frame; finally, performs interpolation enhancement on the initial enhanced image frame to obtain an enhanced video frame. The present invention determines the corresponding low-light image based on the noise level of the image, and performs filtering, detail enhancement and interpolation processing on the low-light image, which significantly reduces the computational complexity, realizes super-resolution enhancement of the low-light image, and enhances the contrast of the image and the detail information in the image.

[0168] Embodiment 2

[0169] like Figure 2 As shown, in the second embodiment of the present invention, a low-light-enhanced camera imaging enhancement system is provided. The system adopts the low-light-enhanced camera imaging enhancement as described in the first embodiment. The system includes:

[0170] Processing module 1 is used to obtain a target video captured by a microarray compound eye lens camera, split the target video frame by frame to obtain a plurality of target video frames, and pre-process the corresponding target video frames to obtain a plurality of target images;

[0171] Filtering module 2, used for establishing an initial filter, updating the initial filter to obtain an updated filter, and performing adaptive filtering on the target image based on the updated filter to obtain a filtered image;

[0172] An identification module 3, configured to identify a noise level of the filtered image and determine a low-light image frame based on the noise level;

[0173] A detail module 4 performs Gaussian filtering and detail enhancement on the low-light image frame to obtain an initial enhanced image frame;

[0174] The enhancement module 5 is used to perform interpolation enhancement processing on the initial enhanced image frame to obtain an enhanced video frame.

[0175] The processing module 1 comprises:

[0176] The splitting submodule is used to label each sub-camera in the micro-array compound eye lens camera, and split the target video corresponding to each sub-camera frame by frame to obtain several groups of target video frames;

[0177] A filtering submodule, used for sequentially performing median filtering and histogram equalization processing on each group of target video frames to obtain processed image frames;

[0178] The stitching submodule is used to perform image registration and image stitching processing on the processed image frames of the corresponding frames in sequence according to the label of each sub-camera to obtain a plurality of target images.

[0179] The filtering module 2 comprises:

[0180] An initialization submodule, used to initialize the weight vector of the initial filter and the target image;

[0181] The error submodule is used to calculate the error signal of the initial filter. :

[0182] ;

[0183] In the formula, represents the expected output signal, Indicates The weight vector after iterations is Indicates The target image signal after iterations;

[0184] The weight submodule is used to determine the weight update formula:

[0185] ;

[0186] In the formula, Indicates The weight vector after iterations is Represents the learning rate parameter;

[0187] The iterative submodule is used to iteratively update the weight vector of the initial filter based on the weight update formula until an iteration stop condition is met to obtain an updated filter.

[0188] The identification module 3 comprises:

[0189] A transformation submodule, used for setting a number of vertices in each of the filter images and performing graph domain transformation respectively to obtain a number of undirected graphs;

[0190] Matrix calculation submodule, used to calculate the Laplace matrix corresponding to each undirected graph :

[0191] ;

[0192] In the formula, , Respectively represent The degree matrix and adjacency matrix of an undirected graph;

[0193] Equation submodule for the Laplace matrix based on Determine the equation to solve:

[0194] ;

[0195] In the formula, represents the matrix determinant function, Indicates The eigenvalue vector of an undirected graph, represents the identity matrix;

[0196] A permutation submodule for solving the eigenvalue vector of each undirected graph based on the solution equation , and arrange the eigenvalues ​​in the eigenvalue vector in descending order to obtain the arrangement vector :

[0197] ;

[0198] In the formula, Indicates The first eigenvalues, Indicates The number of vertices in an undirected graph;

[0199] The maximum eigenvalue submodule is used to determine the maximum eigenvalue in each of the permutation vectors. :

[0200] ;

[0201] Output submodule for the maximum eigenvalue Determine the low-light image frame.

[0202] The output submodule comprises:

[0203] The first output unit is used if , then the corresponding filtered image is a low-noise image;

[0204] The second output unit is used if , then the corresponding filtered image is a medium noise image;

[0205] The third output unit is used if , the corresponding filtered image is a high-noise image, and the high-noise image is used as the low-light image frame.

[0206] The detail module 4 includes:

[0207] Gaussian submodule, used for the low-light image frame Perform Gaussian filtering to obtain a Gaussian filtered image :

[0208] ;

[0209] ;

[0210] In the formula, , Respectively represent the pixels in the low-light image frame and the Gaussian filter image The gray value of is the Gaussian standard deviation, is the Gaussian kernel function, Indicated in pixels The grayscale value of the pixel in the neighborhood centered at is the difference threshold;

[0211] A scale submodule is used to scale the low-light image frame Perform domain conversion and scale fusion to obtain a scale fused image :

[0212] ;

[0213] In the formula, For scale, The scale is The weight factor under , They represent inverse Fourier transform and Fourier transform respectively. The scale is Gaussian function under ;

[0214] Initial enhancement submodule for filtering images based on the Gaussian filter Fusion image with the scale determining an initial enhanced image frame;

[0215] The initial enhancer module comprises:

[0216] A transformation unit for transforming the Gaussian filter image Perform linear transformation enhancement to obtain a linear transformation image :

[0217] ;

[0218] In the formula, , Respectively represent the minimum gray level and maximum gray level in the linear transformation image, Respectively represent the minimum gray level and maximum gray level in the Gaussian filter image, Respectively represent the first partition threshold and the second partition threshold in the Gaussian filter image, Respectively represent the first partition threshold and the second partition threshold in the linear transformation image, is the gray level of the linear transformation image;

[0219] A brightness enhancement unit is used to enhance the scale fusion image Perform brightness enhancement processing to obtain a brightness enhanced image :

[0220] ;

[0221] In the formula, Indicated in pixels In the neighborhood of the center, except for the pixel point The average brightness of the remaining pixels, Represents the brightness mean of the scale fusion image;

[0222] An image fusion unit is used to transform the linear transformation image , Brightness Enhanced Image Fusion is performed to obtain the initial enhanced image frame :

[0223] ;

[0224] In the formula, Indicates Gaussian templates, represents the number of Gaussian templates, Represent the first proportional coefficient and the second proportional coefficient respectively.

[0225] The enhancement module 5 comprises:

[0226] A mapping submodule is used to determine the floating point coordinates of the pixels in the initial enhanced image frame mapped to the target image according to a preset scaling ratio. :

[0227] ; ;

[0228] In the formula, represents the pixel coordinates in the low-light image frame, , Respectively represent the width and height of the target image, , Indicates the width and height of the low-light image frame;

[0229] Function submodule for obtaining floating point coordinates in the target image Determine the neighborhood range for the central pixel and determine the bicubic interpolation function;

[0230] A weight calculation submodule, used to determine the interpolation weights between the pixels within the neighborhood and the central pixel based on the bicubic interpolation function;

[0231] The enhancement submodule is used to perform weighted averaging on the interpolation weights of the pixels within the neighborhood to obtain enhanced pixels, and determine an enhanced video frame based on the enhanced pixels.

[0232] In some other embodiments of the present invention, the embodiments of the present invention provide the following technical solution: a computer, comprising a memory 102, a processor 101, and a computer program stored in the memory 102 and executable on the processor 101; when the processor 101 executes the computer program, the low-light-enhanced camera imaging enhancement method as described above is implemented.

[0233] Specifically, the processor 101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0234] Among them, the memory 102 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 102 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 102 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 102 may be inside or outside the data processing device. In a specific embodiment, the memory 102 is a non-volatile memory. In a specific embodiment, the memory 102 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0235] The memory 102 may be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 101 .

[0236] The processor 101 implements the above-mentioned low-light-enhanced camera imaging enhancement method by reading and executing the computer program instructions stored in the memory 102 .

[0237] In some embodiments, the computer may further include a communication interface 103 and a bus 100. Figure 3 As shown, the processor 101, the memory 102, and the communication interface 103 are connected via a bus 100 and communicate with each other.

[0238] The communication interface 103 is used to implement communication between the modules, devices, units and / or equipment in the embodiment of the present invention. The communication interface 103 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0239] The bus 100 includes hardware, software or both, and couples the components of the computer device to each other. The bus 100 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 100 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, bus 100 may include one or more buses. Although embodiments of the present invention describe and illustrate a particular bus, the present invention contemplates any suitable bus or interconnect.

[0240] The computer can execute the low-light-enhanced camera imaging enhancement method of the present invention based on the acquired low-light-enhanced camera imaging enhancement system, thereby realizing low-light-enhanced camera imaging enhancement.

[0241] In some further embodiments of the present invention, in combination with the above-mentioned low-light-enhanced camera imaging enhancement method, the embodiments of the present invention provide the following technical solution: a storage medium having a computer program stored thereon, and the computer program implements the above-mentioned low-light-enhanced camera imaging enhancement method when executed by a processor.

[0242] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0243] More specific examples of readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0244] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0245] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0246] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A low-light enhancement camera imaging enhancement method, characterized in that: include: Obtain a target video captured by a microarray compound eye lens camera, split the target video frame by frame to obtain a number of target video frames, and pre-process the corresponding target video frames to obtain a number of target images; Establishing an initial filter, updating the initial filter to obtain an updated filter, and performing adaptive filtering on the target image based on the updated filter to obtain a filtered image; identifying a noise level of the filtered image, and determining a low-light image frame based on the noise level; Performing Gaussian filtering and detail enhancement on the low-light image frame to obtain an initial enhanced image frame; Performing interpolation enhancement processing on the initial enhanced image frame to obtain an enhanced video frame; The step of performing Gaussian filtering and detail enhancement on the low-light image frame to obtain an initial enhanced image frame comprises: The low-light image frame Perform Gaussian filtering to obtain a Gaussian filtered image : ； ； In the formula, , Respectively represent the pixels in the low-light image frame and the Gaussian filter image The gray value of is the Gaussian standard deviation, is the Gaussian kernel function, Indicated in pixels The grayscale value of the pixel in the neighborhood as the center, is the difference threshold; The low-light image frame Perform domain conversion and scale fusion to obtain a scale fused image : ； In the formula, For scale, The scale is The weight factor under , They represent inverse Fourier transform and Fourier transform respectively. The scale is Gaussian function under ; Based on the Gaussian filter image Fusion image with the scale An initial enhanced image frame is determined.

2. The imaging enhancement method for a low-light-level enhancement camera according to claim 1, characterized in that: The step of performing frame-by-frame splitting processing on the target video to obtain a plurality of target video frames, and preprocessing the corresponding target video frames to obtain a plurality of target images comprises: Label each sub-camera in the micro-array compound eye lens camera, and split the target video corresponding to each sub-camera frame by frame to obtain several groups of target video frames; Performing median filtering and histogram equalization processing on each group of target video frames in sequence to obtain processed image frames; According to the label of each sub-camera, the processed image frames of the corresponding frames are sequentially processed with image registration and image stitching to obtain a number of target images.

3. The imaging enhancement method for a low-light-level enhancement camera according to claim 1, characterized in that: The step of updating the initial filter to obtain an updated filter comprises: Initializing the weight vector of the initial filter and the target image; Calculate the error signal of the initial filter : ； In the formula, represents the expected output signal, Indicates The weight vector after iterations is Indicates The target image signal after iterations; Determine the weight update formula: ； In the formula, Indicates The weight vector after iterations is Represents the learning rate parameter; The weight vector of the initial filter is iteratively updated based on the weight update formula until an iteration stop condition is met to obtain an updated filter.

4. The imaging enhancement method for a low-light-level enhancement camera according to claim 1, characterized in that: The step of identifying the noise level of the filtered image and determining the low-light image frame based on the noise level comprises: Setting a plurality of vertices in each of the filter images and performing graph domain transformation respectively to obtain a plurality of undirected graphs; Calculate the Laplacian matrix corresponding to each undirected graph : ； In the formula, , Respectively represent The degree matrix and adjacency matrix of an undirected graph; Based on the Laplace matrix Determine the equation to solve: ； In the formula, represents the matrix determinant function, Indicates The eigenvalue vector of an undirected graph, represents the identity matrix; Solve the eigenvalue vector of each undirected graph based on the solution equation , and arrange the eigenvalues ​​in the eigenvalue vector in descending order to obtain the arrangement vector : ； In the formula, Indicates The first eigenvalues, Indicates The number of vertices in an undirected graph; Determine the largest eigenvalue in each of the permutation vectors : ； Based on the maximum eigenvalue Determine the low-light image frame.

5. The imaging enhancement method for a low-light-level enhancement camera according to claim 4, characterized in that: The maximum eigenvalue The steps of determining the low-light image frame include: like , then the corresponding filtered image is a low-noise image; like , then the corresponding filtered image is a medium noise image; like , the corresponding filtered image is a high-noise image, and the high-noise image is used as the low-light image frame.

6. The imaging enhancement method for a low-light-level enhancement camera according to claim 1, characterized in that: The Gaussian filter image Fusion image with the scale The steps of determining the initial enhanced image frame include: For the Gaussian filter image Perform linear transformation enhancement to obtain a linear transformation image : ； In the formula, , Respectively represent the minimum gray level and maximum gray level in the linear transformation image, Respectively represent the minimum gray level and maximum gray level in the Gaussian filter image, Respectively represent the first partition threshold and the second partition threshold in the Gaussian filter image, Respectively represent the first partition threshold and the second partition threshold in the linear transformation image, is the gray level of the linear transformation image; Fusion of images at the scale Perform brightness enhancement processing to obtain a brightness enhanced image : ； In the formula, Indicated in pixels In the neighborhood of the center, except for the pixel point The average brightness of the remaining pixels, Represents the brightness mean of the scale fusion image; The linear transformation image , Brightness Enhanced Image Fusion is performed to obtain the initial enhanced image frame : ； In the formula, Indicates Gaussian templates, represents the number of Gaussian templates, Represent the first proportional coefficient and the second proportional coefficient respectively.

7. The imaging enhancement method for a low-light-level enhancement camera according to claim 1, characterized in that: The step of performing interpolation enhancement processing on the initial enhanced image frame to obtain an enhanced video frame comprises: Determine the floating point coordinates of the pixels of the initial enhanced image frame mapped to the target image according to the preset scaling ratio : ； ； In the formula, represents the pixel coordinates in the low-light image frame, , Respectively represent the width and height of the target image, , Indicates the width and height of the low-light image frame; In the target image, the floating point coordinates Determine the neighborhood range for the central pixel and determine the bicubic interpolation function; Determine the interpolation weights between the pixels in the neighborhood and the central pixel based on the bicubic interpolation function; The interpolation weights of the pixels within the neighborhood are weighted averaged to obtain enhanced pixels, and an enhanced video frame is determined based on the enhanced pixels.

8. A low-light-enhanced camera imaging enhancement system, the system adopts the low-light-enhanced camera imaging enhancement method as claimed in claim 1, characterized in that: The system comprises: A processing module is used to obtain a target video captured by a microarray compound eye lens camera, split the target video frame by frame to obtain a plurality of target video frames, and pre-process the corresponding target video frames to obtain a plurality of target images; A filtering module, used to establish an initial filter, update the initial filter to obtain an updated filter, and adaptively filter the target image based on the updated filter to obtain a filtered image; an identification module, configured to identify a noise level of the filtered image and determine a low-light image frame based on the noise level; A detail module performs Gaussian filtering and detail enhancement on the low-light image frame to obtain an initial enhanced image frame; The enhancement module is used to perform interpolation enhancement processing on the initial enhanced image frame to obtain an enhanced video frame.

9. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the low-light-enhanced camera imaging enhancement method according to any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the low-light-enhanced camera imaging enhancement method according to any one of claims 1 to 7 is implemented.

Images