Image processing method and device in extremely low light environment

Abstract

The disclosure provides an image processing method and device in an extremely low light environment, and belongs to the technical field of image processing. The method collects voltage sequences corresponding to each pixel point in the image to be processed; the voltage sequence is the change of the light signal intensity of the corresponding pixel point; according to the voltage sequence of the pixel point, the statistical characteristics of the voltage sequence corresponding to the pixel point are determined, compared with the static numerical value at the corresponding moment in the related art, the voltage values in all voltage sequences are comprehensively considered, and the accuracy of the light signal intensity is improved; further, according to the voltage sequence of the pixel point and the voltage sequence of each reference pixel point in the preset neighborhood, the correlation characteristics corresponding to the pixel point are determined, the correlation characteristics integrate the voltage sequences of the reference pixel points in the preset neighborhood, and the abnormal pixel points of the image are eliminated; according to the statistical characteristics and the correlation characteristics, the image to be processed is reconstructed to obtain a target image, and the quality of the corresponding image under low light conditions is improved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, specifically to an image processing method and apparatus for extremely low light environments. Background Technology

[0002] Traditional image signal processing (TSP) transforms raw RAW data captured by sensors into RGB images that are understandable to humans / machines. Low-light images are images acquired in extremely low-light environments.

[0003] In related technologies, the processing of low-light images typically only processes the static values ​​of the sensor voltage timing signal at the corresponding moment. However, the static values ​​obtained by the above methods cannot represent all the information of the voltage timing signal, and therefore cannot improve the quality of low-light images. Summary of the Invention

[0004] This disclosure proposes an image processing method and apparatus for extremely low light environments.

[0005] The first aspect of this disclosure provides an image processing method for extremely low-light environments, including:

[0006] The voltage sequence corresponding to each pixel in the image to be processed is acquired; the voltage sequence represents the change in the light signal intensity of the corresponding pixel.

[0007] For any pixel, based on the voltage sequence corresponding to the pixel, the statistical characteristics of the voltage sequence corresponding to the pixel are determined, and based on the voltage sequence corresponding to the pixel and the voltage sequences of each reference pixel in a preset neighborhood, the correlation characteristics of the pixel are determined; wherein, the reference pixels are other pixels in the preset neighborhood besides the pixel; the pixel is the center pixel of the preset neighborhood;

[0008] Based on the statistical features and the correlation features, the image to be processed is reconstructed to obtain the target image.

[0009] In this embodiment of the disclosure, determining the statistical characteristics of the voltage sequence corresponding to the pixel based on the voltage sequence corresponding to the pixel includes:

[0010] Extract the peak value of the voltage sequence corresponding to the pixel, where the peak value represents the maximum light signal intensity of the pixel;

[0011] Extract the rate of change of the voltage sequence; the rate of change characterizes the intensity of the optical signal change at the pixel.

[0012] The maximum value of the optical signal intensity and the intensity of the optical signal change are used as the statistical features.

[0013] In this embodiment of the disclosure, the step of basing the correlation features between the voltage sequence corresponding to the pixel and the voltage sequences of each reference pixel in a preset neighborhood includes:

[0014] For any reference pixel within the preset neighborhood, calculate the similarity between the voltage sequence corresponding to the reference pixel and the voltage sequence corresponding to the pixel.

[0015] A target weight is generated based on the similarity corresponding to each reference pixel, and the target weight represents the degree of influence of each reference pixel on the pixel.

[0016] The associated features are generated based on the target weights and the voltage sequences of the pixels.

[0017] In this embodiment of the disclosure, the image to be processed is reconstructed based on the statistical features and the correlation features to obtain the target image, including:

[0018] Based on the statistical features and the correlation features, the fusion features of the image to be processed are generated;

[0019] Based on the fusion features, the image to be processed is reconstructed to obtain the target image.

[0020] In this embodiment of the disclosure, generating the fusion features of the image to be processed based on the statistical features and the correlation features includes:

[0021] The fused feature is obtained by weighted fusion based on the preset weights corresponding to the statistical features and the correlation features respectively; wherein the preset weights represent the importance of the corresponding features.

[0022] In this embodiment of the disclosure, the step of reconstructing the image to be processed based on the fusion features to obtain the target image includes:

[0023] The image to be processed is reconstructed based on the fusion features to obtain an initial image;

[0024] The initial image is enhanced to obtain the target image; the enhancement operation includes at least one of interpolation, white balance, contrast equalization, brightness adjustment, and frequency domain artifact suppression.

[0025] In this embodiment of the disclosure, the acquisition of the voltage timing corresponding to each pixel in the image to be processed includes:

[0026] When acquiring the image to be processed, the voltage sequence of each pixel is obtained.

[0027] The target voltage sequence is obtained by removing the pre-acquired reference voltage sequence from the voltage sequence to be processed.

[0028] In this embodiment of the disclosure, acquiring the voltage sequence to be processed for each pixel when acquiring the image to be processed includes:

[0029] When acquiring the image to be processed, the initial voltage sequence corresponding to each pixel is obtained;

[0030] The initial voltage sequence is amplified to obtain an amplified initial voltage sequence;

[0031] The amplified initial voltage sequence is denoised to obtain the voltage sequence to be processed.

[0032] In this embodiment of the disclosure, the method further includes:

[0033] The reference voltage sequence is periodically acquired, and the reference voltage sequence acquired in the current period is used to replace the reference voltage sequence acquired in the previous period.

[0034] A second aspect of this disclosure provides an image processing apparatus for extremely low light environments, the apparatus comprising:

[0035] The acquisition module is used to acquire the voltage sequence corresponding to each pixel in the image to be processed; the voltage sequence represents the change in the light signal intensity of the corresponding pixel.

[0036] The feature extraction module is used to, for any pixel, determine the statistical features of the voltage sequence corresponding to the pixel based on the voltage sequence corresponding to the pixel, and determine the association features of the pixel based on the voltage sequence corresponding to the pixel and the voltage sequences of each reference pixel in a preset neighborhood; wherein, the reference pixels are other pixels in the preset neighborhood besides the pixel; and the pixel is the center pixel of the preset neighborhood.

[0037] The reconstruction module is used to reconstruct the image to be processed based on the statistical features and the correlation features to obtain the target image.

[0038] An embodiment of the third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect or any optional embodiment of the first aspect.

[0039] An embodiment of the fourth aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the methods described in the first aspect and any optional embodiment of the first aspect.

[0040] The technical solutions provided in this disclosure have at least the following technical effects or advantages:

[0041] The process involves acquiring voltage sequences corresponding to each pixel in the image to be processed. These voltage sequences characterize the changes in light signal intensity at the corresponding pixel. Based on this, for any given pixel, statistical features of its corresponding voltage sequence are determined. These statistical features characterize the temporal characteristics of the pixel. Compared to related techniques that only use static values ​​at a given moment, this method comprehensively considers voltage values ​​from all voltage sequences, thus improving the accuracy of light signal intensity to some extent. Furthermore, based on the voltage sequences corresponding to the pixel and the voltage sequences of reference pixels within a preset neighborhood, a correlation feature is determined. This correlation feature integrates the voltage sequences of the reference pixels within the preset neighborhood, thus eliminating abnormal pixels in the image to some extent. Finally, based on the statistical features and the correlation features, the image to be processed is reconstructed to obtain the target image, improving the image quality under low-light conditions.

[0042] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description or may be learned by practice of this disclosure. Attached Figure Description

[0043] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this disclosure. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0044] Figure 1 A flowchart illustrating an image processing method under extremely low light conditions provided in an embodiment of this disclosure is shown.

[0045] Figure 2 A schematic diagram of an image processing method under extremely low light conditions provided in an embodiment of this disclosure is shown;

[0046] Figure 3 A schematic diagram of an image processing apparatus for extremely low light environments provided in an embodiment of this disclosure is shown.

[0047] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure is shown;

[0048] Figure 5 A schematic diagram of a storage medium provided according to an embodiment of the present disclosure is shown. Detailed Implementation

[0049] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0050] It should be noted that, unless otherwise stated, the technical or scientific terms used in this disclosure shall have the ordinary meaning as understood by one of ordinary skill in the art to which this disclosure pertains.

[0051] This disclosure proposes an image processing method for extremely low-light environments, such as... Figure 1 The image shown is an embodiment of an image processing method for extremely low light environments, comprising the following steps:

[0052] In step S11, the voltage sequence corresponding to each pixel in the image to be processed is acquired.

[0053] The voltage sequence represents the change in the light signal intensity of the corresponding pixel.

[0054] For example, in this embodiment of the disclosure, the image to be processed can be a two-dimensional light intensity distribution formed on the focal plane of a CMOS sensor. A pixel can be an independent photosensitive unit of 2.8µm × 2.8µm in a CMOS array, corresponding to a spatial position (x, y). The target voltage sequence is a one-dimensional signal V(x, y, t) formed by arranging the analog voltages output by the same pixel over time within consecutive sampling periods. The light signal intensity is the photon flux arriving at the pixel per unit area per unit time.

[0055] On a microstructure CMOS array, each pixel performs photoelectric conversion on the incident photons, generating photogenerated charge proportional to the light signal intensity, and accumulating it on the integrating capacitor to form a microvolt-level analog voltage, which is the voltage sequence in the embodiments of this disclosure.

[0056] In some embodiments, the voltage timing sequence corresponding to each pixel in the image to be processed is acquired, and the voltage sequence to be processed corresponding to each pixel when the image to be processed is acquired can also be obtained; the pre-acquired reference voltage sequence is removed from the voltage sequence to be processed to obtain the target voltage sequence.

[0057] For example, the pre-acquired reference voltage sequence is a digital bitstream output from dark frames acquired under completely dark conditions with the same temperature and integration time, via the same signal chain, used to characterize dark current and system bias. Before the actual shooting, one or more completely black images can be continuously acquired while keeping the shutter closed and the temperature and exposure parameters constant, generating a "reference voltage sequence" for each pixel.

[0058] A reference voltage sequence is calculated for each pixel and subtracted point-by-point in subsequent sampling to ensure that the zero baseline of the voltage sequence corresponds to zero photon input. In some embodiments, the reference voltage sequence can be acquired periodically, and the reference voltage sequence acquired in the current period can be used to replace the reference voltage sequence acquired in the previous period to ensure the accuracy of the reference voltage sequence. For example, the corresponding reference voltage sequence can be acquired every 60 frames.

[0059] In some embodiments, the voltage sequence to be processed can be determined in the following way to improve its accuracy: The initial voltage sequence corresponding to each pixel is acquired when the image to be processed is collected; the initial voltage sequence is magnified to obtain a magnified initial voltage sequence; the magnified initial voltage sequence is denoised to obtain the voltage sequence to be processed.

[0060] For example, the initial voltage sequence can be a one-dimensional signal composed of microvolt-level analog voltages output by CMOS pixels in chronological order after exposure integration, before any amplification or filtering. Amplification can be performed by a programmable gain amplifier (PGA) to linearly boost the initial voltage sequence in the analog domain, bringing its amplitude into the optimal quantization range of the subsequent analog-to-digital converter (ADC). Quantization noise, high-frequency clock noise, and low-frequency drift introduced during amplification are suppressed using analog or digital filtering methods, thereby obtaining a clean voltage sequence to be processed.

[0061] After amplification and denoising, the digital voltage time series can be directly used for subsequent baseline correction and algorithm analysis as the voltage sequence to be processed.

[0062] In step S12, for any pixel, the statistical characteristics of the voltage sequence corresponding to the pixel are determined based on the voltage sequence corresponding to the pixel, and the correlation characteristics of the pixel are determined based on the voltage sequence corresponding to the pixel and the voltage sequences of each reference pixel in the preset neighborhood.

[0063] The reference pixel is any pixel other than the pixel itself within the preset neighborhood; the pixel is the center pixel of the preset neighborhood.

[0064] For example, statistical features can be spatially context-free information extracted solely from the pixel's own voltage sequence, such as mean, variance, kurtosis, etc. The preset neighborhood is a rectangular or cross-shaped window (e.g., 3×3 or 5×5) with radius r centered on the current pixel, used to define a set of reference pixels. Reference pixels are all pixels within the preset neighborhood except the center pixel, and their voltage sequences are denoted as V(i, j, t). Association features can be quantities describing the degree of synchronization or difference between the voltage sequence of the center pixel and the voltage sequences of each reference pixel in the neighborhood.

[0065] For the voltage sequence V(x, y, t) of the center pixel (x, y), calculate its first and second-order statistics, which may include mean, variance, skewness, and kurtosis, to form the statistical feature vector S(x, y) of that pixel. Based on a preset neighborhood window, read the voltage sequences V(i, j, t) of all reference pixels (i, j) within the neighborhood, ensuring that each sequence has the same length as the center sequence and is time-aligned.

[0066] The Pearson correlation coefficient ρ(x, y, i, j) between the voltage sequence V(x, y, t) of the center pixel (x, y) and the voltage sequences V(i, j, t) of each reference pixel (i, j) can be calculated to obtain the correlation coefficient matrix. The spatiotemporal cross-correlation peak can also be used as a similarity score to quantify the degree of matching between the two in waveform shape and delay.

[0067] In some embodiments, the determination of the statistical characteristics of the voltage sequence corresponding to the pixel in step S12 can also be achieved by the following methods: extracting the peak value of the voltage sequence corresponding to the pixel, wherein the peak value represents the maximum value of the light signal intensity of the pixel; extracting the rate of change of the voltage sequence; wherein the rate of change represents the intensity of the light signal change of the pixel; and using the maximum value of the light signal intensity and the intensity of the light signal change as statistical characteristics.

[0068] For example, for the voltage sequence of a pixel, a linear scan is performed within a time window, and the maximum value is recorded as the "maximum value of optical signal intensity". For the same voltage sequence, a first-order difference or a 1×2 convolution kernel [-1, 1] is used to calculate the difference ΔV(t) = V(t+1) – V(t) between adjacent sampling points; the maximum absolute value of ΔV(t), ΔV_max, is taken as the "intensity of optical signal change". The maximum value of optical signal intensity and the intensity of optical signal change are concatenated or cascaded to form the two-dimensional statistical feature of the pixel.

[0069] In some embodiments, the correlation feature between the voltage sequence corresponding to the pixel and the voltage sequences of each reference pixel in the preset neighborhood in step S12 can also be implemented in the following way: for any reference pixel in the preset neighborhood, calculate the similarity between the voltage sequence corresponding to the reference pixel and the voltage sequence corresponding to the pixel; generate a target weight based on the similarity corresponding to each reference pixel, the target weight representing the degree of influence of each reference pixel on the pixel; generate correlation features based on the target weight and the voltage sequence of the pixel.

[0070] For example, the voltage sequence of the center pixel is compared with the voltage sequence of any reference pixel in the preset neighborhood for each pair. Specifically, the linear distance, spectral density correlation, and cosine similarity measured by a deep learning network can be calculated to determine the corresponding correlation coefficient. The correlation coefficient is then normalized and mapped to a positive weight, with the sum of all neighborhood weights being 1; this is the "target weight," which quantifies the influence of the reference pixel on the center pixel.

[0071] The voltage sequences of all reference pixels in the neighborhood are weighted by the target weight to obtain a weighted neighborhood sequence. The weighted neighborhood sequence is then concatenated with the voltage sequence corresponding to the center pixel or the cross-energy is calculated to form the final associated feature vector.

[0072] In step S13, the image to be processed is reconstructed based on statistical and correlation features to obtain the target image.

[0073] For example, statistical features and correlation features can be input into a spatio-temporal reconstruction network (STRN); the STRN performs upsampling in the spatial domain and 1×1 convolution compression in the temporal domain, and finally outputs a target image with the same resolution as the original.

[0074] In some embodiments, step S13 includes: generating fusion features of the image to be processed based on statistical features and correlation features; and reconstructing the image to be processed based on the fusion features to obtain the target image.

[0075] For example, the correlation coefficient matrix vector can be compressed into a fixed-length associated feature vector C(x, y), such as by taking the mean, maximum, or weighted sum of the neighborhood values, to ensure that the feature dimensions are consistent with the subsequent network input. The statistical feature vector S(x, y) and the associated feature vector C(x, y) are concatenated or fed in parallel into the downstream self-attention fusion module to complete the initial fusion features of the spatiotemporal information of the pixel.

[0076] The initial fused features can be further simplified by feeding them into a lightweight self-attention module (LSAM) for channel attention weighting, outputting the final fused features, whose weights are learned end-to-end by the network. Image reconstruction is then performed based on the final fused features.

[0077] In some embodiments, generating fusion features of the image to be processed based on statistical features and correlation features includes: performing weighted fusion based on preset weights corresponding to the statistical features and correlation features respectively to obtain fusion features; wherein the preset weights represent the importance of the corresponding features.

[0078] In some embodiments, reconstructing the image to be processed based on fusion features to obtain a target image includes: reconstructing the image to be processed based on fusion features to obtain an initial image; performing enhancement operations on the initial image to obtain the target image; the enhancement operations include at least one of interpolation, white balance, contrast equalization, brightness adjustment, and frequency domain artifact suppression.

[0079] For example, after inputting the fused features into the corresponding reconstruction network, the reconstructed initial image is obtained. Specifically, the reconstruction network can be a deep network consisting of three levels of downsampling convolution, residual mapping units (including physical regularization loss consistent with the sensor response curve), and three levels of upsampling convolution.

[0080] The initial image contains RGB information. To enhance the visual appeal and quality stability of the initial image, further post-processing can be performed on it in this embodiment. This includes operations such as interpolation, white balance adjustment, contrast equalization, brightness adjustment, and frequency domain artifact suppression. In this process, the initial image can be fed into a preset large model to generate corresponding adjustment parameters for interpolation, white balance, contrast equalization, brightness adjustment, and frequency domain artifact suppression. The initial image is then adjusted based on these adjustment parameters.

[0081] like Figure 2 The diagram shown is an exemplary schematic of the image processing method corresponding to an embodiment of this disclosure. The method involves acquiring the voltage sequence corresponding to the image to be processed, amplifying and filtering the voltage sequence, and removing the reference voltage sequence to obtain the target voltage sequence. Feature preprocessing is then performed on the target voltage sequence, specifically including extracting statistical and correlation features of the target voltage sequence followed by feature fusion. Image reconstruction is then performed on the fused features. Image reconstruction can sequentially employ a three-level downsampling convolution module, residual mapping, and a three-level upsampling convolution module to obtain an initial image. Finally, the initial image is preprocessed to obtain the corresponding target image.

[0082] The image processing method under extremely low light conditions according to the embodiments of this application acquires the voltage sequence corresponding to each pixel in the image to be processed. The voltage sequence represents the change in light signal intensity of the corresponding pixel. Based on this, for any pixel, the statistical characteristics of the voltage sequence corresponding to the pixel are determined according to the voltage sequence corresponding to the pixel. This statistical characteristic can represent the temporal characteristics of the pixel. Compared with related technologies that only use static values ​​at the corresponding time, this method comprehensively considers the voltage values ​​in all voltage sequences, thus improving the accuracy of light signal intensity to a certain extent. Furthermore, based on the voltage sequence corresponding to the pixel and the voltage sequences of each reference pixel in a preset neighborhood, the correlation characteristics corresponding to the pixel are determined. This correlation characteristic integrates the voltage sequences of the reference pixels in the preset neighborhood, thus eliminating abnormal pixels in the image to a certain extent. Finally, based on the statistical characteristics and correlation characteristics, the image to be processed is reconstructed to obtain the target image, improving the quality of the corresponding image under low light conditions.

[0083] Corresponding to the above implementation methods, this disclosure also provides an image processing apparatus for extremely low light environments, used to perform the above-described methods. Figure 1 The illustrated embodiment of the image processing method in an extremely low light environment, such as Figure 3 As shown, the image processing apparatus for extremely low light environments includes:

[0084] The acquisition module 301 is used to acquire the voltage sequence corresponding to each pixel in the image to be processed; the voltage sequence represents the change in the light signal intensity of the corresponding pixel.

[0085] The feature extraction module 302 is used to, for any pixel, determine the statistical features of the voltage sequence corresponding to the pixel based on the voltage sequence corresponding to the pixel, and determine the association features of the pixel based on the voltage sequence corresponding to the pixel and the voltage sequences of each reference pixel in a preset neighborhood; wherein, the reference pixels are other pixels in the preset neighborhood besides the pixel; and the pixel is the center pixel of the preset neighborhood.

[0086] The reconstruction module 303 is used to reconstruct the image to be processed based on the statistical features and the correlation features to obtain the target image.

[0087] The image processing apparatus for extremely low light environments provided in the above embodiments of this disclosure and the image processing method for extremely low light environments provided in the embodiments of this disclosure are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0088] This disclosure also provides an electronic device for performing the above-described method. Please refer to... Figure 4This illustrates a schematic diagram of an electronic device provided by some embodiments of the present disclosure. For example... Figure 4 As shown, the electronic device includes: a processor 400, a memory 401, a bus 402, and a communication interface 403. The processor 400, the communication interface 403, and the memory 401 are connected via the bus 402. The memory 401 stores a computer program that can run on the processor 400. When the processor 400 runs the computer program, it executes the aforementioned functions described in this disclosure. Figure 1 The method provided by any of the illustrated embodiments.

[0089] The memory 401 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 403 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0090] Bus 402 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Memory 401 is used to store programs, and the processor 400 executes the programs after receiving execution instructions. Figure 1 The illustrated method can be applied to or implemented by the processor 400 in any of the embodiments shown.

[0091] The processor 400 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 400 or by instructions in software form. The processor 400 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 401. The processor 400 reads the information in memory 401 and, in conjunction with its hardware, completes the steps of the above method.

[0092] The electronic devices and methods provided in this disclosure are based on the same inventive concept and have the same beneficial effects as the methods they employ, operate, or implement.

[0093] This disclosure also provides a computer-readable storage medium corresponding to the method provided in the foregoing embodiments. Please refer to... Figure 5 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored, which, when run by a processor, executes the methods provided in any of the foregoing embodiments.

[0094] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0095] The computer-readable storage medium provided in the above embodiments of this disclosure and the methods provided in the embodiments of this disclosure are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0096] It should be noted that:

[0097] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this disclosure may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0098] Similarly, it should be understood that, in order to simplify this disclosure and aid in understanding one or more of the various inventive aspects, in the foregoing description of exemplary embodiments of this disclosure, various features of this disclosure are sometimes grouped together in a single embodiment, figure, or description thereof. However, this approach to disclosure should not be construed as reflecting a schematic diagram in which the claimed disclosure requires more features than are explicitly stated in each embodiment. The inventive aspect lies in fewer than all features of a single foregoing disclosed embodiment. Therefore, the implementation following the detailed description is hereby explicitly incorporated into this detailed description, wherein each embodiment itself serves as a separate embodiment of this disclosure.

[0099] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this disclosure and form different embodiments.

[0100] The above description is merely a preferred embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An image processing method under extremely low light conditions, characterized in that, The method includes: The voltage sequence corresponding to each pixel in the image to be processed is acquired; the voltage sequence represents the change in the light signal intensity of the corresponding pixel. For any pixel, based on the voltage sequence corresponding to the pixel, the statistical characteristics of the voltage sequence corresponding to the pixel are determined, and based on the voltage sequence corresponding to the pixel and the voltage sequences of each reference pixel in a preset neighborhood, the correlation characteristics of the pixel are determined; wherein, the reference pixels are other pixels in the preset neighborhood besides the pixel; the pixel is the center pixel of the preset neighborhood; Based on the statistical features and the correlation features, the image to be processed is reconstructed to obtain the target image; The step of determining the statistical characteristics of the voltage sequence corresponding to the pixel based on the voltage sequence corresponding to the pixel includes: Extract the peak value of the voltage sequence corresponding to the pixel, where the peak value represents the maximum light signal intensity of the pixel; Extract the rate of change of the voltage sequence; the rate of change characterizes the intensity of the optical signal change at the pixel. The maximum value of the optical signal intensity and the intensity of the optical signal change are used as the statistical features; The step of determining the association features of the pixel based on the voltage sequence corresponding to the pixel and the voltage sequences of each reference pixel in a preset neighborhood includes: For any reference pixel within the preset neighborhood, calculate the similarity between the voltage sequence corresponding to the reference pixel and the voltage sequence corresponding to the pixel. A target weight is generated based on the similarity corresponding to each reference pixel, and the target weight represents the degree of influence of each reference pixel on the pixel. The associated features are generated based on the target weights and the voltage sequences of the pixels.

2. The method according to claim 1, characterized in that, Based on the statistical features and the correlation features, the image to be processed is reconstructed to obtain the target image, including: Based on the statistical features and the correlation features, the fusion features of the image to be processed are generated; Based on the fusion features, the image to be processed is reconstructed to obtain the target image.

3. The method according to claim 2, characterized in that, The step of generating fusion features for the image to be processed based on the statistical features and the correlation features includes: The fused feature is obtained by weighted fusion based on the preset weights corresponding to the statistical features and the correlation features respectively; wherein the preset weights represent the importance of the corresponding features.

4. The method according to claim 3, characterized in that, The process of reconstructing the image to be processed based on the fusion features to obtain the target image includes: The image to be processed is reconstructed based on the fusion features to obtain an initial image; The initial image is enhanced to obtain the target image; the enhancement operation includes at least one of interpolation, white balance, contrast equalization, brightness adjustment, and frequency domain artifact suppression.

5. The method according to claim 1, characterized in that, The acquisition of the voltage sequence corresponding to each pixel in the image to be processed includes: When acquiring the image to be processed, the voltage sequence of each pixel is obtained. The voltage sequence is obtained by removing the pre-acquired reference voltage sequence from the voltage sequence to be processed.

6. The method according to claim 5, characterized in that, The step of acquiring the processing voltage sequence for each pixel in the image to be processed includes: When acquiring the image to be processed, the initial voltage sequence corresponding to each pixel is obtained; The initial voltage sequence is amplified to obtain an amplified initial voltage sequence; The amplified initial voltage sequence is denoised to obtain the voltage sequence to be processed.

7. The method according to claim 5, characterized in that, The method further includes: The reference voltage sequence is periodically acquired, and the reference voltage sequence acquired in the current period is used to replace the reference voltage sequence acquired in the previous period.

8. An image processing device for extremely low light environments, characterized in that, The device includes: The acquisition module is used to acquire the voltage sequence corresponding to each pixel in the image to be processed; the voltage sequence represents the change in the light signal intensity of the corresponding pixel. The feature extraction module is used to, for any pixel, determine the statistical features of the voltage sequence corresponding to the pixel based on the voltage sequence corresponding to the pixel, and determine the association features of the pixel based on the voltage sequence corresponding to the pixel and the voltage sequences of each reference pixel in a preset neighborhood; wherein, the reference pixels are other pixels in the preset neighborhood besides the pixel; and the pixel is the center pixel of the preset neighborhood. The reconstruction module is used to reconstruct the image to be processed based on the statistical features and the correlation features to obtain the target image; The step of determining the statistical characteristics of the voltage sequence corresponding to the pixel based on the voltage sequence corresponding to the pixel includes: Extract the peak value of the voltage sequence corresponding to the pixel, where the peak value represents the maximum light signal intensity of the pixel; Extract the rate of change of the voltage sequence; the rate of change characterizes the intensity of the optical signal change at the pixel. The maximum value of the optical signal intensity and the intensity of the optical signal change are used as the statistical features; The step of determining the association features of the pixel based on the voltage sequence corresponding to the pixel and the voltage sequences of each reference pixel in a preset neighborhood includes: For any reference pixel within the preset neighborhood, calculate the similarity between the voltage sequence corresponding to the reference pixel and the voltage sequence corresponding to the pixel. A target weight is generated based on the similarity corresponding to each reference pixel, and the target weight represents the degree of influence of each reference pixel on the pixel. The associated features are generated based on the target weights and the voltage sequences of the pixels.

Images