Image processing method and device in extremely low illumination environment

Abstract

The invention provides an image processing method and device in an extremely low light environment, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a voltage sequence corresponding to each pixel point in a to-be-processed image; the voltage sequence is the change of the optical signal intensity of the corresponding pixel point; according to the voltage sequences of the pixel points, the statistical characteristics of the voltage sequences corresponding to the pixel points are determined, compared with the prior art that only static values at corresponding moments are adopted, voltage values in all the voltage sequences are comprehensively considered, and the accuracy of the optical signal intensity is improved; furthermore, according to the voltage sequence corresponding to the pixel point and the voltage sequence of each reference pixel point in the preset neighborhood, the correlation feature corresponding to the pixel point is determined, the correlation feature integrates the voltage sequence of the reference pixel points in the preset neighborhood, and the abnormal pixel points of the image are eliminated; and according to the statistical features and the associated features, the to-be-processed image is reconstructed to obtain the target image, so that the quality of the corresponding image under the low-light condition is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method and device in an extremely low-light environment. Background Art

[0002] Traditional Image Signal Processing (TSP) converts the raw RAW data captured by the sensor into RGB images that can be understood by humans and machines. Low-light images are images acquired in extremely low-light environments.

[0003] In related technologies, low-light image processing typically only processes the static values ​​of the sensor voltage timing signal at the corresponding moment. However, these static values ​​cannot represent the entire voltage timing signal information and cannot improve the quality of low-light images. Summary of the Invention

[0004] The present disclosure provides an image processing method and device in an extremely low-light environment.

[0005] The first embodiment of the present disclosure provides an image processing method in an extremely low-light environment, including: Collecting a voltage sequence corresponding to each pixel in the image to be processed; the voltage sequence represents a change in the intensity of the light signal of the corresponding pixel; For any pixel point, based on the voltage sequence corresponding to the pixel point, determine the statistical characteristics of the voltage sequence corresponding to the pixel point, and based on the voltage sequence corresponding to the pixel point and the voltage sequence of each reference pixel point in a preset neighborhood, determine the association characteristics of the pixel point; wherein the reference pixel point is a pixel point other than the pixel point in the preset neighborhood; and the pixel point is the center pixel point of the preset neighborhood; The image to be processed is reconstructed according to the statistical features and the associated features to obtain a target image.

[0006] In the embodiment of the present disclosure, determining the statistical features of the voltage sequence corresponding to the pixel points according to the voltage sequence corresponding to the pixel points includes: Extracting a peak value of a voltage sequence corresponding to the pixel point, wherein the peak value represents a maximum value of an optical signal intensity of the pixel point; Extracting the rate of change of the voltage sequence; the rate of change represents the intensity of the change of the light signal of the pixel point; The maximum optical signal intensity and the optical signal variation intensity are used as the statistical features.

[0007] In the embodiment of the present disclosure, the correlation feature between the voltage sequence corresponding to the pixel point and the voltage sequence of each reference pixel point in a preset neighborhood includes: For any reference pixel point within the preset neighborhood, calculating the similarity between the voltage sequence corresponding to the reference pixel point and the voltage sequence corresponding to the pixel point; Generate a target weight according to the similarity corresponding to each reference pixel point, wherein the target weight represents the influence degree of each reference pixel point on the pixel point; The associated feature is generated according to the target weight and the voltage sequence of the pixel point.

[0008] In an embodiment of the present disclosure, reconstructing the image to be processed based on the statistical features and the correlation features to obtain a target image includes: generating a fusion feature of the image to be processed according to the statistical features and the correlation features; Based on the fusion features, the image to be processed is reconstructed to obtain the target image.

[0009] In the embodiment of the present disclosure, generating the fusion feature of the image to be processed based on the statistical feature and the correlation feature includes: The fused feature is obtained by performing weighted fusion according to the preset weights corresponding to the statistical feature and the associated feature respectively; wherein the preset weight represents the importance of the corresponding feature.

[0010] In the embodiment of the present disclosure, reconstructing the image to be processed based on the fusion features to obtain the target image includes: Reconstructing the image to be processed based on the fusion features to obtain an initial image; An enhancement operation is performed on the initial image to obtain the target image; the enhancement operation includes at least one of interpolation, white balance, balanced contrast, brightness adjustment, and frequency domain artifact suppression.

[0011] In the embodiment of the present disclosure, the step of collecting the voltage timing corresponding to each pixel in the image to be processed includes: Acquiring a voltage sequence to be processed corresponding to each pixel point when collecting the image to be processed; The pre-acquired reference voltage sequence is removed from the voltage sequence to be processed to obtain a target voltage sequence.

[0012] In the embodiment of the present disclosure, when acquiring the image to be processed, the voltage sequence to be processed corresponding to each pixel point includes: Acquiring an initial voltage sequence corresponding to each pixel when collecting the image to be processed; amplifying the initial voltage sequence to obtain an amplified initial voltage sequence; The amplified initial voltage sequence is denoised to obtain the voltage sequence to be processed.

[0013] In an embodiment of the present disclosure, the method further includes: The reference voltage sequence is periodically acquired, and the reference voltage sequence acquired in the previous cycle is replaced by the reference voltage sequence acquired in the current cycle.

[0014] An embodiment of a second aspect of the present disclosure provides an image processing device in an extremely low-light environment, the device comprising: An acquisition module is used to acquire a voltage sequence corresponding to each pixel in the image to be processed; the voltage sequence represents a change in the intensity of the light signal of the corresponding pixel; a feature extraction module, configured to determine, for any pixel point, statistical features of a voltage sequence corresponding to the pixel point based on the voltage sequence corresponding to the pixel point, and determine associated features of the pixel point based on the voltage sequence corresponding to the pixel point and voltage sequences of reference pixels within a preset neighborhood; wherein the reference pixel point is a pixel point within the preset neighborhood other than the pixel point; and the pixel point is a central pixel point of the preset neighborhood; A reconstruction module is used to reconstruct the image to be processed according to the statistical features and the correlation features to obtain a target image.

[0015] An embodiment of the third aspect of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect or any optional embodiment of the first aspect.

[0016] An embodiment of the fourth aspect of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the method described in the first aspect and any optional implementation manner of the first aspect.

[0017] The technical solutions provided in the embodiments of the present disclosure have at least the following technical effects or advantages: A voltage sequence corresponding to each pixel in the image to be processed is collected; wherein the voltage sequence represents the change in the light signal intensity of the corresponding pixel. On this basis, for any pixel, according to the voltage sequence corresponding to the pixel, the statistical characteristics of the voltage sequence corresponding to the pixel are determined. The statistical characteristics can represent the temporal characteristics of the pixel. Compared with the related art which only uses the static numerical value at the corresponding moment, the voltage values ​​in all voltage sequences are comprehensively considered, thereby improving the accuracy of the light signal intensity to a certain extent. Furthermore, according to the voltage sequence corresponding to the pixel and the voltage sequence of each reference pixel in a preset neighborhood, the correlation characteristics corresponding to the pixel are determined. The correlation characteristics integrate the voltage sequences of the reference pixel in the preset neighborhood, thereby eliminating abnormal pixels in the image to a certain extent. Finally, according to the statistical characteristics and the correlation characteristics, the image to be processed is reconstructed to obtain the target image, thereby improving the quality of the corresponding image under low light conditions.

[0018] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will become apparent from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present disclosure. The same reference symbols are used throughout the drawings to represent the same components. In the drawings: Figure 1 A flowchart of an image processing method in an extremely low-light environment provided by an embodiment of the present disclosure is shown; Figure 2 A schematic diagram showing an image processing method in an extremely low-light environment provided by an embodiment of the present disclosure is shown; Figure 3 A schematic diagram of an image processing device in an extremely low-light environment provided by an embodiment of the present disclosure is shown; Figure 4 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown; Figure 5 A schematic diagram of a storage medium provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0020] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0021] It should be noted that, unless otherwise specified, the technical or scientific terms used in the present disclosure should have the common meanings understood by those skilled in the art to which the present disclosure belongs.

[0022] The present disclosure provides an image processing method in an extremely low light environment. Figure 1 An image processing method in an extremely low-light environment provided by an embodiment of the present disclosure includes the following steps: In step S11 , a voltage sequence corresponding to each pixel in the image to be processed is collected.

[0023] The voltage sequence represents the change in the intensity of the light signal at the corresponding pixel point.

[0024] For example, in the disclosed embodiments, the image to be processed can be a two-dimensional light intensity distribution formed in the focal plane of a CMOS sensor. A pixel can be an independent 2.8µm × 2.8µm photosensitive unit in a CMOS array, corresponding to a spatial location (x, y). The target voltage sequence is a one-dimensional signal V(x, y, t) composed of the analog voltages output by the same pixel over time during consecutive sampling periods. The light signal intensity is the photon flux reaching the pixel per unit area and per unit time.

[0025] On the microstructure CMOS array, each pixel performs photoelectric conversion on the incident photons, generates photogenerated charges proportional to the intensity of the light signal, and accumulates them into microvolt analog voltages on the integrating capacitor. This microvolt analog voltage is the voltage sequence in the embodiment of the present disclosure.

[0026] In some embodiments, the voltage timing corresponding to each pixel in the image to be processed is collected, and the voltage sequence to be processed corresponding to each pixel when the image to be processed is collected can also be obtained; the pre-acquired reference voltage sequence is removed from the voltage sequence to be processed to obtain the target voltage sequence.

[0027] For example, the pre-acquired reference voltage sequence is a dark frame captured in complete darkness, at the same temperature and integration time. The digital bitstream output through the same signal chain is used to characterize dark current and system bias. Before the actual capture, one or more completely black frames can be continuously captured, with the shutter closed and temperature and exposure parameters unchanged, to generate a "reference voltage sequence" for each pixel.

[0028] A reference voltage sequence is calculated for each pixel and deducted point by point in subsequent sampling to ensure that the zero baseline of the voltage sequence corresponds to zero photon input. In some embodiments, the reference voltage sequence can be acquired periodically, and the reference voltage sequence acquired in the current cycle can replace the reference voltage sequence acquired in the previous cycle to ensure the accuracy of the reference voltage sequence. For example, the corresponding reference voltage sequence can be acquired every 60 frames.

[0029] In some embodiments, the voltage sequence to be processed can be determined in the following manner to improve the accuracy of the voltage sequence to be processed: obtaining an initial voltage sequence corresponding to each pixel when acquiring the image to be processed; amplifying the initial voltage sequence to obtain an amplified initial voltage sequence; and denoising the amplified initial voltage sequence to obtain the voltage sequence to be processed.

[0030] For example, the initial voltage sequence can be a one-dimensional signal consisting of microvolt-level analog voltages output by CMOS pixels after exposure integration is complete but before any amplification or filtering. Amplification can be performed by a programmable gain amplifier (PGA) in the analog domain, linearly boosting the gain of the initial voltage sequence to bring its amplitude within the optimal quantization range of the subsequent analog-to-digital converter (ADC). Analog or digital filtering is used to suppress quantization noise, high-frequency clock noise, and low-frequency drift introduced during the amplification process, resulting in a pure voltage sequence to be processed.

[0031] After amplification and denoising, the digitized voltage time series can be directly used for subsequent baseline correction and algorithm analysis as the voltage sequence to be processed.

[0032] In step S12, for any pixel point, the statistical characteristics of the voltage sequence corresponding to the pixel point are determined based on the voltage sequence corresponding to the pixel point, and the association characteristics of the pixel point are determined based on the voltage sequence corresponding to the pixel point and the voltage sequences of each reference pixel point in a preset neighborhood.

[0033] The reference pixel point is a pixel point other than the pixel point in the preset neighborhood; the pixel point is a central pixel point in the preset neighborhood.

[0034] For example, statistical features can be spatially context-free information extracted solely from the pixel's voltage sequence, such as mean, variance, and kurtosis. A preset neighborhood is a rectangular or cross-shaped window (e.g., 3×3 or 5×5) with a radius r centered on the current pixel, which defines the set of reference pixels. Reference pixels are all pixels within the preset neighborhood except the center pixel, and their voltage sequences are denoted as V(i, j, t). Correlation features can be quantities that describe the degree of synchronization or difference between the voltage sequence of the center pixel and the voltage sequences of each reference pixel in the neighborhood.

[0035] Calculate the first- and second-order statistics of the voltage sequence V(x, y, t) at the central pixel (x, y), including the mean, variance, skewness, and kurtosis, to form the pixel's statistical feature vector S(x, y). Read the voltage sequences V(i, j, t) of all reference pixels (i, j) within the preset neighborhood window, ensuring that each sequence is of the same length as the central sequence and is time-aligned.

[0036] The Pearson correlation coefficient ρ(x, y, i, j) between the voltage sequence V(x, y, t) of the central pixel (x, y) and the voltage sequence V(i, j, t) of each reference pixel (i, j) can be calculated to obtain a correlation coefficient matrix. The spatiotemporal cross-correlation peak can also be used as a similarity score to quantify the degree of match between the two in waveform shape and delay.

[0037] In some embodiments, determining the statistical characteristics of the voltage sequence corresponding to the pixel point in the above step S12 based on the voltage sequence corresponding to the pixel point can also be achieved in the following way: extracting the peak value of the voltage sequence corresponding to the pixel point, the peak value represents the maximum value of the light signal intensity of the pixel point; extracting the change rate of the voltage sequence; the change rate represents the change intensity of the light signal of the pixel point; and using the maximum value of the light signal intensity and the light signal change intensity as statistical characteristics.

[0038] For example, a linear scan is performed on the pixel voltage sequence within a time window, and the maximum value is recorded as the "maximum optical signal intensity." For the same voltage sequence, a first-order difference or a 1×2 convolution kernel [-1, 1] is used to calculate the difference between adjacent sampling points: ΔV(t) = V(t+1) – V(t). The maximum absolute value of ΔV(t), ΔV_max, is taken as the "optical signal variation intensity." The maximum optical signal intensity and the optical signal variation intensity are concatenated or cascaded to form the two-dimensional statistical features of the pixel.

[0039] In some embodiments, the correlation feature between the voltage sequence corresponding to the pixel point and the voltage sequence of each reference pixel point in the preset neighborhood in the above step S12 can also be achieved in the following way: for any reference pixel point in the preset neighborhood, calculate the similarity between the voltage sequence corresponding to the reference pixel point and the voltage sequence corresponding to the pixel point; generate a target weight based on the similarity corresponding to each reference pixel point, and the target weight represents the degree of influence of each reference pixel point on the pixel point; generate the correlation feature based on the target weight and the voltage sequence of the pixel point.

[0040] For example, the voltage sequence of the central pixel is calculated pairwise with the voltage sequence of any reference pixel in a preset neighborhood. Specifically, linear distance, spectral density correlation, and cosine similarity using a deep learning network are calculated to determine the corresponding correlation coefficient. The correlation coefficient is normalized and mapped to a positive weight, with the sum of all neighborhood weights being 1; this is the "target weight," which quantifies the influence of the reference pixel on the central pixel.

[0041] The target weight is used to perform weighted averaging on the voltage sequences of all reference pixels in the neighborhood to obtain a weighted neighborhood sequence. The weighted neighborhood sequence is then concatenated with the voltage sequence corresponding to the central pixel or the mutual energy is calculated to form the final correlation feature vector.

[0042] In step S13, the image to be processed is reconstructed according to the statistical features and the correlation features to obtain a target image.

[0043] For example, the statistical features and correlation features can be input into the Spatio-Temporal Reconstruction Network (STRN); the SPATIAL-Temporal Reconstruction Network performs upsampling in the spatial domain and 1×1 convolution compression in the temporal domain, and finally outputs a target image with the same resolution as the original.

[0044] In some embodiments, the above step S13 includes: generating fusion features of the image to be processed according to the statistical features and the correlation features; and reconstructing the image to be processed based on the fusion features to obtain a target image.

[0045] For example, the correlation coefficient matrix vector can be compressed into a fixed-length correlation feature vector C(x, y), for example by taking the neighborhood mean, maximum value, or weighted sum, to ensure that the feature dimension is consistent with the subsequent network input. The statistical feature vector S(x, y) and the correlation feature vector C(x, y) are concatenated or fed in parallel to the downstream self-attention fusion module to complete the initial fusion feature of the pixel's spatiotemporal information.

[0046] The initial fused features can be further streamlined and fed into a lightweight self-attention module (LSAM) for channel-weighted attention. The final fused features are then output, with weights learned end-to-end by the network. Image reconstruction is then performed based on the final fused features.

[0047] In some embodiments, based on statistical features and correlation features, fusion features of the image to be processed are generated, including: performing weighted fusion to obtain fusion features based on preset weights corresponding to the statistical features and correlation features respectively; wherein the preset weights represent the importance of the corresponding features.

[0048] In some embodiments, the processed image is reconstructed based on the fusion features to obtain a target image, including: reconstructing the processed image based on the fusion features to obtain an initial image; performing an enhancement operation on the initial image to obtain a target image; the enhancement operation includes at least one of interpolation, white balance, balanced contrast, brightness adjustment, and frequency domain artifact suppression.

[0049] For example, after the fused features are input into the corresponding reconstruction network, the reconstructed initial image is obtained. Specifically, the reconstruction network can be a deep network composed of three levels of downsampling convolution, a residual mapping unit (including a physical regularization loss consistent with the sensor response curve), and three levels of upsampling convolution.

[0050] The initial image contains RGB information. To enhance the visual quality and quality stability of the initial image, the disclosed embodiments may further post-process the initial image. Specifically, the initial image may be subjected to operations such as interpolation, white balance, contrast balancing, brightness adjustment, and frequency domain artifact suppression. During implementation, the initial image may be fed into a pre-set large model to generate corresponding adjustment parameters for interpolation, white balance, contrast balancing, brightness adjustment, and frequency domain artifact suppression. The initial image may then be adjusted based on these adjustment parameters.

[0051] like Figure 2 As shown, it is an exemplary schematic diagram of the image processing method corresponding to the embodiment of the present disclosure, in which a voltage sequence to be processed corresponding to the image to be processed is acquired, and the voltage sequence to be processed is amplified and filtered, and the reference voltage sequence is removed to obtain a target voltage sequence; feature preprocessing is performed on the target voltage sequence, which may specifically include extracting statistical features and correlation features of the target voltage sequence and then performing feature fusion; image reconstruction is performed based on the fused features, and the image reconstruction may sequentially use a three-stage downsampling convolution module, a residual mapping, and a three-stage upsampling convolution module to obtain an initial image; finally, the initial image is preprocessed to obtain the corresponding target image.

[0052] Through the image processing method in an extremely low-light environment of the embodiment of the present application, a voltage sequence corresponding to each pixel point in the image to be processed is collected; wherein the voltage sequence represents the change in the light signal intensity of the corresponding pixel point, and on this basis, for any pixel point, according to the voltage sequence corresponding to the pixel point, the statistical characteristics of the voltage sequence corresponding to the pixel point are determined, and the statistical characteristics can represent the timing characteristics of the pixel point. Compared with the related art that only uses the static numerical value at the corresponding moment, the voltage values ​​in all voltage sequences are comprehensively considered, and the accuracy of the light signal intensity is improved to a certain extent; further, according to the voltage sequence corresponding to the pixel point and the voltage sequence of each reference pixel point in a preset neighborhood, the correlation feature corresponding to the pixel point is determined, and the correlation feature integrates the voltage sequence of the reference pixel points in the preset neighborhood, and eliminates abnormal pixels of the image to a certain extent; finally, according to the statistical characteristics and the correlation features, the image to be processed is reconstructed to obtain the target image, thereby improving the quality of the corresponding image under low-light conditions.

[0053] Corresponding to the implementation of the above method, the embodiment of the present disclosure further provides an image processing device in an extremely low light environment, for performing the above Figure 1 The image processing method in an extremely low light environment of any embodiment is shown, Figure 3 As shown, the image processing device for extremely low light environment includes: The acquisition module 301 is used to acquire a voltage sequence corresponding to each pixel in the image to be processed; the voltage sequence represents the change in the light signal intensity of the corresponding pixel; A feature extraction module 302 is configured to determine, for any pixel, statistical features of a voltage sequence corresponding to the pixel based on the voltage sequence corresponding to the pixel, and to determine associated features of the pixel based on the voltage sequence corresponding to the pixel and voltage sequences of reference pixels within a preset neighborhood; wherein the reference pixel is a pixel other than the pixel within the preset neighborhood; and the pixel is the center pixel of the preset neighborhood; The reconstruction module 303 is configured to reconstruct the image to be processed according to the statistical features and the correlation features to obtain a target image.

[0054] The image processing device in an extremely low-light environment provided by the above-mentioned embodiment of the present disclosure and the image processing method in an extremely low-light environment provided by the embodiment of the present disclosure are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0055] The present disclosure also provides an electronic device for executing the above method. Figure 4 , which shows a schematic diagram of an electronic device provided by some embodiments of the present disclosure. Figure 4As shown, the electronic device includes: a processor 400, a memory 401, a bus 402 and a communication interface 403, wherein the processor 400, the communication interface 403 and the memory 401 are connected via the bus 402; the memory 401 stores a computer program that can be run on the processor 400, and the processor 400 executes the aforementioned Figure 1 The method provided in any embodiment is illustrated.

[0056] Memory 401 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage. Communication between the system network element and at least one other network element is achieved through at least one communication interface 403 (which may be wired or wireless), and may utilize the Internet, a wide area network, a local area network, a metropolitan area network, or the like.

[0057] The bus 402 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 401 is used to store programs. The processor 400 executes the program after receiving the execution instruction. Figure 1 The method disclosed in any of the illustrated embodiments may be applied to the processor 400 or implemented by the processor 400 .

[0058] The processor 400 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method may be completed by hardware integrated logic circuits or software instructions in the processor 400. The processor 400 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It may implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present disclosure may be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 401 , and the processor 400 reads the information in the memory 401 and completes the steps of the above method in combination with its hardware.

[0059] The electronic device provided by the embodiments of the present disclosure and the method provided by the embodiments of the present disclosure are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented by them.

[0060] The present disclosure also provides a computer-readable storage medium corresponding to the method provided in the above embodiment. Figure 5 The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, the method provided by any of the aforementioned embodiments is executed.

[0061] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0062] The computer-readable storage medium provided by the above-mentioned embodiments of the present disclosure and the method provided by the embodiments of the present disclosure are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0063] It should be noted that: In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present disclosure can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0064] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present disclosure, various features of the present disclosure are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed approach should not be interpreted as reflecting a schematic diagram that the claimed disclosure requires more features than those explicitly recited in each embodiment. Inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the implementation methods that follow the specific embodiments are hereby expressly incorporated into the specific embodiments, with each embodiment itself serving as a separate embodiment of the present disclosure.

[0065] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features and not other features included in other embodiments, the combination of features from different embodiments is meant to be within the scope of this disclosure and to form different embodiments.

[0066] The above description is only a preferred specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or replacements that can be easily conceived by any technician familiar with this technical field within the technical scope disclosed in this disclosure should be covered by the protection scope of the present disclosure.

Claims

1. An image processing method in an extremely low light environment, characterized in that: The method comprises: Collecting a voltage sequence corresponding to each pixel in the image to be processed; the voltage sequence represents a change in the intensity of the light signal of the corresponding pixel; For any pixel point, based on the voltage sequence corresponding to the pixel point, determine the statistical characteristics of the voltage sequence corresponding to the pixel point, and based on the voltage sequence corresponding to the pixel point and the voltage sequence of each reference pixel point in a preset neighborhood, determine the association characteristics of the pixel point; wherein the reference pixel point is a pixel point other than the pixel point in the preset neighborhood; and the pixel point is the center pixel point of the preset neighborhood; The image to be processed is reconstructed according to the statistical features and the associated features to obtain a target image.

2. The method according to claim 1, characterized in that The determining, based on the voltage sequence corresponding to the pixel point, the statistical characteristics of the voltage sequence corresponding to the pixel point includes: Extracting a peak value of a voltage sequence corresponding to the pixel point, wherein the peak value represents a maximum value of an optical signal intensity of the pixel point; Extracting the rate of change of the voltage sequence; the rate of change represents the intensity of the change of the light signal of the pixel point; The maximum optical signal intensity and the optical signal variation intensity are used as the statistical features.

3. The method according to claim 1, characterized in that The correlation feature between the voltage sequence corresponding to the pixel point and the voltage sequence of each reference pixel point in a preset neighborhood includes: For any reference pixel point within the preset neighborhood, calculating the similarity between the voltage sequence corresponding to the reference pixel point and the voltage sequence corresponding to the pixel point; Generate a target weight according to the similarity corresponding to each reference pixel point, wherein the target weight represents the influence degree of each reference pixel point on the pixel point; The associated feature is generated according to the target weight and the voltage sequence of the pixel point.

4. The method according to any one of claims 1 to 3, characterized in that Reconstructing the image to be processed according to the statistical features and the correlation features to obtain a target image, including: generating a fusion feature of the image to be processed according to the statistical features and the correlation features; Based on the fusion features, the image to be processed is reconstructed to obtain the target image.

5. The method according to claim 4, characterized in that Generating the fusion features of the image to be processed according to the statistical features and the correlation features includes: The fused feature is obtained by performing weighted fusion according to the preset weights corresponding to the statistical feature and the associated feature respectively; wherein the preset weight represents the importance of the corresponding feature.

6. The method according to claim 5, characterized in that The reconstructing the image to be processed based on the fusion feature to obtain the target image includes: Reconstructing the image to be processed based on the fusion features to obtain an initial image; An enhancement operation is performed on the initial image to obtain the target image; the enhancement operation includes at least one of interpolation, white balance, balanced contrast, brightness adjustment, and frequency domain artifact suppression.

7. The method according to claim 1, characterized in that The collecting of the voltage sequence corresponding to each pixel in the image to be processed includes: Acquiring a voltage sequence to be processed corresponding to each pixel point when collecting the image to be processed; The voltage sequence is obtained by removing the pre-acquired reference voltage sequence from the voltage sequence to be processed.

8. The method according to claim 7, characterized in that The acquiring and collecting of the image to be processed, the voltage sequence to be processed corresponding to each pixel point, includes: Acquiring an initial voltage sequence corresponding to each pixel when collecting the image to be processed; amplifying the initial voltage sequence to obtain an amplified initial voltage sequence; The amplified initial voltage sequence is denoised to obtain the voltage sequence to be processed.

9. The method according to claim 7, characterized in that The method further comprises: The reference voltage sequence is periodically acquired, and the reference voltage sequence acquired in the previous cycle is replaced by the reference voltage sequence acquired in the current cycle.

10. An image processing device for extremely low light environments, characterized in that: The device comprises: An acquisition module is used to acquire a voltage sequence corresponding to each pixel in the image to be processed; the voltage sequence represents a change in the intensity of the light signal of the corresponding pixel; a feature extraction module, configured to determine, for any pixel point, statistical features of a voltage sequence corresponding to the pixel point based on the voltage sequence corresponding to the pixel point, and determine associated features of the pixel point based on the voltage sequence corresponding to the pixel point and voltage sequences of reference pixels within a preset neighborhood; wherein the reference pixel point is a pixel point within the preset neighborhood other than the pixel point; and the pixel point is a central pixel point of the preset neighborhood; A reconstruction module is used to reconstruct the image to be processed according to the statistical features and the correlation features to obtain a target image.

Images