A system and method for real-time image denoising

Abstract

This disclosure discloses a system and method for real-time image denoising. The system and method may be included in a combined processing apparatus, which may further include a general interconnect interface and other processing devices. The computing device interacts with the other processing devices to jointly perform user-specified computational operations. The combined processing apparatus may also include a storage device connected to the device and other processing devices respectively, for storing data from the device and other processing devices. The solution disclosed herein can improve the operational efficiency of computations in various data processing fields, including, for example, artificial intelligence, thereby reducing the overall overhead and cost of computation.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of computer vision, and more particularly, to image denoising in computer vision. BACKGROUND

[0002] Image and video denoising plays a very important role in the field of computer vision, and is also the core algorithm in ISP (image signal processing). Traditional ISP denoising algorithms in the Bayer domain or YUV domain usually use a combination of various traditional filters and adjusted parameters to achieve denoising.

[0003] Due to the lack of data sets and differences in sensors, in recent years, image and video methods based on deep learning mainly do the work of removing additive Gaussian white noise on RGB images, that is, artificial Gaussian white noise is added to clean RGB pictures, and then a neural network is used to output the denoised pictures. In deep learning video denoising, the inter-frame alignment method based on optical flow estimation is the mainstream method.

[0004] The prior art has the following defects:

[0005] The inter-frame alignment method using optical flow motion estimation has a large amount of calculation, and in the presence of noise, the error of the optical flow calculation result is too large. The existing denoising method is more based on Gaussian white noise, which cannot accurately describe the noise in actual deployment, and needs to input the noise intensity information, etc., and cannot realize blind denoising. In addition, the model in the prior art is too large or needs to rely on the result of optical flow, so real-time denoising cannot be achieved.

[0006] For denoising algorithms in the RGB or YUV domain, since the image has been processed by ISP upstream demosaicing algorithms, white balance algorithms, etc., the noise distribution is very complex, and it is difficult to describe it with a simple distribution compared to the Raw picture in the Bayer domain. The ground truth of the training data set is also difficult to obtain. SUMMARY

[0007] One object of the present disclosure is to overcome the defect that the prior art cannot perform real-time denoising on images.

[0008] According to a first aspect of the present disclosure, a system for real-time denoising of images is provided, comprising: a motion alignment module configured to obtain a second image group from a first image group comprising a plurality of first image frames and a noise description of the first image group, the second image group containing motion information between the plurality of first image frames to align the first image frames; and a denoising module configured to obtain a third image after noise reduction from the second image group.

[0009] According to a second aspect of the present disclosure, there is provided a method for real-time denoising of images, comprising: obtaining a second image group from a first image group comprising a plurality of first image frames and a noise description of the first image group, the second image group containing running information between the plurality of first image frames to facilitate alignment of the first image frames; and obtaining a third image after denoising according to the second image group.

[0010] According to a third aspect of the present disclosure, there is provided an electronic device, comprising: one or more processors; and a memory having computer-executable instructions stored therein, which when executed by the one or more processors, cause the electronic device to perform the method as described above.

[0011] According to a fourth aspect of the present disclosure, there is provided a computer-readable storage medium comprising computer-executable instructions, which when executed by one or more processors, perform the method as described above.

[0012] The scheme of the present disclosure can implicitly learn the motion information between frames without using the optical flow method in the prior art, and thereby align the adjacent frames. Finally, the noise in the images can be efficiently removed according to the aligned adjacent frames and the obtained noise description.

[0013] In addition, the present disclosure can remove the noise in the images in real time when the noise description and the parameters are known. BRIEF DESCRIPTION OF DRAWINGS

[0014] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0015] Figure 1 A schematic diagram of a system for real-time denoising of images according to an embodiment of the present disclosure is shown;

[0016] Figure 2 A schematic diagram of division of a first image group according to an embodiment of the present disclosure is shown;

[0017] Figure 3 A schematic diagram of division of a first image group according to an embodiment of the present disclosure is shown;

[0018] Figures 4a to 4c A schematic diagram of division of a first image group according to other embodiments of the present disclosure is shown;

[0019] Figure 5A method of real-time denoising of images is shown according to one embodiment of the present disclosure;

[0020] Figure 6 A combination processing device is shown; and

[0021] Figure 7 An exemplary board card is shown. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present disclosure.

[0023] It should be understood that the terms "first", "second", "third", and "fourth" and the like in the claims, the specification, and the drawings of the present disclosure are used to distinguish different objects, rather than to describe a particular order. The terms "include" and "contain" used in the specification and claims of the present disclosure indicate the presence of described features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or sets thereof.

[0024] It should also be understood that the terms used in the specification of the present disclosure are only for the purpose of describing specific embodiments, and are not intended to limit the present disclosure. As used in the specification and claims of the present disclosure, the singular forms "a", "an", and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should be further understood that the term "and / or" used in the specification and claims of the present disclosure means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0025] As used in the specification and claims, the term "if" can be interpreted as "when" or "upon" or "in response to a determination" or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if detected [the described condition or event]" can be interpreted as meaning "upon determining" or "in response to determining" or "upon detecting [the described condition or event]" or "in response to detecting [the described condition or event]" depending on the context.

[0026] First, an example of noise involved in the present application is introduced.

[0027] A common type of noise is additive white Gaussian noise (AWGN). Assuming the observed image is In, the noise-free image is I, and the noise is n, then their relationship can be expressed as:

[0028] In = I + n

[0029] Under this assumption, the probability distribution of AWGN noise can be described as follows:

[0030] n i ~N(0, σ 2 )

[0031] Where, in the above formula n i The noise value at pixel i follows a mean of 0 and a variance of σ. 2 w is a Gaussian distribution.

[0032] AWGN noise is independent of the signal, therefore it cannot describe signal-dependent photon noise. A Poisson distribution can be used to describe this part of the noise. The expression for the Poisson distribution is:

[0033] n i ~αP(I i )-I i

[0034] Among them, I i The signal is noise-free, and the mean and variance of the noise are both I. i α is the amplification factor of the sensor.

[0035] The noise signal contains both signal-dependent Poisson noise and signal-independent Gaussian noise. Therefore, the model that uses a joint Poisson-Gaussian approach to describe the noise is:

[0036] n i ~αP(I i )-I i +N(0,σ) 2 )

[0037] A more general approach is to approximate this using a heteroscedastic Gaussian model:

[0038] n i ~N(0, α) 2 I i +δ 2 )

[0039] The heteroscedastic Gaussian model is often referred to as the noise level function (NLF) and is used to describe noise levels and pixel brightness.

[0040] However, due to the lack of datasets and differences in sensors, recent deep learning-based image and video methods have mainly focused on removing additive white Gaussian noise from RGB images, while less attention has been paid to signal-related Poisson noise.

[0041] Figure 1 A schematic diagram of a system for real-time image denoising according to an embodiment of the present disclosure is shown, including: a motion alignment module 120 configured to obtain a second image group based on a first image group including a plurality of first image frames and a noise description of the first image group, the second image group containing running information between the plurality of first image frames to facilitate alignment of the first image frames; and a denoising module 130 configured to obtain a denoised third image based on the second image group.

[0042] Noise descriptions can be directly obtained from image acquisition devices (such as cameras and camcorders). These descriptions can include, for example, the camera's ISO (sensitivity); the camera's gain, including analog gain, digital gain, etc.; noise parameters, such as parameters of Poisson or Gaussian distributions (e.g., noise mean, variance, etc.); and noise intensity. Obtaining detailed noise description information will be beneficial for aligning adjacent images.

[0043] In existing technologies, optical flow methods are typically used to align adjacent frames in order to fuse inter-frame motion information during video denoising. However, optical flow is difficult to calculate accurately in the presence of noise. For example, if noise in a video frame is represented as black in the color gamut, it is difficult to determine whether the movement is due to noise or the object (e.g., hair) in the image. Therefore, optical flow is difficult to calculate in the presence of noise.

[0044] In this disclosure, according to one embodiment, the first image group can be divided into a plurality of first image frame sets, each first image frame set including a portion of the plurality of first image frames, and the motion alignment module 120 can be configured to sequentially receive each first image frame set and a corresponding noise description in order to align the first image frames.

[0045] Figure 2 A schematic diagram illustrating the division of a first image group according to one embodiment of the present disclosure is shown.

[0046] like Figure 2The first image group shown includes n image frames, namely image frame 0 to image frame n-1. These image frames can be divided into multiple sets. For example, image frame 0 and image frame 1 form the first image frame set 0, image frame 1 and image frame 2 form the first image frame set 1, ..., image frame n-2 and image frame n-1 form the first image frame set m. By analyzing the changes between each image frame, the motion relationships between the image frames can be obtained.

[0047] According to one embodiment of this disclosure, adjacent first image frame sets may include at least one identical image frame.

[0048] by Figure 2 For example, the first image frame set 0 includes image frame 0 and image frame 1, the first image frame set 1 includes image frame 1 and image frame 2, and the first image frame set 2 (not shown) includes image frame 2 and image frame 3. Therefore, adjacent first image frame sets 0 and 1 include the same image frame 1, and adjacent first image frame sets 1 and 2 include the same image frame 2. With this configuration, the motion alignment module 120 can obtain the inter-frame motion information from these image frames and align the various image frames.

[0049] It is necessary to understand that Figure 2 For illustrative purposes only, each set of image frames is shown to include two image frames. In embodiments of this disclosure, the number of image frames included in each set of image frames is not limited, and different sets of image frames may include the same or different numbers of image frames.

[0050] According to a preferred embodiment of the present disclosure, at least one first image frame at the end of the preceding first image frame set is located at the beginning of the subsequent first image frame set.

[0051] Figure 3 A schematic diagram illustrating the division of a first image group according to one embodiment of the present disclosure is shown.

[0052] Figure 2 A schematic diagram illustrating the division of the first image group is provided as a whole. Figure 3 A more specific example of an implementation method is given. For example... Figure 3As shown, the first image group exemplarily includes image frame t-2, image frame t-1, image frame t, image frame t+1, and image frame t+2, where t is time. In this case, these five image frames can be divided into three image sets: first image frame set 1 includes image frame t-2, image frame t-1, and image frame t; first image frame set 2 includes image frame t-1, image frame t, and image frame t+1; and first image frame set 3 includes image frame t, image frame t+1, and image frame t+2. It can be seen that each first image frame set includes three image frames. Therefore, when these image frames are input to the motion alignment module 120, the motion alignment module 120 can obtain the motion information between the frames from these image frames and then align these image frames.

[0053] Further as Figure 3 As shown, the first image frame set 1 is first input to the motion alignment module 120. The motion alignment module 120 generates the second image 1 based on the three image frames t-2, t-1, and t and the noise description corresponding to each image frame. Next, the first image frame set 2 is input to the motion alignment module 120. The motion alignment module 120 generates the second image 2 based on the three image frames t-1, t, and t+1 and the noise description corresponding to each image frame. Then, the first image frame set 3 is input to the motion alignment module 120. The motion alignment module 120 generates the second image 3 based on the three image frames t, t+1, and t+2 and the noise description corresponding to each image frame.

[0054] It needs to be understood that, although Figure 3 The image shows three motion alignment modules 120, but this is only for ease of understanding. In reality, the motion alignment module 120 can be a single module, which receives the first image frame set 1 to the first image frame set 3 one by one. Thus, the motion alignment module 120 can know the changes between the image frames through these image frames and the noise description, thereby facilitating the alignment of information in the image frames.

[0055] It is also important to understand that there are multiple ways to group image frames, and it is not limited to the cases described above. For example, there can be other numbers of image frames, and the combination of each set of image frames is not limited to the methods described above.

[0056] Figures 4a to 4c A schematic diagram illustrating the division of a first image group according to other embodiments of the present disclosure is shown.

[0057] like Figure 4aAs shown, the first image group exemplarily includes image frame t-3, image frame t-2, image frame t-1, image frame t, image frame t+1, image frame t+2, and image frame t+3, where t is time. In this case, these seven image frames can be divided into five image sets: the first image frame set 1 includes image frames t-3, t-2, and t-1; the first image frame set 2 includes image frames t-2, t-1, and t; the first image frame set 3 includes image frames t-1, t, and t+1; the first image frame set 4 includes image frames t, t+1, and t+2; and the first image frame set 5 includes image frames t+1, t+2, and t+3.

[0058] like Figure 4b As shown, the first image group exemplarily includes image frame t-3, image frame t-2, image frame t-1, image frame t, image frame t+1, image frame t+2, and image frame t+3, where t is time. In this case, these seven image frames can be divided into four image sets: the first image frame set 1 includes image frame t-3, image frame t-2, image frame t-1, and image frame t; the first image frame set 2 includes image frame t-2, image frame t-1, image frame t, and image frame t+1; the first image frame set 3 includes image frame t-1, image frame t, image frame t+1, and image frame t+2; and the first image frame set 4 includes image frame t, image frame t+1, image frame t+2, and image frame t+3.

[0059] like Figure 4c As shown, the first image group exemplarily includes image frame t-3, image frame t-2, image frame t-1, image frame t, image frame t+1, image frame t+2, and image frame t+3, where t is time. In this case, these seven image frames can be divided into four image sets: the first image frame set 1 includes image frame t-3 and image frame t-2; the first image frame set 2 includes image frame t-2, image frame t-1, and image frame t; the first image frame set 3 includes image frame t, image frame t+1, and image frame t+2; and the first image frame set 4 includes image frame t+1, image frame t+2, and image frame t+3. Figure 4c The illustrated implementation differs from other implementations in that the number of image frames in each first image frame set is not uniform. Those skilled in the art can choose the desired partitioning method according to actual needs; this disclosure does not impose any limitations on the number of image frames in each image group or its partitioning method.

[0060] According to one embodiment of this disclosure, the motion alignment module can be a neural network with a Unet structure.

[0061] The Unet network architecture is widely used in image processing, such as medical image processing. It typically includes a contracting path and an expanding path. The contracting path is mainly used to capture context information in the image, while the expanding path is used to accurately localize the parts of the image that need to be segmented.

[0062] Next, the denoising module 130 can receive multiple images output by the motion alignment module 120 and fuse these images to obtain the final denoised image. Since the second image group output by the motion alignment module 120 already contains motion information between image frames, after alignment by the motion alignment module 120, the denoising module 130 can clearly identify the noise part, thereby removing the noise from the image.

[0063] The denoising module 130 can adopt the same network structure as the motion alignment module 120, or the denoising module 130 can be implemented by a temporal attention-based fusion method. The temporal attention-based fusion method can be obtained from the article "EDVR: Video Restoration with Enhanced Deformable Convolutional Networks" by Xintao Wang et al., which is incorporated herein by reference in its entirety.

[0064] Figure 5 A method for real-time denoising of an image according to one embodiment of the present disclosure is shown, comprising: in operation S520, obtaining a second image group based on a first image group including a plurality of first image frames and a noise description of the first image group, the second image group including running information between the plurality of first image frames to facilitate alignment of the first image frames; and in operation S530, obtaining a denoised third image based on the second image group.

[0065] This disclosure also provides an electronic device, including: one or more processors; and a memory storing computer-executable instructions that, when executed by the one or more processors, cause the electronic device to perform the method described above.

[0066] This disclosure also provides a computer-readable storage medium including computer-executable instructions that, when executed by one or more processors, perform the method described above.

[0067] In this application, the technical solution disclosed herein can be applied to images of various formats, including but not limited to Raw format, RGB format, and YUV format, etc.

[0068] The disclosed method can implicitly learn inter-frame motion information without employing existing optical flow methods, thereby aligning adjacent frames. Finally, based on the aligned adjacent frames and the obtained noise description, noise in the image can be efficiently removed.

[0069] Furthermore, this disclosure enables the removal of noise from an image in real time, provided that the noise description and parameters are known.

[0070] In this disclosure, the motion alignment module 120 and the denoising module 130 can be trained independently to obtain the desired parameter values; however, the modules are preferably trained together. Training them together yields better training results, resulting in better output during inference.

[0071] The technical solution disclosed herein can be applied to the field of artificial intelligence, and can be implemented as or in an artificial intelligence chip. This chip can exist independently or be included in a computing device.

[0072] Figure 6 A combined processing apparatus 600 is shown, comprising the aforementioned computing device 602, a general interconnect interface 604, and other processing devices 606. The computing device according to this disclosure interacts with the other processing devices to jointly perform user-specified operations. Figure 6 This is a schematic diagram of the combined processing device.

[0073] Other processing devices include one or more processor types such as central processing unit (CPU), graphics processing unit (GPU), and neural network processor. There is no limit to the number of processors included in other processing devices. These other processing devices serve as interfaces between the machine learning computing device and external data and control, including data transfer and basic control such as starting and stopping the machine learning computing device. Other processing devices can also collaborate with the machine learning computing device to complete computational tasks.

[0074] A general interconnect interface is used to transfer data and control commands between a computing device (including, for example, a machine learning computing device) and other processing devices. The computing device can obtain required input data from other processing devices and write it to on-chip storage; it can obtain control commands from other processing devices and write them to on-chip control caches; and it can read data from the computing device's storage modules and transmit it to other processing devices.

[0075] Optionally, the structure may further include a storage device 608, which is connected to both the computing device and the other processing device. The storage device is used to store data in the computing device and the other processing device, and is particularly suitable for data that cannot be fully stored in the internal storage of the computing device or other processing device, requiring computation.

[0076] This combined processing device can serve as a System-on-a-Chip (SoC) for devices such as mobile phones, robots, drones, and video surveillance equipment, effectively reducing the core area of ​​the control unit, increasing processing speed, and lowering overall power consumption. In this case, the universal interconnect interface of the combined processing device connects to certain components of the device, such as cameras, monitors, mice, keyboards, network cards, and Wi-Fi interfaces.

[0077] In some embodiments, this disclosure also discloses a chip package structure that includes the aforementioned chip.

[0078] In some embodiments, this disclosure also discloses a circuit board that includes the above-described chip packaging structure. See also... Figure 7 The present invention provides an exemplary board, which, in addition to the chip 702, may also include other supporting components, including but not limited to: a storage device 704, an interface device 706, and a controller 708.

[0079] The storage device is connected to the chip within the chip package structure via a bus for storing data. The storage device may include multiple sets of storage cells 710. Each set of storage cells is connected to the chip via a bus. It is understood that each set of storage cells may be DDR SDRAM (Double Data Rate SDRAM).

[0080] DDR can double the speed of SDRAM without increasing the clock frequency. DDR allows data to be read on both the rising and falling edges of the clock pulse. DDR is twice as fast as standard SDRAM. In one embodiment, the storage device may include four groups of storage cells. Each group of storage cells may include multiple DDR4 chips. In one embodiment, the chip may internally include four 72-bit DDR4 controllers, of which 64 bits are used for data transmission and 8 bits are used for ECC verification. In one embodiment, each group of storage cells includes multiple Double Data Rate Synchronous Dynamic Random Access Memory (DRAM) units connected in parallel. DDR can transmit data twice per clock cycle. A controller for controlling DDR is provided in the chip for controlling data transmission and data storage in each storage cell.

[0081] The interface device is electrically connected to the chip within the chip package structure. The interface device is used to enable data transmission between the chip and an external device 712 (e.g., a server or computer). For example, in one embodiment, the interface device can be a standard PCIe interface. For instance, data to be processed is transferred from the server to the chip via a standard PCIe interface, realizing data transfer. In another embodiment, the interface device can also be other interfaces; this disclosure does not limit the specific form of these other interfaces, as long as the interface unit can perform the switching function. Furthermore, the calculation results of the chip are still transmitted back to the external device (e.g., the server) by the interface device.

[0082] The controller is electrically connected to the chip. The controller monitors the state of the chip. Specifically, the chip and the controller can be electrically connected via an SPI interface. The controller may include a microcontroller (MCU). The chip may include multiple processing chips, multiple processing cores, or multiple processing circuits, capable of driving multiple loads. Therefore, the chip can operate in different states, such as high load and low load. The controller can regulate the operating states of multiple processing chips, multiple processing cores, and / or multiple processing circuits within the chip.

[0083] In some embodiments, this disclosure also discloses an electronic device or apparatus that includes the aforementioned board.

[0084] Electronic devices or apparatuses include data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, mobile phones, dashcams, navigators, sensors, cameras, servers, cloud servers, cameras, camcorders, projectors, watches, headphones, mobile storage, wearable devices, vehicles, home appliances, and / or medical devices.

[0085] The means of transportation include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical equipment includes MRI scanners, ultrasound scanners and / or electrocardiographs.

[0086] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this disclosure. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0087] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0088] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, optical, acoustic, magnetic, or other forms.

[0089] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0090] Furthermore, the functional units in the various embodiments disclosed herein can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0091] If the integrated unit is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, when the technical solution disclosed herein can be embodied in the form of a software product, the computer software product is stored in a storage device and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage device includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0092] The embodiments of this disclosure have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this disclosure. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this disclosure. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this disclosure. Therefore, the content of this specification should not be construed as a limitation of this disclosure.

Claims

1. A system for real-time denoising of images, comprising: a motion alignment module configured to learn changes between a plurality of first image frames of a first image group and a noise description of the first image group, and to align information in the first image frames to obtain a second image group containing motion information between the plurality of first image frames to facilitate removal of noise from the aligned first image frames, wherein adjacent frames corresponding to the plurality of first image frames in the second image group are aligned, and the first image group is divided into a plurality of first image frame sets, each first image frame set including a temporally consecutive portion of the plurality of first image frames, at least one first image frame at a back of a first image frame set that temporally precedes, and at a front of a first image frame set that temporally succeeds; and a denoising module configured to obtain third images with reduced noise from the second image group.

2. The system of claim 1, wherein: the motion alignment module is configured to receive each first image frame set and corresponding noise description sequentially to facilitate alignment of the first image frames.

3. The system of claim 2, wherein, at least one same image frame is included in adjacent first image frame sets.

4. The system of any of claims 1-3, wherein, the motion alignment module is a neural network of a Unet structure.

5. The system of any of claims 1-3, wherein, the denoising module is of a same structure as the motion alignment module; or the denoising module is implemented by a fusion manner based on temporal attention.

6. The system of any of claims 1-3, wherein, the noise is Gaussian white noise and / or Poisson distribution noise.

7. The system of any of claims 1-3, wherein, the noise description includes a mean and a variance of the noise.

8. A method for real-time denoising of images, comprising: learning changes between a plurality of first image frames of a first image group and a noise description of the first image group, and aligning information in the first image frames to obtain a second image group containing motion information between the plurality of first image frames to facilitate removal of noise from the aligned first image frames, wherein adjacent frames corresponding to the plurality of first image frames in the second image group are aligned, and the first image group is divided into a plurality of first image frame sets, each first image frame set including a temporally consecutive portion of the plurality of first image frames, at least one first image frame at a back of a first image frame set that temporally precedes, and at a front of a first image frame set that temporally succeeds; and obtaining third images with reduced noise from the second image group.

9. An electronic device, comprising: one or more processors; and a memory having computer-executable instructions stored therein that, when executed by the one or more processors, cause the electronic device to perform the method of claim 8.

10. A computer-readable storage medium comprising computer-executable instructions that, when executed by one or more processors, perform the method of claim 8. ​

Images