An image denoising method, device and electronic equipment

Abstract

The application provides an image denoising method and device and electronic equipment. The method comprises: inputting adjacent image groups into a motion detection architecture to obtain a first motion mask corresponding to the adjacent image groups output by the motion detection architecture; wherein the adjacent image groups comprise a to-be-denoised image and a previous frame image of the to-be-denoised image, the motion detection architecture comprises a multi-layer motion detection network, the multi-layer motion detection network is used for multi-scale motion detection on the adjacent image groups, and information transmission exists between the multi-layer motion detection network; and the to-be-denoised image is denoised according to the first motion mask to obtain a second denoised image corresponding to the to-be-denoised image. By inputting the adjacent image groups into the motion detection architecture, motion detection on adjacent frames under multiple scales is realized, and information flow exists between the multiple scales, so that the accuracy of motion detection between adjacent frames can be improved. Therefore, the effect of denoising the image can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to an image noise reduction method, apparatus, and electronic device. Background Technology

[0002] In the imaging process of image acquisition devices, noise level has become one of the core factors affecting image quality. For example, the noise level of an image can significantly affect the human eye's perception and the ability to acquire information from the image; and excessively high noise levels increase the error rate of subsequent image structuring algorithms.

[0003] In existing technologies, image denoising methods include single-frame denoising and multi-frame denoising. Multi-frame denoising, by utilizing information from adjacent frames, offers better inter-frame smoothness. Depending on the specific algorithm used, these two methods can be further divided into traditional denoising and neural network denoising, with neural network methods often outperforming traditional methods. However, combining multi-frame denoising with neural network denoising results in lower accuracy for motion detection between multiple frames, leading to poorer denoising performance. Summary of the Invention

[0004] The purpose of this application is to provide an image denoising method, apparatus, and electronic device to solve the technical problem in the prior art where the accuracy of motion detection between multiple frames of images is low, resulting in poor denoising effect.

[0005] In a first aspect, embodiments of this application provide an image denoising method, comprising: inputting a group of adjacent images into a motion detection architecture to obtain a first motion mask output by the motion detection architecture corresponding to the group of adjacent images; wherein the group of adjacent images includes an image to be denoised and a previous frame image of the image to be denoised, the motion detection architecture includes a multi-layer motion detection network, the multi-layer motion detection network is used to perform multi-scale motion detection on the group of adjacent images, and there is information transmission between the multi-layer motion detection networks; denoising the image to be denoised according to the first motion mask to obtain a second denoised image corresponding to the image to be denoised.

[0006] In the above scheme, motion detection between adjacent frames is achieved by inputting adjacent image groups into the motion detection architecture. During this motion detection process, motion detection is performed on adjacent frames at multiple scales, and information flows between these scales, thereby improving the accuracy of motion detection between adjacent frames. Therefore, the effect of image denoising can be improved.

[0007] In an optional implementation, the step of inputting adjacent image groups into the motion detection architecture to obtain a first motion mask output by the motion detection architecture corresponding to the adjacent image groups includes: inputting the processed image group corresponding to the first layer of the motion detection network into the first layer of the motion detection network to obtain a first motion mask output by the first layer of the motion detection network; wherein, the processed image group corresponding to each layer of the motion detection network includes an image obtained by downsampling the image to be denoised and the previous frame image, and the resolution of the multi-layer motion detection network increases layer by layer; starting from the second layer of the motion detection network to the penultimate layer of the motion detection network, the processed image group is processed sequentially. The processed image group corresponding to the layer and the second motion mask corresponding to the previous layer motion detection network are input into the motion detection network of this layer to obtain the first motion mask output by the motion detection network of this layer; wherein, the second motion mask corresponding to the previous layer motion detection network is an image obtained by upsampling the first motion mask corresponding to the previous layer motion detection network, and the resolution of the second motion mask is equal to the resolution of the processed image group corresponding to the previous layer motion detection network; the adjacent image group and the second motion mask corresponding to the previous layer motion detection network are input into the last layer motion detection network to obtain the first motion mask output by the last layer motion detection network.

[0008] In the above scheme, the motion detection architecture can include a multi-layer motion detection network, which can perform motion detection at multiple scales on adjacent frames. The first motion mask output by the previous layer is used as the input to the next layer, thereby reducing the computational load of the first layer. Furthermore, using the first motion mask output by the previous layer as the input to the next layer also enables information flow between the multiple layers, thus improving the accuracy of motion detection between adjacent frames. Therefore, the noise reduction effect on the image can be improved while reducing computational load.

[0009] In an optional implementation, the step of denoising the image to be denoised according to the first motion mask to obtain a second denoised image corresponding to the image to be denoised includes: fusing the image to be denoised and a third denoised image corresponding to the previous frame image according to the first motion mask to obtain the second denoised image.

[0010] In the above scheme, temporal denoising can be performed on the image to be denoised based on the first motion mask output by the motion detection architecture. Specifically, by inputting adjacent image groups into the motion detection architecture, motion detection is performed on adjacent frames at multiple scales, and information flows between these scales, thereby improving the accuracy of motion detection between adjacent frames. Therefore, the denoising effect on the image can be improved.

[0011] In an optional implementation, the image denoising method further includes: training a first neural network architecture using the following process to obtain the motion detection architecture: inputting a first sample image into the first neural network architecture to obtain a first predicted motion mask output by each layer of the first neural network model in the neural network architecture; determining a first predicted image corresponding to the first neural network model of each layer based on the first predicted motion mask corresponding to each layer of the first neural network model, the first sample image, and the previous frame image of the first sample image; calculating a loss based on the first predicted image corresponding to each layer of the first neural network model and the actual denoised image, and adjusting the parameters in the first neural network architecture based on the loss until the motion detection architecture is obtained.

[0012] In the above scheme, motion detection at each scale can calculate the denoising result (i.e., the first predicted image) at that scale, and the loss is calculated using the first predicted image and the actual denoised image, so as to train the motion detection architecture based on the loss. In particular, since the denoising results at each scale play a supervisory role in the training process, the accuracy of the trained motion detection architecture can be improved.

[0013] In an optional implementation, the image denoising method further includes: inputting the image to be denoised into a spatial domain denoising network to obtain a first denoised image output by the spatial domain denoising network; the step of denoising the image to be denoised according to the first motion mask to obtain a second denoised image corresponding to the image to be denoised includes: fusing the first denoised image and a third denoised image corresponding to the previous frame image according to the first motion mask to obtain the second denoised image.

[0014] In the above scheme, the spatial denoising result based on the first motion mask and the denoising result of the previous frame image can be fused to obtain the second denoised image. Since the first motion mask is obtained by performing multi-scale motion detection on adjacent frames, a good temporal denoising foundation is provided for image denoising, thereby improving the denoising effect.

[0015] In an optional implementation, the image denoising method further includes: training a second neural network model and a second neural network architecture using the following process to obtain the spatial domain denoising network and the motion detection architecture: inputting a second sample image into the second neural network model to obtain a second predicted image output by the second neural network model; inputting the second sample image and the previous frame image of the second sample image into the second neural network architecture to obtain a second predicted motion mask output by each layer of the third neural network model in the second neural network architecture; determining a third predicted image corresponding to the third neural network model at each layer based on the second predicted motion mask, the second sample image, and the second predicted image; calculating a loss based on the third predicted image corresponding to each layer of the third neural network model and the actual denoised image, and adjusting the parameters in the second neural network model and the second neural network architecture based on the loss until the spatial domain denoising network and the motion detection architecture are obtained.

[0016] In the above scheme, motion detection at each scale can calculate the denoising result (i.e., the third predicted image) at that scale, and the loss is calculated using the third predicted image and the actual denoised image, so as to train the motion detection architecture based on the loss. In particular, since the denoising results at each scale play a supervisory role in the training process, the accuracy of the trained motion detection architecture can be improved.

[0017] Secondly, embodiments of this application provide an image denoising apparatus, comprising: a first input module, configured to input adjacent image groups into a motion detection architecture to obtain a first motion mask output by the motion detection architecture corresponding to the adjacent image groups; wherein, the adjacent image groups include an image to be denoised and a previous frame image of the image to be denoised, the motion detection architecture includes a multi-layer motion detection network, the multi-layer motion detection network is used to perform multi-scale motion detection on the adjacent image groups, and there is information transmission between the multi-layer motion detection networks; and a denoising module, configured to denoise the image to be denoised according to the first motion mask to obtain a second denoised image corresponding to the image to be denoised.

[0018] In the above scheme, motion detection between adjacent frames is achieved by inputting adjacent image groups into the motion detection architecture. During this motion detection process, motion detection is performed on adjacent frames at multiple scales, and information flows between these scales, thereby improving the accuracy of motion detection between adjacent frames. Therefore, the effect of image denoising can be improved.

[0019] In an optional implementation, the first input module is specifically used for: inputting the processed image group corresponding to the first layer motion detection network into the first layer motion detection network to obtain the first motion mask output by the first layer motion detection network; wherein, the processed image group corresponding to each layer motion detection network includes an image obtained by downsampling the image to be denoised and the previous frame image, and the resolution of the multi-layer motion detection network increases layer by layer; starting from the second layer motion detection network to the penultimate layer motion detection network, sequentially inputting the processed image group corresponding to the layer and the second motion mask corresponding to the previous layer motion detection network into the layer motion detection network to obtain the first motion mask output by the layer motion detection network; wherein, the second motion mask corresponding to the previous layer motion detection network is an image obtained by upsampling the first motion mask corresponding to the previous layer motion detection network, and the resolution of the second motion mask is equal to the resolution of the processed image group corresponding to the previous layer motion detection network; inputting the adjacent image group and the second motion mask corresponding to the previous layer motion detection network into the last layer motion detection network to obtain the first motion mask output by the last layer motion detection network.

[0020] In the above scheme, the motion detection architecture can include a multi-layer motion detection network, which can perform motion detection at multiple scales on adjacent frames. The first motion mask output by the previous layer is used as the input to the next layer, thereby reducing the computational load of the first layer. Furthermore, using the first motion mask output by the previous layer as the input to the next layer also enables information flow between the multiple layers, thus improving the accuracy of motion detection between adjacent frames. Therefore, the noise reduction effect on the image can be improved while reducing computational load.

[0021] In an optional implementation, the noise reduction module is specifically used to: fuse the image to be denoised and the third noise-reduced image corresponding to the previous frame image according to the first motion mask to obtain the second noise-reduced image.

[0022] In the above scheme, temporal denoising can be performed on the image to be denoised based on the first motion mask output by the motion detection architecture. Specifically, by inputting adjacent image groups into the motion detection architecture, motion detection is performed on adjacent frames at multiple scales, and information flows between these scales, thereby improving the accuracy of motion detection between adjacent frames. Therefore, the denoising effect on the image can be improved.

[0023] In an optional embodiment, the image denoising device further includes: a first training module, configured to train a first neural network architecture using the following process to obtain the motion detection architecture: inputting a first sample image into the first neural network architecture to obtain a first predicted motion mask output by each layer of the first neural network model in the neural network architecture; determining a first predicted image corresponding to the first neural network model of each layer based on the first predicted motion mask corresponding to each layer of the first neural network model, the first sample image, and the previous frame image of the first sample image; calculating a loss based on the first predicted image corresponding to each layer of the first neural network model and the actual denoised image, and adjusting the parameters in the first neural network architecture based on the loss until the motion detection architecture is obtained.

[0024] In the above scheme, motion detection at each scale can calculate the denoising result (i.e., the first predicted image) at that scale, and the loss is calculated using the first predicted image and the actual denoised image, so as to train the motion detection architecture based on the loss. In particular, since the denoising results at each scale play a supervisory role in the training process, the accuracy of the trained motion detection architecture can be improved.

[0025] In an optional embodiment, the image denoising device further includes: a second input module, used to input the image to be denoised into a spatial domain denoising network to obtain a first denoised image output by the spatial domain denoising network; the denoising module is specifically used to: fuse the first denoised image and a third denoised image corresponding to the previous frame image according to the first motion mask to obtain the second denoised image.

[0026] In the above scheme, the spatial denoising result based on the first motion mask and the denoising result of the previous frame image can be fused to obtain the second denoised image. Since the first motion mask is obtained by performing multi-scale motion detection on adjacent frames, a good temporal denoising foundation is provided for image denoising, thereby improving the denoising effect.

[0027] In an optional embodiment, the image denoising device further includes: a second training module, configured to train a second neural network model and a second neural network architecture using the following process to obtain the spatial domain denoising network and the motion detection architecture: inputting a second sample image into the second neural network model to obtain a second predicted image output by the second neural network model; inputting the second sample image and the previous frame image of the second sample image into the second neural network architecture to obtain a second predicted motion mask output by each layer of the third neural network model in the second neural network architecture; determining a third predicted image corresponding to the third neural network model at each layer based on the second predicted motion mask, the second sample image, and the second predicted image; calculating a loss based on the third predicted image corresponding to each layer of the third neural network model and the actual denoised image, and adjusting the parameters in the second neural network model and the second neural network architecture based on the loss until the spatial domain denoising network and the motion detection architecture are obtained.

[0028] In the above scheme, motion detection at each scale can calculate the denoising result (i.e., the third predicted image) at that scale, and the loss is calculated using the third predicted image and the actual denoised image, so as to train the motion detection architecture based on the loss. In particular, since the denoising results at each scale play a supervisory role in the training process, the accuracy of the trained motion detection architecture can be improved.

[0029] Thirdly, embodiments of this application provide a computer program product, including computer program instructions, which, when read and executed by a processor, perform the image noise reduction method as described in the first aspect.

[0030] Fourthly, embodiments of this application provide an electronic device, including: a processor, a memory, and a bus; the processor and the memory communicate with each other via the bus; the memory stores computer program instructions executable by the processor, and the processor can execute the image noise reduction method as described in the first aspect by calling the computer program instructions.

[0031] Fifthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions that, when executed by a computer, cause the computer to perform the image noise reduction method as described in the first aspect.

[0032] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, embodiments of this application are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 A flowchart of an image noise reduction method provided in an embodiment of this application;

[0035] Figure 2 A schematic diagram of an image denoising model provided in an embodiment of this application;

[0036] Figure 3 A schematic diagram of another image denoising model provided in an embodiment of this application;

[0037] Figure 4 A structural block diagram of an image noise reduction device provided in an embodiment of this application;

[0038] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0039] In recent years, significant progress has been made in research on technologies based on artificial intelligence, such as computer vision, deep learning, machine learning, image processing, and image recognition. Artificial intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies, and application systems to simulate and extend human intelligence. AI is a comprehensive discipline involving numerous technologies, including chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, and neural networks. Computer vision, as an important branch of AI, specifically enables machines to recognize the world. Computer vision technologies typically include face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, object detection, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, behavior recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, and robot navigation and localization. With the research and advancement of artificial intelligence technology, this technology has been applied in numerous fields, such as security, urban management, traffic management, building management, park management, facial recognition access control, facial recognition attendance, logistics management, warehouse management, robotics, intelligent marketing, computational photography, mobile imaging, cloud services, smart homes, wearable devices, autonomous driving, autonomous driving, smart healthcare, facial payment, facial unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile internet, live streaming, beautification, makeup, medical aesthetics, and intelligent temperature measurement.

[0040] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0041] Please refer to Figure 1 , Figure 1 This application provides a flowchart of an image denoising method, which may include the following steps:

[0042] Step S101: Input the adjacent image group into the motion detection architecture to obtain the first motion mask output by the motion detection architecture corresponding to the adjacent image group.

[0043] Step S102: Denoise the image to be denoised according to the first motion mask to obtain the second denoised image corresponding to the image to be denoised.

[0044] Specifically, in step S101 above, the adjacent image group includes the image to be denoised and the previous frame image of the image to be denoised. During the image denoising process, different images that need denoising can be continuously acquired; the image to be denoised refers to the current image that needs denoising; the previous frame image of the image to be denoised refers to the image preceding the image to be denoised in the continuously acquired images.

[0045] It should be noted that the embodiments of this application do not specifically limit the implementation method for acquiring adjacent image groups, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, adjacent image groups sent by other devices can be received; or, pre-stored adjacent image groups can be read from local storage or the cloud; or, the previous frame image of the continuous image to be denoised and the image to be denoised can be acquired, etc.

[0046] It is understandable that when the image to be denoised is the first frame, that is, when there is no previous frame, the image to be denoised can be itself determined as the previous frame. In other words, an adjacent image group includes two images to be denoised.

[0047] The motion detection architecture includes a multi-layer motion detection network for extracting a first motion mask corresponding to adjacent image groups. Specifically, the multi-layer motion detection network performs multi-scale motion detection on adjacent image groups; that is, it extracts a first motion mask between the multi-scale image to be denoised and the previous frame of the image to be denoised.

[0048] It is understandable that the aforementioned different scales can refer to different resolutions. In other words, each layer of the motion detection network is used to extract a first motion mask between the image to be denoised at a certain resolution and the previous frame of that image at that resolution. Since images of different resolutions have a relatively small impact on the motion mask, the differences between the multiple first motion masks extracted by different motion detection networks are small.

[0049] For example, one layer of motion detection network is used to extract the first motion mask between the image to be denoised and the previous frame image of the image to be denoised, and another layer of motion detection network is used to extract the first motion mask between an image with a resolution of 1 / 2 that of the image to be denoised and an image with a resolution of 1 / 2 that of the previous frame image of the image to be denoised, and so on.

[0050] Information is transmitted between the aforementioned multi-layer motion detection networks. It should be noted that the embodiments of this application do not specifically limit the implementation of the information transmitted between the multi-layer motion detection networks, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, the information transmitted between the multi-layer motion detection networks may include: motion masks, images to be denoised at different resolutions, and the previous frame image of the images to be denoised at different resolutions, etc.

[0051] It is understandable that the multi-layer motion detection networks included in the above motion detection architecture have the same structure, but different parameters; their specific parameters can be adjusted appropriately according to actual needs and training conditions.

[0052] It should be noted that the embodiments of this application do not impose specific limitations on the specific implementation of the above-described motion detection network structure, and those skilled in the art can make adjustments based on actual conditions and existing technology.

[0053] In step S102 above, based on the first motion mask obtained in step S101 above, the image to be denoised can be denoised, thereby obtaining a second denoised image corresponding to the image to be denoised.

[0054] It should be noted that the embodiments of this application do not specifically limit the specific implementation of determining the second denoised image based on the first motion mask, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, the second denoised image can be determined based on the first motion mask and adjacent image groups; or, the second denoised image can be determined based on the first motion mask and the result obtained after spatial domain denoising of the image to be denoised, etc.

[0055] In the above scheme, motion detection between adjacent frames is achieved by inputting adjacent image groups into the motion detection architecture. During this motion detection process, motion detection is performed on adjacent frames at multiple scales, and information flows between these scales, thereby improving the accuracy of motion detection between adjacent frames. Therefore, the effect of image denoising can be improved.

[0056] Furthermore, based on the above embodiments, a specific implementation of obtaining the first motion mask will be described in detail below. In this implementation, step S101 may specifically include the following steps:

[0057] Step 1) Input the processed image group corresponding to the first layer motion detection network into the first layer motion detection network to obtain the first motion mask output by the first layer motion detection network.

[0058] Step 2): Starting from the second layer of the motion detection network and going to the penultimate layer of the motion detection network, the processing image group corresponding to this layer and the second motion mask corresponding to the previous layer of the motion detection network are sequentially input into this layer of the motion detection network to obtain the first motion mask output by this layer of the motion detection network.

[0059] Step 3) Input the adjacent image groups and the second motion mask corresponding to the previous motion detection network into the last motion detection network to obtain the first motion mask output by the last motion detection network.

[0060] Specifically, in the embodiments of this application, the processing image group corresponding to each layer of motion detection network includes the image to be denoised and the image obtained by downsampling the previous frame image, and the resolution corresponding to the multi-layer motion detection network increases layer by layer.

[0061] In other words, the resolution of the images in the image processing group corresponding to the first layer of the motion detection network is also the lowest; the resolution of the images in the image processing groups corresponding to the second to penultimate layers of the motion detection network increases layer by layer; the image processing group corresponding to the last layer of the motion detection network is the adjacent image group itself.

[0062] Taking a motion detection architecture comprising a four-layer motion detection network as an example: As one implementation method, adjacent image groups can be directly downsampled to obtain the processed image group corresponding to each layer of the motion detection network. For instance, downsampling adjacent image groups at a rate of 1 / 2 yields the processed image group corresponding to the second layer of the motion detection network; downsampling adjacent image groups at a rate of 1 / 4 yields the processed image group corresponding to the third layer of the motion detection network; and downsampling adjacent image groups at a rate of 1 / 8 yields the processed image group corresponding to the fourth layer of the motion detection network.

[0063] As another implementation, the processing image group corresponding to the next layer of motion detection network can also be downsampled to obtain the processing image group corresponding to that layer of motion detection network. For example, adjacent image groups can be downsampled at a downsampling rate of 1 / 2 to obtain the processing image group corresponding to the second layer of motion detection network; the processing image group corresponding to the second layer of motion detection network can be downsampled at a downsampling rate of 1 / 2 to obtain the processing image group corresponding to the third layer of motion detection network; the processing image group corresponding to the third layer of motion detection network can be downsampled at a downsampling rate of 1 / 2 to obtain the processing image group corresponding to the fourth layer of motion detection network.

[0064] It should be noted that the specific downsampling rate described above is merely an example provided in this application embodiment, and those skilled in the art can make appropriate adjustments to the downsampling rate according to actual circumstances. For example, the downsampling rate can be 1 / 2, 1 / 4, 1 / 16, etc.

[0065] Furthermore, the second motion mask corresponding to the previous motion detection network is an image obtained by upsampling the first motion mask corresponding to the previous motion detection network, and the resolution of the second motion mask is equal to the resolution of the processed image group corresponding to the previous motion detection network.

[0066] Taking a motion detection architecture consisting of four layers of motion detection networks as an example: the first motion mask corresponding to the first layer of motion detection network is upsampled at a rate of 1 / 2 to obtain the second motion mask corresponding to the first layer of motion detection network; the first motion mask corresponding to the second layer of motion detection network is upsampled at a rate of 1 / 2 to obtain the second motion mask corresponding to the second layer of motion detection network; the first motion mask corresponding to the third layer of motion detection network is upsampled at a rate of 1 / 2 to obtain the second motion mask corresponding to the third layer of motion detection network.

[0067] In step 1) above, the processed image group corresponding to the first layer motion detection network can be input into the first layer motion detection network in the motion detection architecture to obtain the first motion mask output by the first layer motion detection network. Then, the first motion mask corresponding to the first layer motion detection network can be upsampled to obtain the second motion mask corresponding to the first layer motion detection network.

[0068] Step 2) involves inputting the processed image group corresponding to the second-layer motion detection network and the second motion mask corresponding to the first-layer motion detection network into the second-layer motion detection network of the motion detection architecture, thereby obtaining the first motion mask output by the second-layer motion detection network. Then, the first motion mask corresponding to the second-layer motion detection network can be upsampled to obtain the second motion mask corresponding to the second-layer motion detection network.

[0069] The processed image group corresponding to the third-layer motion detection network and the second motion mask corresponding to the second-layer motion detection network can be input into the third-layer motion detection network in the motion detection architecture to obtain the first motion mask output by the third-layer motion detection network. Then, the first motion mask corresponding to the third-layer motion detection network can be upsampled to obtain the second motion mask corresponding to the third-layer motion detection network.

[0070] Repeat the above steps until the processed image group corresponding to the penultimate motion detection network and the second motion mask corresponding to the penultimate motion detection network are input into the third motion detection network in the motion detection architecture, thereby obtaining the first motion mask output by the penultimate motion detection network. Then, the first motion mask corresponding to the penultimate motion detection network can be upsampled to obtain the second motion mask corresponding to the penultimate motion detection network.

[0071] Step 3) Input the adjacent image groups and the second motion mask corresponding to the penultimate motion detection network into the last motion detection network to obtain the first motion mask output by the last motion detection network.

[0072] Taking a motion detection architecture comprising a four-layer motion detection network as an example: Inputting the processed image group corresponding to the first-layer motion detection network into the first-layer motion detection network of the motion detection architecture yields the first motion mask output by the first-layer motion detection network; upsampling the first motion mask corresponding to the first-layer motion detection network yields the second motion mask corresponding to the first-layer motion detection network; inputting the processed image group corresponding to the second-layer motion detection network and the second motion mask corresponding to the first-layer motion detection network into the second-layer motion detection network of the motion detection architecture yields the first motion mask output by the second-layer motion detection network; upsampling the second motion mask corresponding to the second-layer motion detection network... The first motion mask is upsampled to obtain the second motion mask corresponding to the second layer motion detection network; the processed image group corresponding to the third layer motion detection network and the second motion mask corresponding to the second layer motion detection network are input into the third layer motion detection network in the motion detection architecture to obtain the first motion mask output by the third layer motion detection network; the first motion mask corresponding to the third layer motion detection network is upsampled to obtain the second motion mask corresponding to the third layer motion detection network; the adjacent image group and the second motion mask corresponding to the third layer motion detection network are input into the fourth layer motion detection network to obtain the first motion mask output by the fourth layer motion detection network.

[0073] In the above scheme, the motion detection architecture can include a multi-layer motion detection network, which can perform motion detection at multiple scales on adjacent frames. The first motion mask output by the previous layer is used as the input to the next layer, thereby reducing the computational load of the first layer. Furthermore, using the first motion mask output by the previous layer as the input to the next layer also enables information flow between the multiple layers, thus improving the accuracy of motion detection between adjacent frames. Therefore, the noise reduction effect on the image can be improved while reducing computational load.

[0074] Furthermore, based on the above embodiments, step S102 may specifically include the following steps:

[0075] The second denoised image is obtained by fusing the image to be denoised and the third denoised image corresponding to the previous frame image using the first motion mask.

[0076] Specifically, the third denoised image refers to the denoised image obtained after applying the image denoising method provided in this application embodiment to the previous frame of the image to be denoised. The second denoised image refers to the denoised image obtained after applying the image denoising method provided in this application embodiment to the image to be denoised.

[0077] In one implementation, the image to be denoised can be multiplied by the difference between a preset boundary value and a first motion mask to obtain a first fused image, and the third denoised image can be multiplied by the first motion mask to obtain a second fused image; then, the first fused image and the second fused image can be added together to obtain the aforementioned second denoised image.

[0078] It should be noted that the embodiments of this application do not limit the specific implementation of the above-mentioned preset boundary values, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, the preset boundary values ​​can be 1, 0.8, etc.

[0079] In the above scheme, temporal denoising can be performed on the image to be denoised based on the first motion mask output by the motion detection architecture. Specifically, by inputting adjacent image groups into the motion detection architecture, motion detection is performed on adjacent frames at multiple scales, and information flows between these scales, thereby improving the accuracy of motion detection between adjacent frames. Therefore, the denoising effect on the image can be improved.

[0080] Furthermore, based on the above embodiments, the image denoising method provided in this application can also train the first neural network architecture to obtain the above motion detection architecture, wherein the training process may include the following steps:

[0081] Step 1) Input the first sample image into the first neural network architecture to obtain the first predicted motion mask output by each layer of the first neural network model in the neural network architecture.

[0082] Step 2) Determine the first predicted image corresponding to the first neural network model of each layer based on the first predicted motion mask, the first sample image, and the previous frame image of the first sample image.

[0083] Step 3) Calculate the loss based on the first predicted image and the actual denoised image corresponding to each layer of the first neural network model, and adjust the parameters in the first neural network architecture based on the loss until the motion detection architecture is obtained.

[0084] Specifically, the difference between the above training process and the application process is that in the application process, it is only necessary to obtain the first motion mask output by each layer of the motion detection network, without further determining the denoising result corresponding to each layer of the motion detection network; while in the training process, in addition to obtaining the first predicted motion mask corresponding to each layer of the first neural network model, it is also necessary to further determine the denoising result corresponding to each layer of the first neural network model based on the first predicted motion mask, that is, the first predicted image in step 2) above.

[0085] It is understandable that, as one implementation method, the aforementioned first sample image can be obtained by adding noise to the actual denoised image, thereby eliminating the need for motion segmentation and making it easier to obtain sample data.

[0086] In the above scheme, motion detection at each scale can calculate the denoising result (i.e., the first predicted image) at that scale, and the loss is calculated using the first predicted image and the actual denoised image, so as to train the motion detection architecture based on the loss. In particular, since the denoising results at each scale play a supervisory role in the training process, the accuracy of the trained motion detection architecture can be improved.

[0087] Please refer to Figure 2 , Figure 2 This is a schematic diagram of an image denoising model provided in an embodiment of this application. The image denoising model includes a motion detection architecture, and the motion detection architecture includes a four-layer motion detection network. Based on the above image denoising model, the image denoising method provided in this embodiment of the application may include the following steps:

[0088] The first step is to downsample adjacent image groups to obtain the processed image groups corresponding to the first, second, and third motion detection networks, respectively. The resolution of the processed image group corresponding to the third motion detection network is 1 / 2 of that of the adjacent image groups, the resolution of the processed image group corresponding to the second motion detection network is 1 / 4 of that of the adjacent image groups, and the resolution of the processed image group corresponding to the first motion detection network is 1 / 8 of that of the adjacent image groups.

[0089] The second step involves inputting the processed image group corresponding to the first layer motion detection network into the first layer motion detection network in the motion detection architecture to obtain the first motion mask output by the first layer motion detection network. Then, the first motion mask corresponding to the first layer motion detection network is upsampled to obtain the second motion mask corresponding to the first layer motion detection network.

[0090] The third step involves inputting the processed image group corresponding to the second layer motion detection network and the second motion mask corresponding to the first layer motion detection network into the second layer motion detection network in the motion detection architecture to obtain the first motion mask output by the second layer motion detection network. Then, the first motion mask corresponding to the second layer motion detection network is upsampled to obtain the second motion mask corresponding to the second layer motion detection network.

[0091] The fourth step involves inputting the processed image group corresponding to the third layer motion detection network and the second motion mask corresponding to the second layer motion detection network into the third layer motion detection network in the motion detection architecture to obtain the first motion mask output by the third layer motion detection network. The first motion mask corresponding to the third layer motion detection network is then upsampled to obtain the second motion mask corresponding to the third layer motion detection network.

[0092] The fifth step involves inputting adjacent image groups and the second motion mask corresponding to the third layer motion detection network into the fourth layer motion detection network to obtain the first motion mask output by the fourth layer motion detection network.

[0093] The sixth step is to fuse the image to be denoised and the third denoised image corresponding to the previous frame image according to the first motion mask to obtain the second denoised image.

[0094] Furthermore, based on the above embodiments, the image noise reduction method provided in this application may further include the following steps:

[0095] The image to be denoised is input into the spatial domain denoising network to obtain the first denoised image output by the spatial domain denoising network.

[0096] It should be noted that the embodiments of this application do not impose specific limitations on the specific implementation of the above-described spatial noise reduction network, and those skilled in the art can make adjustments based on actual conditions and existing technology. For example, the second noise reduction model can be an Encoder-Decoder structure.

[0097] Accordingly, in this embodiment, step S10 may specifically include the following steps:

[0098] The first denoised image and the third denoised image corresponding to the previous frame are fused using the first motion mask to obtain the second denoised image.

[0099] Specifically, as one implementation, the first denoised image can be multiplied by the first motion mask to obtain the first fused image, and the third denoised image can be multiplied by the difference between the preset boundary value and the first motion mask to obtain the second fused image; then, the first fused image and the second fused image can be added together to obtain the aforementioned second denoised image.

[0100] It should be noted that the embodiments of this application do not limit the specific implementation of the above-mentioned preset boundary values, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, the preset boundary values ​​can be 1, 0.8, etc.

[0101] In the above scheme, the spatial denoising result based on the first motion mask and the denoising result of the previous frame image can be fused to obtain the second denoised image. Since the first motion mask is obtained by performing multi-scale motion detection on adjacent frames, a good temporal denoising foundation is provided for image denoising, thereby improving the denoising effect.

[0102] Furthermore, based on the above embodiments, the image denoising method provided in this application can also train the second neural network model and the second neural network architecture to obtain a spatial domain denoising network and a motion detection architecture. The training process may include the following steps:

[0103] Step 1) Input the second sample image into the second neural network model to obtain the second predicted image output by the second neural network model.

[0104] Step 2) Input the second sample image and the previous frame image of the second sample image into the second neural network architecture to obtain the second predicted motion mask output by the third neural network model of each layer in the second neural network architecture.

[0105] Step 3) Determine the third prediction image corresponding to the third neural network model of each layer based on the second predicted motion mask, the second sample image, and the second prediction image.

[0106] Step 4) Calculate the loss based on the third predicted image and the actual denoised image corresponding to each layer of the third neural network model, and adjust the parameters in the second neural network model and the second neural network architecture based on the loss until the spatial domain denoising network and motion detection architecture are obtained.

[0107] Specifically, the difference between the above training process and the application process is that in the application process, it is only necessary to obtain the first motion mask output by each layer of the motion detection network, without further determining the denoising result corresponding to each layer of the motion detection network; while in the training process, in addition to obtaining the second predicted motion mask corresponding to each layer of the second neural network model, it is also necessary to further determine the denoising result corresponding to each layer of the second neural network model based on the above second predicted motion mask, that is, the third predicted image in step 3) above.

[0108] It is understandable that, as one implementation method, the aforementioned second sample image can be achieved by adding noise to the actual denoised image, thereby eliminating the need for motion segmentation and making it easier to obtain sample data.

[0109] In the above scheme, motion detection at each scale can calculate the denoising result (i.e., the third predicted image) at that scale, and the loss is calculated using the third predicted image and the actual denoised image, so as to train the motion detection architecture based on the loss. In particular, since the denoising results at each scale play a supervisory role in the training process, the accuracy of the trained motion detection architecture can be improved.

[0110] Please refer to Figure 3 , Figure 3 This is a schematic diagram of another image denoising model provided in an embodiment of this application. The image denoising model includes a spatial domain denoising network and a motion detection architecture, and the motion detection architecture includes a four-layer motion detection network. Based on the above image denoising model, the image denoising method provided in this embodiment of the application may include the following steps:

[0111] The first step is to input the image to be denoised into the network to obtain the first denoised image output by the spatial domain denoising network.

[0112] The second step involves downsampling adjacent image groups to obtain the processed image groups corresponding to the first, second, and third motion detection networks, respectively. The resolution of the processed image group corresponding to the third motion detection network is 1 / 2 of that of the adjacent image groups, the resolution of the processed image group corresponding to the second motion detection network is 1 / 4 of that of the adjacent image groups, and the resolution of the processed image group corresponding to the first motion detection network is 1 / 8 of that of the adjacent image groups.

[0113] The third step is to input the processed image group corresponding to the first layer motion detection network into the first layer motion detection network in the motion detection architecture to obtain the first motion mask output by the first layer motion detection network, and to upsample the first motion mask corresponding to the first layer motion detection network to obtain the second motion mask corresponding to the first layer motion detection network.

[0114] The fourth step involves inputting the processed image group corresponding to the second layer motion detection network and the second motion mask corresponding to the first layer motion detection network into the second layer motion detection network in the motion detection architecture to obtain the first motion mask output by the second layer motion detection network. The first motion mask corresponding to the second layer motion detection network is then upsampled to obtain the second motion mask corresponding to the second layer motion detection network.

[0115] The fifth step involves inputting the processed image group corresponding to the third layer motion detection network and the second motion mask corresponding to the second layer motion detection network into the third layer motion detection network in the motion detection architecture to obtain the first motion mask output by the third layer motion detection network. The first motion mask corresponding to the third layer motion detection network is then upsampled to obtain the second motion mask corresponding to the third layer motion detection network.

[0116] The sixth step involves inputting the adjacent image groups and the second motion mask corresponding to the third layer motion detection network into the fourth layer motion detection network to obtain the first motion mask output by the fourth layer motion detection network.

[0117] Step 7: Based on the first motion mask, the first denoised image and the third denoised image corresponding to the previous frame are fused to obtain the second denoised image.

[0118] Please refer to Figure 4 , Figure 4 This application provides a structural block diagram of an image denoising device. The image denoising device may include: a first input module 401, used to input adjacent image groups into a motion detection architecture to obtain a first motion mask output by the motion detection architecture corresponding to the adjacent image groups; wherein the adjacent image groups include an image to be denoised and a previous frame image of the image to be denoised; the motion detection architecture includes a multi-layer motion detection network, the multi-layer motion detection network is used to perform multi-scale motion detection on the adjacent image groups, and information is transmitted between the multi-layer motion detection networks; and a denoising module 402, used to denoise the image to be denoised according to the first motion mask to obtain a second denoised image corresponding to the image to be denoised.

[0119] In the above scheme, motion detection between adjacent frames is achieved by inputting adjacent image groups into the motion detection architecture. During this motion detection process, motion detection is performed on adjacent frames at multiple scales, and information flows between these scales, thereby improving the accuracy of motion detection between adjacent frames. Therefore, the effect of image denoising can be improved.

[0120] Furthermore, based on the above embodiments, the first input module 401 is specifically used for: inputting the processed image group corresponding to the first layer motion detection network into the first layer motion detection network to obtain the first motion mask output by the first layer motion detection network; wherein, the processed image group corresponding to each layer motion detection network includes an image obtained by downsampling the image to be denoised and the previous frame image, and the resolution of the multi-layer motion detection network increases layer by layer; starting from the second layer motion detection network to the penultimate layer motion detection network, sequentially inputting the processed image group corresponding to the layer and the second motion mask corresponding to the previous layer motion detection network into the layer motion detection network to obtain the first motion mask output by the layer motion detection network; wherein, the second motion mask corresponding to the previous layer motion detection network is an image obtained by upsampling the first motion mask corresponding to the previous layer motion detection network, and the resolution of the second motion mask is equal to the resolution of the processed image group corresponding to the previous layer motion detection network; inputting the adjacent image group and the second motion mask corresponding to the previous layer motion detection network into the last layer motion detection network to obtain the first motion mask output by the last layer motion detection network.

[0121] In the above scheme, the motion detection architecture can include a multi-layer motion detection network, which can perform motion detection at multiple scales on adjacent frames. The first motion mask output by the previous layer is used as the input to the next layer, thereby reducing the computational load of the first layer. Furthermore, using the first motion mask output by the previous layer as the input to the next layer also enables information flow between the multiple layers, thus improving the accuracy of motion detection between adjacent frames. Therefore, the noise reduction effect on the image can be improved while reducing computational load.

[0122] Furthermore, based on the above embodiments, the noise reduction module 402 is specifically used to: fuse the image to be denoised and the third noise-reduced image corresponding to the previous frame image according to the first motion mask to obtain the second noise-reduced image.

[0123] In the above scheme, temporal denoising can be performed on the image to be denoised based on the first motion mask output by the motion detection architecture. Specifically, by inputting adjacent image groups into the motion detection architecture, motion detection is performed on adjacent frames at multiple scales, and information flows between these scales, thereby improving the accuracy of motion detection between adjacent frames. Therefore, the denoising effect on the image can be improved.

[0124] Furthermore, based on the above embodiments, the image denoising device 400 further includes: a first training module, used to train a first neural network architecture using the following process to obtain the motion detection architecture: inputting a first sample image into the first neural network architecture to obtain a first predicted motion mask output by each layer of the first neural network model in the neural network architecture; determining a first predicted image corresponding to the first neural network model of each layer based on the first predicted motion mask corresponding to each layer of the first neural network model, the first sample image, and the previous frame image of the first sample image; calculating a loss based on the first predicted image corresponding to each layer of the first neural network model and the actual denoised image, and adjusting the parameters in the first neural network architecture based on the loss until the motion detection architecture is obtained.

[0125] In the above scheme, motion detection at each scale can calculate the denoising result (i.e., the first predicted image) at that scale, and the loss is calculated using the first predicted image and the actual denoised image, so as to train the motion detection architecture based on the loss. In particular, since the denoising results at each scale play a supervisory role in the training process, the accuracy of the trained motion detection architecture can be improved.

[0126] Furthermore, based on the above embodiments, the image denoising device 400 further includes: a second input module, used to input the image to be denoised into a spatial domain denoising network to obtain a first denoised image output by the spatial domain denoising network; the denoising module 402 is specifically used to: fuse the first denoised image and the third denoised image corresponding to the previous frame image according to the first motion mask to obtain the second denoised image.

[0127] In the above scheme, the spatial denoising result based on the first motion mask and the denoising result of the previous frame image can be fused to obtain the second denoised image. Since the first motion mask is obtained by performing multi-scale motion detection on adjacent frames, a good temporal denoising foundation is provided for image denoising, thereby improving the denoising effect.

[0128] Furthermore, based on the above embodiments, the image denoising device 400 further includes: a second training module, used to train the second neural network model and the second neural network architecture using the following process to obtain the spatial domain denoising network and the motion detection architecture: inputting a second sample image into the second neural network model to obtain a second predicted image output by the second neural network model; inputting the second sample image and the previous frame image of the second sample image into the second neural network architecture to obtain a second predicted motion mask output by each layer of the third neural network model in the second neural network architecture; determining a third predicted image corresponding to the third neural network model of each layer based on the second predicted motion mask, the second sample image, and the second predicted image; calculating a loss based on the third predicted image corresponding to each layer of the third neural network model and the actual denoised image, and adjusting the parameters in the second neural network model and the second neural network architecture based on the loss until the spatial domain denoising network and the motion detection architecture are obtained.

[0129] In the above scheme, motion detection at each scale can calculate the denoising result (i.e., the third predicted image) at that scale, and the loss is calculated using the third predicted image and the actual denoised image, so as to train the motion detection architecture based on the loss. In particular, since the denoising results at each scale play a supervisory role in the training process, the accuracy of the trained motion detection architecture can be improved.

[0130] Please refer to Figure 5 , Figure 5 This application provides a structural block diagram of an electronic device 500, which includes at least one processor 501, at least one communication interface 502, at least one memory 503, and at least one communication bus 504. The communication bus 504 enables direct communication between these components, the communication interface 502 facilitates signaling or data communication with other node devices, and the memory 503 stores machine-readable instructions executable by the processor 501. When the electronic device 500 is running, the processor 501 communicates with the memory 503 via the communication bus 504, and the image denoising method described above is executed when the machine-readable instructions are invoked by the processor 501.

[0131] For example, the processor 501 in this embodiment of the application can read a computer program from the memory 503 via the communication bus 504 and execute the computer program to implement the following method: Step S101: Input adjacent image groups into the motion detection architecture to obtain a first motion mask output by the motion detection architecture corresponding to the adjacent image groups. Step S102: Denoise the image to be denoised according to the first motion mask to obtain a second denoised image corresponding to the image to be denoised.

[0132] The processor 501 comprises one or more, and can be an integrated circuit chip with signal processing capabilities. The processor 501 can be a general-purpose processor, including a Central Processing Unit (CPU), a Microcontroller Unit (MCU), a Network Processor (NP), or other conventional processors; it can also be a special-purpose processor, including a Neural-network Processing Unit (NPU), a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. Furthermore, when there are multiple processors 501, some can be general-purpose processors, and others can be special-purpose processors.

[0133] The memory 503 includes one or more, which may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0134] Understandable. Figure 5 The structure shown is for illustrative purposes only; the electronic device 500 may also include components that are more advanced than those shown. Figure 5 The more or fewer components shown, or having the same Figure 5 The different configurations shown. Figure 5The components shown can be implemented using hardware, software, or a combination thereof. In the embodiments of this application, electronic device 500 can be, but is not limited to, physical devices such as desktop computers, laptops, smartphones, smart wearable devices, and in-vehicle devices, or virtual devices such as virtual machines. Furthermore, electronic device 500 is not necessarily a single device; it can be a combination of multiple devices, such as a server cluster, etc.

[0135] This application also provides a computer program product, including a computer program stored on a computer-readable storage medium. The computer program includes computer program instructions. When the computer program instructions are executed by a computer, the computer can perform the steps of the image denoising method described in the above embodiments, such as: inputting adjacent image groups into a motion detection architecture to obtain a first motion mask output by the motion detection architecture corresponding to the adjacent image groups; wherein, the adjacent image groups include an image to be denoised and a previous frame image of the image to be denoised, the motion detection architecture includes a multi-layer motion detection network, the multi-layer motion detection network is used to perform multi-scale motion detection on the adjacent image groups, and there is information transmission between the multi-layer motion detection networks; denoising the image to be denoised according to the first motion mask to obtain a second denoised image corresponding to the image to be denoised.

[0136] This application also provides a computer-readable storage medium that stores computer program instructions. When the computer program instructions are executed by a computer, the computer performs the image noise reduction method described in the foregoing method embodiments.

[0137] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0138] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0139] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0140] It should be noted that if the function is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0142] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An image denoising method, characterized in that, include: The adjacent image group is input into the motion detection architecture to obtain the first motion mask output by the motion detection architecture corresponding to the adjacent image group; wherein, the adjacent image group includes the image to be denoised and the previous frame image of the image to be denoised, the motion detection architecture includes a multi-layer motion detection network, the multi-layer motion detection network is used to perform multi-scale motion detection on the adjacent image group, and there is information transmission between the multi-layer motion detection networks. The image to be denoised is denoised according to the first motion mask to obtain a second denoised image corresponding to the image to be denoised. The step of denoising the image to be denoised according to the first motion mask to obtain a second denoised image corresponding to the image to be denoised includes: The second denoised image is determined based on the first motion mask and the adjacent image group, or the second denoised image is determined based on the first motion mask and the result obtained after spatial denoising of the image to be denoised. Determining the second denoised image based on the first motion mask and the adjacent image group includes: The image to be denoised and the third denoised image corresponding to the previous frame image are fused according to the first motion mask to obtain the second denoised image; The step of inputting adjacent image groups into the motion detection architecture to obtain a first motion mask output by the motion detection architecture corresponding to the adjacent image groups includes: The image processing group corresponding to the first layer motion detection network is input into the first layer motion detection network to obtain the first motion mask output by the first layer motion detection network; wherein, the image processing group corresponding to each layer motion detection network includes the image obtained by downsampling the image to be denoised and the previous frame image, and the resolution of the multi-layer motion detection network increases layer by layer; Starting from the second-to-last motion detection network layer, the processing image group corresponding to this layer and the second motion mask corresponding to the previous layer motion detection network are sequentially input into this layer motion detection network to obtain the first motion mask output by this layer motion detection network; wherein, the second motion mask corresponding to the previous layer motion detection network is an image obtained by upsampling the first motion mask corresponding to the previous layer motion detection network, and the resolution of the second motion mask is equal to the resolution of the processing image group corresponding to the previous layer motion detection network; The adjacent image groups and the second motion mask corresponding to the previous layer motion detection network are input into the last layer motion detection network to obtain the first motion mask output by the last layer motion detection network.

2. The image denoising method of claim 1, wherein, The image noise reduction method further includes: The motion detection architecture is obtained by training the first neural network architecture using the following process: The first sample image is input into the first neural network architecture to obtain the first predicted motion mask output by each layer of the first neural network model in the neural network architecture; The first predicted image corresponding to the first neural network model of each layer is determined based on the first predicted motion mask corresponding to the first layer of the first neural network model, the first sample image, and the previous frame image of the first sample image. The loss is calculated based on the first predicted image and the actual denoised image corresponding to each layer of the first neural network model, and the parameters in the first neural network architecture are adjusted based on the loss until the motion detection architecture is obtained.

3. The image denoising method of claim 1, wherein, Determining the second denoised image based on the first motion mask and the result obtained after spatial denoising of the image to be denoised includes: The image to be denoised is input into the spatial domain denoising network to obtain the first denoised image output by the spatial domain denoising network; The first denoised image and the third denoised image corresponding to the previous frame image are fused according to the first motion mask to obtain the second denoised image.

4. The image denoising method of claim 3, wherein, The image noise reduction method further includes: The second neural network model and the second neural network architecture are trained using the following process to obtain the spatial domain denoising network and the motion detection architecture: The second sample image is input into the second neural network model to obtain the second predicted image output by the second neural network model; The second sample image and the previous frame image of the second sample image are input into the second neural network architecture to obtain the second predicted motion mask output by each layer of the third neural network model in the second neural network architecture. The third prediction image corresponding to the third neural network model of each layer is determined based on the second predicted motion mask, the second sample image, and the second predicted image. The loss is calculated based on the third predicted image and the actual denoised image corresponding to the third neural network model of each layer, and the parameters in the second neural network model and the second neural network architecture are adjusted based on the loss until the spatial domain denoising network and the motion detection architecture are obtained.

5. An image noise reduction apparatus, characterized by comprising: include: The first input module is used to input adjacent image groups into the motion detection architecture to obtain a first motion mask output by the motion detection architecture corresponding to the adjacent image groups; wherein, the adjacent image groups include the image to be denoised and the previous frame image of the image to be denoised, the motion detection architecture includes a multi-layer motion detection network, the multi-layer motion detection network is used to perform multi-scale motion detection on the adjacent image groups, and there is information transmission between the multi-layer motion detection networks. The noise reduction module is used to reduce the noise of the image to be denoised according to the first motion mask to obtain a second noise-reduced image corresponding to the image to be denoised. Specifically, the noise reduction module is used for: The second denoised image is determined based on the first motion mask and the adjacent image group, or the second denoised image is determined based on the first motion mask and the result obtained after spatial denoising of the image to be denoised. The image to be denoised and the third denoised image corresponding to the previous frame image are fused according to the first motion mask to obtain the second denoised image; The first input module is specifically used for: The image processing group corresponding to the first layer motion detection network is input into the first layer motion detection network to obtain the first motion mask output by the first layer motion detection network; wherein, the image processing group corresponding to each layer motion detection network includes the image obtained by downsampling the image to be denoised and the previous frame image, and the resolution of the multi-layer motion detection network increases layer by layer; Starting from the second-to-last motion detection network layer, the processing image group corresponding to this layer and the second motion mask corresponding to the previous layer motion detection network are sequentially input into this layer motion detection network to obtain the first motion mask output by this layer motion detection network; wherein, the second motion mask corresponding to the previous layer motion detection network is an image obtained by upsampling the first motion mask corresponding to the previous layer motion detection network, and the resolution of the second motion mask is equal to the resolution of the processing image group corresponding to the previous layer motion detection network; The adjacent image groups and the second motion mask corresponding to the previous layer motion detection network are input into the last layer motion detection network to obtain the first motion mask output by the last layer motion detection network.

6. A computer program product, characterized in that, It includes computer program instructions, which, when read and executed by a processor, perform the image denoising method as described in any one of claims 1-4.

7. An electronic device, characterized in that, include: Processor, memory, and bus; The processor and the memory communicate with each other via the bus; The memory stores computer program instructions that can be executed by the processor, and the processor can execute the image noise reduction method as described in any one of claims 1-4 by calling the computer program instructions.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a computer, cause the computer to perform the image noise reduction method as described in any one of claims 1-4.

Images