Image processing method, device, imaging device, electronic device and storage medium

Abstract

The present disclosure proposes an image processing method, apparatus, imaging device, electronic device and storage medium, wherein the apparatus first obtains an original frame and at least one reference frame in a video, wherein the at least one reference frame includes a previous frame of the original frame, and then performs spatial denoising on at least one pixel group in the original frame to obtain a corresponding spatial denoising pixel group, and then determines a corresponding fusion coefficient based on the temporal similarity between the spatial denoising pixel group and the corresponding pixel group in at least one reference frame, and fuses the spatial denoising pixel group with the corresponding pixel group in the previous frame according to the corresponding fusion coefficient to obtain a corresponding target pixel group, thereby generating a target frame after denoising the original frame. This achieves spatiotemporal denoising using different denoising intensities for different pixel groups in the original frame, and the denoising intensities are also different for different original frames in the video, thereby avoiding the use of a fixed denoising intensity for denoising and improving the denoising effect of the video.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, apparatus, imaging device, electronic device, and storage medium. Background Art

[0002] The goal of video denoising is to reduce noise in video images, addressing the problem of image quality degradation caused by noise interference. Effective video denoising methods not only improve image quality but also better reflect the effective information within the video. Therefore, video denoising has long been a key research area in image quality processing and holds significant significance.

[0003] Video noise reduction methods in related technologies usually adopt the spatiotemporal filtering method, that is, joint filtering in the spatial and temporal domains, and noise reduction is performed by utilizing the correlation between each frame image in the spatial and temporal domains in the video. However, the noise reduction intensity used by this filtering method is usually fixed, and the noise reduction effect of the video is poor. Summary of the Invention

[0004] This disclosure proposes an image processing method, apparatus, imaging device, electronic device, and storage medium. The specific solution is as follows:

[0005] In one aspect, an embodiment of the present disclosure proposes an image processing device, comprising: a video frame acquisition module, configured to acquire an original frame in a video and at least one reference frame corresponding to the original frame, wherein the at least one reference frame includes a previous frame of the original frame, and the at least one reference frame is a denoised image frame; a spatial denoising module, configured to perform spatial denoising on at least one pixel group in the original frame to obtain a corresponding spatial denoised pixel group, wherein the pixel group includes at least one pixel point; a denoising intensity determination module, configured to determine a corresponding fusion coefficient based on a temporal similarity between the spatial denoised pixel group and a corresponding pixel group in the at least one reference frame; a fusion denoising module, configured to fuse the spatial denoised pixel group with the corresponding pixel group in the previous frame according to the corresponding fusion coefficient to obtain a corresponding target pixel group; and an image generation module, configured to generate a target frame after denoising the original frame based on a target pixel group corresponding to the at least one pixel group in the original frame.

[0006] On the other hand, an embodiment of the present disclosure proposes an image processing method, including: obtaining an original frame in a video and at least one reference frame corresponding to the original frame, wherein the at least one reference frame includes a previous frame of the original frame, and the at least one reference frame is a denoised image frame; performing spatial denoising on at least one pixel group in the original frame to obtain a corresponding spatial denoised pixel group, wherein the pixel group includes at least one pixel point; determining a corresponding fusion coefficient based on the temporal similarity between the spatial denoised pixel group and the corresponding pixel group in the at least one reference frame; fusing the spatial denoised pixel group with the corresponding pixel group in the previous frame according to the corresponding fusion coefficient to obtain a corresponding target pixel group; and generating a target frame after the original frame is denoised based on the target pixel group corresponding to the at least one pixel group in the original frame.

[0007] Another embodiment of the present disclosure provides an imaging device, comprising: an image acquisition device for acquiring video; and an image processing device as described in the embodiment of the first aspect of the present disclosure, for performing noise reduction processing on the video.

[0008] Another embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the method described in the embodiment of the second aspect of the present disclosure.

[0009] Another aspect of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the second aspect of the present disclosure.

[0010] Another aspect of the present disclosure provides a computer program product, including a computer program. When the computer program is executed by a processor, the computer program implements the method described in the second aspect of the present disclosure.

[0011] The image processing method, apparatus, imaging device, electronic device and storage medium of the embodiments of the present disclosure implement spatiotemporal noise reduction using different noise reduction intensities for different pixel groups in the original frame, and the noise reduction intensities are also different for different original frames in the video, thereby avoiding the use of a fixed noise reduction intensity for noise reduction and improving the noise reduction effect of the video.

[0012] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and / or additional aspects and advantages of the present disclosure will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0014] Figure 1 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0015] Figure 2 A flowchart of another image processing method provided by an embodiment of the present disclosure;

[0016] Figure 3 A flowchart of another image processing method provided by an embodiment of the present disclosure;

[0017] Figure 4 A flowchart of another image processing method provided by an embodiment of the present disclosure;

[0018] Figure 5 A schematic diagram of the mapping relationship between the average value and the content coefficient provided in the embodiment of the present disclosure;

[0019] Figure 6 A flowchart of another image processing method provided by an embodiment of the present disclosure;

[0020] Figure 7 A schematic diagram of the mapping relationship between target similarity and structure coefficient provided in an embodiment of the present disclosure;

[0021] Figure 8 A schematic structural diagram of an image processing device provided by an embodiment of the present disclosure;

[0022] Figure 9 A schematic structural diagram of an imaging device provided by an embodiment of the present disclosure;

[0023] Figure 10 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] The following describes in detail embodiments of the present disclosure, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0025] Video noise reduction methods in related technologies usually adopt the spatiotemporal filtering method, that is, joint filtering in the spatial and temporal domains, and noise reduction is performed by utilizing the correlation between each frame image in the spatial and temporal domains in the video. However, the noise reduction intensity used by this filtering method is usually fixed, and the noise reduction effect of the video is poor.

[0026] The present disclosure addresses the above-mentioned issues and provides an image processing method for video noise reduction based on spatiotemporal filtering. The method first obtains an original frame in a video and at least one reference frame corresponding to the original frame, wherein the at least one reference frame includes a previous frame of the original frame, and the at least one reference frame is a denoised image frame. Spatial noise reduction is performed on at least one pixel group in the original frame to obtain a corresponding spatially denoised pixel group, wherein the pixel group includes at least one pixel point. A corresponding fusion coefficient is determined based on the temporal similarity between the spatially denoised pixel group and the corresponding pixel group in the at least one reference frame. The spatially denoised pixel group is fused with the corresponding pixel group in the previous frame according to the corresponding fusion coefficient to obtain a corresponding target pixel group. Furthermore, a target frame after the original frame noise reduction is generated based on the target pixel group corresponding to the at least one pixel group in the original frame. Thus, spatiotemporal noise reduction is performed using different noise reduction intensities for different pixel groups in the original frame, and the noise reduction intensities are also different for different original frames in the video, thereby avoiding the use of a fixed noise reduction intensity for noise reduction and improving the noise reduction effect of the video.

[0027] The image processing method, apparatus, imaging device, electronic device, and storage medium provided by the present disclosure are described in detail below with reference to the accompanying drawings.

[0028] Figure 1 This is a flow chart of an image processing method provided by an embodiment of the present disclosure. The method includes:

[0029] Step 101: Acquire an original frame in a video and at least one reference frame corresponding to the original frame, wherein the at least one reference frame includes a previous frame of the original frame, and the at least one reference frame is a noise-reduced image frame.

[0030] The original frame is any one or more frames of images to be processed in a video other than the first frame. The present embodiment is described using an example in which the original frame is any one or more frames of images to be processed in a video other than the first frame. When the original frames are multiple frames of images to be processed in a video, each original frame can be processed separately according to the image processing method provided in the present embodiment to obtain a target frame after noise reduction for each original frame.

[0031] The previous frame of the original frame is an image frame obtained by performing noise reduction on the previous frame image of the original frame in the video (i.e., an image frame that is adjacent to the original frame and precedes the original frame in the video) using the image processing method of the embodiment of the present disclosure. The previous frame of the original frame is an image frame after noise reduction on the previous frame image of the original frame in the video. The noise reduction here refers to noise reduction in the spatiotemporal domain.

[0032] The at least one reference frame corresponding to the original frame includes an image frame after denoising the previous frame image of the original frame in the video (i.e., the previous frame of the original frame), or an image frame after denoising the previous multiple frames image of the original frame in the video (i.e., the continuous multiple frames image including the previous frame image before the original frame in the video), and the present disclosure does not impose any restrictions on this. Among them, each reference frame corresponding to the original frame is an image frame after spatiotemporal denoising using the image processing method of an embodiment of the present disclosure. The number of the at least one reference frame corresponding to the original frame can be set as needed, and the present disclosure does not impose any restrictions on this.

[0033] For example, assuming the original frame is the 10th frame in a video, the frame preceding the original frame is the 9th frame in the video after denoising. The at least one reference frame corresponding to the original frame may include only the 9th frame in the video after denoising, or may include the 8th frame in the video after denoising and the 9th frame in the video after denoising, or may include the 7th frame in the video after denoising, the 8th frame in the video after denoising, and the 9th frame in the video after denoising, etc. The denoising here refers to spatiotemporal denoising.

[0034] Step 102 : performing spatial denoising on at least one pixel group in the original frame to obtain a corresponding spatial denoised pixel group, wherein the pixel group includes at least one pixel point.

[0035] In one embodiment of the present disclosure, the original frame and the reference frame can be divided into multiple pixel groups in the same division method. Each pixel group can include one pixel point, or include two or more adjacent pixel points, and the pixel groups in the same position in different image frames are called corresponding pixel groups in different image frames.

[0036] For example, taking the case where there are two reference frames and a pixel group includes two adjacent pixels, assuming that the original frame and the two reference frames are both 11 (rows) * 10 (columns) image frames, that is, the original frame and the two reference frames each include 11 * 10 pixels. Then the original frame and the two reference frames can be divided into 11 * 5 pixel groups, where the first and second pixels of each row form a pixel group, the third and fourth pixels of each row form a pixel group, the fifth and sixth pixels of each row form a pixel group, the seventh and eighth pixels of each row form a pixel group, and the ninth and tenth pixels of each row form a pixel group. Moreover, the pixel groups at the same position in the original frame and the two reference frames, such as the pixel group consisting of the first and second pixels of each row in the original frame and the two reference frames, are called the corresponding pixel groups in the original frame and the two reference frames.

[0037] In one embodiment of the present disclosure, spatial denoising can be performed on each pixel group in the original frame to obtain a corresponding spatial denoised pixel group. The pixel value of each pixel in the spatial denoised pixel group is the pixel value of the corresponding pixel in the corresponding pixel group in the original frame after spatial denoising.

[0038] Among them, any filtering method such as NLM (Non-Local Mean Filter), bilateral filtering, Gaussian filtering, etc. can be used to perform spatial noise reduction on each pixel group in the original frame to obtain the spatial noise reduction pixel group corresponding to each pixel group.

[0039] Step 103 : determining a corresponding fusion coefficient according to a temporal similarity between the spatial denoised pixel group and a corresponding pixel group in at least one reference frame.

[0040] Among them, the temporal similarity is used to characterize the similarity in content between the spatial denoised pixel group and the corresponding pixel group in the reference frame, or to characterize the similarity in local structure of pixel points between the spatial denoised pixel group and the corresponding pixel group in the reference frame, or to characterize the similarity in content and local structure of pixel points between the spatial denoised pixel group and the corresponding pixel group in the reference frame.

[0041] In one embodiment of the present disclosure, after spatial denoising is performed on each pixel group in the original frame to obtain a corresponding spatial denoised pixel group, the temporal similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame can be determined. Furthermore, based on the temporal similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame, a fusion coefficient corresponding to the spatial denoised pixel group can be determined. Specifically, for each spatial denoised pixel group, a fusion coefficient corresponding to the spatial denoised pixel group can be determined based on each temporal similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame.

[0042] Step 104 : Fusing the spatial domain denoised pixel group with the corresponding pixel group in the previous frame according to the corresponding fusion coefficient to obtain the corresponding target pixel group.

[0043] In one embodiment of the present disclosure, for each pixel group in the original frame, the spatial denoising pixel group corresponding to the pixel group is obtained, and after determining the fusion coefficient corresponding to the spatial denoising pixel group, the spatial denoising pixel group can be fused with the pixel group at the corresponding position in the previous frame according to the fusion coefficient to obtain the target pixel group corresponding to the pixel group in the original frame.

[0044] Among them, for any pixel group in the original frame, when the corresponding spatial denoising pixel group is fused with the pixel group at the corresponding position in the previous frame according to the fusion coefficient corresponding to the spatial denoising pixel group, the pixel value of each pixel point in the spatial denoising pixel group can be fused with the pixel value of the corresponding pixel point in the pixel group at the corresponding position in the previous frame based on the fusion coefficient to obtain the pixel value of the pixel point at the corresponding position in the target pixel group, and thus the target pixel group corresponding to the pixel group in the original frame is obtained according to the pixel value of each pixel point in the target pixel group.

[0045] Step 105 : generating a target frame after noise reduction of the original frame according to a target pixel group corresponding to at least one pixel group in the original frame.

[0046] In one embodiment of the present disclosure, after obtaining the corresponding target pixel group for each pixel group in the original frame, a target frame after noise reduction of the original frame can be generated according to the target pixel groups corresponding to each pixel group in the original frame.

[0047] For example, assuming the original frame is the 10th frame in a video, the at least one reference frame corresponding to the original frame includes the denoised image frame of the 8th frame in the video (referred to herein as reference frame 1) and the denoised image frame of the 9th frame (referred to herein as reference frame 2, i.e., the frame immediately preceding the original frame). Furthermore, assuming the original frame and the two reference frames each include 11*10 pixels, the original frame and the two reference frames are divided into 11*5 pixel groups according to the above example.

[0048] In an embodiment of the present disclosure, spatial denoising can be performed on the pixel group consisting of the first and second pixels in the first row of the 10th frame image to obtain a corresponding spatial denoised pixel group, and then based on the temporal similarity between the spatial denoised pixel group and the pixel group consisting of the first and second pixels in the first row of reference frame 1, and the temporal similarity between the spatial denoised pixel group and the pixel group consisting of the first and second pixels in the first row of reference frame 2, the fusion coefficient corresponding to the spatial denoised pixel group is determined, and then the spatial denoised pixel group is fused with the pixel group consisting of the first and second pixels in the first row of the image frame after denoising the 9th frame image (i.e., reference frame 2) according to the corresponding fusion coefficient to obtain a target pixel group corresponding to the pixel group consisting of the first and second pixels in the first row of the 10th frame image.

[0049] Furthermore, for the pixel group consisting of the 3rd and 4th pixel points in the first row of the 10th frame image, the corresponding target pixel group can be determined in the above manner, and the above process of determining the target pixel group can be repeated for other pixel groups in the 10th frame image until the corresponding target pixel groups have been determined for all pixel groups in the 10th frame image. Then, the target frame after noise reduction of the 10th frame image can be generated based on the target pixel groups corresponding to each pixel group in the 10th frame image.

[0050] It should be noted that, for the first frame image in the video, only spatial denoising can be performed on the image, and the image frame after spatial denoising can be determined as the target frame after denoising of the first frame image; for the second frame image in the video, the second frame image can be used as the original frame, and the target frame after denoising of the first frame image can be used as the reference frame corresponding to the original frame and the previous frame of the original frame included in the reference frame, and the target frame after denoising of the second frame image can be obtained through the method shown in the above steps 102-105; similarly, taking the number of reference frames as 1 as an example, for the third frame image and other subsequent image frames in the video, the image frame can be used as the original frame, and the target frame after denoising of the previous frame image can be used as the reference frame corresponding to the original frame and the previous frame of the original frame included in the reference frame, and the target frame after denoising of the image frame can be obtained through the method shown in the above steps 102-105.

[0051] The image processing method provided by the embodiment of the present disclosure uses a spatial denoising pixel group corresponding to a pixel group in the original frame when determining a target frame after denoising the original frame, which is the result of spatial denoising, wherein spatial denoising is planar two-dimensional denoising. In addition, the method determines the target frame after denoising the original frame based on the spatial denoising and combines the previous frame after denoising, which is equivalent to performing temporal denoising based on spatial denoising and combining time domain information. Therefore, the image processing method provided by the embodiment of the present disclosure is a three-dimensional denoising method in the spatiotemporal domain. Since different fusion coefficients can be determined for the spatial denoising pixel groups corresponding to different pixel groups in the original frame of the video according to the temporal similarity between the corresponding pixel groups in at least one reference frame, the different spatial denoising pixel groups are fused with the corresponding pixel groups in the previous frame according to their respective fusion coefficients to obtain their respective corresponding target pixel groups, thereby achieving temporal and spatial denoising with different noise reduction intensities for different pixel groups in the original frame, and the noise reduction intensities are also different for different original frames in the video, thereby avoiding the use of a fixed noise reduction intensity for noise reduction, improving the noise reduction effect of the video, and achieving adaptive temporal and spatial denoising. In addition, this solution processes pixel groups. When performing spatiotemporal noise reduction on the original frames in the video, it is performed in the Bayer domain. Compared with noise reduction in the YUV (a color encoding method, where Y represents brightness and U and V represent color chrominance) domain or RGB (a color mode, where R, G, and B represent red, green, and blue color channels, respectively) domain, it can avoid the influence of various modules of the ISP (Image Signal Processor) on the properties of the original noise, avoid the problem of ineffective removal of video picture noise, and achieve better video noise reduction effect.

[0052] In summary, the image processing method provided by the embodiment of the present disclosure first obtains the original frame in the video and at least one reference frame corresponding to the original frame, wherein the at least one reference frame includes the previous frame of the original frame, and the at least one reference frame is a denoised image frame, and then performs spatial denoising on at least one pixel group in the original frame to obtain a corresponding spatial denoised pixel group, wherein the pixel group includes at least one pixel point, and then determines the corresponding fusion coefficient based on the temporal similarity between the spatial denoised pixel group and the corresponding pixel group in the at least one reference frame, and fuses the spatial denoised pixel group with the corresponding pixel group in the previous frame according to the corresponding fusion coefficient to obtain a corresponding target pixel group, and then generates a target frame after the original frame denoises the target pixel group corresponding to the at least one pixel group in the original frame. In this way, different denoising intensities are used for temporal and spatial denoising for different pixel groups in the original frame, and the denoising intensities are also different for different original frames in the video, thereby avoiding the use of a fixed denoising intensity for denoising and improving the denoising effect of the video.

[0053] Through the above analysis, it can be known that the temporal similarity in the present disclosure is used to characterize the similarity in content between the spatial denoised pixel group and the corresponding pixel group in the reference frame, or to characterize the similarity in local structure of pixel points between the spatial denoised pixel group and the corresponding pixel group in the reference frame, or to characterize the similarity in content and local structure of pixel points between the spatial denoised pixel group and the corresponding pixel group in the reference frame. In one possible implementation form, the temporal similarity may include temporal content similarity that characterizes the similarity in content between the spatial denoised pixel group and the corresponding pixel group in the reference frame, and temporal structural similarity that characterizes the similarity in local structure of pixel points between the spatial denoised pixel group and the corresponding pixel group in the reference frame. The following is a case where the temporal similarity includes temporal content similarity and temporal structural similarity, combined with Figure 2 , further illustrating the image processing method provided by the embodiment of the present disclosure.

[0054] Figure 2 FIG. 1 is a flow chart of another image processing method provided by an embodiment of the present disclosure. Figure 2 As shown, the method includes:

[0055] Step 201 : Acquire an original frame in a video and at least one reference frame corresponding to the original frame, wherein the at least one reference frame includes a previous frame of the original frame, and the at least one reference frame is a noise-reduced image frame.

[0056] Step 202 : performing spatial denoising on at least one pixel group in the original frame to obtain a corresponding spatial denoised pixel group, wherein the pixel group includes at least one pixel point.

[0057] The specific implementation process and principles of steps 201-202 can be referred to the description of the above embodiment and will not be repeated here.

[0058] Step 203 : determining a corresponding content coefficient according to the temporal content similarity between the spatial denoised pixel group and a corresponding pixel group in at least one reference frame.

[0059] Among them, the temporal content similarity is used to characterize the similarity in content between the spatial domain denoised pixel group and the corresponding pixel group in the reference frame.

[0060] In one embodiment of the present disclosure, after spatial denoising is performed on each pixel group in the original frame to obtain a corresponding spatial denoised pixel group, the temporal content similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame can be determined. Furthermore, based on the temporal content similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame, a content coefficient corresponding to the spatial denoised pixel group can be determined. Specifically, for each spatial denoised pixel group, a content coefficient corresponding to the spatial denoised pixel group can be determined based on each temporal content similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame.

[0061] In one embodiment of the present disclosure, for any reference frame, the motion intensity of each pixel point in the corresponding pixel group in the original frame can be determined based on the temporal content similarity between the spatial denoising pixel group and the corresponding pixel group in the reference frame, and the content coefficient corresponding to the spatial denoising group can be determined based on the motion intensity of each pixel point in the corresponding pixel group in the original frame.

[0062] Among them, for any reference frame, the greater the temporal content similarity between the spatial denoising pixel group and the corresponding pixel group in the reference frame, the lower the motion intensity of each pixel point in the corresponding pixel group in the original frame; the smaller the temporal content similarity between the spatial denoising pixel group and the corresponding pixel group in the reference frame, the higher the motion intensity of each pixel point in the corresponding pixel group in the original frame.

[0063] When the motion intensity of each pixel in a certain pixel group in the original frame is high, the content coefficient of the spatial denoising pixel group corresponding to this pixel group is low; when the motion intensity of each pixel in a certain pixel group in the original frame is low, the content coefficient of the spatial denoising pixel group corresponding to this pixel group is high. Accordingly, the content coefficient corresponding to the spatial denoising pixel group can be used to measure the motion intensity of each pixel in the corresponding pixel group in the original frame. A larger content coefficient corresponding to the spatial denoising pixel group indicates a lower motion intensity of each pixel in the corresponding pixel group in the original frame, and a smaller content coefficient corresponding to the spatial denoising pixel group indicates a higher motion intensity of each pixel in the corresponding pixel group in the original frame.

[0064] Step 204 : determining a corresponding structure coefficient based on the temporal structural similarity between the spatial denoised pixel group and a corresponding pixel group in at least one reference frame.

[0065] Among them, the temporal structural similarity is used to characterize the similarity in the local structure of the pixel points between the spatial denoising pixel group and the corresponding pixel group in the reference frame.

[0066] In one embodiment of the present disclosure, after spatial denoising is performed on each pixel group in the original frame to obtain a corresponding spatial denoised pixel group, the temporal structural similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame can be determined. Furthermore, based on the temporal structural similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame, a structural coefficient corresponding to the spatial denoised pixel group can be determined. Specifically, for each spatial denoised pixel group, a structural coefficient corresponding to the spatial denoised pixel group can be determined based on each temporal structural similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame.

[0067] In one embodiment of the present disclosure, for any reference frame, the motion intensity of each pixel point in the corresponding pixel group in the original frame can be determined based on the temporal structural similarity between the spatial denoising pixel group and the corresponding pixel group in the reference frame, and the structural coefficient corresponding to the spatial denoising group can be determined based on the motion intensity of each pixel point in the corresponding pixel group in the original frame.

[0068] Among them, for any reference frame, the greater the temporal structural similarity between the spatial denoising pixel group and the corresponding pixel group in the reference frame, the lower the motion intensity of each pixel point in the corresponding pixel group in the original frame; the smaller the temporal structural similarity between the spatial denoising pixel group and the corresponding pixel group in the reference frame, the higher the motion intensity of each pixel point in the corresponding pixel group in the original frame.

[0069] When the motion intensity of each pixel in a certain pixel group in the original frame is high, the structure coefficient of the spatial denoising pixel group corresponding to this pixel group is low; when the motion intensity of each pixel in a certain pixel group in the original frame is low, the structure coefficient of the spatial denoising pixel group corresponding to this pixel group is high. Accordingly, the structure coefficient corresponding to the spatial denoising pixel group can be used to measure the motion intensity of each pixel in the corresponding pixel group in the original frame. A larger structure coefficient corresponding to the spatial denoising pixel group indicates a lower motion intensity of each pixel in the corresponding pixel group in the original frame, and a smaller structure coefficient corresponding to the spatial denoising pixel group indicates a higher motion intensity of each pixel in the corresponding pixel group in the original frame.

[0070] Step 205: Fusing the content coefficient and the structure coefficient to obtain a fusion coefficient.

[0071] In one embodiment of the present disclosure, the content coefficient and the structure coefficient may be multiplied together to obtain a fusion coefficient.

[0072] Among them, the fusion coefficient can be used to measure the motion intensity of each pixel in the original frame.

[0073] It should be noted that, in one embodiment of the present disclosure, the time domain similarity may also only include the time domain content similarity or the time domain structure similarity. Accordingly, the content coefficient or the structure coefficient may be directly used as a fusion coefficient for subsequent video noise reduction processing.

[0074] Step 206 : Determine the weight corresponding to each first pixel in the spatial denoising pixel group and the weight corresponding to each second pixel in the corresponding pixel group in the previous frame according to the fusion coefficient.

[0075] In one embodiment of the present disclosure, it is assumed that the fusion coefficient corresponding to the spatial denoising pixel group is W motion , you can use W motion As the weight of each second pixel in the corresponding pixel group in the previous frame of the original frame, 1-W motion As the weight corresponding to each first pixel in the spatial denoising pixel group.

[0076] In step 207, the pixel value of each first pixel point in the spatial denoising pixel group is fused with the pixel value of the second pixel point at the corresponding position in the corresponding pixel group in the previous frame based on the corresponding weights to obtain the target pixel value of the target pixel point at the corresponding position in the target pixel group.

[0077] In one embodiment of the present disclosure, step 207 may be implemented as shown in the following formula (1):

[0078] F denoised (n) = F(n-1) × W motion +F spa (n)×(1-W motion ) (1)

[0079] Among them, F spa (n) represents the pixel value of any first pixel in the spatial denoised pixel group obtained after spatial denoising of any pixel group in the nth frame image in the video, i.e., the original frame, that is, the pixel value of the pixel at the same position as the first pixel in the original frame after spatial denoising; F(n-1) represents the pixel value of the second pixel corresponding to the first pixel position in the pixel group corresponding to the position of the spatial denoised pixel group in the n-1th frame image in the video after spatiotemporal denoising, i.e., the previous frame; F denoised (n) is the target pixel value of the target pixel corresponding to the first pixel and the second pixel in the target pixel group, that is, the pixel value of the pixel at the same position as the first pixel in the original frame after spatiotemporal denoising; W motion is the fusion coefficient corresponding to the spatial denoising pixel group. motion It can also be called temporal noise reduction strength.

[0080] Step 208 : generating a target pixel group according to the target pixel value of each target pixel point in the target pixel group.

[0081] Step 209 : generating a target frame after noise reduction of the original frame according to a target pixel group corresponding to at least one pixel group in the original frame.

[0082] The specific implementation process and principle of step 209 can be referred to the description of the above embodiment and will not be repeated here.

[0083] refer to Figure 3 , taking the example that the reference frame corresponding to the original frame only includes the previous frame of the original frame, wherein the previous frame is the image frame after denoising (the denoising here refers to spatiotemporal denoising). In the embodiment of the present disclosure, adaptive spatial filtering can be performed on each pixel group in the original frame to obtain a spatial denoised pixel group after spatial denoising, and then the temporal content similarity and temporal structure similarity between the spatial denoised pixel group corresponding to the pixel group and the corresponding pixel group in the previous frame are calculated, and the content coefficient corresponding to the spatial denoised pixel group is determined according to the temporal content similarity, and the structure coefficient corresponding to the spatial denoised pixel group is determined according to the temporal structure similarity, and then the fusion coefficient is determined according to the content coefficient and the structure coefficient, and the spatial denoised pixel group and the corresponding pixel group in the previous frame are fused in the spatiotemporal domain based on the fusion coefficient to obtain the corresponding target pixel group, and then the target frame after the original frame denoising is generated according to the target pixel group corresponding to each pixel group in the original frame.

[0084] In the embodiment of the present disclosure, a corresponding content coefficient is determined based on the temporal content similarity between the spatial denoised pixel group and the corresponding pixel group in at least one reference frame, and a corresponding structure coefficient is determined based on the temporal structure similarity between the spatial denoised pixel group and the corresponding pixel group in at least one reference frame. The content coefficient and the structure coefficient are fused to obtain a fusion coefficient. This achieves the combination of the content similarity between the spatial denoised pixel group and the corresponding pixel group in the reference frame and the local structure similarity of the pixel points between the spatial denoised pixel group and the corresponding pixel group in the reference frame, and jointly judges the motion intensity of each pixel point in the corresponding pixel group in the original frame, thereby improving the accuracy of the motion and stillness judgment of each pixel point in the original frame, and thereby improving the accuracy of the determined fusion coefficient. By determining the weight corresponding to each first pixel point in the spatial denoising pixel group and the weight corresponding to each second pixel point in the corresponding pixel group in the previous frame according to the fusion coefficient, the pixel value of each first pixel point in the spatial denoising pixel group and the pixel value of the second pixel point at the corresponding position in the corresponding pixel group in the previous frame are fused based on the corresponding weights to obtain the target pixel value of the target pixel point at the corresponding position in the target pixel group, and a target pixel group is generated according to the target pixel value of each target pixel point in the target pixel group. According to the target pixel group corresponding to at least one pixel group in the original frame, a target frame after denoising of the original frame is generated, thereby realizing spatiotemporal denoising with different denoising intensities for different pixel groups in the original frame, and the denoising intensities are also different for different original frames in the video, thereby avoiding the use of a fixed denoising intensity for denoising and improving the denoising effect of the video.

[0085] Through the above analysis, it can be seen that in the embodiment of the present disclosure, after performing spatial denoising on at least one pixel group in the original frame and obtaining the corresponding spatial denoised pixel group, the corresponding content coefficient can be determined based on the temporal content similarity between the spatial denoised pixel group and at least one reference frame. Figure 4 , a detailed description is given of the process of determining the temporal content similarity between any spatial denoised pixel group and the corresponding pixel group in at least one reference frame and determining the content coefficient based on the temporal content similarity in the embodiment of the present disclosure.

[0086] Figure 4 FIG. 1 is a flow chart of another image processing method provided by an embodiment of the present disclosure. Figure 4 As shown, Figure 2 Before step 203, the following steps 401-403 may also be included:

[0087] Step 401 : determining a first image block of a preset size in an original frame, wherein the central pixel points of the first image block include the pixel points in a pixel group corresponding to the spatial domain noise reduction pixel group in the original frame.

[0088] Among them, the preset size can be set arbitrarily as needed, and the present disclosure does not limit this.

[0089] The central pixel point of the first image block is a pixel point located at the center position of the first image block.

[0090] For example, taking the image denoising process as one pixel point at a time, where the spatial denoising pixel group and the corresponding pixel group in the original frame include one pixel point, as an example, assuming the preset size is 3 (rows) * 3 (columns), that is, the first image block includes 3*3 pixels, and the original frame is an image frame of 11 (rows) * 10 (columns), that is, the original frame includes 11*10 pixels. Then, for the pixel group consisting of a pixel point in the 5th row and 6th column of the original frame, after determining the spatial denoising pixel group corresponding to the pixel group, the first image block including 3*3 pixels in the original frame can be determined, wherein the center pixel point of the first image block includes a pixel point in the 5th row and 6th column of the original frame. That is, the first image block includes the pixels of the 4th row and 5th column, the 4th row and 6th column, the 4th row and 7th column, the 5th row and 5th column, the 5th row and 6th column, the 5th row and 7th column, the 6th row and 5th column, the 6th row and 6th column, and the 6th row and 7th column in the original frame.

[0091] Taking the image denoising process as an example in which spatiotemporal fusion is performed on two adjacent pixels each time, that is, the spatial denoising pixel group and the corresponding pixel group in the original frame include two adjacent pixels, assuming that the preset size is 3 (rows) * 4 (columns), that is, the first image block includes 3*4 pixels, and the original frame is an image frame of 11 (rows) * 10 (columns), that is, the original frame includes 11*10 pixels. Then, for the pixel group consisting of a pixel in the 5th row and 5th column and a pixel in the 5th row and 6th column in the original frame, after determining the spatial denoising pixel group corresponding to the pixel group, the first image block including 3*4 pixels in the original frame can be determined, wherein the center pixel of the first image block includes a pixel in the 5th row and 5th column and a pixel in the 5th row and 6th column in the original frame. That is, the first image block includes the pixels of the 4th row and 4th column, the 4th row and 5th column, the 4th row and 6th column, the 4th row and 7th column, the 5th row and 4th column, the 5th row and 5th column, the 5th row and 6th column, the 5th row and 7th column, the 6th row and 4th column, the 6th row and 5th column, the 6th row and 6th column, and the 6th row and 7th column in the original frame.

[0092] Step 402 : Using the pixel value of each pixel in the spatial denoising pixel group, update the pixel value of the corresponding pixel in the first image block to obtain a second image block.

[0093] In one embodiment of the present disclosure, the pixel values ​​of the corresponding pixels in the first image block can be updated using the pixel values ​​of each pixel in the spatial denoising pixel group. Specifically, the pixel value of the central pixel of the first image block is updated to the pixel value of the corresponding pixel in the spatial denoising pixel group, thereby obtaining a second image block. Accordingly, the pixel value of the central pixel of the second image block is the pixel value of the corresponding pixel in the original frame after spatial denoising, and the pixel values ​​of the other pixels in the second image block are the original pixel values ​​of the corresponding pixels in the original frame.

[0094] It should be noted that, in the embodiment of the present disclosure, the central pixel point of the second image block is taken as the pixel value of the corresponding pixel point in the original frame after spatial denoising, and the pixel values ​​of other pixel points of the second image block are the original pixel values ​​of the corresponding pixel points in the original frame. Without considering the processing efficiency and processing complexity, spatial denoising can also be performed on all pixel points in the second image block except the central pixel point, and then subsequent steps are performed based on the second image block after spatial denoising to improve the accuracy of the temporal content similarity, thereby improving the denoising effect of the video. The present disclosure does not impose any restrictions on this.

[0095] Step 403: For any reference frame, determine a third image block in the reference frame that corresponds to the position of the first image block, and determine the temporal content similarity between the spatial denoising pixel group and the corresponding pixel group in the reference frame based on the content similarity between the second image block and the third image block.

[0096] In one embodiment of the present disclosure, taking one of the reference frames as an example, a third image block corresponding to the position of the first image block in the reference frame can be determined, that is, the third image block and the first image block have the same size, both of which are preset sizes, and the position of the third image block in the reference frame is the same as the position of the first image block in the original frame, and the central pixel point of the third image block includes the pixel points in the pixel group corresponding to the spatial denoising pixel group in the reference frame. Furthermore, the content similarity between the second image block and the third image block can be determined, and the content similarity is determined as the temporal content similarity between the spatial denoising pixel group and the corresponding pixel group in the reference frame. The content similarity between the second image block and the third image block is used to characterize the degree of similarity in content between the second image block and the third image block.

[0097] The content similarity between the second image block and the third image block can be specifically represented by the pixel distance between the second image block and the third image block. The pixel distance between the second image block and the third image block can be determined by the method shown in formula (2):

[0098]

[0099] Among them, diff' avgrepresents the pixel distance between the second image block and the third image block, which can represent the content similarity between the second image block and the third image block; ABS represents the absolute value; n*m represents the preset size of the first image block, that is, the first image block includes n*m pixels; P refi,k represents the RGB value or grayscale value of the i-th pixel in the third image block in the k-th reference frame; curi Represents the RGB value or grayscale value of the i-th pixel in the second image block.

[0100] That is, the process of determining the content similarity between the second image block and the third image block can be: performing a weighted summation on the pixel values ​​of each pixel in the second image block to obtain a first accumulated value; performing a weighted summation on the pixel values ​​of each pixel in the third image block to obtain a second accumulated value; determining an average value of the difference between the first accumulated value and the second accumulated value, and determining the content similarity between the second image block and the third image block based on the average value of the difference.

[0101] In an embodiment of the present disclosure, after determining the temporal content similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame, the content coefficient corresponding to the spatial denoised pixel group can be determined according to the temporal content similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame through the method shown in the following steps 404-405.

[0102] Step 404 : Determine an average value of temporal content similarity between the spatially denoised pixel group and a corresponding pixel group in at least one reference frame.

[0103] In one embodiment of the present disclosure, taking the temporal content similarity between the spatial denoised pixel group and the corresponding pixel group in any reference frame as represented by the pixel distance shown in formula (2) as an example, step 404 can be implemented as shown in the following formula (3):

[0104]

[0105] Here, K represents the number of reference frames.

[0106] Step 405: Determine the content coefficient according to the mapping relationship between the average value and the content coefficient.

[0107] It can be understood that the smaller the pixel distance between the second image block and the third image block, the greater the temporal content similarity between the spatially denoised pixel group and the corresponding pixel group in the reference frame, the lower the motion intensity of each pixel in the corresponding pixel group in the original frame, and the higher the content coefficient; the larger the pixel distance between the second image block and the third image block, the smaller the temporal content similarity between the spatially denoised pixel group and the corresponding pixel group in the reference frame, the higher the motion intensity of each pixel in the corresponding pixel group in the original frame, and the lower the content coefficient. Based on this, a mapping relationship can be set between the average value of the temporal content similarity between the spatially denoised pixel group and the corresponding pixel groups in each reference frame and the content coefficient, and based on this mapping relationship, the content coefficient is determined based on the average value.

[0108] In one embodiment of the present disclosure, three thresholds, a first threshold, a second threshold, and a third threshold, may be set, and the mapping relationship between the average value and the content coefficient may be determined in combination with the three thresholds. For example, taking the case where the temporal content similarity between the spatial denoised pixel group and the corresponding pixel group in the reference frame is represented by the pixel distance between the second image block and the third image block, the mapping relationship between the average value of the temporal content similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame and the content coefficient may be set as follows: Figure 5 The curve form shown.

[0109] in, Figure 5 Where thr1 represents the first threshold, thr2 represents the second threshold, and thr3 represents the third threshold; diff avg represents the average value of the temporal content similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame; w1 represents the content coefficient; k1 and k2 represent the slopes respectively.

[0110] Figure 5 The curve shown can be expressed as the following formula (4):

[0111]

[0112] Among them, b1 and b2 are preset values; k1 and k2 represent slopes respectively, and b1, b2, k1 and k2 can be set and adjusted as needed. The average diff of the temporal content similarity between the spatial domain denoised pixel group and the corresponding pixel group in each reference frame is calculated. avg Substituting the values ​​of x and y into formula (4) gives the content coefficient w1.

[0113] Refer to formula (4) and Figure 5 The curve shown in the figure shows that the average value of the temporal content similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame is diff avgWhen it is greater than or equal to the third threshold thr3, it can be determined that the motion intensity of each pixel in the corresponding pixel group in the original frame is 1, that is, each pixel in the corresponding pixel group in the original frame can be divided into a motion pixel, and the content coefficient w1 is 0 at this time; the average value diff of the temporal content similarity between the spatial domain denoised pixel group and the corresponding pixel group in each reference frame avg When it is less than or equal to the first threshold thr1, it can be determined that the motion intensity of each pixel in the corresponding pixel group in the original frame is 0, that is, each pixel in the corresponding pixel group in the original frame can be divided into a stationary pixel, and the content coefficient w1 is 1 at this time; the average value diff of the temporal content similarity between the spatial domain denoised pixel group and the corresponding pixel group in each reference frame avg When it is greater than the first threshold thr1 and less than the third threshold thr3, it can be determined that the motion intensity of each pixel in the corresponding pixel group in the original frame is a value between 0 and 1, that is, each pixel in the corresponding pixel group in the original frame can be divided into a pixel point between motion and stillness. At this time, the content coefficient w1 is a value between 0 and 1. According to the average value diff avg The mapping relationship between the content coefficient w1 and the content coefficient w1 can determine the value of the content coefficient w1.

[0114] Thus, through a simple piecewise linear relationship, the content coefficient can be easily determined based on the temporal content similarity between the spatial denoising pixel group and the corresponding pixel group in each reference frame, and then the fusion coefficient can be obtained based on the content coefficient. This reduces the amount of computation and significantly reduces the hardware resources required to determine the temporal similarity and fusion coefficient through hardware circuits, reducing the cost of video noise reduction and better adapting to scenes with frequently changing light in static backgrounds. Furthermore, by determining a lower content coefficient when the temporal content similarity between the spatial denoising pixel group and the corresponding pixel group in each reference frame is low, that is, when the motion intensity of each pixel point in the corresponding pixel group in the original frame is high, the temporal denoising intensity can be reduced, thereby reducing the generation of motion artifacts and improving the noise reduction effect of the video.

[0115] In the disclosed embodiment, a first image block of a preset size is determined in an original frame, wherein the central pixel of the first image block includes a pixel in a pixel group corresponding to a spatial denoising pixel group in the original frame. The pixel value of each pixel in the spatial denoising pixel group is used to update the pixel value of the corresponding pixel in the first image block to obtain a second image block. For any reference frame, a third image block corresponding to the position of the first image block is determined in the reference frame. Based on the content similarity between the second image block and the third image block, the temporal content similarity between the spatial denoising pixel group and the corresponding pixel group in the reference frame is determined. The average value of the temporal content similarity between the spatial denoising pixel group and the corresponding pixel group in at least one reference frame is determined. The content coefficient is determined based on a mapping relationship between the average value and the content coefficient. In this way, the content coefficient is determined based on the content similarity between the spatial denoising pixel group and the corresponding pixel group in the reference frame, and the motion intensity of each pixel in the corresponding pixel group in the original frame is determined. Furthermore, an appropriate noise reduction intensity is determined based on the motion intensity of each pixel in the original frame, thereby performing spatiotemporal noise reduction on the original frame and improving the noise reduction effect of the video. Moreover, by adopting the pixel value or grayscale value of each pixel in the image block rather than the pixel value or grayscale value of a single pixel to determine the content coefficient and the motion intensity of each pixel in the corresponding pixel group in the original frame, the influence of a single noise point is avoided, and the accuracy of the content coefficient and the motion and stillness judgment of each pixel in the original frame is improved, thereby improving the accuracy of the determined fusion coefficient and further improving the noise reduction effect of the video.

[0116] Through the above analysis, it can be seen that in the embodiment of the present disclosure, after performing spatial denoising on at least one pixel group in the original frame and obtaining the corresponding spatial denoised pixel group, the corresponding structure coefficient can be determined based on the temporal structural similarity between the spatial denoised pixel group and at least one reference frame. Figure 6 Taking the image denoising process in which spatiotemporal fusion is performed on at least two adjacent pixels in the original frame each time, that is, a spatial denoising pixel group includes at least two adjacent pixels as an example, the process of determining the temporal structural similarity between any spatial denoising pixel group and a corresponding pixel group in at least one reference frame and determining the structural coefficient based on the temporal structural similarity in the embodiment of the present disclosure is described in detail.

[0117] Figure 6 FIG. 1 is a flow chart of another image processing method provided by an embodiment of the present disclosure. Figure 6 As shown, Figure 2 Before step 204, the following steps 601-603 may be included:

[0118] Step 601: Determine a fourth image block in the original frame, where the fourth image block includes pixel points corresponding to at least two adjacent pixel points in the original frame.

[0119] The at least two adjacent pixels are pixels included in the spatial noise reduction pixel group.

[0120] Taking the example of performing spatiotemporal fusion on two adjacent pixels each time during the image denoising process, i.e., the spatial denoising pixel group and the corresponding pixel group in the original frame include two adjacent pixels, assuming that the original frame is an 11 (row) * 10 (column) image frame, i.e., the original frame includes 11 * 10 pixels. Then, for a pixel group consisting of a pixel in the 5th row and 5th column and a pixel in the 5th row and 6th column in the original frame, after determining the spatial denoising pixel group corresponding to this pixel group, the fourth image block in the original frame can be determined, where the fourth image block includes a pixel in the 5th row and 5th column and a pixel in the 5th row and 6th column in the original frame.

[0121] It should be noted that this example uses the fourth image block as an example to illustrate the structural information of the pixels in the two horizontal directions, which only include the pixels corresponding to two adjacent pixels in the spatial noise reduction pixel group in the original frame. In practical applications, it can also be determined as needed that the fourth image block includes structural information in four horizontal and vertical cross directions, such as the fourth image block includes pixels in four directions of the upper, lower, left, and right sides of the pixels corresponding to two adjacent pixels in the spatial noise reduction pixel group in the original frame, or the fourth image block includes structural information in eight directions, such as the fourth image block includes pixels in eight directions of the upper, lower, left, right, left oblique upper, right oblique upper, left oblique lower, and right oblique lower sides of the pixels corresponding to two adjacent pixels in the spatial noise reduction pixel group in the original frame. The present disclosure does not limit this. Wherein, when the fourth image block includes pixels in four directions or eight directions, the pixels in the four directions and the pixels in the eight directions can all be subjected to spatial filtering, that is, the pixel values ​​of the pixels in the four directions and the pixels in the eight directions can all be pixel values ​​after spatial filtering.

[0122] It should be noted that, in the image denoising process, each time a pixel point in the original frame is subjected to spatiotemporal fusion, that is, when the spatial denoising pixel group includes only one pixel point, the fourth image block may include structural information in four directions of a horizontal and vertical cross, such as the fourth image block includes pixels in four directions of the upper side, lower side, left side and right side of the pixel point corresponding to one pixel point in the spatial denoising pixel group in the original frame, or the fourth image block includes structural information in eight directions, such as the fourth image block includes pixels in eight directions of the upper side, lower side, left side, right side, left oblique upper side, right oblique upper side, left oblique lower side and right oblique lower side of the pixel point corresponding to one pixel point in the spatial denoising pixel group in the original frame. The present disclosure does not impose any restrictions on this. Wherein, when the fourth image block includes pixels in four directions or eight directions, the pixels in the four directions and the pixels in the eight directions may all be subjected to spatial filtering, that is, the pixel values ​​of the pixels in the four directions and the pixels in the eight directions may all be pixel values ​​after spatial filtering.

[0123] Step 602: Update the pixel values ​​of corresponding pixels in the fourth image block using the pixel values ​​of at least two adjacent pixels to obtain a fifth image block.

[0124] The at least two adjacent pixels are pixels in the spatial domain noise reduction pixel group.

[0125] In one embodiment of the present disclosure, the pixel values ​​of each pixel in the spatial denoising pixel group can be used to update the pixel values ​​of corresponding pixels in the fourth image block. Specifically, the pixel values ​​of at least two pixels in the fourth image block are updated to the pixel values ​​of the corresponding pixels in the spatial denoising pixel group, thereby obtaining a fifth image block. Accordingly, the pixel values ​​of each pixel in the fifth image block are the pixel values ​​of the corresponding pixels in the original frame after spatial denoising.

[0126] Step 603: For any reference frame, determine the sixth image block corresponding to the position of the fourth image block in the reference frame, and determine the temporal structural similarity between the spatial denoising pixel group and the corresponding pixel group in the reference frame based on the structural similarity between the fifth image block and the sixth image block.

[0127] In one embodiment of the present disclosure, taking one of the reference frames as an example, a sixth image block corresponding to the position of the fourth image block in the reference frame can be determined, that is, the sixth image block has the same size as the fourth image block, and the position of the sixth image block in the reference frame is the same as the position of the fourth image block in the original frame, and the sixth image block includes pixels in the reference frame that correspond to at least two adjacent pixels in the spatial denoising pixel group. Furthermore, the structural similarity between the fifth image block and the sixth image block can be determined, and the structural similarity is determined as the temporal structural similarity between the spatial denoising pixel group and the corresponding pixel group in the reference frame. The structural similarity between the fifth image block and the sixth image block is used to characterize the degree of similarity between the fifth image block and the sixth image block in the local structure of the pixel points.

[0128] Taking the fifth image block including two adjacent pixels as an example, the structural similarity between the fifth image block and the sixth image block can be specifically characterized by the structural offset distance between the fifth image block and the sixth image block. The SAD (Sum of absolute differences, binocular stereo vision matching) algorithm can be used to determine the structural offset distance between the fifth image block and the sixth image block in the manner shown in formula (5):

[0129] diff'=MAX(ABS(P ref(x,y,k) -P cur(x,y) ),ABS(P ref(x,y+1,k) -P cur(x,y+1) )) (5)

[0130] Wherein, diff' represents the structural offset distance between the fifth image block and the sixth image block, which can represent the structural similarity between the fifth image block and the sixth image block; ABS represents the absolute value; MAX represents the maximum value; P ref(x,y,k) represents the pixel value of the pixel with coordinates (x, y) in the sixth image block in the k-th reference frame; P cur(x,y) represents the pixel value of the pixel point with coordinates (x, y) in the fifth image block; ref(x,y+1,k) represents the pixel value of the pixel with coordinates (x, y+1) in the sixth image block in the k-th reference frame; cur(x,y+1) Represents the pixel value of the pixel with coordinates (x, y+1) in the fifth image block.

[0131] That is, the process of determining the structural similarity between the fifth image block and the sixth image block can be: determining the difference between the pixel value of each pixel in the fifth image block and the pixel value of the corresponding pixel in the sixth image block; and determining the structural similarity between the fifth image block and the sixth image block based on the difference.

[0132] In an embodiment of the present disclosure, after determining the temporal structural similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame, the structural coefficient corresponding to the spatial denoised pixel group can be determined according to the temporal structural similarity between the spatial denoised pixel group and the corresponding pixel group in each reference frame by the method shown in the following steps 604-605.

[0133] Step 604 : Determine a target similarity in the temporal structural similarity between the spatially denoised pixel group and a corresponding pixel group in at least one reference frame.

[0134] In one embodiment of the present disclosure, taking the temporal structural similarity between the spatial denoised pixel group and the corresponding pixel group in any reference frame as represented by the structural offset distance shown in formula (5) as an example, step 604 can be implemented as shown in the following formula (6):

[0135] diff=MAX(ABS(P ref(x,y,k) -P cur(x,y) ),ABS(P ref(x,y+1,k) -P cur(x,y+1) ),...) (6)

[0136] Wherein, k=1,…K, K represents the number of reference frames.

[0137] That is, the maximum temporal structural similarity among the temporal structural similarities between the spatial denoised pixel group and the corresponding pixel group in at least one reference frame may be determined as the target similarity.

[0138] Step 605: Determine the structural coefficient according to the mapping relationship between the target similarity and the structural coefficient.

[0139] It can be understood that the smaller the structural offset distance between the fifth image block and the sixth image block, the greater the temporal structural similarity between the spatially denoised pixel group and the corresponding pixel group in the reference frame, the lower the motion intensity of each pixel in the corresponding pixel group in the original frame, and the higher the structural coefficient; the larger the structural offset distance between the fifth image block and the sixth image block, the smaller the temporal structural similarity between the spatially denoised pixel group and the corresponding pixel group in the reference frame, the higher the motion intensity of each pixel in the corresponding pixel group in the original frame, and the lower the structural coefficient. Based on this, a mapping relationship between the target similarity and the structural coefficient can be set, and the structural coefficient can be determined based on the target similarity according to this mapping relationship.

[0140] In one embodiment of the present disclosure, two thresholds, a fourth threshold and a fifth threshold, may be set, and the mapping relationship between the target similarity and the structural coefficient may be determined in combination with the two thresholds. For example, taking the temporal structural similarity between the spatial denoised pixel group and the corresponding pixel group in the reference frame as represented by the structural offset distance between the fifth image block and the sixth image block, the mapping relationship between the target similarity and the structural coefficient may be set as follows: Figure 7 The curve form shown.

[0141] in, Figure 7 Where thr4 represents the fourth threshold, thr5 represents the fifth threshold; diff represents the target similarity; w2 represents the structural coefficient; and k3 represents the slope.

[0142] Figure 7 The curve shown can be expressed as the following formula (7):

[0143]

[0144] Where b3 is a preset value and k3 represents the slope. b3 and k3 can be set and adjusted as needed. Substituting the target similarity diff into the values ​​of x and y in formula (7) is the structural coefficient w2.

[0145] Refer to formula (7) and Figure 7 As can be seen from the curve shown, when the target similarity diff is greater than or equal to the fifth threshold thr5, it can be determined that the motion intensity of each pixel point in the corresponding pixel group in the original frame is 1, that is, each pixel point in the corresponding pixel group in the original frame can be divided into a motion pixel point, and the structure coefficient w2 is 0 at this time; when the target similarity diff is less than or equal to the fourth threshold thr4, it can be determined that the motion intensity of each pixel point in the corresponding pixel group in the original frame is 0, that is, each pixel point in the corresponding pixel group in the original frame can be divided into a stationary pixel point, and the structure coefficient w2 is 1 at this time; when the target similarity diff is greater than the fourth threshold thr4 and less than the fifth threshold thr5, it can be determined that the motion intensity of each pixel point in the corresponding pixel group in the original frame is a value between 0-1, that is, each pixel point in the corresponding pixel group in the original frame can be divided into a pixel point transitioning between motion and stationary, and the structure coefficient w2 is a value between 0-1. According to the mapping relationship between the target similarity diff and the structure coefficient w2, the value of the structure coefficient w2 can be determined.

[0146] Thus, through a simple linear relationship, the structural coefficient can be easily determined based on the temporal structural similarity between the spatial denoising pixel group and the corresponding pixel group in each reference frame, and the fusion coefficient can then be obtained based on the structural coefficient. This reduces the amount of computation and significantly reduces the hardware resources required to determine the temporal similarity and fusion coefficient through hardware circuits, thereby reducing the cost of video noise reduction. Furthermore, by determining a lower structural coefficient when the temporal structural similarity between the spatial denoising pixel group and the corresponding pixel group in each reference frame is low, that is, when the motion intensity of each pixel in the corresponding pixel group in the original frame is high, the temporal denoising intensity can be reduced, thereby reducing the occurrence of motion artifacts and improving the noise reduction effect of the video.

[0147] In the disclosed embodiment, a fourth image block in an original frame is determined, the fourth image block including pixels corresponding to at least two adjacent pixels in the original frame. The pixel values ​​of the corresponding pixels in the fourth image block are updated using the pixel values ​​of the at least two adjacent pixels to obtain a fifth image block. For any reference frame, a sixth image block corresponding to the position of the fourth image block is determined in the reference frame. Based on the structural similarity between the fifth image block and the sixth image block, a temporal structural similarity between a spatially denoised pixel group and a corresponding pixel group in the reference frame is determined. A target similarity in the temporal structural similarity between the spatially denoised pixel group and a corresponding pixel group in at least one reference frame is determined. Based on a mapping relationship between the target similarity and the structural coefficient, a structural coefficient is determined. In this way, the structural coefficient is determined based on the structural similarity between the spatially denoised pixel group and the corresponding pixel group in the reference frame, and the motion intensity of each pixel in the corresponding pixel group in the original frame is determined. Furthermore, an appropriate noise reduction intensity is determined based on the motion intensity of each pixel in the original frame, thereby performing spatiotemporal noise reduction on the original frame and improving the noise reduction effect of the video. Moreover, by using the structural information of each pixel in the image block rather than the pixel value or grayscale value of a single pixel to determine the structural coefficient and the motion intensity of each pixel in the corresponding pixel group in the original frame, the influence of individual noise points is avoided, and the accuracy of the structural coefficient and the motion and stillness judgment of each pixel in the original frame is improved, thereby improving the accuracy of the determined fusion coefficient and further improving the noise reduction effect of the video.

[0148] In order to implement the above embodiments, the present disclosure also provides an image processing device.

[0149] Figure 8 A schematic structural diagram of an image processing device provided in an embodiment of the present disclosure.

[0150] like Figure 8 As shown, the image processing device 800 includes: a video frame acquisition module 810, a spatial noise reduction module 820, a noise reduction intensity determination module 830, a fusion noise reduction module 840 and an image generation module 850.

[0151] The video frame acquisition module 810 is configured to acquire an original frame in a video and at least one reference frame corresponding to the original frame, wherein the at least one reference frame includes a frame preceding the original frame, and the at least one reference frame is a noise-reduced image frame.

[0152] A spatial noise reduction module 820 is configured to perform spatial noise reduction on at least one pixel group in the original frame to obtain a corresponding spatial noise reduction pixel group, wherein the pixel group includes at least one pixel point;

[0153] a noise reduction strength determination module 830 for determining a corresponding fusion coefficient based on a temporal similarity between a spatially denoised pixel group and a corresponding pixel group in at least one reference frame;

[0154] A fusion noise reduction module 840 is configured to fuse the spatial noise reduction pixel group with the corresponding pixel group in the previous frame according to the corresponding fusion coefficient to obtain the corresponding target pixel group;

[0155] The image generation module 850 is configured to generate a target frame after noise reduction from the original frame according to a target pixel group corresponding to at least one pixel group in the original frame.

[0156] As a possible implementation of the embodiment of the present disclosure, the temporal similarity includes temporal content similarity and temporal structure similarity; the noise reduction strength determination module 830 includes:

[0157] a content coefficient determination unit, configured to determine a corresponding content coefficient based on a temporal content similarity between a spatial domain denoised pixel group and a corresponding pixel group in at least one reference frame;

[0158] a structural coefficient determination unit, configured to determine a corresponding structural coefficient based on a temporal structural similarity between a spatial denoised pixel group and a corresponding pixel group in at least one reference frame;

[0159] The coefficient fusion unit is used to fuse the content coefficient and the structure coefficient to obtain a fusion coefficient.

[0160] As a possible implementation of the embodiment of the present disclosure, the noise reduction strength determination module 830 further includes:

[0161] A first image block determining unit is configured to determine a first image block of a preset size in an original frame, wherein a central pixel point of the first image block includes a pixel point in a pixel group corresponding to a spatial domain noise reduction pixel group in the original frame;

[0162] A first pixel updating unit is configured to update the pixel value of the corresponding pixel point in the first image block by using the pixel value of each pixel point in the spatial denoising pixel group to obtain a second image block;

[0163] The first similarity determination unit is configured to determine, for any reference frame, a third image block corresponding to a position of the first image block in the reference frame, and determine, based on the content similarity between the second image block and the third image block, a temporal content similarity between the spatial denoising pixel group and a corresponding pixel group in the reference frame.

[0164] As a possible implementation of the embodiment of the present disclosure, the content coefficient determination unit is configured to:

[0165] determining an average of temporal content similarities between the spatially denoised pixel group and a corresponding pixel group in at least one reference frame;

[0166] The content coefficient is determined according to the mapping relationship between the average value and the content coefficient.

[0167] As a possible implementation of the embodiment of the present disclosure, the spatial denoising pixel group includes at least two adjacent pixel points; the denoising strength determination module 830 further includes:

[0168] a second image block determining unit, configured to determine a fourth image block in the original frame, where the fourth image block includes pixel points corresponding to at least two adjacent pixel points in the original frame;

[0169] a second pixel updating unit, configured to update the pixel values ​​of corresponding pixels in the fourth image block using the pixel values ​​of at least two adjacent pixels to obtain a fifth image block;

[0170] The second similarity determination unit is configured to determine, for any reference frame, a sixth image block corresponding to a position of the fourth image block in the reference frame, and determine, based on the structural similarity between the fifth image block and the sixth image block, a temporal structural similarity between the spatial denoising pixel group and a corresponding pixel group in the reference frame.

[0171] As a possible implementation of the embodiment of the present disclosure, the structural coefficient determination unit is configured to:

[0172] determining a target similarity in a temporal structural similarity between a spatially denoised pixel group and a corresponding pixel group in at least one reference frame;

[0173] The structural coefficient is determined according to the mapping relationship between the target similarity and the structural coefficient.

[0174] As a possible implementation of the embodiment of the present disclosure, the fusion noise reduction module 840 includes:

[0175] a weight determination unit, configured to determine, based on the fusion coefficient, a weight corresponding to each first pixel in the spatial denoising pixel group and a weight corresponding to each second pixel in the corresponding pixel group in the previous frame;

[0176] a pixel fusion unit, configured to fuse the pixel value of each first pixel point in the spatial denoising pixel group with the pixel value of the second pixel point at the corresponding position in the corresponding pixel group in the previous frame based on corresponding weights, to obtain a target pixel value of the target pixel point at the corresponding position in the target pixel group;

[0177] The pixel generating unit is used to generate a target pixel group according to the target pixel value of each target pixel point in the target pixel group.

[0178] The image processing device of the embodiment of the present disclosure first obtains the original frame in the video and at least one reference frame corresponding to the original frame, wherein the at least one reference frame includes the previous frame of the original frame, and the at least one reference frame is a denoised image frame, then performs spatial denoising on at least one pixel group in the original frame to obtain a corresponding spatial denoised pixel group, wherein the pixel group includes at least one pixel point, then determines a corresponding fusion coefficient based on the temporal similarity between the spatial denoised pixel group and the corresponding pixel group in the at least one reference frame, fuses the spatial denoised pixel group with the corresponding pixel group in the previous frame according to the corresponding fusion coefficient, obtains a corresponding target pixel group, and then generates a target frame after the original frame is denoised based on the target pixel group corresponding to the at least one pixel group in the original frame. In this way, different denoising intensities are used for temporal and spatial denoising of different pixel groups in the original frame, and the denoising intensities are also different for different original frames in the video, thereby avoiding the use of a fixed denoising intensity for denoising and improving the denoising effect of the video.

[0179] It should be noted that the aforementioned explanation of the embodiment of the image processing method is also applicable to the image processing device of this embodiment, and will not be repeated here.

[0180] In order to implement the above embodiments, the present disclosure also provides an imaging device. Figure 9 A schematic structural diagram of an imaging device provided by an embodiment of the present disclosure.

[0181] like Figure 9 As shown, the imaging device 900 may include:

[0182] An image acquisition device 910, for acquiring video; and

[0183] The image processing device 920 is used to perform noise reduction processing on the video.

[0184] The imaging device 900 may be any electronic device with an imaging function, such as a camera, a computer, or a monitoring device.

[0185] The image processing device 920 may be a chip, such as an ISP chip.

[0186] Image processing device 920 has the same structure and functions as image processing device 800. Image processing device 920 can use the image processing method described in any of the aforementioned embodiments to perform video noise reduction. The explanations of the image processing methods in the aforementioned embodiments also apply to image processing device 920 in this embodiment and are not further described here.

[0187] In order to implement the above embodiments, the present disclosure also proposes an electronic device, such as Figure 10 As shown, Figure 10 FIG. 1 is a block diagram of an electronic device for implementing image processing according to an exemplary embodiment. Figure 10 As shown, the electronic device 1000 may include:

[0188] The memory 1010 and the processor 1020, and the bus 1030 connecting different components (including the memory 1010 and the processor 1020), the memory 1010 stores a computer program, and when the processor 1020 executes the program, the image processing method described in the embodiment of the present disclosure is implemented.

[0189] Bus 1030 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0190] The electronic device 1000 typically includes a variety of computer-readable media. These media can be any available media that can be accessed by the electronic device 1000, including volatile and non-volatile media, removable and non-removable media.

[0191] The memory 1010 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 1040 and / or cache memory 1050. The electronic device 1000 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 1060 may be used to read and write non-removable, non-volatile magnetic media ( Figure 10 Not shown, often called a "hard drive"). Although Figure 10Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 1030 via one or more data medium interfaces. Memory 1010 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present disclosure.

[0192] A program / utility 1080 having a set (at least one) of program modules 1070 may be stored, for example, in memory 1010. Such program modules 1070 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 1070 generally implement the functions and / or methods of the embodiments described herein.

[0193] The electronic device 1000 may also communicate with one or more external devices 1090 (e.g., a keyboard, a pointing device, a display 1091, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 1000, and / or any device that enables the electronic device 1000 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed through an input / output (I / O) interface 1092. Furthermore, the electronic device 1000 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 1093. Figure 10 As shown, the network adapter 1093 communicates with other modules of the electronic device 1000 via the bus 1030. Figure 10 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 1000, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0194] The processor 1020 executes various functional applications and data processing by running programs stored in the memory 1010 .

[0195] In order to implement the above embodiments, the present disclosure further provides a non-transitory computer-readable storage medium storing computer instructions, the computer instructions being used to enable a computer to execute the present disclosure. Figures 1 to 7 The method described in the embodiment.

[0196] In order to implement the above embodiments, the present disclosure also proposes a computer program product, including a computer program, which implements the present disclosure when executed by a processor. Figures 1 to 7 The method described in the embodiment.

[0197] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and features of different embodiments or examples, unless they are mutually inconsistent.

[0198] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0199] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0200] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0201] It should be understood that various parts of the present disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0202] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0203] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0204] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. A person of ordinary skill in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. An image processing device, characterized in that: include: A video frame acquisition module, configured to acquire an original frame in a video and at least one reference frame corresponding to the original frame, wherein the at least one reference frame includes a frame preceding the original frame, and the at least one reference frame is a denoised image frame; a spatial noise reduction module, configured to perform spatial noise reduction on at least one pixel group in the original frame to obtain a corresponding spatial noise reduction pixel group, wherein the pixel group includes at least one pixel point; a noise reduction strength determination module, configured to determine a corresponding fusion coefficient based on a temporal similarity between the spatially denoised pixel group and a corresponding pixel group in the at least one reference frame; A fusion noise reduction module is used to fuse the spatial domain noise reduction pixel group with the corresponding pixel group in the previous frame according to the corresponding fusion coefficient to obtain a corresponding target pixel group; The image generation module is configured to generate a target frame after noise reduction of the original frame according to a target pixel group corresponding to the at least one pixel group in the original frame.

2. The device according to claim 1, characterized in that The temporal similarity includes temporal content similarity and temporal structure similarity; the noise reduction strength determination module includes: a content coefficient determination unit, configured to determine a corresponding content coefficient based on the temporal content similarity between the spatial domain denoised pixel group and a corresponding pixel group in the at least one reference frame; a structural coefficient determining unit, configured to determine a corresponding structural coefficient according to the temporal structural similarity between the spatial denoised pixel group and a corresponding pixel group in the at least one reference frame; The coefficient fusion unit is used to fuse the content coefficient and the structure coefficient to obtain the fusion coefficient.

3. The device according to claim 2, characterized in that The noise reduction intensity determination module further includes: a first image block determining unit, configured to determine a first image block of a preset size in the original frame, wherein a central pixel point of the first image block includes a pixel point in a pixel group in the original frame corresponding to the spatial domain noise reduction pixel group; a first pixel updating unit, configured to update the pixel value of the corresponding pixel point in the first image block by using the pixel value of each pixel point in the spatial denoising pixel group, to obtain a second image block; a first similarity determination unit, configured to determine, for any one of the reference frames, a third image block corresponding to a position of the first image block in the reference frame, and determine, based on content similarity between the second image block and the third image block, the temporal content similarity between the spatial denoising pixel group and a corresponding pixel group in the reference frame.

4. The device according to claim 2 or 3, characterized in that The content coefficient determination unit is configured to: determining an average value of the temporal content similarity between the spatially denoised pixel group and a corresponding pixel group in the at least one reference frame; The content coefficient is determined according to a mapping relationship between the average value and the content coefficient.

5. The device according to claim 2, characterized in that The spatial denoising pixel group includes at least two adjacent pixel points; the denoising strength determination module further includes: a second image block determining unit, configured to determine a fourth image block in the original frame, wherein the fourth image block includes pixel points in the original frame corresponding to the at least two adjacent pixel points; a second pixel updating unit, configured to update the pixel values ​​of corresponding pixels in the fourth image block using the pixel values ​​of the at least two adjacent pixels to obtain a fifth image block; a second similarity determination unit, configured to determine, for any one of the reference frames, a sixth image block corresponding to a position of the fourth image block in the reference frame, and determine, based on the structural similarity between the fifth image block and the sixth image block, the temporal structural similarity between the spatial denoising pixel group and the corresponding pixel group in the reference frame.

6. The device according to claim 2 or 5, characterized in that The structural coefficient determination unit is used to: Determining a target similarity in the temporal structural similarity between the spatial denoised pixel group and a corresponding pixel group in the at least one reference frame; The structural coefficient is determined according to a mapping relationship between the target similarity and the structural coefficient.

7. The device according to any one of claims 1 to 3, characterized in that The fusion noise reduction module includes: a weight determination unit, configured to determine, based on the fusion coefficient, a weight corresponding to each first pixel in the spatial denoising pixel group and a weight corresponding to each second pixel in the corresponding pixel group in the previous frame; a pixel fusion unit, configured to fuse the pixel value of each first pixel point in the spatial denoising pixel group with the pixel value of the second pixel point at the corresponding position in the corresponding pixel group in the previous frame based on corresponding weights to obtain a target pixel value of the target pixel point at the corresponding position in the target pixel group; The pixel generating unit is configured to generate the target pixel group according to the target pixel value of each target pixel point in the target pixel group.

8. An image processing method, characterized in that: The following steps are involved: Acquire an original frame in a video and at least one reference frame corresponding to the original frame, wherein the at least one reference frame includes a frame preceding the original frame, and the at least one reference frame is a noise-reduced image frame; Performing spatial noise reduction on at least one pixel group in the original frame to obtain a corresponding spatial noise reduction pixel group, wherein the pixel group includes at least one pixel point; Determining a corresponding fusion coefficient based on a temporal similarity between the spatial denoised pixel group and a corresponding pixel group in the at least one reference frame; Fusing the spatial domain denoised pixel group with the corresponding pixel group in the previous frame according to the corresponding fusion coefficient to obtain a corresponding target pixel group; A target frame after noise reduction of the original frame is generated according to the target pixel group corresponding to the at least one pixel group in the original frame.

9. An imaging device, characterized in that include: An image acquisition device, for acquiring video; as well as The image processing device according to any one of claims 1 to 7, configured to perform noise reduction processing on the video.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to claim 8.

11. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to claim 8.

Images