Image denoising method and device, terminal, base station and chip

Abstract

The application relates to an image denoising method and device, a terminal, a base station and a chip. The method comprises the following steps: determining a fusion weight of a first target image layer corresponding to multiple images according to pixel differences between the multiple images, wherein the multiple images represent multiple images continuously captured; the first target image layer comprises a first image layer corresponding to each image; determining a fusion weight of each target image layer corresponding to the multiple images based on the fusion weight of the first target image layer; respectively performing fusion processing on each target image layer according to the fusion weight of each target image layer to obtain a fusion image layer of each target image layer; performing multiple fusion processing on the fusion image layer of each target image layer to obtain a processed image layer corresponding to the multiple images; and determining a denoised image corresponding to the multiple images according to the processed image layer. The method can improve the image denoising capability, effectively suppress noise and retain image details.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image denoising method, apparatus, terminal, base station, and chip. Background Technology

[0002] Currently, in order to improve image quality, efficient image denoising is of paramount importance.

[0003] In traditional techniques, single-frame denoising is generally used when denoising images; however, this single-frame denoising method has low image denoising capability and is difficult to effectively suppress noise and preserve image details. Summary of the Invention

[0004] Therefore, it is necessary to provide an image denoising method, apparatus, terminal, base station, and chip that can improve image denoising capabilities, effectively suppress noise, and preserve image details, in order to address the aforementioned technical problems.

[0005] In a first aspect, this application provides an image denoising method, comprising:

[0006] Based on the pixel differences between multiple frames of images, the fusion weights of the first target image layer corresponding to the multiple frames of images are determined; the multiple frames of images are used to represent images captured in multiple consecutive frames; the first target image layer includes the first image layer corresponding to each frame of image;

[0007] Based on the fusion weights of the first target image layer, the fusion weights of each target image layer corresponding to the multi-frame images are determined;

[0008] Each target image layer is fused according to its fusion weight to obtain a fused image layer for each target image layer.

[0009] The fusion image layer of each target image layer is subjected to multiple fusion processes to obtain the processed image layer corresponding to the multiple frames of images;

[0010] Based on the processed image layer, the denoised image corresponding to the multi-frame image is determined.

[0011] In one embodiment, determining the fusion weights for each target image layer corresponding to the multi-frame images based on the fusion weights of the first target image layer includes:

[0012] The fusion weights of the first target image layer are upsampled to obtain the processed fusion weights;

[0013] Based on the processed fusion weights, the fusion weights of the next target image layer of the first target image layer are determined;

[0014] The next target image layer is taken as the new first target image layer, and the process jumps to the step of upsampling the fusion weights of the first target image layer to obtain the processed fusion weights, until the fusion weights of the next target image layer are the fusion weights of the last target image layer corresponding to the multi-frame images.

[0015] In one embodiment, the step of performing multiple fusion processes on the fused image layer for each target image layer to obtain the processed image layer corresponding to the multiple frames of images includes:

[0016] The fused image layer of the first target image layer is upsampled to obtain the first processed fused image layer;

[0017] The first processed fused image layer is fused with the fused image layer of the next target image layer of the first target image layer to obtain the second processed fused image layer of the next target image layer.

[0018] The second processed fused image layer is used as the fused image layer of the new first target image layer, and the process jumps to the step of upsampling the fused image layer of the first target image layer to obtain the first processed fused image layer, until the obtained second processed fused image layer is the second processed fused image layer of the last target image layer corresponding to the multi-frame image. Then the obtained second processed fused image layer is used as the processed image layer corresponding to the multi-frame image.

[0019] In one embodiment, the upsampling process performed on the fused image layer of the first target image layer to obtain a first processed fused image layer includes:

[0020] Denoising is performed on the fused image layer of the first target image layer to obtain the denoised fused image layer of the first target image layer.

[0021] The denoised and fused image layer is enhanced to obtain the enhanced fused image layer of the first target image layer;

[0022] The enhanced fused image layer is upsampled to obtain the first processed fused image layer.

[0023] In one embodiment, determining the denoised image corresponding to the multi-frame image based on the processed image layer includes:

[0024] The processed image layer is then subjected to denoising processing to obtain a denoised image layer;

[0025] The denoised image layer is then enhanced to obtain the enhanced image layer.

[0026] Based on the enhanced image layer, the denoised image corresponding to the multi-frame image is obtained.

[0027] In one embodiment, determining the fusion weights of the first target image layer corresponding to the multiple frames of images based on the pixel differences between the multiple frames includes:

[0028] Identify the reference image in each frame;

[0029] The weights of the other images are determined based on the pixel differences between the reference image and other images; the other images are used to represent the images other than the reference image in each frame.

[0030] Based on the base weights of the reference image, the fusion weights of the first image layer corresponding to the reference image are obtained, and based on the weights of the other images, the fusion weights of the first image layer corresponding to the other images are obtained;

[0031] Based on the fusion weights of the first image layer corresponding to the reference image and the fusion weights of the first image layers corresponding to the other images, the fusion weights of the first target image layer corresponding to the multi-frame images are obtained.

[0032] Secondly, this application also provides an image denoising apparatus, comprising:

[0033] The first weight determination module is used to determine the fusion weight of the first target image layer corresponding to the multiple frames of images based on the pixel differences between the multiple frames of images; the multiple frames of images are used to represent images captured in multiple consecutive frames; the first target image layer includes the first image layer corresponding to each frame of image;

[0034] The second weight determination module is used to determine the fusion weight of each target image layer corresponding to the multi-frame images based on the fusion weight of the first target image layer.

[0035] The first image fusion module is used to perform fusion processing on each target image layer according to the fusion weight of each target image layer to obtain a fused image layer of each target image layer;

[0036] The second image fusion module is used to perform multiple fusion processes on the fused image layer of each target image layer to obtain the processed image layer corresponding to the multiple frames of images.

[0037] The denoised image determination module is used to determine the denoised image corresponding to the multi-frame image based on the processed image layer.

[0038] Thirdly, this application also provides a terminal, including a memory, a processor, and a camera, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0039] Based on the pixel differences between multiple frames of images, the fusion weights of the first target image layer corresponding to the multiple frames of images are determined; the multiple frames of images are used to represent images captured in multiple consecutive frames; the first target image layer includes the first image layer corresponding to each frame of image;

[0040] Based on the fusion weights of the first target image layer, the fusion weights of each target image layer corresponding to the multi-frame images are determined;

[0041] Each target image layer is fused according to its fusion weight to obtain a fused image layer for each target image layer.

[0042] The fusion image layer of each target image layer is subjected to multiple fusion processes to obtain the processed image layer corresponding to the multiple frames of images;

[0043] Based on the processed image layer, the denoised image corresponding to the multi-frame image is determined.

[0044] Fourthly, this application also provides a base station, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0045] Based on the pixel differences between multiple frames of images, the fusion weights of the first target image layer corresponding to the multiple frames of images are determined; the multiple frames of images are used to represent images captured in multiple consecutive frames; the first target image layer includes the first image layer corresponding to each frame of image;

[0046] Based on the fusion weights of the first target image layer, the fusion weights of each target image layer corresponding to the multi-frame images are determined;

[0047] Each target image layer is fused according to its fusion weight to obtain a fused image layer for each target image layer.

[0048] The fusion image layer of each target image layer is subjected to multiple fusion processes to obtain the processed image layer corresponding to the multiple frames of images;

[0049] Based on the processed image layer, the denoised image corresponding to the multi-frame image is determined.

[0050] Fifthly, this application also provides a chip, the chip including at least one processor, which executes a computer program to perform the following steps:

[0051] Based on the pixel differences between multiple frames of images, the fusion weights of the first target image layer corresponding to the multiple frames of images are determined; the multiple frames of images are used to represent images captured in multiple consecutive frames; the first target image layer includes the first image layer corresponding to each frame of image;

[0052] Based on the fusion weights of the first target image layer, the fusion weights of each target image layer corresponding to the multi-frame images are determined;

[0053] Each target image layer is fused according to its fusion weight to obtain a fused image layer for each target image layer.

[0054] The fusion image layer of each target image layer is subjected to multiple fusion processes to obtain the processed image layer corresponding to the multiple frames of images;

[0055] Based on the processed image layer, the denoised image corresponding to the multi-frame image is determined.

[0056] The aforementioned image denoising method, apparatus, terminal, base station, and chip first determine the fusion weights of the first target image layer corresponding to the multiple frames of images based on the pixel differences between the multiple frames of images. The multiple frames of images represent images captured consecutively. The first target image layer includes the first image layer corresponding to each frame of images. Then, based on the fusion weights of the first target image layer, the fusion weights of each target image layer corresponding to the multiple frames of images are determined. Next, each target image layer is fused according to its fusion weight to obtain a fused image layer for each target image layer. Then, the fused image layer of each target image layer is fused multiple times to obtain the processed image layer corresponding to the multiple frames of images. Finally, the denoised image corresponding to the multiple frames of images is determined based on the processed image layer. In this way, when denoising an image, the fusion weights of the first target image layer corresponding to multiple frames can be accurately determined based on the pixel differences between the multiple frames. Reusing these fusion weights allows for efficient determination of the fusion weights of each target image layer across the multiple frames. By performing fusion processing on each target image layer according to its respective fusion weight, the fused image layer of each target image layer can be obtained more accurately. Furthermore, multiple fusion processes can be performed on each target image layer to more accurately obtain the processed image layer corresponding to the multiple frames. Based on the processed image layer, the denoised image corresponding to the multiple frames can be determined more accurately, which helps improve the denoising quality of the denoised image, thereby enhancing the image denoising capability, effectively suppressing noise and preserving image details. Moreover, the entire process employs a multi-frame denoising approach, avoiding the shortcomings of traditional single-frame denoising methods, which have lower denoising capabilities and struggle to effectively suppress noise and preserve image details. This further improves the image denoising capability, effectively suppressing noise and preserving image details. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a flowchart illustrating an image denoising method in one embodiment;

[0059] Figure 2 This is a schematic diagram of the multi-frame pyramid fusion denoising process in one embodiment;

[0060] Figure 3This is a flowchart illustrating the steps for determining the fusion weights of each target image layer corresponding to multiple frames in one embodiment.

[0061] Figure 4 This is a flowchart illustrating the steps for obtaining the processed image layer corresponding to multiple frames of images in one embodiment.

[0062] Figure 5 This is a flowchart illustrating an image denoising method in another embodiment;

[0063] Figure 6 This is a structural block diagram of an image denoising device in one embodiment;

[0064] Figure 7 This is a diagram of the internal structure of a terminal in one embodiment. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0066] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0067] Before introducing the specific embodiments of this application, the technical terms involved in this application will be explained:

[0068] YUV (Luminance, Chrominance): YUV is a color coding mode where Y represents luminance (grayscale value), and UV represents chrominance (color value) and chroma (color density) respectively.

[0069] NLM (Non-Local Means): The Non-Local Means filtering algorithm is a classic image denoising technique. Its core idea is to use redundant information in the image, calculate the similarity between pixel blocks, and perform a weighted average of similar pixel blocks to remove noise and preserve the detailed features of the image.

[0070] ISO (International Organization for Standardization Sensitivity): Sensitivity, also known as ISO, refers to the camera's sensitivity to light.

[0071] In one exemplary embodiment, such as Figure 1 As shown, an image denoising method is provided. This embodiment illustrates the application of this method to a server. It is understood that this method can also be applied to a terminal, or to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. The terminal refers to a digital device with a camera function, which can be, but is not limited to, various personal computers, laptops, smartphones, tablets, digital SLR cameras, and other digital devices with photo and video recording capabilities. The server can be a standalone server or a server cluster composed of multiple servers. In this embodiment, the method includes the following steps:

[0072] Step S101: Determine the fusion weights of the first target image layer corresponding to the multiple frames based on the pixel differences between the multiple frames.

[0073] In this context, "multi-frame images" refers to images captured consecutively. It should be noted that this application employs a multi-frame denoising method. Multi-frame denoising is an image processing technique designed to reduce noise and improve image quality by simultaneously utilizing multiple consecutively captured images. On mobile devices, due to limitations of camera sensors and uncontrollable shooting conditions, images often contain various types of noise. Traditional single-frame denoising algorithms typically only process the currently captured image, failing to fully utilize information from multiple consecutively captured images. Multi-frame denoising algorithms, however, fuse multiple images, utilizing the correlation within the image sequence from a temporal perspective to reduce noise and improve image quality. The basic principle of multi-frame denoising algorithms is to extract an image with a higher signal-to-noise ratio by aligning and fusing multiple images. The advantage of multi-frame denoising algorithms lies in their ability to effectively reduce noise in images, especially high-frequency noise, thereby improving image clarity and detail. It is widely used on mobile devices such as digital cameras, smartphones, and camcorders, providing users with better image quality and shooting experience. For example... Figure 2 As shown, the multiple frames are image input1, image input2, image input3, and image input4.

[0074] Pixel difference is used to represent the difference in pixel value at the same pixel location in multiple frames of images.

[0075] The target image layer represents the set of image layers in the image pyramid corresponding to multiple frames. Specifically, the first target image layer includes the first image layer corresponding to each frame; the second target image layer includes the second image layer corresponding to each frame; the third target image layer includes the third image layer corresponding to each frame; and the fourth target image layer includes the fourth image layer corresponding to each frame. For example... Figure 2 As shown, each frame contains images input1, input2, input3, and input4 (all of size h×w). The first image layer corresponding to input1 is image layer G31 (h / 8×w / 8), the first image layer corresponding to input2 is image layer G32 (h / 8×w / 8), the first image layer corresponding to input3 is image layer G33 (h / 8×w / 8), and the first image layer corresponding to input4 is image layer G34 (h / 8×w / 8).

[0076] The fusion weight is also called the fusion weight plane. It's important to note that frames with smaller pixel differences are assigned larger fusion weights, while frames with larger pixel differences are assigned smaller fusion weights. For example... Figure 2 As shown, the fusion weights corresponding to image layer G31 (h / 8×w / 8) are W31 (h / 8×w / 8), image layer G32 (h / 8×w / 8) are W32 (h / 8×w / 8), image layer G33 (h / 8×w / 8) are W33 (h / 8×w / 8), and image layer G34 (h / 8×w / 8) are W34 (h / 8×w / 8).

[0077] For example, the server determines the similarity between multiple frames of images based on the pixel differences between them; then, the server queries the correspondence between similarity and weight to obtain the weight of the similarity between the multiple frames of images; then, the server uses the weight of the similarity between the multiple frames of images as the fusion weight of the first target image layer corresponding to the multiple frames of images.

[0078] Step S102: Based on the fusion weight of the first target image layer, determine the fusion weight of each target image layer corresponding to multiple frames of images.

[0079] For example, the server determines the hierarchical relationship between each target image layer corresponding to multiple frames and the first target image layer. Then, based on the fusion weight of the first target image layer and the hierarchical relationship between each target image layer and the first target image layer, the server determines the fusion weight of each target image layer corresponding to multiple frames. For instance, the fusion weight of the second target image layer corresponding to multiple frames is specifically obtained by upsampling the fusion weight of the first target image layer corresponding to multiple frames; the fusion weight of the third target image layer corresponding to multiple frames is specifically obtained by upsampling the fusion weight of the second target image layer corresponding to multiple frames; and the fusion weight of the fourth target image layer corresponding to multiple frames is specifically obtained by upsampling the fusion weight of the third target image layer corresponding to multiple frames.

[0080] For example, such as Figure 2As shown, the image pyramid corresponding to image input1 includes the first image layer G31 (h / 8×w / 8), the second image layer L21 (h / 4×w / 4), the third image layer L11 (h / 2×w / 2), and the fourth image layer L01 (h×w); the image pyramid corresponding to image input2 includes the first image layer G32 (h / 8×w / 8), the second image layer L22 (h / 4×w / 4), the third image layer L12 (h / 2×w / 2), and the fourth image layer L02 (h×w). The image pyramid corresponding to image input3 includes the first image layer G33 (h / 8×w / 8), the second image layer L23 (h / 4×w / 4), the third image layer L13 (h / 2×w / 2), and the fourth image layer L03 (h×w); the image pyramid corresponding to image input4 includes the first image layer G34 (h / 8×w / 8), the second image layer L24 (h / 4×w / 4), the third image layer L14 (h / 2×w / 2), and the fourth image layer L04 (h×w). Taking the image pyramid corresponding to image input1 as an example, the fusion weight W21(h / 4×w / 4) of the second image layer L21(h / 4×w / 4) is obtained by upsampling the fusion weight W31(h / 8×w / 8) of the first image layer G31(h / 8×w / 8); the fusion weight W11(h / 2×w / 2) of the third image layer L11(h / 2×w / 2) is obtained by upsampling the fusion weight W21(h / 4×w / 4) of the second image layer L21(h / 4×w / 4); and the fusion weight W01(h×w) of the fourth image layer L01(h×w) is obtained by upsampling the fusion weight W11(h / 2×w / 2) of the third image layer L11(h / 2×w / 2). It should be noted that the method for determining the fusion weights of each target image layer corresponding to image input2, image input3, and image input4 is similar to that of image input1, and will not be repeated here.

[0081] Step S103: Perform fusion processing on each target image layer according to the fusion weight of each target image layer to obtain the fused image layer of each target image layer.

[0082] The fused image layer refers to the image layer obtained by fusing each target image layer according to its fusion weights. It should be noted that there is only one fused image layer for each target image layer.

[0083] For example, the server performs a weighted summation on each target image layer according to the fusion weight of each target image layer to obtain a fused image layer for each target image layer.

[0084] For example, the fused image layer of the first target image layer can be calculated using the following formula:

[0085] Equation (1)

[0086] in, This refers to the fused image layer of the first target image layer. This refers to the fusion weights of the first target image layer. This refers to the weight of the reference frame in the first target image layer, which is fixed at 255 by default. This refers to the first image layer corresponding to the reference frame in the first target image layer. It refers to the first image layer corresponding to all frames in the first target image layer except the reference frame.

[0087] For example, the fused image layer of the second target image layer can be calculated using the following formula:

[0088] Equation (2)

[0089] in, This refers to the fused image layer of the second target image layer. This refers to the fusion weights of the second target image layer. This refers to the weights of the reference frame in the second target image layer; This refers to the second image layer corresponding to the reference frame in the second target image layer. This refers to the second image layer corresponding to all frames in the second target image layer other than the reference frame.

[0090] Equation (3)

[0091] in, This refers to the fused image layer of the third target image layer. This refers to the fusion weights of the third target image layer. This refers to the weights of the reference frames in the third target image layer; This refers to the third image layer corresponding to the reference frame in the third target image layer. This refers to the third image layer corresponding to all frames other than the reference frame in the third target image layer.

[0092] Equation (4)

[0093] in, This refers to the fused image layer of the fourth target image layer. This refers to the fusion weights of the fourth target image layer. This refers to the weights of the reference frames in what is also known as the fourth target image layer; This refers to the fourth image layer corresponding to the reference frame in the fourth target image layer. This refers to the fourth image layer corresponding to all frames other than the reference frame in the fourth target image layer.

[0094] For example, see reference. Figure 2 Taking the first target image layer corresponding to multiple frames as an example, the image layers G31(h / 8×w / 8), G32(h / 8×w / 8), G33(h / 8×w / 8), and G34(h / 8×w / 8) are fused based on the fusion weights W31(h / 8×w / 8), G32(h / 8×w / 8), G33(h / 8×w / 8), and G34(h / 8×w / 8) to obtain the fused image layer of the first target image layer. It should be noted that the determination methods for the fused image layers of the second, third, and fourth target image layers are similar to those for the fused image layer of the first target image layer, and will not be repeated here.

[0095] Step S104: Perform multiple fusion processes on the fused image layer of each target image layer to obtain the processed image layer corresponding to multiple frames of images.

[0096] The processed image layer refers to the image layer obtained by performing multiple fusion processes on the fused image layer of each target image layer.

[0097] For example, the server determines the fused image layer of the previous target image layer corresponding to each target image layer; then, the server performs summation processing on the fused image layer of each target image layer and the fused image layer of the previous target image layer corresponding to each target image layer to obtain the summed image layer of each target image layer, until each target image layer is the last target image layer corresponding to multiple frames of images, then the summed image layer of the last target image layer is used as the processed image layer corresponding to multiple frames of images.

[0098] Step S105: Determine the denoised images corresponding to multiple frames based on the processed image layers.

[0099] Among them, the denoised image is used to represent the denoising result corresponding to multiple frames of images.

[0100] For example, the server preprocesses the processed image layer to obtain a preprocessed processed image layer; then, the server uses the preprocessed processed image layer as the denoised image corresponding to the multi-frame image.

[0101] In the above image denoising method, the fusion weights of the first target image layer corresponding to the multi-frame images are first determined based on the pixel differences between the multi-frame images. The multi-frame images represent images captured in multiple consecutive frames. The first target image layer includes the first image layer corresponding to each frame image. Then, based on the fusion weights of the first target image layer, the fusion weights of each target image layer corresponding to the multi-frame images are determined. Next, each target image layer is fused according to its fusion weight to obtain a fused image layer for each target image layer. Then, the fused image layer of each target image layer is fused multiple times to obtain the processed image layer corresponding to the multi-frame images. Finally, the denoised image corresponding to the multi-frame images is determined based on the processed image layer. In this way, when denoising an image, the fusion weights of the first target image layer corresponding to multiple frames can be accurately determined based on the pixel differences between the multiple frames. Reusing these fusion weights allows for efficient determination of the fusion weights of each target image layer across the multiple frames. By performing fusion processing on each target image layer according to its respective fusion weight, the fused image layer of each target image layer can be obtained more accurately. Furthermore, multiple fusion processes can be performed on each target image layer to more accurately obtain the processed image layer corresponding to the multiple frames. Based on the processed image layer, the denoised image corresponding to the multiple frames can be determined more accurately, which helps improve the denoising quality of the denoised image, thereby enhancing the image denoising capability, effectively suppressing noise and preserving image details. Moreover, the entire process employs a multi-frame denoising approach, avoiding the shortcomings of traditional single-frame denoising methods, which have lower denoising capabilities and struggle to effectively suppress noise and preserve image details. This further improves the image denoising capability, effectively suppressing noise and preserving image details.

[0102] In one exemplary embodiment, such as Figure 3 As shown, step S102 above, based on the fusion weights of the first target image layer, determines the fusion weights of each target image layer corresponding to multiple frames of images, specifically including the following steps:

[0103] Step S301: Upsample the fusion weights of the first target image layer to obtain the processed fusion weights.

[0104] Step S302: Based on the processed fusion weights, determine the fusion weights of the next target image layer of the first target image layer.

[0105] Step S303: Take the next target image layer as the new first target image layer, and jump to the step of upsampling the fusion weights of the first target image layer to obtain the processed fusion weights, until the obtained fusion weights of the next target image layer are the fusion weights of the last target image layer corresponding to the multi-frame images.

[0106] Among them, the post-processing fusion weight refers to the fusion weight of the first target image layer after upsampling.

[0107] The next target image layer represents a target image layer with a higher resolution than the first target image layer and that is directly adjacent to the first target image layer. For example, see [link to example]. Figure 2 The next target image layer after image layer G31 (h / 8×w / 8) is image layer L21 (h / 4×w / 4), the next target image layer after image layer L21 (h / 4×w / 4) is image layer L11 (h / 2×w / 2), and the next target image layer after image layer L11 (h / 2×w / 2) is image layer L01 (h×w).

[0108] The last target image layer represents the highest-resolution target image layer in the image pyramid, whose size is identical to that of the multiple frames. For example, see [reference needed]. Figure 2 The last target image layer is image layer L01(h×w).

[0109] For example, the server upsamples the fusion weights of the first target image layer using an interpolation algorithm to obtain processed fusion weights. Then, the server uses the processed fusion weights as the fusion weights of the next target image layer based on the first target image layer. Next, the server judges the fusion weights of the next target image layer. If the fusion weights of the next target image layer are not the same as the fusion weights of the last target image layer corresponding to the multi-frame images, the server uses the next target image layer as the new first target image layer and jumps to the step of upsampling the fusion weights of the first target image layer to obtain processed fusion weights, until the obtained fusion weights of the next target image layer are the same as the fusion weights of the last target image layer corresponding to the multi-frame images.

[0110] In this embodiment, by calculating the fusion weight only once in the first low-resolution target image layer, and then reusing the weight in all subsequent high-resolution layers through upsampling, the waste of computing power caused by repeated calculations is avoided, and the processing efficiency is significantly improved. This not only greatly reduces the computational complexity of multi-frame denoising, but also ensures the consistency and accuracy of the fusion weights of the target image layers across all scales.

[0111] In one exemplary embodiment, such as Figure 4 As shown, step S104 above involves performing multiple fusion processes on the fused image layer of each target image layer to obtain the processed image layer corresponding to multiple frames of images. Specifically, it includes the following steps:

[0112] Step S401: Upsample the fused image layer of the first target image layer to obtain the first processed fused image layer.

[0113] Step S402: The first processed fused image layer is fused with the fused image layer of the next target image layer of the first target image layer to obtain the second processed fused image layer of the next target image layer.

[0114] Step S403: The second processed fused image layer is used as the fused image layer of the new first target image layer, and the process jumps to the step of upsampling the fused image layer of the first target image layer to obtain the first processed fused image layer. This process continues until the obtained second processed fused image layer is the second processed fused image layer of the last target image layer corresponding to the multi-frame image. Then, the obtained second processed fused image layer is used as the processed image layer corresponding to the multi-frame image.

[0115] The first post-processed fused image layer refers to the fused image layer of the first target image layer after upsampling.

[0116] The second post-processed fused image layer refers to the image layer obtained by fusing the first post-processed fused image layer with the fused image layer of the next target image layer of the first target image layer. In practical scenarios, the second post-processed fused image layer is also called the reconstructed Gaussian image layer.

[0117] For example, the server upsamples the fused image layer of the first target image layer using an interpolation algorithm to obtain a first processed fused image layer. Then, the server sums the weights of the first processed fused image layer and the fused image layer of the next target image layer according to the weights of the first processed fused image layer and the weights of the fused image layer of the next target image layer, to obtain a second processed fused image layer of the next target image layer. Next, the server judges the obtained second processed fused image layer. If the obtained second processed fused image layer is not the second processed fused image layer of the last target image layer corresponding to the multi-frame images, the server uses the second processed fused image layer as the new fused image layer of the first target image layer and jumps to the step of upsampling the fused image layer of the first target image layer to obtain the first processed fused image layer, until the obtained second processed fused image layer is the second processed fused image layer of the last target image layer corresponding to the multi-frame images. Then, the obtained second processed fused image layer is used as the processed image layer corresponding to the multi-frame images.

[0118] In this embodiment, by repeatedly performing upsampling and fusion processing on the first low-resolution target image layer and the fusion image layer, there is no need to recalculate the fusion results for each layer. This greatly reduces redundant calculations in multi-frame fusion, while ensuring the continuity of the low-resolution layer structure when transferring to the high-resolution layer, avoiding information gaps between layers, and improving the denoising effect of the image.

[0119] In an exemplary embodiment, step S401, which involves upsampling the fused image layer of the first target image layer to obtain a first processed fused image layer, specifically includes the following: denoising the fused image layer of the first target image layer to obtain a denoised fused image layer of the first target image layer; enhancing the denoised fused image layer to obtain an enhanced fused image layer of the first target image layer; and upsampling the enhanced fused image layer to obtain the first processed fused image layer.

[0120] Among them, noise reduction processing can refer to spatial noise reduction processing.

[0121] Among them, the denoised fused image layer refers to the fused image layer of the first target image layer after denoising.

[0122] Among these, enhancement processing can refer to detail enhancement processing.

[0123] Among them, the enhanced fused image layer refers to the denoised fused image layer after enhancement processing.

[0124] For example, the server identifies the noise type of each image layer region in the fused image layer of the first target image layer, and determines the spatial denoising method for each image layer region in the fused image layer of the first target image layer based on the noise type. Then, the server performs spatial denoising on each image layer region in the fused image layer of the first target image layer according to the denoising method, obtaining a denoised fused image layer of the first target image layer. Next, the server extracts the structural feature map of the denoised fused image layer using the Laplacian operator, and performs detail enhancement processing on the denoised fused image layer based on the structural feature map, obtaining an enhanced fused image layer of the first target image layer. Finally, the server performs upsampling processing on the enhanced fused image layer using an interpolation algorithm, obtaining a first processed fused image layer.

[0125] For example, the second processed fused image layer of the next target image layer can be calculated using the following formula:

[0126] Equation (5)

[0127] Equation (6)

[0128] Equation (7)

[0129] in, This indicates spatial domain noise reduction processing. This indicates edge enhancement processing. Indicates upsampling processing; This refers to the reconstructed Gaussian image layer corresponding to the second target image layer. This refers to the reconstructed Gaussian image layer corresponding to the third target image layer. This refers to the reconstructed Gaussian image layer corresponding to the fourth target image layer.

[0130] In this embodiment, a three-step progressive processing approach of denoising, enhancement, and upsampling is adopted to achieve comprehensive optimization of the fused image quality, which conveys accurate basic information for the subsequent reconstruction of high-resolution layers. The overall process not only optimizes the fusion quality of low-resolution layers, but also lays a high-quality data foundation for full-scale image reconstruction, which is conducive to improving the image processing quality.

[0131] In an exemplary embodiment, step S105, which determines the denoised image corresponding to multiple frames of images based on the processed image layer, specifically includes the following: performing denoising processing on the processed image layer to obtain a denoised image layer; performing enhancement processing on the denoised image layer to obtain an enhanced image layer; and obtaining the denoised image corresponding to multiple frames of images based on the enhanced image layer.

[0132] The denoised image layer refers to the image layer after denoising.

[0133] The enhanced image layer refers to the denoised image layer after enhancement processing.

[0134] For example, the server identifies the noise type of each image layer region in the processed image layer, and determines the spatial denoising method for each image layer region based on the noise type. Then, the server performs spatial denoising on each image layer region in the processed image layer according to the spatial denoising method, resulting in a denoised image layer. Next, the server extracts the structural feature map of the denoised image layer using the Laplacian operator, and performs detail enhancement processing on the denoised image layer based on the structural feature map, resulting in an enhanced image layer. Finally, the server uses the enhanced image layer as the denoised image corresponding to multiple frames.

[0135] For example, the denoised image corresponding to multiple frames can be calculated using the following formula:

[0136] Equation (8)

[0137] in, It refers to the denoised image corresponding to multiple frames.

[0138] In this embodiment, by performing denoising and enhancement on the processed image layer, residual noise that was not completely filtered in the previous pyramid layer fusion can be accurately eliminated, avoiding the impact of this noise on the purity of the final image and compensating for the possible loss of detail in the previous fusion. This also maximizes the improvement of the visual quality of the final image and is beneficial to improving the image denoising effect.

[0139] In an exemplary embodiment, step S101, which determines the fusion weights of the first target image layer corresponding to the multiple frames of images based on the pixel differences between the multiple frames, specifically includes the following: determining the reference image in each frame; determining the weights of other images based on the pixel differences between the reference image and other images; obtaining the fusion weights of the first image layer corresponding to the reference image based on the base weights of the reference image, and obtaining the fusion weights of the first image layers corresponding to the other images based on the weights of the other images; and obtaining the fusion weights of the first target image layer corresponding to the multiple frames of images based on the fusion weights of the first image layer corresponding to the reference image and the fusion weights of the first image layers corresponding to the other images.

[0140] The reference image is also called the reference frame. In practical scenarios, the reference image refers to image input1.

[0141] The "other images" section refers to the images in each frame other than the reference image. In a real-world scenario, the other images are image input2, image input3, and image input4.

[0142] The base weight, also known as the reference frame weight, is fixed at 255 by default.

[0143] For example, the server performs format unification processing on each frame of image to obtain a unified image for each frame; then, the server randomly selects one frame from each unified image as the reference image in each frame, and uses the images in each frame other than the reference image as other images in each frame; then, the server determines the pixel difference between the reference image and other images based on the pixel values ​​of the reference image and other images; then, the server determines the weight of the other images based on the pixel difference between the reference image and other images; then, the server uses the base weight of the reference image as the fusion weight of the first image layer corresponding to the reference image, and uses the weight of the other images as the fusion weight of the first image layer corresponding to the other images; finally, the server uses the fusion weight of the first image layer corresponding to the reference image and the fusion weight of the first image layer corresponding to the other images as the fusion weight of the first target image layer corresponding to the multi-frame images.

[0144] For example, the fusion weights of the first target image layer corresponding to multiple frames can be calculated using the following formula:

[0145] Equation (9)

[0146] in, Adjust the parameters to enhance noise reduction.

[0147] In this embodiment, by using the reference image as the core anchor point, disordered weight calculation without reference between multiple frames is avoided. By comparing the pixel differences between the reference image and other images, the weight allocation of other images is made more consistent with the credibility between frames, providing an accurate and reliable initial basis for the reuse of weights in the subsequent full-scale target image layer. Thus, while processing efficiently, the overall quality and structural consistency of multi-frame fusion denoising can be improved.

[0148] In an exemplary embodiment, before determining the fusion weight of the first target image layer corresponding to the multi-frame images based on the pixel differences between the multi-frame images, step S101 specifically includes the following: acquiring multiple initial images; performing image alignment processing on the multiple initial images to obtain aligned multiple initial images; and obtaining multiple images based on the aligned multiple initial images.

[0149] Among them, the initial multi-frame image is used to represent images that have been captured in succession without alignment.

[0150] For example, in response to an image denoising instruction for multiple initial images sent by a terminal (such as the camera of a handheld device), the server acquires the multiple initial images sent by the terminal; then, the server inputs the multiple initial images into a trained image alignment model, performs image alignment processing on the multiple initial images through the trained image alignment model, and obtains aligned multiple initial images; then, the server uses the aligned multiple initial images as the multiple images.

[0151] In this embodiment, by accurately aligning multiple input images in space to facilitate subsequent image fusion and processing, the slight movement or shaking during the shooting process is avoided, which would cause slight offset and rotation between multiple images. This would result in lower image denoising capability in subsequent image processing, making it difficult to effectively suppress noise and preserve image details. This approach lays a crucial foundation for generating high-quality denoised images.

[0152] In one exemplary embodiment, such as Figure 5 As shown, another image denoising method is provided. Taking the application of this method to a server as an example, the specific steps include:

[0153] Step S501: Determine the fusion weights of the first target image layer corresponding to the multiple frames of images based on the pixel differences between the multiple frames of images; the multiple frames of images are used to represent images captured in multiple consecutive frames; the first target image layer includes the first image layer corresponding to each frame of images.

[0154] Step S502: Upsample the fusion weights of the first target image layer to obtain the processed fusion weights.

[0155] Step S503: Based on the processed fusion weights, determine the fusion weights of the next target image layer of the first target image layer.

[0156] Step S504: Take the next target image layer as the new first target image layer, and jump to the step of upsampling the fusion weights of the first target image layer to obtain the processed fusion weights, until the obtained fusion weights of the next target image layer are the fusion weights of the last target image layer corresponding to the multi-frame images.

[0157] Step S505: Perform fusion processing on each target image layer according to the fusion weight of each target image layer to obtain the fused image layer of each target image layer.

[0158] Step S506: Upsample the fused image layer of the first target image layer to obtain the first processed fused image layer.

[0159] Step S507: The first processed fused image layer is fused with the fused image layer of the next target image layer of the first target image layer to obtain the second processed fused image layer of the next target image layer.

[0160] Step S508: The second processed fused image layer is used as the fused image layer of the new first target image layer, and the process jumps to the step of upsampling the fused image layer of the first target image layer to obtain the first processed fused image layer. This process continues until the obtained second processed fused image layer is the second processed fused image layer of the last target image layer corresponding to the multi-frame image. Then, the obtained second processed fused image layer is used as the processed image layer corresponding to the multi-frame image.

[0161] Step S509: Denoise the processed image layer to obtain a denoised image layer; enhance the denoised image layer to obtain an enhanced image layer; and obtain denoised images corresponding to multiple frames based on the enhanced image layer.

[0162] In the aforementioned image denoising method, when denoising an image, the fusion weights of the first target image layer corresponding to multiple frames can be accurately determined based on the pixel differences between multiple frames. Reusing these fusion weights allows for efficient determination of the fusion weights of each target image layer across multiple frames. Each target image layer is then fused according to its respective fusion weight, resulting in a more accurate fused image layer. This allows for multiple fusion processes of each target image layer, leading to a more accurate processed image layer corresponding to the multiple frames. Based on this processed image layer, the denoised image corresponding to the multiple frames can be more accurately determined, improving the denoising quality and thus enhancing the image denoising capability, effectively suppressing noise while preserving image details. Furthermore, the multi-frame denoising approach avoids the shortcomings of traditional single-frame denoising methods, which suffer from lower denoising capabilities and difficulty in effectively suppressing noise and preserving image details, further improving the image denoising ability and effectively suppressing noise while preserving image details.

[0163] In an exemplary embodiment, to more clearly illustrate the image denoising method provided by this application, the following specific embodiment will be used to describe the image denoising method in detail. In one embodiment, this application also provides a pyramid-based multi-frame image denoising method. Specifically, it includes the following:

[0164] Multi-frame image denoising algorithms typically consist of two steps:

[0165] (1) Image Alignment. Due to the slight movement or shaking of the camera on a handheld device during shooting, there may be slight offsets and rotations between multiple images. Image alignment is the accurate spatial alignment of multiple input images for subsequent image fusion and processing. Therefore, the goal of image alignment is to find the optimal alignment transformation that aligns multiple images at the pixel level.

[0166] (2) Image fusion. Image fusion is the process of merging aligned images to reduce noise and improve image quality. The goal of image fusion is to calculate the fusion weight of each frame based on the pixel values ​​of multiple images, and then generate a final image with a higher signal-to-noise ratio by weighted averaging.

[0167] The multi-frame image denoising algorithm proposed in this scheme operates in the YUV domain of the image, achieving a certain denoising effect on all three channels (Y, U, and V). Common image alignment methods include feature point matching and block matching. Feature point matching extracts feature points from the image and calculates the alignment transformation by matching these feature points. Block matching divides the image into small blocks and calculates the alignment transformation by matching these blocks. This scheme adopts a block matching-based approach for image alignment. The advantage of block matching is that it performs better on locally moving objects, such as portraits and vehicles, and is less prone to artifacts. Image alignment is not the focus of this scheme and will not be discussed in detail. The focus of this scheme is on the image fusion part. It uses an image pyramid approach for multi-frame image fusion, employing weight sharing to reduce computational load, integrating single-frame spatial domain denoising, and edge enhancement modules to achieve the target image effect with lower performance overhead. The flowchart of the image fusion scheme is shown below. Figure 2 As shown.

[0168] Taking four frames as an example, assuming the inputs (inputs 1-4) are already aligned and distinguished by different colors, the specific implementation steps are as follows:

[0169] (1) Construct a pyramid from the four input images to obtain Laplacian layers of different sizes and a top Gaussian layer. Calculate the fusion weights in the top Gaussian image layer. Assume... As a reference frame, with Using the reference image as a baseline, the pixel differences between the corresponding pixel values ​​in the other three frames and the reference frame are calculated. Frames with smaller pixel differences are assigned larger fusion weights, while those with larger differences are assigned smaller fusion weights. The fusion weight plane W3 for all G3 frames is obtained, and then a multi-frame fused Gaussian layer G3' is obtained through pixel-weighted averaging. The formula for calculating G3' is shown below:

[0170] Equation (9)

[0171] Equation (1)

[0172] in, Adjust the parameters to enhance noise reduction. The reference frame weight is fixed at 255 by default.

[0173] (2) Upsample the W3 weight plane of each frame calculated by the Gaussian image layer, and reuse the weights on the L2, L1, and L0 Laplacian layers respectively to obtain the fused L2', L1', and L0' of multiple frames. The calculation formulas for L2', L1', and L0' are as follows:

[0174] Equation (2)

[0175] Equation (3)

[0176] Equation (4)

[0177] (3) After obtaining G3', L2', L1', and L0', pyramid reconstruction begins. Starting from the top layer, single-frame spatial denoising and edge enhancement are performed on G3', followed by a 2x upsampling and the addition of L2' to obtain G2'. Similarly, G2' undergoes the same steps until the final output image is reconstructed. The final output image reconstruction process is shown in the following formula:

[0178] Equation (5)

[0179] Equation (6)

[0180] Equation (7)

[0181] Equation (8)

[0182] in, This indicates spatial domain noise reduction processing. This indicates edge enhancement processing. This indicates upsampling processing. For spatial denoising and edge enhancement algorithms, different processing methods can be selected based on actual performance requirements and acceptable effect range. For example, in scenes with low noise, simple guided filtering or bilateral filtering can be used, while in scenes with high noise, the fast NLM algorithm can be used. Furthermore, the noise level can be controlled separately for different brightness levels and frequency bands by partitioning the scene based on brightness and gradient.

[0183] In the above embodiments, when denoising an image, the fusion weights of the first target image layer corresponding to the multi-frame images can be accurately determined based on the pixel differences between multiple frames. Reusing these fusion weights allows for efficient determination of the fusion weights of each target image layer across the multi-frame images. By performing fusion processing on each target image layer according to its respective fusion weight, the fused image layer of each target image layer can be obtained more accurately. Furthermore, multiple fusion processes can be performed on each fused image layer to more accurately obtain the processed image layer corresponding to the multi-frame images. Based on the processed image layer, the denoised image corresponding to the multi-frame images can be determined more accurately, which helps improve the denoising quality of the denoised image, thereby enhancing the image denoising capability, effectively suppressing noise and preserving image details. Moreover, the entire process employs a multi-frame denoising approach, avoiding the shortcomings of traditional single-frame denoising methods, which have lower image denoising capabilities and struggle to effectively suppress noise and preserve image details. This further improves the image denoising capability, effectively suppressing noise and preserving image details.

[0184] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0185] Based on the same inventive concept, this application also provides an image denoising apparatus for implementing the image denoising method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image denoising apparatus embodiments provided below can be found in the limitations of the image denoising method described above, and will not be repeated here.

[0186] In one exemplary embodiment, such as Figure 6 As shown, an image denoising device is provided, comprising: a first weight determination module 601, a second weight determination module 602, a first image fusion module 603, a second image fusion module 604, and a denoised image determination module 605, wherein:

[0187] The first weight determination module 601 is used to determine the fusion weight of the first target image layer corresponding to the multi-frame images based on the pixel differences between the multi-frame images; the multi-frame images are used to represent images captured in multiple consecutive frames; the first target image layer includes the first image layer corresponding to each frame image.

[0188] The second weight determination module 602 is used to determine the fusion weight of each target image layer corresponding to multiple frames of images based on the fusion weight of the first target image layer.

[0189] The first image fusion module 603 is used to perform fusion processing on each target image layer according to the fusion weight of each target image layer to obtain a fused image layer of each target image layer.

[0190] The second image fusion module 604 is used to perform multiple fusion processes on the fusion image layer of each target image layer to obtain the processed image layer corresponding to multiple frames of images.

[0191] The denoised image determination module 605 is used to determine the denoised image corresponding to multiple frames of images based on the processed image layers.

[0192] In an exemplary embodiment, the second weight determination module 602 is further configured to perform upsampling processing on the fusion weight of the first target image layer to obtain the processed fusion weight; determine the fusion weight of the next target image layer based on the processed fusion weight; take the next target image layer as the new first target image layer, and jump to the step of upsampling processing on the fusion weight of the first target image layer to obtain the processed fusion weight, until the obtained fusion weight of the next target image layer is the fusion weight of the last target image layer corresponding to the multi-frame images.

[0193] In an exemplary embodiment, the second image fusion module 604 is further configured to perform upsampling processing on the fused image layer of the first target image layer to obtain a first processed fused image layer; perform fusion processing on the first processed fused image layer and the fused image layer of the next target image layer of the first target image layer to obtain a second processed fused image layer of the next target image layer; use the second processed fused image layer as the new fused image layer of the first target image layer, and jump to the step of performing upsampling processing on the fused image layer of the first target image layer to obtain the first processed fused image layer, until the obtained second processed fused image layer is the second processed fused image layer of the last target image layer corresponding to the multi-frame image, then use the obtained second processed fused image layer as the processed image layer corresponding to the multi-frame image.

[0194] In an exemplary embodiment, the second image fusion module 604 is further configured to perform denoising processing on the fused image layer of the first target image layer to obtain a denoised fused image layer of the first target image layer; perform enhancement processing on the denoised fused image layer to obtain an enhanced fused image layer of the first target image layer; and perform upsampling processing on the enhanced fused image layer to obtain a first processed fused image layer.

[0195] In an exemplary embodiment, the denoised image determination module 605 is further configured to perform denoising processing on the processed image layer to obtain a denoised image layer; perform enhancement processing on the denoised image layer to obtain an enhanced image layer; and obtain denoised images corresponding to multiple frames of images based on the enhanced image layer.

[0196] In an exemplary embodiment, the first weight determination module 601 is further configured to determine a reference image in each frame of images; determine the weights of other images based on the pixel differences between the reference image and other images; the other images are used to represent images other than the reference image in each frame of images; obtain the fusion weight of the first image layer corresponding to the reference image based on the base weight of the reference image, and obtain the fusion weight of the first image layer corresponding to the other images based on the weights of the other images; obtain the fusion weight of the first target image layer corresponding to multiple frames of images based on the fusion weight of the first image layer corresponding to the reference image and the fusion weight of the first image layer corresponding to the other images.

[0197] In an exemplary embodiment, the image denoising device further includes a multi-frame image processing module for acquiring multiple initial images; performing image alignment processing on the multiple initial images to obtain aligned multiple initial images; and obtaining multiple images based on the aligned multiple initial images.

[0198] Each module in the aforementioned image denoising device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0199] In one exemplary embodiment, a terminal is provided, the internal structure of which can be as follows: Figure 7As shown, the terminal includes a processor, memory, input / output interface, communication interface, display unit, input device, and camera. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an image denoising method.

[0200] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the terminal to which the present application is applied. A specific terminal may include more or fewer components than those shown in the figure, or may combine certain components, or may have different component arrangements.

[0201] This embodiment also provides a base station, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described image alignment method.

[0202] This embodiment also provides a chip, which includes at least one processor. When the processor executes a computer program, it implements the steps of the above-described image denoising method.

[0203] This embodiment also provides a chip module coupled to a memory, which is used to implement the steps of the above-described image denoising method when executing a computer program stored in the memory.

[0204] This embodiment also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described image denoising method.

[0205] This embodiment also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described image denoising method.

[0206] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to mutually. In addition, different parts between embodiments can also be combined with each other, and this invention does not limit this.

[0207] The image denoising method, apparatus, terminal, base station, and chip provided in this embodiment include: determining the fusion weights of a first target image layer corresponding to the multiple frames of images based on the pixel differences between the multiple frames of images; the multiple frames of images are used to represent images captured in multiple consecutive frames; the first target image layer includes a first image layer corresponding to each frame of images; determining the fusion weights of each target image layer corresponding to the multiple frames of images based on the fusion weights of the first target image layer; performing fusion processing on each target image layer according to the fusion weights of each target image layer to obtain a fused image layer of each target image layer; performing multiple fusion processing on the fused image layer of each target image layer to obtain a processed image layer corresponding to the multiple frames of images; and determining the denoised image corresponding to the multiple frames of images based on the processed image layer. When denoising images, the fusion weights of the first target image layer corresponding to multiple frames can be accurately determined based on the pixel differences between the frames. These fusion weights are then reused to efficiently determine the fusion weights of each target image layer across the multiple frames. Each target image layer is then fused according to its respective weight, resulting in a more accurate fused image layer. This fusion process can be repeated multiple times to obtain the processed image layer corresponding to the multiple frames. Based on this processed image layer, the denoised image corresponding to the multiple frames can be determined more accurately, improving the denoising quality and enhancing the image denoising capability. This effectively suppresses noise while preserving image details. Furthermore, the multi-frame denoising approach avoids the shortcomings of traditional single-frame denoising methods, which suffer from lower denoising capabilities and difficulty in effectively suppressing noise and preserving image details. This further improves the image denoising ability, effectively suppressing noise and preserving image details.

[0208] Regarding the modules / units included in the various devices and products described in the above embodiments, they can be software modules / units, hardware modules / units, or a combination of both. For example, for various devices and products applied to or integrated into a chip, all of their modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits; for various devices and products applied to or integrated into a chip module, all of their modules / units can be implemented using hardware methods such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The components can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, each of its components / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or in different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.

[0209] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0210] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0211] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image denoising method, characterized in that, The method includes: Based on the pixel differences between multiple frames of images, the fusion weights of the first target image layer corresponding to the multiple frames of images are determined; the multiple frames of images are used to represent images captured in multiple consecutive frames; the first target image layer includes the first image layer corresponding to each frame of image; The fusion weights of the first target image layer are upsampled to obtain processed fusion weights; the processed fusion weights are used as the fusion weights of the next target image layer; the next target image layer is used as the new first target image layer, and the process jumps to the step of upsampling the fusion weights of the first target image layer to obtain processed fusion weights, until the obtained fusion weights of the next target image layer are the fusion weights of the last target image layer corresponding to the multi-frame images; Each target image layer is fused according to its fusion weight to obtain a fused image layer for each target image layer. The fusion image layer of each target image layer is subjected to multiple fusion processes to obtain the processed image layer corresponding to the multiple frames of images; Spatial denoising and detail enhancement are performed on the processed image layer to obtain the denoised image corresponding to the multi-frame image.

2. The method according to claim 1, characterized in that, Before determining the fusion weights of the first target image layer corresponding to the multiple frames based on the pixel differences between the multiple frames, the process includes: Acquire multiple initial images; The initial multi-frame images are subjected to image alignment processing to obtain aligned initial multi-frame images; The multi-frame images are obtained based on the aligned initial multi-frame images.

3. The method according to claim 1, characterized in that, The process of performing multiple fusion processes on the fused image layer for each target image layer to obtain the processed image layer corresponding to the multiple frames of images includes: The fused image layer of the first target image layer is upsampled to obtain the first processed fused image layer; The first processed fused image layer is fused with the fused image layer of the next target image layer of the first target image layer to obtain the second processed fused image layer of the next target image layer. The second processed fused image layer is used as the fused image layer of the new first target image layer, and the process jumps to the step of upsampling the fused image layer of the first target image layer to obtain the first processed fused image layer, until the obtained second processed fused image layer is the second processed fused image layer of the last target image layer corresponding to the multi-frame image. Then the obtained second processed fused image layer is used as the processed image layer corresponding to the multi-frame image.

4. The method according to claim 3, characterized in that, The upsampling process performed on the fused image layer of the first target image layer to obtain the first processed fused image layer includes: Denoising is performed on the fused image layer of the first target image layer to obtain the denoised fused image layer of the first target image layer. The denoised and fused image layer is enhanced to obtain the enhanced fused image layer of the first target image layer; The enhanced fused image layer is upsampled to obtain the first processed fused image layer.

5. The method according to claim 1, characterized in that, The step of performing spatial denoising and detail enhancement on the processed image layer to obtain the denoised image corresponding to the multi-frame image includes: Spatial domain denoising is performed on the processed image layer to obtain a denoised image layer; The denoised image layer is then subjected to detail enhancement processing to obtain the enhanced image layer. Based on the enhanced image layer, the denoised image corresponding to the multi-frame image is obtained.

6. The method according to any one of claims 1 to 5, characterized in that, The step of determining the fusion weights of the first target image layer corresponding to the multiple frames of images based on the pixel differences between the multiple frames includes: Identify the reference image in each frame; The weights of the other images are determined based on the pixel differences between the reference image and other images; the other images are used to represent the images other than the reference image in each frame. Based on the base weights of the reference image, the fusion weights of the first image layer corresponding to the reference image are obtained, and based on the weights of the other images, the fusion weights of the first image layer corresponding to the other images are obtained; Based on the fusion weights of the first image layer corresponding to the reference image and the fusion weights of the first image layers corresponding to the other images, the fusion weights of the first target image layer corresponding to the multi-frame images are obtained.

7. An image denoising device, characterized in that, The device includes: The first weight determination module is used to determine the fusion weight of the first target image layer corresponding to the multiple frames of images based on the pixel differences between the multiple frames of images; the multiple frames of images are used to represent images captured in multiple consecutive frames; the first target image layer includes the first image layer corresponding to each frame of image; The second weight determination module is used to upsample the fusion weights of the first target image layer to obtain processed fusion weights; use the processed fusion weights as the fusion weights of the next target image layer of the first target image layer; use the next target image layer as the new first target image layer, and jump to the step of upsampling the fusion weights of the first target image layer to obtain processed fusion weights, until the obtained fusion weights of the next target image layer are the fusion weights of the last target image layer corresponding to the multi-frame images; The first image fusion module is used to perform fusion processing on each target image layer according to the fusion weight of each target image layer to obtain a fused image layer of each target image layer; The second image fusion module is used to perform multiple fusion processes on the fused image layer of each target image layer to obtain the processed image layer corresponding to the multiple frames of images. The denoised image determination module is used to perform spatial denoising and detail enhancement processing on the processed image layer to obtain the denoised image corresponding to the multi-frame image.

8. A terminal comprising a memory, a processor, and a camera, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A base station, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A chip, characterized in that, The chip includes at least one processor, which executes a computer program to implement the steps of the method according to any one of claims 1 to 6.

Images