Image denoising method based on image multi-scale information and electronic device
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHENGDU CK TECH
- Filing Date
- 2022-12-29
- Publication Date
- 2026-05-12
Smart Images
Figure CN116012244B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image denoising technology, and more specifically to an image denoising method and electronic device based on multi-scale image information. Background Technology
[0002] Noise is a significant cause of image interference. An image in practical applications may contain various types of noise, which may be generated during transmission or during processing such as quantization. Common spatial filtering algorithms based on spatial pixel similarity are often affected by noise when matching similar pixels, impacting the accuracy of the spatial filtering and thus the overall image denoising effect. Summary of the Invention
[0003] This application is proposed to address the aforementioned problems. According to one aspect of this application, an image denoising method based on multi-scale image information is provided. The method includes: acquiring an image to be processed; constructing an image pyramid based on the image to be processed; for each current pixel in the image to be processed that needs filtering: finding associated pixels related to the current pixel from the image pyramid; calculating spatial filtering weights for the associated pixels based on the current pixel and the associated pixels; and obtaining a filtering result for the current pixel based on the spatial filtering weights; and obtaining a denoising result for the image to be processed based on the filtering result for each current pixel.
[0004] In one embodiment of this application, constructing an image pyramid based on the image to be processed includes: defining the number of layers in the image pyramid, the image pyramid including a first layer and other layers; assigning the image to be processed to the first layer; and assigning the image to the other layers after convolving and downsampling the image based on a defined filter kernel.
[0005] In one embodiment of this application, the method further includes: initializing a vector with a length equal to the number of layers of the image pyramid, for storing each layer of the image pyramid.
[0006] In one embodiment of this application, the step of finding associated pixels related to the current pixel in the image pyramid includes: calculating the pixel corresponding to the current pixel in each layer of the image pyramid as the center pixel; for each layer of the image pyramid, taking the center pixel in that layer as the center, taking all pixels within the defined spatial noise reduction radius as associated pixels related to the current pixel.
[0007] In one embodiment of this application, the step of calculating the spatial filtering weight of the associated pixel based on the current pixel and the associated pixel includes: calculating the pixel value difference between each associated pixel and the current pixel; and calculating the spatial filtering weight of each associated pixel based on the absolute value of the pixel value difference.
[0008] In one embodiment of this application, obtaining the filtering result of the current pixel based on the spatial filtering weights includes: calculating the weighted pixel value of each associated pixel according to the spatial filtering weight of each associated pixel; and using the ratio of the sum of the weighted pixel values of all associated pixels to the sum of all the spatial filtering weights as the filtering result of the current pixel.
[0009] In one embodiment of this application, the image to be processed is a grayscale image or a brightness image.
[0010] According to another aspect of this application, an electronic device is provided, the electronic device including a memory and a processor, the memory storing a computer program executed by the processor, the computer program, when executed by the processor, causing the processor to perform the above-described image denoising method based on multi-scale image information.
[0011] According to another aspect of this application, an image denoising device based on multi-scale image information is provided. The device includes a pyramid construction module, a weight calculation module, and a filtering module. The pyramid construction module is used to acquire an image to be processed and construct an image pyramid based on the image to be processed. The weight calculation module is used to find related pixels in the image pyramid for each current pixel in the image to be filtered, and calculate the spatial filtering weights of the related pixels based on the current pixel and the related pixels. The filtering module is used to obtain the filtering result of the current pixel based on the spatial filtering weights, and obtain the denoising result of the image to be processed based on the filtering result of each current pixel.
[0012] In one embodiment of this application, the pyramid construction module constructs an image pyramid based on the image to be processed, including: defining the number of layers of the image pyramid, the image pyramid including a first layer and other layers; assigning the image to be processed to the first layer; and assigning the image to the other layers after convolving and downsampling the image to be processed based on a defined filtering kernel.
[0013] In one embodiment of this application, the pyramid construction module is further configured to: initialize a vector with a length equal to the number of layers of the image pyramid, for storing each layer of the image pyramid.
[0014] In one embodiment of this application, the weight calculation module searches for associated pixels related to the current pixel in the image pyramid, including: calculating the pixel corresponding to the current pixel in each layer of the image pyramid as the center pixel; for each layer of the image pyramid, taking the center pixel in that layer as the center, taking all pixels within the defined spatial noise reduction radius as associated pixels related to the current pixel.
[0015] In one embodiment of this application, the weight calculation module calculates the spatial filtering weight of the associated pixel based on the current pixel and the associated pixel, including: calculating the pixel value difference between each associated pixel and the current pixel; and calculating the spatial filtering weight of each associated pixel based on the absolute value of the pixel value difference.
[0016] In one embodiment of this application, the filtering module obtains the filtering result of the current pixel based on the spatial filtering weights, including: calculating the weighted pixel value of each associated pixel according to the spatial filtering weight of each associated pixel; and using the ratio of the sum of the weighted pixel values of all associated pixels to the sum of all spatial filtering weights as the filtering result of the current pixel.
[0017] In one embodiment of this application, the image to be processed is a grayscale image or a brightness image.
[0018] According to another aspect of this application, a storage medium is provided, on which a computer program is stored, which, when run, causes the processor to execute the above-described image denoising method based on multi-scale image information.
[0019] According to another aspect of this application, a computer program product is provided, the computer program product including a computer program, which, when run by a processor, causes the processor to perform the above-described image denoising method based on multi-scale image information.
[0020] The image denoising method and apparatus based on multi-scale image information of this application expands the original single-scale pixel matching to multiple scales by constructing an image pyramid. This not only utilizes the multi-scale information of the image but also allows more points to participate in spatial filtering, thereby enhancing the effect of spatial filtering, enabling better removal of image noise, and better preservation of image detail information. Attached Figure Description
[0021] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0022] Figure 1 A schematic block diagram of an example electronic device is shown for implementing the image denoising method and apparatus based on multi-scale image information according to embodiments of the present invention.
[0023] Figure 2 A schematic flowchart of an image denoising method based on multi-scale image information according to an embodiment of this application is shown.
[0024] Figure 3 A schematic structural block diagram of an image noise reduction apparatus based on multi-scale image information according to an embodiment of this application is shown.
[0025] Figure 4 A schematic structural block diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation
[0026] To make the objectives, technical solutions, and advantages of this application more apparent, exemplary embodiments according to this application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein. Based on the embodiments of this application described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of this application.
[0027] In recent years, significant progress has been made in research on technologies based on artificial intelligence, such as computer vision, deep learning, machine learning, image processing, and image recognition. Artificial intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies, and application systems to simulate and extend human intelligence. AI is a comprehensive discipline involving numerous technologies, including chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, and neural networks. Computer vision, as an important branch of AI, specifically enables machines to recognize the world. Computer vision technologies typically include face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, object detection, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, behavior recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, and robot navigation and localization. With the research and advancement of artificial intelligence technology, this technology has been applied in numerous fields, such as security, urban management, traffic management, building management, park management, facial recognition access control, facial recognition attendance, logistics management, warehouse management, robotics, intelligent marketing, computational photography, mobile imaging, cloud services, smart homes, wearable devices, autonomous driving, autonomous driving, smart healthcare, facial payment, facial unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile internet, live streaming, beautification, makeup, medical aesthetics, and intelligent temperature measurement.
[0028] Below, refer to Figure 1 This describes an example electronic device 100 for implementing the image denoising method and apparatus based on multi-scale image information according to embodiments of the present invention.
[0029] like Figure 1 As shown, the electronic device 100 includes one or more processors 102, one or more storage devices 104, input devices 106, and output devices 108, which are interconnected via a bus system 110 and / or other forms of connection mechanisms (not shown). It should be noted that... Figure 1 The components and structure of the electronic device 100 shown are merely exemplary and not limiting; the electronic device may also have other components and structures as needed.
[0030] The processor 102 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.
[0031] The storage device 104 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 102 may execute the program instructions to implement the client functions (implemented by the processor) in the embodiments of the present invention described below, and / or other desired functions. Various applications and various data may also be stored in the computer-readable storage medium, such as various data used and / or generated by the applications.
[0032] The input device 106 can be a device used by a user to input commands, and may include one or more of a keyboard, mouse, microphone, and touchscreen. Furthermore, the input device 106 can also be any interface for receiving information.
[0033] The output device 108 can output various information (e.g., images or sounds) to the outside (e.g., a user), and may include one or more of a display, speaker, etc. Furthermore, the output device 108 can also be any other device with output functionality.
[0034] For example, the example electronic device used to implement the image denoising method and apparatus based on multi-scale image information according to embodiments of the present invention can be implemented in terminals such as smartphones, tablets, cameras, in-vehicle devices, and intelligent robots.
[0035] Below, we will refer to Figure 2 This application describes an image denoising method 200 based on multi-scale image information according to embodiments of the present application. For example... Figure 2 As shown, the image denoising method 200 based on multi-scale image information may include the following steps:
[0036] In step S210, the image to be processed is obtained, and an image pyramid is constructed based on the image to be processed.
[0037] In step S220, for each current pixel in the image to be processed that needs to be filtered: find the associated pixels related to the current pixel in the image pyramid, calculate the spatial filtering weights of the associated pixels based on the current pixel and the associated pixels, and obtain the filtering result of the current pixel based on the spatial filtering weights.
[0038] In step S230, the noise reduction result of the image to be processed is obtained based on the filtering result of each current pixel.
[0039] In the embodiments of this application, after acquiring the image to be processed (e.g., a grayscale image or a brightness image), for each current pixel that needs filtering, instead of directly searching for a matching pixel in the image based on spatial pixel similarity and then performing spatial filtering for image denoising based on such single-scale pixel matching, the inventors have discovered that natural images themselves possess strong fractal characteristics. Image patterns that exist at small scales also exist at large scales, and noise levels decrease exponentially at large scales. Based on this, the inventors have proposed the method 200 of this application, which is a method for improving image denoising effects by applying multi-scale information of the image. It extends the original single-scale pixel matching to multiple scales by constructing an image pyramid. Specifically, in the embodiments of this application, after constructing an image pyramid based on the image to be processed, for each current pixel in the image to be processed that needs filtering, a matching associated pixel can be found in each layer of the image pyramid. The spatial filtering weight of each associated pixel is calculated. Finally, based on the pixel value of the current pixel, the pixel values of all associated pixels related to the current pixel, and the spatial filtering weight of each associated pixel, the spatial filtering result of the current pixel can be calculated. By performing this operation on each pixel in the image to be processed that needs filtering, the filtered results of all pixels can be obtained, thus obtaining the filtered result of the entire image to be processed, i.e., the denoised image. This allows the original single-scale pixel matching to be extended to multiple scales through the image pyramid. It not only utilizes the multi-scale information of the image but also allows more points to participate in the spatial filtering, thereby enhancing the effect of spatial filtering, enabling better removal of image noise, and better preservation of image detail information.
[0040] In the embodiments of this application, the construction of an image pyramid based on the image to be processed in step S210 may include: defining the number of layers in the image pyramid, wherein the image pyramid includes a first layer and other layers; assigning the image to be processed to the first layer; and assigning the image to be processed to other layers after convolving and downsampling the image based on a defined filtering kernel. Here, downsampling means taking a value every other point.
[0041] For example, assuming the image to be processed is represented as y, the number of layers in the image pyramid P is defined as K, where K is a natural number greater than 1. Each layer can be represented by k, where k = 1 to K. For k = 1 (i.e., the first layer), the original image (i.e., the image to be processed) y can be directly assigned to the first layer of the image pyramid P, i.e., P[k = 1] = y. For k... ≠1 (i.e., other layers that are not the first layer, i.e., layers 2 to K), can be convolved with a predefined filter kernel on the image to be processed, and the convolved y is downsampled to obtain yk, and yk is assigned to the kth layer of the pyramid: P[k ≠ 1] = yk.
[0042] For example, for k=2 (i.e., the second layer), a predefined kernel can be used to convolve the image y to be processed, for example, a two-dimensional convolution, which can be expressed by the formula y'=conv2d(kernel,y), where conv2d represents the two-dimensional convolution operation. Then, the resulting y' is downsampled by 1 / 2 to obtain y2, and y2 is assigned to the second layer of the pyramid. For k=3 (i.e., the third layer), a predefined kernel can be used to convolve the image y to be processed, for example, a two-dimensional convolution, which can be expressed by the formula y'=conv2d(kernel,y), where conv2d represents the two-dimensional convolution operation. Then, the resulting y' is downsampled by 1 / 4 to obtain y3, and y3 is assigned to the third layer of the pyramid. And so on, for k=K (i.e., the Kth layer), a predefined kernel can be used to convolve the image y to be processed, for example, a two-dimensional convolution, which can be expressed by the formula y'=conv2d(kernel,y), where conv2d represents the two-dimensional convolution operation. Then, the resulting y' after convolution is downsampled by 1 / 2^(K-1) to obtain yK, and yK is assigned to the Kth layer of the pyramid. At this point, the image pyramid is complete. A vector P of length K can be initialized to store the layers of the image pyramid.
[0043] The predefined filter kernel mentioned above can include anisotropic filter kernels kernel_x, kernel_y, and isotropic filter kernels kernel. Here, kernel = ones(n,n) means defining a matrix with n rows and n columns, where all elements are 1; kernel = kernel / (n*n) means dividing each element in the kernel by n*n. Here, n is the size of the filter kernel.
[0044] In the embodiments of this application, the step S220 of finding the associated pixel points related to the current pixel point from the image pyramid may include: calculating the pixel point corresponding to the current pixel point in each layer of the image pyramid as the center pixel point; for each layer of the image pyramid, taking the center pixel point in that layer as the center, taking all pixels within the defined spatial noise reduction radius as the associated pixel points related to the current pixel point.
[0045] For example, a current pixel can be represented as (i,j), where i∈[1,……,h] and j∈[1,……,w], where h and w refer to the height and width of the image y to be processed, respectively. Therefore, the pixel corresponding to the current pixel in each layer of the image pyramid is ((i / 2^(k-1), j / 2^(k-1)). For example, for k=1, which is the first layer of the image pyramid, since the original image y is assigned to the first layer, the pixel corresponding to the current pixel (i,j) in this layer is ((i / 2^(1-1), j / 2^(1-1)), which is the current pixel itself. For k=2, which is the second layer of the image pyramid, the pixel corresponding to the current pixel (i,j) in this layer is ((i / 2^(2-1), j / 2^(2-1)), which is (i / 2,j / 2). For k=3, which is the third layer of the image pyramid, the pixel corresponding to the current pixel (i,j) in this layer is ((i / 2^(3-1), j / 2^(3-1)), which is (i / 4,j / 4). And so on.
[0046] After obtaining the center pixel, all pixels within the defined spatial denoising radius r can be considered as associated pixels related to the current pixel. For example, these associated pixels can be stored in val[k,s1,s2], where k represents the current layer of the image pyramid, s1 represents the distance of the x-coordinate of the associated pixel in this layer relative to the x-coordinate of the center pixel, and s2 represents the distance of the y-coordinate of the associated pixel in this layer relative to the y-coordinate of the center pixel.
[0047] In the embodiments of this application, the step S220 of calculating the spatial filtering weight of the associated pixel based on the current pixel and the associated pixel may include: calculating the pixel value difference between each associated pixel and the current pixel; and calculating the spatial filtering weight of each associated pixel based on the absolute value of the pixel value difference.
[0048] Continuing with the example above, the absolute value of the pixel value difference between each associated pixel and the current pixel can be: d[k,s1,s2]=abs(y(i,j)-P[k,i / 2^(k-1)+s1,j / 2^(k-1]+s2]). The spatial filtering weight of each associated pixel based on the absolute value of the pixel value difference can be expressed as: w[k,s1,s2]=exp(-((d[k,s1,s2])^2) / s[k]). Where s[k] is the intensity value of the pixel in each layer of the image pyramid participating in the spatial filtering, which is predefined, and s[k]∈R,k∈[1,……,K].
[0049] In an embodiment of this application, obtaining the filtering result of the current pixel based on the spatial filtering weight in step S220 may include: calculating the weighted pixel value of each associated pixel according to the spatial filtering weight of each associated pixel; and using the ratio of the sum of the weighted pixel values of all associated pixels to the sum of all spatial filtering weights as the filtering result of the current pixel.
[0050] Continuing with the example above, the weighted pixel value of each associated pixel, calculated based on its spatial filtering weight, can be represented as w.*val, where * indicates element-wise multiplication. The ratio of the sum of the weighted pixel values of all associated pixels to the sum of all spatial filtering weights is taken as the filtering result for the current pixel, which can be expressed as the spatial filtering result for point (i,j) new_y(i,j) = sum(w.*val) / sum(w).
[0051] In the embodiments of this application, the denoising result of the image to be processed is obtained based on the filtering result of each current pixel. That is, the denoising result of the image to be processed is obtained by obtaining the filtering result of each current pixel in the manner described above.
[0052] Based on the above description, the image denoising method based on multi-scale image information according to the embodiments of this application constructs an image pyramid, which expands the original single-scale pixel matching to multiple scales. This not only utilizes the multi-scale information of the image, but also allows more points to participate in spatial filtering, thereby enhancing the effect of spatial filtering, enabling better removal of image noise, and better preservation of image detail information.
[0053] The following is combined Figures 3 to 4 This application describes an image denoising apparatus based on multi-scale image information, which can be used to perform the image denoising method based on multi-scale image information according to the embodiments of this application described above. The specific operation process of the image denoising method based on multi-scale image information has been described in detail above; therefore, for the sake of brevity, the specific details will not be described again below, only some main operations will be described.
[0054] Figure 3 A schematic structural block diagram of an image denoising apparatus 300 based on multi-scale image information according to an embodiment of this application is shown. The image denoising apparatus 300 based on multi-scale image information according to an embodiment of this application can be used to perform the image denoising method 200 based on multi-scale image information according to an embodiment of this application described above. The image denoising method 300 based on multi-scale image information has been described in detail above; for brevity, only the structure and main operations of the image denoising apparatus 300 based on multi-scale image information are described here, and other details are not repeated.
[0055] like Figure 3 As shown, the image denoising device 300 based on multi-scale image information may include a pyramid construction module 310, a weight calculation module 320, and a filtering module 330. The pyramid construction module 310 acquires the image to be processed and constructs an image pyramid based on the image. The weight calculation module 320, for each current pixel in the image that needs filtering, searches for related pixels in the image pyramid and calculates the spatial filtering weights of the related pixels based on the current pixel and the related pixels. The filtering module 330 obtains the filtering result of the current pixel based on the spatial filtering weights and obtains the denoising result of the image to be processed based on the filtering result of each current pixel.
[0056] In the embodiments of this application, after acquiring the image to be processed (e.g., a grayscale image or a brightness image), for each current pixel that needs filtering, instead of directly searching for a matching pixel in the image based on spatial pixel similarity and then performing spatial filtering for image denoising based on such single-scale pixel matching, the inventors have discovered that natural images themselves possess strong fractal characteristics. Image patterns that exist at small scales also exist at large scales, and noise levels decrease exponentially at large scales. Based on this, the inventors have proposed the device 300 of this application, which is a device for improving image denoising based on multi-scale information of applied images. It constructs an image pyramid through a pyramid construction module 310, extending the original single-scale pixel matching to multiple scales. Specifically, in the embodiments of this application, after the image pyramid construction module 310 constructs an image pyramid based on the image to be processed, the weight calculation module 320, for each current pixel in the image to be processed that needs filtering, can find the associated pixels that match it in each layer of the image pyramid, calculate the spatial filtering weight of each associated pixel, and finally, the filtering module 330 calculates the spatial filtering result of the current pixel based on the pixel value of the current pixel, the pixel values of all associated pixels related to the current pixel, and the spatial filtering weight of each associated pixel. By performing this operation on each pixel in the image to be processed that needs filtering, the filtered results of all pixels can be obtained, thereby obtaining the filtered result of the entire image to be processed, i.e., the denoised image. This allows the original single-scale pixel matching to be extended to multiple scales through the image pyramid, which not only utilizes the multi-scale information of the image but also allows more points to participate in the spatial filtering, thereby enhancing the effect of spatial filtering, enabling better removal of image noise, and better preservation of image detail information.
[0057] In one embodiment of this application, the pyramid construction module 310 constructs an image pyramid based on the image to be processed, including: defining the number of layers in the image pyramid, the image pyramid including a first layer and other layers; assigning the image to be processed to the first layer; and assigning the image to be processed to other layers after convolving and downsampling the image based on a defined filtering kernel. Here, downsampling means taking a value every other point.
[0058] For example, assuming the image to be processed is represented as y, the number of layers in the image pyramid P is defined as K, where K is a natural number greater than 1. Each layer can be represented by k, where k = 1 to K. For k = 1 (i.e., the first layer), the original image (i.e., the image to be processed) y can be directly assigned to the first layer of the image pyramid P, i.e., P[k = 1] = y. For k... ≠ 1 (i.e., other layers that are not the first layer, i.e., layers 2 to K), can be convolved with a predefined filter kernel on the image to be processed, and the convolved y is downsampled to obtain yk, and yk is assigned to the kth layer of the pyramid: P[k ≠ 1] = yk.
[0059] For example, for k=2 (i.e., the second layer), a predefined kernel can be used to convolve the image y to be processed, for example, a two-dimensional convolution, which can be expressed by the formula y'=conv2d(kernel,y), where conv2d represents the two-dimensional convolution operation. Then, the resulting y' is downsampled by 1 / 2 to obtain y2, and y2 is assigned to the second layer of the pyramid. For k=3 (i.e., the third layer), a predefined kernel can be used to convolve the image y to be processed, for example, a two-dimensional convolution, which can be expressed by the formula y'=conv2d(kernel,y), where conv2d represents the two-dimensional convolution operation. Then, the resulting y' is downsampled by 1 / 4 to obtain y3, and y3 is assigned to the third layer of the pyramid. And so on, for k=K (i.e., the Kth layer), a predefined kernel can be used to convolve the image y to be processed, for example, a two-dimensional convolution, which can be expressed by the formula y'=conv2d(kernel,y), where conv2d represents the two-dimensional convolution operation. Then, the y' obtained after convolution is downsampled by 1 / 2^(K-1) to obtain yK, and yK is assigned to the Kth layer of the pyramid. At this point, the image pyramid construction is complete. The pyramid construction module 310 can also initialize a vector P of length K to store the layers of the image pyramid.
[0060] The predefined filter kernel mentioned above can include anisotropic filter kernels kernel_x, kernel_y, and isotropic filter kernels kernel. Here, kernel = ones(n,n) means defining a matrix with n rows and n columns, where all elements are 1; kernel = kernel / (n*n) means dividing each element in the kernel by n*n. Here, n is the size of the filter kernel.
[0061] In one embodiment of this application, the weight calculation module 320 searches for associated pixels related to the current pixel in the image pyramid, including: calculating the pixel corresponding to the current pixel in each layer of the image pyramid as the center pixel; for each layer of the image pyramid, taking the center pixel in that layer as the center, taking all pixels within the defined spatial noise reduction radius as associated pixels related to the current pixel.
[0062] For example, a current pixel can be represented as (i,j), where i∈[1,……,h] and j∈[1,……,w], where h and w refer to the height and width of the image y to be processed, respectively. Therefore, the pixel corresponding to the current pixel in each layer of the image pyramid is ((i / 2^(k-1), j / 2^(k-1)). For example, for k=1, which is the first layer of the image pyramid, since the original image y is assigned to the first layer, the pixel corresponding to the current pixel (i,j) in this layer is ((i / 2^(1-1), j / 2^(1-1)), which is the current pixel itself. For k=2, which is the second layer of the image pyramid, the pixel corresponding to the current pixel (i,j) in this layer is ((i / 2^(2-1), j / 2^(2-1)), which is (i / 2,j / 2). For k=3, which is the third layer of the image pyramid, the pixel corresponding to the current pixel (i,j) in this layer is ((i / 2^(3-1), j / 2^(3-1)), which is (i / 4,j / 4). And so on.
[0063] After obtaining the center pixel, all pixels within the defined spatial denoising radius r can be considered as associated pixels related to the current pixel. For example, these associated pixels can be stored in val[k,s1,s2], where k represents the current layer of the image pyramid, s1 represents the distance of the x-coordinate of the associated pixel in this layer relative to the x-coordinate of the center pixel, and s2 represents the distance of the y-coordinate of the associated pixel in this layer relative to the y-coordinate of the center pixel.
[0064] In one embodiment of this application, the weight calculation module 320 calculates the spatial filtering weight of the associated pixel based on the current pixel and the associated pixel, including: calculating the pixel value difference between each associated pixel and the current pixel; and calculating the spatial filtering weight of each associated pixel based on the absolute value of the pixel value difference.
[0065] Continuing with the example above, the absolute value of the pixel value difference between each associated pixel and the current pixel can be: d[k,s1,s2]=abs(y(i,j)-P[k,i / 2^(k-1)+s1,j / 2^(k-1]+s2]). The spatial filtering weight of each associated pixel based on the absolute value of the pixel value difference can be expressed as: w[k,s1,s2]=exp(-((d[k,s1,s2])^2) / s[k]). Where s[k] is the intensity value of the pixel in each layer of the image pyramid participating in the spatial filtering, which is predefined, and s[k]∈R,k∈[1,……,K].
[0066] In one embodiment of this application, the filtering module 330 obtains the filtering result of the current pixel based on the spatial domain filtering weights, including: calculating the weighted pixel value of each associated pixel according to the spatial domain filtering weight of each associated pixel; and using the ratio of the sum of the weighted pixel values of all associated pixels to the sum of all spatial domain filtering weights as the filtering result of the current pixel.
[0067] Continuing with the example above, the weighted pixel value of each associated pixel, calculated based on its spatial filtering weight, can be represented as w.*val, where * indicates element-wise multiplication. The ratio of the sum of the weighted pixel values of all associated pixels to the sum of all spatial filtering weights is taken as the filtering result for the current pixel, which can be expressed as the spatial filtering result for point (i,j) new_y(i,j) = sum(w.*val) / sum(w).
[0068] In the embodiments of this application, the denoising result of the image to be processed is obtained based on the filtering result of each current pixel. That is, the denoising result of the image to be processed is obtained by obtaining the filtering result of each current pixel in the manner described above.
[0069] Based on the above description, the image noise reduction device based on multi-scale image information according to the embodiments of this application expands the original single-scale pixel matching to multiple scales by constructing an image pyramid. This not only utilizes the multi-scale information of the image but also allows more points to participate in spatial filtering, thereby enhancing the effect of spatial filtering, enabling better removal of image noise, and better preservation of image detail information.
[0070] Figure 4A schematic block diagram of an electronic device 400 according to an embodiment of this application is shown. Figure 4 As shown, the electronic device 400 according to an embodiment of this application may include a memory 410 and a processor 420. The memory 410 stores a computer program executed by the processor 420. When the computer program is executed by the processor 420, the processor 420 performs the image denoising method 200 based on multi-scale image information described above according to an embodiment of this application. Those skilled in the art can understand the specific operation of the electronic device 400 according to the embodiments of this application in conjunction with the foregoing description. For the sake of brevity, specific details will not be repeated here.
[0071] Furthermore, according to embodiments of this application, a storage medium is also provided, on which program instructions are stored. When executed by a computer or processor, these program instructions are used to perform corresponding steps of the image noise reduction method based on multi-scale image information according to embodiments of this application. The storage medium may, for example, include a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.
[0072] Furthermore, according to embodiments of this application, a computer program product is also provided, the computer program product including a computer program, which, when run by a processor, causes the processor to execute the image denoising method based on multi-scale image information according to embodiments of this application.
[0073] Based on the above description, the image denoising method, apparatus and electronic device based on multi-scale image information according to the embodiments of this application construct an image pyramid, which expands the original single-scale pixel matching to multiple scales. This not only utilizes the multi-scale information of the image, but also allows more points to participate in spatial filtering, thereby enhancing the effect of spatial filtering, enabling better removal of image noise and better preservation of image detail information.
[0074] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of this application. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of this application. All such changes and modifications are intended to be included within the scope of this application as claimed in the appended claims.
Claims
1. An image denoising method based on multi-scale image information, characterized in that, The method includes: Obtain the image to be processed, and construct an image pyramid based on the image to be processed; For each current pixel in the image to be processed that needs filtering: find the associated pixels related to the current pixel in the image pyramid, including: calculating the pixel corresponding to the current pixel in each layer of the image pyramid as the center pixel; for each layer of the image pyramid, taking the center pixel in that layer as the center, taking all pixels within the defined spatial noise reduction radius as the associated pixels related to the current pixel; Calculating the spatial filtering weight of the associated pixel based on the current pixel and the associated pixel includes: calculating the pixel value difference between each associated pixel and the current pixel, and calculating the spatial filtering weight of each associated pixel based on the absolute value of the pixel value difference and the intensity value of the pixels in each layer of the image pyramid participating in spatial filtering, as defined in a predefined manner. The filtering result of the current pixel is obtained based on the spatial filtering weights. Based on the filtering result of each current pixel, the noise reduction result of the image to be processed is obtained.
2. The method according to claim 1, characterized in that, The construction of the image pyramid based on the image to be processed includes: Define the number of layers in the image pyramid, which includes a first layer and other layers; For the first layer, the image to be processed is assigned to the first layer; For the other layers, the image to be processed is convolved and downsampled based on the defined filter kernel and then assigned to the other layers.
3. The method according to claim 2, characterized in that, The method further includes: Initialize a vector of length equal to the number of layers in the image pyramid to store each layer of the image pyramid.
4. The method according to claim 3, characterized in that, The step of obtaining the filtering result of the current pixel based on the spatial domain filtering weights includes: Calculate the spatial filtering weights for each associated pixel and calculate the weighted pixel value for each associated pixel. The ratio of the sum of the weighted pixel values of all the associated pixels to the sum of all the spatial filtering weights is taken as the filtering result of the current pixel.
5. The method according to claim 1, characterized in that, The image to be processed is either a grayscale image or a brightness image.
6. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program executed by the processor, the computer program, when executed by the processor, causing the processor to perform the image denoising method based on multi-scale image information as described in any one of claims 1-5.
7. A storage medium, characterized in that, The storage medium stores a computer program that, when run, causes a processor to execute the image denoising method based on multi-scale image information as described in any one of claims 1-5.
8. A computer program product, characterized in that, The computer program product includes a computer program that, when run by a processor, causes the processor to perform the image denoising method based on multi-scale image information as described in any one of claims 1-5.