Image denoising method and device, terminal equipment and storage medium

Abstract

The invention relates to the technical field of image denoising, and discloses an image denoising method and device, terminal equipment and a storage medium, and the method comprises the steps: carrying out the region decomposition of a target image through an image denoising hybrid model, and obtaining a flat region, a gradual change region and an image edge region; the image denoising hybrid model comprises a first model, a second model and a third model; when the flat area is denoised, the first model is used for denoising; when the gradual change area is denoised, the first model and the second model are used for denoising; when the edge area of the image is denoised, the first model, the second model and the third model are used for denoising; and outputting a de-noised image gray value, and performing up-sampling for image reconstruction to obtain a de-noised image. According to the method, region decomposition is carried out on the target image, and different model combinations are used for processing different regions, so that targeted denoising is carried out, and the denoising quality is improved.

Description

Technical Field

[0001] The present application relates to the technical field of image denoising, and in particular to an image denoising method, apparatus, terminal device and storage medium. Background Art

[0002] Given that noise signals are not a priori and their generation mechanisms are diverse, in the current image denoising process, the restored and reconstructed images often lose edge information. Therefore, how to maintain image edge information while denoising has become a key issue. Current image denoising technology mainly relies on diffusion models based on partial differential equations. The core concept of such models is to regard the pixel values ​​in the image as some physical quantities (such as temperature or material concentration) and describe the change process of these physical quantities over time through diffusion equations to achieve noise elimination. For example, in the Perona-Malik model, different values ​​of the threshold parameter will produce significantly different processing effects. A lower threshold parameter makes the model more sensitive to edges, which may lead to excessive retention of edges and difficulty in effectively removing noise; on the contrary, a higher threshold parameter may cause the image to be over-smoothed, thereby losing some edge and detail information. Therefore, it is usually necessary to optimize the parameter settings through repeated trials and according to specific image processing requirements, which is a very cumbersome process. Summary of the invention

[0003] In view of this, an embodiment of the present application provides an image denoising method, which can effectively solve the problem of losing image edge information.

[0004] In a first aspect, an embodiment of the present application provides an image denoising method, comprising: Performing regional decomposition on the target image through an image denoising hybrid model to obtain a flat region, a gradient region and an image edge region; the image denoising hybrid model includes a first model, a second model and a third model; When denoising the flat area, the first model is used for denoising; when denoising the gradient area, the first model and the second model are used for denoising; when denoising the edge area of ​​the image, the first model, the second model and the third model are used for denoising; Output the grayscale value of the denoised image and perform upsampling to reconstruct the image to obtain the denoised image.

[0005] In some embodiments, the first model is a PM model, the second model is a YK model, and the third model is a weighted anisotropic diffusion model.

[0006] In some embodiments, the performing regional decomposition on the target image by using the image denoising hybrid model includes: Downsampling the target image to obtain image features, and calculating image gradients according to grayscale values ​​of the image features; An image region corresponding to the image feature is determined according to the image gradient.

[0007] In some embodiments, determining the image region corresponding to the image feature according to the image gradient includes: Get the preset threshold parameters; The image gradient is The image features are divided into the flat area; The image gradient is The image features are divided into the gradient area; The image gradient is The image features are divided into the image edge area; In the formula, is the modulus of the image gradient, K is the threshold parameter, is the parameter of the threshold parameter.

[0008] In some embodiments, when the first model is used for denoising, the diffusion coefficient function of the image denoising hybrid model is the diffusion coefficient function of the first model; When the first model and the second model are used for denoising, the diffusion coefficient function of the image denoising hybrid model is the sum of the diffusion coefficient function of the first model and the diffusion coefficient function of the second model; When the first model, the second model and the third model are used for denoising, the diffusion coefficient function of the image denoising hybrid model is the sum of the diffusion coefficient functions of the first model, the second model and the third model.

[0009] In some embodiments, the diffusion coefficient function expression of the first model is: ; In the formula, ∇ is the gradient operator, div is the divergence operator, is a hyperparameter, is the magnitude of the image gradient, is the diffusion coefficient function; The expression of the diffusion coefficient function of the second model is: ; The expression of the diffusion coefficient function of the third model is: ; In the formula, is a weight function used to reduce the staircase artifacts at the edge of the image. is the diffusion coefficient, which is used to control the convergence speed.

[0010] In some embodiments, the weight function is calculated as: ; In the formula, is a fixed parameter; The expression of the diffusion coefficient is: ; ; In the formula, K is the threshold parameter, , are diffusion coefficients with different convergence trends.

[0011] In a second aspect, the present application also provides an image denoising device, comprising: An image decomposition module is used to perform regional decomposition of the target image through an image denoising hybrid model to obtain a flat area, a gradient area and an image edge area; An image denoising module, configured to use the first model for denoising the flat area; use the first model and the second model for denoising the gradient area; and use the first model, the second model and the third model for denoising the edge area of ​​the image; The image reconstruction module is used to output the grayscale value of the denoised image and perform upsampling to reconstruct the image to obtain a denoised image.

[0012] In a third aspect, the present application further provides a terminal device, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the computer program to implement the image denoising method.

[0013] In a fourth aspect, the present application further provides a computer-readable storage medium storing a computer program, which implements the image denoising method when executed on a processor.

[0014] The embodiments of the present application have the following beneficial effects: The present application divides the target image into regions through an image denoising hybrid model. For different types of image regions, the image denoising hybrid model uses different denoising model combinations to perform denoising, so as to perform targeted denoising and improve the denoising quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A schematic diagram of a process flow of an image denoising method according to an embodiment of the present application is shown; Figure 2 A schematic diagram of image region division results according to an embodiment of the present application is shown; Figure 3 A schematic diagram of the structure of an image denoising device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0018] The components of the embodiments of the present application generally described and shown in the drawings herein may be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0019] Hereinafter, the terms "including", "having" and their cognates that can be used in various embodiments of the present application are intended only to indicate specific features, numbers, steps, operations, elements, components or a combination of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or a combination of the foregoing items or increasing the possibility of one or more features, numbers, steps, operations, elements, components or a combination of the foregoing items. In addition, the terms "first", "second", "third" and the like are only used to distinguish descriptions and cannot be understood as indicating or implying relative importance.

[0020] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present application.

[0021] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0022] The present application is applied to the scenario of image denoising. The image denoising method of the present application constructs an image denoising hybrid model. When performing image denoising, the image is divided into regions. Different diffusion function coefficients are set for different regions to perform denoising. Different denoising methods are implemented for different regions by utilizing the characteristics of different diffusion function coefficients, thereby improving the quality of denoising and avoiding the loss of too much image information during denoising.

[0023] The image denoising method is described below in conjunction with some specific embodiments.

[0024] Figure 1 A flow chart of an image denoising method according to an embodiment of the present application is shown. Exemplarily, the image denoising method comprises the following steps: Step S100, performing regional decomposition on the target image through an image denoising hybrid model to obtain a flat region, a gradient region and an image edge region.

[0025] The image denoising hybrid model is composed of a mixture of multiple models, including a first model, a second model and a third model.

[0026] The target image is an image that needs to be denoised. Before denoising the target image, the image region is divided by the image denoising hybrid model, mainly into a flat region, a gradient region and an image edge region.

[0027] First, downsampling is performed. The purpose of downsampling is to reduce the resolution, obtain image features from the target image, and then calculate the image gradient based on the image features obtained by downsampling.

[0028] Image gradient refers to the rate of change of brightness of an image at a certain point. It is a vector that contains information about the magnitude and direction of the gradient. In a two-dimensional image, the gradient is usually represented by partial derivatives in the horizontal and vertical directions.

[0029] The flat areas of the image usually have small changes in pixel values, so their gradient values ​​are also small and relatively uniform.

[0030] The pixel values ​​in the gradient area change to a certain extent, but the change is relatively slow and smooth, so the gradient value is at a medium level and relatively stable.

[0031] The edge region is where the pixel values ​​in the image change dramatically, resulting in larger gradient values ​​and possibly a significant change in the gradient direction.

[0032] According to the image gradient, the target image can be divided into regions, wherein the region division of the image can be performed according to a threshold parameter K obtained through a priori experiments. The K value is mainly used to control the balance between edge preservation and noise removal.

[0033] The image gradient is The image features are divided into the flat area; The image gradient is The image features are divided into the gradient area; The image gradient is The image features are divided into the image edge area; In the formula, is the modulus of the image gradient, is a parameter of the threshold parameter, and the threshold parameter and the parameter are determined through prior experiments.

[0034] It can be understood that the pixel gradients at different positions on the target image are different, so the regions with different gradients in the target image can be divided by the above method to obtain the above three types of image regions, thereby completing the region division of the image.

[0035] Among them, in addition to the above-mentioned division by the interval of the gradient value, clustering can also be used. For example, the gradient value and the position information of the pixel where the gradient value is located form a feature vector. In this way, the feature vector of the entire image can be obtained. Then, the number of clusters is set to 3, and cluster analysis is performed. This can achieve the effect of dividing the target image area into three areas.

[0036] Exemplary, such as Figure 2 As shown in Figure 1, a possible segmentation result after segmenting a target image. In actual segmentation, the segmentation result may be more complicated, and the above three regions will be scattered throughout the image instead of being Figure 2 As shown in the regularity.

[0037] Step S200, when denoising the flat area, use the first model for denoising; when denoising the gradient area, use the first model and the second model for denoising; when denoising the edge area of ​​the image, use the first model, the second model and the third model for denoising.

[0038] The above three types of image regions have different characteristics. This embodiment uses different model combinations to process different image regions.

[0039] The first model is a PM model, the second model is a YK model, and the third model is a weighted anisotropic diffusion model.

[0040] The PM model is a Perona-Malik model, which can effectively retain the edge information of the image while removing noise. It is suitable for various types of noise removal work because it can diffuse along the edge and perpendicular to the edge at the same time, so it has a significant noise removal effect.

[0041] The YK model (Kekulé model) can divide an image into different regions. The fourth-order YK model overcomes the weaknesses of low-order partial differential equations. While achieving the denoising effect, it protects the texture details of the image and eliminates the staircase effect to a certain extent.

[0042] Weighted multi-scale diffusion model: It is a variant model based on the Riya (read-interpret-apply) model. It has a faster decay rate. The diffusion model changes the scale in four directions and has a good effect on edge processing.

[0043] According to the characteristics of the above three models, this embodiment uses different models to process different image areas. For example, when processing flat areas, the PM model can be used in this area, taking into account the efficient denoising ability of the PM model in flat area images. When processing gradient areas, considering the need to eliminate the step effect, the PM model + YK model is used for denoising. When processing the edge area of ​​the image, considering the need to retain more image edge details, the PM model + YK model + weighted multi-scale diffusion model is used for denoising. In this way, when denoising the target image, different model combinations are used for denoising for different area positions of the image, which can reasonably allocate computing power, and at the same time make good use of the characteristics of each model, and complement their advantages and disadvantages to ensure the image denoising quality during denoising, and more image information can be retained to reduce the loss of detail information due to denoising.

[0044] Among them, the above three models all have their own diffusion coefficient functions. Therefore, when performing the above three, the diffusion coefficient function is an important part of the above three models. Therefore, for the image denoising hybrid model, which model is used for denoising is equivalent to using the diffusion coefficient function of which model to perform denoising operations.

[0045] Among them, the diffusion coefficient function expression of the first model is: ; In the formula, div is the divergence operator, is a hyperparameter, is the magnitude of the image gradient, is a function of the diffusion coefficient.

[0046] The expression of the diffusion coefficient function of the second model is: ; Where ∇ is the gradient operator.

[0047] The expression of the diffusion coefficient function of the third model is: .

[0048] In the formula, is the weight function, is the diffusion coefficient.

[0049] Therefore, when denoising a flat area, As a diffusion coefficient function, the PM model is used for denoising. When denoising the gradient area, the As the diffusion coefficient function, the PM model + YK model is used for denoising. When denoising the edge area of ​​the image, the As a diffusion coefficient function, where I is the image and t is the iteration time, is the diffusion coefficient function of the third model.

[0050] Among them, the weight function The expression is: ; It is a fixed parameter, and its value can be 100, etc. This weight function can be used to reduce the staircase artifacts at the edge.

[0051] Diffusion coefficient The expression is: ; , are diffusion coefficients with different convergence trends. When the smooth gradient of the image edge is small, Convergence is slow, The convergence is fast. When denoising at the edge of the image, an appropriate convergence speed can be ensured, ensuring the denoising performance while retaining the image details and avoiding over-smoothing of the image edge.

[0052] Exemplarily, the expressions of these two diffusion coefficients are:

[0053] Wherein, K is the aforementioned threshold parameter.

[0054] In this way, by using three different diffusion coefficient functions, different model combinations can be used for denoising in different image areas. When denoising the image edges, the diffusion coefficient function of the third model is introduced to retain the edges as much as possible. While improving the image denoising effect, it ensures that more details are retained on the image edges without losing them, and at the same time increases the computational efficiency of the denoising model.

[0055] Step S300, outputting the grayscale value of the denoised image, and performing upsampling to reconstruct the image to obtain a denoised image.

[0056] After denoising is completed in the above manner, the grayscale value of the denoised image can be output, and then image reconstruction can be performed. Finally, the complete target image is reconstructed, and the target image at this time is the denoised image.

[0057] The image denoising method of this embodiment divides the denoised image into regions and blocks, effectively utilizes the different denoising characteristics of each model, adopts the efficient denoising ability of the PM model in flat areas, combines the YK model with the ability to include the edge of the image, and uses the PM+YK+weighted multi-scale diffusion model to denoise the edge area of ​​the image to retain the edge as much as possible. Compared with the independent use of a single PM, YK, and weighted isotropic diffusion model, a better denoising effect is achieved. While improving the image denoising effect, it ensures that more details are retained at the edge of the image without loss, while increasing the computational efficiency of the denoising model.

[0058] In order to conveniently illustrate the improvement of this embodiment in image denoising, the PSNR and SSIM indicators are used to evaluate the quality of smoothed images using different methods. PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index) are two commonly used image quality evaluation indicators.

[0059] Compared with other mainstream diffusion models, the PSNR and SSIM of the images with different noises are shown in Table 1 below.

[0060] Table 1:

[0061] The model of this embodiment has a certain improvement in image edge quality compared to the traditional model, and the score is higher than the existing model, which can have a better denoising effect.

[0062] Figure 3 A schematic diagram of the structure of an image denoising device according to an embodiment of the present application is shown. Exemplarily, the device includes: An image decomposition module 10 is used to perform regional decomposition on the target image through an image denoising hybrid model to obtain a flat region, a gradient region and an image edge region; An image denoising module 20, configured to use the first model for denoising the flat area; use the first model and the second model for denoising the gradient area; and use the first model, the second model and the third model for denoising the edge area of ​​the image; The image reconstruction module 30 is used to output the grayscale value of the denoised image and perform upsampling to reconstruct the image to obtain a denoised image.

[0063] It can be understood that the device of this embodiment corresponds to the method of the above embodiment, and the options in the above embodiment are also applicable to this embodiment, so they will not be described repeatedly here.

[0064] The present application also provides a terminal device. Exemplarily, the terminal device includes a processor and a memory, wherein the memory stores a computer program, and the processor runs the computer program to enable the terminal device to execute the functions of each module in the above-mentioned image denoising method or the above-mentioned image denoising device.

[0065] Among them, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or at least one of other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application.

[0066] The memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electric erasable programmable read-only memory (EEPROM), etc. The memory is used to store a computer program, and the processor may execute the computer program accordingly after receiving an execution instruction.

[0067] The present application also provides a computer-readable storage medium for storing the computer program used in the above-mentioned terminal device. For example, the computer-readable storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0068] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow diagram, and the combination of boxes in the structure diagram and / or the flow diagram, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0069] In addition, the functional modules or units in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0070] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0071] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. An image denoising method, characterized in that: include: The target image is decomposed into regions through the image denoising hybrid model to obtain flat regions, gradient regions and image edge regions; The image denoising hybrid model includes a first model, a second model and a third model; When denoising the flat area, the first model is used for denoising; when denoising the gradient area, the first model and the second model are used for denoising; When denoising the edge area of ​​the image, the first model, the second model and the third model are used for denoising; Output the grayscale value of the denoised image and perform upsampling to reconstruct the image to obtain the denoised image.

2. The image denoising method according to claim 1, characterized in that: The first model is a PM model, the second model is a YK model, and the third model is a weighted anisotropic diffusion model.

3. The image denoising method according to claim 1, characterized in that: The method of performing regional decomposition on the target image by using the image denoising hybrid model comprises: Downsampling the target image to obtain image features, and calculating image gradients according to grayscale values ​​of the image features; An image region corresponding to the image feature is determined according to the image gradient.

4. The image denoising method according to claim 3, characterized in that: The determining, according to the image gradient, an image region corresponding to the image feature comprises: Get the preset threshold parameters; The image gradient is The image features are divided into the flat area; The image gradient is The image features are divided into the gradient area; The image gradient is The image features are divided into the image edge area; In the formula, is the modulus of the image gradient, K is the threshold parameter, is the parameter of the threshold parameter.

5. The image denoising method according to claim 1, characterized in that: When the first model is used for denoising, the diffusion coefficient function of the image denoising hybrid model is the diffusion coefficient function of the first model; When the first model and the second model are used for denoising, the diffusion coefficient function of the image denoising hybrid model is the sum of the diffusion coefficient function of the first model and the diffusion coefficient function of the second model; When the first model, the second model and the third model are used for denoising, the diffusion coefficient function of the image denoising hybrid model is the sum of the diffusion coefficient functions of the first model, the second model and the third model.

6. The image denoising method according to claim 5, characterized in that: The diffusion coefficient function expression of the first model is: ； In the formula, ∇ is the gradient operator, div is the divergence operator, is a hyperparameter, is the modulus of the image gradient, is the diffusion coefficient function; The expression of the diffusion coefficient function of the second model is: ； The expression of the diffusion coefficient function of the third model is: ； In the formula, is a weight function used to reduce the staircase artifacts at the edge of the image. is the diffusion coefficient, which is used to control the convergence speed.

7. The image denoising method according to claim 6, characterized in that: The calculation expression of the weight function is: ； In the formula, is a fixed parameter; The expression of the diffusion coefficient is: ； ； In the formula, K is the threshold parameter, , are diffusion coefficients with different convergence trends.

8. An image denoising device, characterized in that: include: An image decomposition module is used to perform regional decomposition of the target image through an image denoising hybrid model to obtain a flat area, a gradient area and an image edge area; An image denoising module, configured to use the first model to denoise the flat area; When denoising the gradient region, using the first model and the second model for denoising; When denoising the edge area of ​​the image, the first model, the second model and the third model are used for denoising; The image reconstruction module is used to output the grayscale value of the denoised image and perform upsampling to reconstruct the image to obtain a denoised image.

9. A terminal device, characterized in that: The terminal device comprises a processor and a memory, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement the image denoising method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer program is stored therein, and when the computer program is executed on a processor, the image denoising method according to any one of claims 1 to 7 is implemented.

Images