Image denoising method and device, electronic equipment and storage medium

Abstract

The invention provides an image denoising method and device, electronic equipment and a storage medium, and relates to the field of digital image processing. The method comprises the following steps: acquiring a noise image, and constructing a first vector matrix corresponding to the noise image; performing optimization processing on the first vector matrix by adopting a feature optimization model in a preset hybrid model to obtain a second vector matrix; performing sparse decomposition processing on the second vector matrix by adopting a sparse decomposition model in a preset mixed model to obtain a third vector matrix; wherein the rank of the third vector matrix is smaller than that of the second vector matrix; and carrying out inverse transformation processing on the third vector matrix to obtain a de-noised image. According to the mode, a dual noise suppression mechanism is provided through the preset hybrid model, high-quality reservation of the denoised image on the image structure can be realized, and the denoising effect is improved while the generalization ability is ensured.

Description

Technical Field

[0001] The present application relates to the field of digital image processing, and in particular to an image denoising method, device, electronic device and storage medium. Background Art

[0002] In recent years, with the continuous evolution of semiconductor chip process nodes, the resolution of image sensors has become higher and higher, and the scenarios in which digital images can be used have become more extensive and diverse. However, digital images still face many interferences and influences in applications, which ultimately lead to a serious degradation of image quality. The most prominent interference is image noise. Therefore, image denoising has always been a core research topic in the field of digital image processing, and strong noise denoising is the key research direction of image denoising, because there is currently a lack of effective technical methods to achieve high-quality retention of image structure while denoising.

[0003] Existing strong noise denoising methods generally use prior knowledge of natural images to summarize some statistical laws of image structure / features, and then use them for denoising, such as non-local similarity of image structure, sparse expression of image structure, etc.

[0004] This method is prone to losing image features during the denoising process and cannot achieve high-quality preservation of image structure, so its denoising effect still needs to be further improved. Summary of the invention

[0005] The present application provides an image denoising method, device, electronic device and storage medium to solve the problem of poor denoising effect in the prior art.

[0006] According to a first aspect of the present application, there is provided an image denoising method, comprising:

[0007] Acquire a noise image, and construct a first vector matrix corresponding to the noise image;

[0008] Using a feature optimization model in a preset hybrid model, the first vector matrix is ​​optimized to obtain a second vector matrix;

[0009] Using a sparse decomposition model in a preset hybrid model, sparse decomposition processing is performed on the second vector matrix to obtain a third vector matrix; wherein the rank of the third vector matrix is ​​less than the rank of the second vector matrix;

[0010] The third vector matrix is ​​subjected to inverse transformation processing to obtain a denoised image.

[0011] Optionally, the feature optimization model in the preset hybrid model includes: an image screening module, an image feature extraction module and a region matching module;

[0012] Wherein, the output end of the image screening module and the output end of the image feature extraction module are both connected to the input end of the region matching module;

[0013] The method of using a feature optimization model in a preset hybrid model to optimize the first vector matrix to obtain a second vector matrix includes:

[0014] By means of the image screening module, according to the first vector matrix, a reference image whose similarity with the noise image reaches a first preset threshold is obtained from a preset reference image library; wherein the variance of the reference image is smaller than the variance of the noise image;

[0015] By means of the image feature extraction module, feature extraction is performed on each first feature vector contained in the first vector matrix to obtain a second feature vector corresponding to the first feature vector;

[0016] Through the region matching module, for each of the second eigenvectors, the second eigenvector is traversed and matched with multiple regions in the reference image to obtain the eigenvector of the matching region, and the second eigenvector is optimized using the eigenvector of the matching region to obtain the third eigenvector corresponding to each of the second eigenvectors, and a second vector matrix composed of all the third eigenvectors is output.

[0017] Optionally, the image screening module includes a denoising unit, a similarity calculation unit and a screening unit connected in sequence;

[0018] The method further comprises: obtaining, by the image screening module, a reference image whose similarity with the noise image reaches a first preset threshold from a preset reference image library according to the first vector matrix, including:

[0019] Performing denoising processing on the first vector matrix by the denoising unit to obtain a denoised first vector matrix;

[0020] By means of the similarity calculation unit, similarity calculation is performed on the first vector matrix after denoising with vector matrices of multiple images in the preset reference image library to obtain respective corresponding similarities;

[0021] The screening unit screens out reference images whose similarity with the noise image reaches a first preset threshold.

[0022] Optionally, the sparse decomposition model in the preset hybrid model includes a cluster matching module and a sparse decomposition module connected in sequence;

[0023] The method adopts a sparse decomposition model in a preset hybrid model to perform sparse decomposition processing on the second vector matrix to obtain a third vector matrix, including:

[0024] By means of the cluster matching module, all the third eigenvectors in the second vector matrix are clustered to obtain a clustering result, and a sparse transformation matrix is ​​constructed based on the clustering result; wherein the rank of the sparse transformation matrix is ​​smaller than the rank of the second vector matrix;

[0025] The sparse transformation matrix is ​​subjected to singular value decomposition by the sparse decomposition module to obtain singular values, and the singular values ​​are subjected to threshold processing to obtain a singular value matrix, which is output as a third vector matrix.

[0026] Optionally, performing threshold processing on the singular values ​​to obtain a singular value matrix includes:

[0027] Through the singular value threshold algorithm, singular values ​​greater than or equal to the second preset threshold are retained, and singular values ​​less than the second preset threshold are set as preset values, and a singular value matrix is ​​obtained after threshold processing.

[0028] Optionally, after performing inverse transformation processing on the third vector matrix to obtain a denoised image, the method further includes:

[0029] Calculate the index value of the denoised image under the evaluation index; wherein the evaluation index includes peak signal-to-noise ratio and / or structural similarity.

[0030] Optionally, constructing a first vector matrix corresponding to the noise image includes:

[0031] Dividing the noise image into a plurality of local image blocks by overlapping; wherein two adjacent local image blocks have overlapping areas;

[0032] Performing vectorization processing on each of the local image blocks respectively to obtain a first eigenvector corresponding to each of the local image blocks;

[0033] The first eigenvectors corresponding to all the local image blocks are concatenated to obtain a first vector matrix corresponding to the noise image; wherein each of the first eigenvectors is a row vector / column vector in the first vector matrix; and the number of columns / rows of the first vector matrix is ​​the number of the local image blocks contained in the noise image.

[0034] According to a second aspect of the present application, there is provided an image denoising device, comprising:

[0035] An image denoising acquisition module, used to acquire a noisy image and construct a first vector matrix corresponding to the noisy image;

[0036] A feature optimization module, used to optimize the first vector matrix by using a feature optimization model in a preset hybrid model to obtain a second vector matrix;

[0037] A sparse decomposition module, used to use a sparse decomposition model in a preset hybrid model to perform sparse decomposition processing on the second vector matrix to obtain a third vector matrix; wherein the rank of the third vector matrix is ​​less than the rank of the second vector matrix;

[0038] The inverse transformation module is used to perform inverse transformation processing on the third vector matrix to obtain a denoised image.

[0039] According to a third aspect of the present application, there is provided an electronic device, comprising: at least one processor and a memory;

[0040] The memory stores computer-executable instructions;

[0041] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the image denoising method as described in the first aspect above.

[0042] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the image denoising method described in the first aspect above.

[0043] According to a fifth aspect of the present application, a computer program product is provided, comprising a computer program, which implements the image denoising method described in the first aspect when executed by a processor.

[0044] An image denoising method provided by the present application includes: acquiring a noisy image and constructing a first vector matrix corresponding to the noisy image; using a feature optimization model in a preset hybrid model to optimize the first vector matrix to obtain a second vector matrix; using a sparse decomposition model in the preset hybrid model to perform sparse decomposition on the second vector matrix to obtain a third vector matrix; wherein the rank of the third vector matrix is ​​less than the rank of the second vector matrix; and performing an inverse transformation on the third vector matrix to obtain a denoised image.

[0045] The embodiments of the present application have the following two advantages: on the one hand, the feature optimization model in the preset hybrid model is adopted to perform initial denoising processing on the first vector matrix from the dimension of feature optimization, and achieve high-quality retention of the image structure by the second vector matrix, thereby achieving high-quality retention of the image structure by the denoised image; on the other hand, the sparse decomposition model in the preset hybrid model is adopted, and on the basis of the initial denoising processing, a secondary denoising processing is performed on it from the dimension of reduced rank, thereby making the final denoised image have a higher filtering effect. Therefore, the dual noise suppression mechanism in the present application can highly retain the image structure, improve the denoising effect while ensuring the generalization ability.

[0046] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0048] Figure 1 A schematic diagram of a process of an image denoising method provided in an embodiment of the present application;

[0049] Figure 2 A schematic diagram of a flow chart of another image denoising method provided in an embodiment of the present application;

[0050] Figure 3 A schematic diagram of the structure of a feature optimization model in a preset hybrid model provided in an embodiment of the present application;

[0051] Figure 4 A flowchart of another image denoising method provided in an embodiment of the present application;

[0052] Figure 5 A schematic diagram of a process flow of a second denoising process for a noise image provided by an embodiment of the present application;

[0053] Figure 6 A flowchart of another image denoising method provided in an embodiment of the present application;

[0054] Figure 7 A schematic diagram of the test effect of the image denoising method provided in an embodiment of the present application on a strong noise image;

[0055] Figure 8 A schematic diagram of the structure of an image denoising device provided in an embodiment of the present application;

[0056] Fig. 9A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0057] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0058] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0059] In recent years, with the continuous evolution of semiconductor chip process nodes, the resolution that image sensors can achieve is getting higher and higher, and the scenarios that digital images can be used in are becoming more and more extensive and diverse. Equipped with mobile communication terminals, it is also easier to obtain digital images. Digital images are widely used. Among them, digital images can be used in many fields such as mobile phone photography, automatic driving, scientific exploration, industrial manufacturing, etc., and have many uses such as recording target morphology, moving trajectory, defect detection, etc. However, digital images are still facing many interferences and influences, among which the most prominent is image noise. There are many reasons for the generation of image noise. From the hardware aspect, the performance of the device is affected by the process quality, which is easy to cause defects in the imaging pixels, or low detection sensitivity (i.e., interference from thermal noise), analog-to-digital conversion circuit accuracy, etc.; from the software aspect, it is affected by the noise introduced during the transmission and conversion of the image digital signal, which eventually interferes with the grayscale value of the pixel; from the environmental aspect, due to the quantum fluctuations of the light quantum itself, the signal itself has a certain randomness, especially in the case of weak signals, the random effect of the pixel detection light signal is amplified, etc. Therefore, image denoising has always been a core research topic in the field of digital image processing, and heavy noise denoising is the key research direction of image denoising, because there is currently a lack of effective technical methods to achieve high-quality preservation of image structure while denoising.

[0060] In the field of image denoising, many new technologies have emerged and achieved certain results. They can be roughly divided into two categories: one is the method of modeling denoising based on image priors, and the other is the image denoising method based on deep learning. The former uses the prior knowledge of natural images to summarize some statistical laws of image structure / features, and then uses them for denoising, such as non-local similarity of image structure, sparse expression of image structure, etc.; the latter also completes image denoising through some characteristics or regular features of the image. The difference is that the acquisition method is not through explicit induction and generalization, but through invisible machine learning-deep neural network, to learn the image features or laws in specific scenes and obtain non-explicit expressions. The difference between the two methods directly leads to the difference in their applications: the former relies on the global similarity and coefficient characteristics of image features, and transforms and inversely transforms local structures or image features, and its feature expression ability is poor. When targeting a specific scene, its effect still needs to be further improved, such as: Non-local Centralized Sparse Representation (NCSR), Nuclear Norm Minimization (NNM), Non-local Means (NLM) and other method models; the latter trains deep neural networks through image training data sets of specific scenes to obtain high-quality features, thereby obtaining end-to-end high denoising performance, and its generalization ability is weak, and it has a high dependence on training data, such as Fastand Flexible Denoising Network (FFD-Net), Convolutional Blind Denoising Network (CBD-Net) and other network models, so its scope of application is limited. In addition, the above two types of methods have problems such as restoration of structural distortion in the process of image denoising.

[0061] In view of the shortcomings of the above two types of methods in denoising strong noise images, the present application provides a method applied in the field of digital image processing to improve the denoising effect.

[0062] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0063] Embodiment 1:

[0064] Figure 1The following is a flow chart of an image denoising method provided in an embodiment of the present application. Figure 1 As shown, the method of this embodiment includes the following steps:

[0065] S10, acquiring a noise image, and constructing a first vector matrix corresponding to the noise image.

[0066] In the embodiment of the present application, the noise image may be referred to as a noisy image, a strong noise image, etc., which may be divided into a plurality of image blocks, and the image blocks may be represented by image vectors after vectorization processing. The embodiment of the present application does not specifically limit the source of the noise in the noisy image. The noise image can be reconstructed into a denoised image after the image denoising process provided in the embodiment of the present application.

[0067] S20. Using a feature optimization model in a preset hybrid model, the first vector matrix is ​​optimized to obtain a second vector matrix.

[0068] In the application embodiment, the preset hybrid model is a pre-trained hybrid model, which is also called a hybrid expression model, including a feature optimization model and a sparse decomposition model, wherein the specific structure of the feature optimization model and its specific workflow are described in the following embodiment 2, which will not be repeated here. The specific structure of the sparse decomposition model and its specific workflow are described in the following embodiment 3, which will not be repeated here.

[0069] Optionally, the preset hybrid model can also add other models based on the feature optimization model and the sparse decomposition model, so the embodiment of the present application does not specifically limit the specific structure of the preset hybrid model. The embodiment of the present application executes the first denoising process of the hybrid model in step S20.

[0070] S30. Using a sparse decomposition model in a preset hybrid model, perform sparse decomposition processing on the second vector matrix to obtain a third vector matrix; wherein the rank of the third vector matrix is ​​smaller than the rank of the second vector matrix.

[0071] In the embodiment of the present application, the second denoising process of the hybrid model is performed in step S30. This step can enhance the image structure (ie, the above-mentioned image block / image vector).

[0072] S40, performing inverse transformation processing on the third vector matrix to obtain a denoised image.

[0073] In the embodiment of the present application, the purpose of the inverse transformation process is to inversely transform the one-dimensional data (i.e., the image vector in the third vector matrix) into a two-dimensional image block, thereby completing the reconstruction of the image and obtaining a denoised image. The strategy is to use the average value of the pixel points of multiple image blocks in the overlapping area as the value of the pixel point, based on the fact that the previous overlapping segmentation makes a certain pixel point belong to several image blocks at the same time, so the inverse transformation process is to use the average value of the pixel values ​​of all image blocks containing this pixel point at this pixel point as the pixel value of this pixel point. The inverse transformation process can be implemented using existing methods, so the embodiment of the present application does not describe the inverse transformation process in detail.

[0074] The image denoising method is a strong noise image denoising method based on a hybrid model, and the method includes the following key steps: (1) using a feature optimization model in a preset hybrid model to achieve a first denoising by optimizing a first vector matrix; (2) after the first denoising is completed, using a sparse decomposition model in a preset hybrid model to achieve a second denoising by sparse decomposing a second vector matrix.

[0075] The embodiments of the present application have the following two advantages: on the one hand, the feature optimization model in the preset hybrid model is adopted to perform initial denoising processing on the first vector matrix from the dimension of feature optimization, and achieve high-quality retention of the image structure by the second vector matrix, thereby achieving high-quality retention of the image structure by the denoised image; on the other hand, the sparse decomposition model in the preset hybrid model is adopted, and on the basis of the initial denoising processing, a secondary denoising processing is performed on it from the dimension of reduced rank, thereby making the final denoised image have a higher filtering effect. Therefore, the dual noise suppression mechanism in the present application can highly retain the image structure, improve the denoising effect while ensuring the generalization ability.

[0076] In a possible implementation, in step S10, constructing a first vector matrix corresponding to the noise image includes the following steps:

[0077] S101, dividing a noise image into a plurality of local image blocks by overlapping, wherein two adjacent local image blocks have overlapping areas.

[0078] It should be understood that division can be understood as segmentation, specifically overlapping segmentation. The local image block is the local structure of the noise image, also known as the local structure, image block, similar block, local block image, etc. For image processing, it is usually difficult to process the noise image in one go, so the embodiment of the present application adopts the image segmentation method to segment the noise image, and then performs subsequent processing on the image blocks obtained after segmentation. The area size of the local image block is usually the size of the segmentation size. For example: the size of the noise image is 512x512, and the noise image can be cut according to the segmentation size of 8x8, and the size of each local structure is 8x8.

[0079] S102 , performing vectorization processing on each local image block respectively to obtain a first eigenvector corresponding to each local image block.

[0080] It should be understood that vectorization processing refers to transforming two-dimensional image block data into one-dimensional data, and the first eigenvector refers to expressing two-dimensional image block data with one-dimensional data. Therefore, the first eigenvector is also called an image vector. The second eigenvector, the third eigenvector, etc. in the subsequent steps can all be described by the concept of image vector.

[0081] Furthermore, the number of image vectors is the same as the number of local image blocks, which is (N-n+1)×(N-n+1), where N is the length and width of the noise image, and n is the length and width of the segmentation size.

[0082] S103. Concatenate the first eigenvectors corresponding to all local image blocks to obtain a first vector matrix corresponding to the noise image; wherein each first eigenvector is a row vector / column vector in the first vector matrix; and the number of columns / rows of the first vector matrix is ​​the number of local image blocks contained in the noise image.

[0083] It should be understood that the first vector matrix is ​​composed of all row strings / column strings, wherein one row string / column string is a first eigenvector corresponding to a local image block, or is called a similarity matrix.

[0084] Optionally, the embodiment of the present application can be divided into blocks according to the segmentation size, and the segmentation size can be customized according to the processor resources and the running time requirements of the algorithm. For example, for a 512×512 noise image, if the segmentation size is 50×50, then the vector converted from the image block is 2500×1 (i.e., 2500-dimensional data). If there are 100 similar blocks, then the similarity matrix is ​​2500x100, and the memory usage is large. If the segmentation size is 10x10, then the vector converted from the image is 100×1 (i.e., 100-dimensional data), which has a small memory usage, but the number of image blocks is 25 times that of the former, and the processing time becomes longer.

[0085] The embodiment of the present application can implement the segmentation of the noise image by executing the above step S101, and can implement the matrix operation of the vector by executing the above steps S102 and S103, thereby providing data support for subsequent operations.

[0086] In a possible implementation, after performing inverse transformation processing on the third vector matrix to obtain a denoised image, the method further includes the following steps:

[0087] S50, calculating the index value of the denoised image under the evaluation index, wherein the evaluation index includes Peak Signal to Noise Ratio (PSNR) and / or Structural Similarity (SSIM).

[0088] In the embodiment of the present application, the peak signal-to-noise ratio PSNR can be defined by the mean square error or by other parameters. The embodiment of the present application does not specifically limit the specific function form adopted by the peak signal-to-noise ratio. Structural similarity is also called structural similarity factor, structural consistency similarity factor, etc. The structural similarity factor is an indicator based on the visual characteristics of the human eye to measure the similarity between a noisy image and a denoised image. The larger the value, the better, and the maximum value is 1.

[0089] By executing step S50, the embodiment of the present application can evaluate the denoising effect of the method according to the index value.

[0090] Based on the above embodiments, the technical solution of the present application is described in more detail below in combination with several specific embodiments.

[0091] Embodiment 2:

[0092] Figure 2 A flowchart of another image denoising method provided in an embodiment of the present application. Figure 1 Based on the embodiment shown, this embodiment focuses on Figure 1 In the embodiment of the present application, if Figure 3 As shown, the feature optimization model in the preset hybrid model includes: an image selection block (ISB) 100, an image feature extraction block (IFEB) 200 and a region matching block (RMB) 300. The output end of the image selection block 100 and the output end of the image feature extraction block 200 are both connected to the input end of the region matching block 300.

[0093] Further, the feature optimization model in the preset hybrid model is an optimization network framework, in which the noise image is input into a bidirectional branch so that the image screening module 100 located on the first branch obtains a reference image so that the image feature extraction module 200 located on the second branch performs feature extraction. Optionally, the image screening module 100, the image feature extraction module 200 and the region matching module 300 can all be neural networks.

[0094] like Figure 2 As shown, step S20, using the feature optimization model in the preset hybrid model to optimize the first vector matrix to obtain the second vector matrix, includes the following steps:

[0095] S201. Obtain, through an image screening module, a reference image whose similarity with the noise image reaches a first preset threshold from a preset reference image library according to a first vector matrix; wherein the variance of the reference image is smaller than the variance of the noise image.

[0096] It should be understood that the reference image refers to a high-quality external image / natural image, which can be obtained through various external clean natural image databases. These databases are open source in the field of image processing, such as: Kodak dataset, Samsung image dataset, standard digital image processing dataset, etc.

[0097] Compared with the strong noise image in this application, a high-quality image refers to a clean image. From an objective point of view, the noise of a clean image is extremely low. Whether a digital image can be regarded as a clean image can be measured by an indicator. For example, for an 8-bit digital image (whose pixel value range is [0, 255]), if the standard deviation σ of its background area (i.e., the smooth part) is less than or equal to 3, it is considered a clean image. From a subjective point of view, the evaluation result is usually consistent with the variance effect, that is, the image is clear and smooth.

[0098] In the embodiment of the present application, the preset reference image library or the preset material library or the external image library refers to one or more natural image databases mentioned above, and can also be customized as needed. The reference image whose similarity with the noise image reaches the first preset threshold is a similar image of the noise image, so it can be simply referred to as a similar image. The number of similar images can be one or more, and the embodiment of the present application does not specifically limit the number of similar images.

[0099] S202 . Perform feature extraction on each first eigenvector included in the first vector matrix through an image feature extraction module to obtain a second eigenvector corresponding to the first eigenvector.

[0100] Among them, the image feature extraction module can be a deep convolutional neural network (DCNN) containing multiple feature extraction layers, or it can be other feature extraction models, and the embodiments of the present application do not impose any limitations on this.

[0101] In an embodiment of the present application, the second feature vector is composed of image features, which may be one or more image features. The image features are features extracted from the local structure of the noise image, so they are local features of the noise image extracted by the image feature extraction module.

[0102] Optionally, the image feature extraction module can use convolution operation to complete the extraction of image features. Usually, image feature extraction is related to the basic configuration parameters of the deep convolutional neural network (for example, the size of the convolution kernel, the number of feature extraction layers, etc.).

[0103] S203. Through the region matching module, for each second eigenvector, the second eigenvector is traversed and matched with multiple regions in the reference image to obtain the eigenvector of the matching region, and the second eigenvector is optimized using the eigenvector of the matching region to obtain the third eigenvector corresponding to each second eigenvector, and a second vector matrix composed of all the third eigenvectors is output.

[0104] It should be understood that the region matching module is also called the local structure matching module. Since the first feature vector is also called the image vector, the second feature vector is the image vector after feature extraction, and the third feature vector is the optimized image vector, which contains the optimized local features, and is also called the optimized image vector or the optimized feature vector.

[0105] In the embodiment of the present application, the matching in step S203 is similar matching, which is used to achieve average noise reduction. Since the reference image is a similar image, and the matching area is an area in the similar image where the local image block corresponding to the second eigenvector is located with a high degree of similarity, the area is an external similar structure of the local image block.

[0106] In the embodiments of the present application, external similar structures generally exist in similar images for the following reasons: first, with the continuous addition of new images in recent years, the external image library contains rich structures; second, natural images have a non-local similarity prior, which indicates that there are still many similar structures in the image itself. When expanded to the outside, the number of these structures is huge and the structural features are richer.

[0107] Therefore, the regional matching module RMM can traverse and match the features extracted by the deep convolutional neural network DCNN in similar images to obtain external similar structures. Finally, the external similar structures are used to optimize the mean of the features extracted by DCNN, reduce the noise in the features, and perform feature vectorization output.

[0108] The embodiment of the present application focuses on describing in detail the first denoising process of the hybrid model: the first denoising process mainly uses a deep convolutional neural network to complete the feature extraction of the noise image, and uses a region matching module to complete the optimization of the extracted features.

[0109] In one possible implementation, since the image screening module 100 is mainly used to implement denoising and image similarity calculation operations, the image screening module 100 designed in the embodiment of the present application may include: a denoising unit, a similarity calculation unit and a screening unit connected in sequence.

[0110] Step S201, using an image screening module, according to a first vector matrix, obtaining a reference image whose similarity with the noise image reaches a first preset threshold from a preset reference image library, including the following steps:

[0111] S2011. Perform denoising processing on the first vector matrix through a denoising unit to obtain a denoised first vector matrix.

[0112] S2012. Using a similarity calculation unit, perform similarity calculations on the first vector matrix after denoising and on vector matrices of multiple images in a preset reference image library to obtain respective corresponding similarities.

[0113] In an embodiment of the present application, the preset reference image library contains a huge number of images (for example, tens of millions). If the first vector matrix after denoising is used to calculate the similarity with the vector matrices of all images in the preset reference image library, there is a problem of huge amount of calculation. In order to minimize the amount of calculation, the embodiment of the present application can reconstruct the preset reference image library. The reconstruction method is: classify and group the images, and the number of groups and the number of images in the group can be customized according to the actual situation. When performing similarity calculation, this embodiment calculates a few images according to the grouping situation, and can derive the similarity between the first vector matrix and all images in the image library, so that there is no need to traverse the calculation, so that the amount of calculation is reduced, thereby improving the calculation efficiency.

[0114] S2013. Screen out, by a screening unit, a reference image whose similarity with the noise image reaches a first preset threshold.

[0115] The embodiment of the present application introduces a high-quality external image as a reference image by executing steps S2011 to S2013 to provide a high-quality external similar structure; through DCNN and RMM, feature extraction of the first eigenvector corresponding to the local image block, matching of the external similar structure, and optimization of the second eigenvector are completed, thereby achieving the purpose of the first denoising.

[0116] In one possible implementation, Figure 3 As shown, the region matching module 300 includes a region matching unit (Region Matching Unit, RMU) 31, a feature optimization unit 32 and an output unit 33. Combined with the specific structure of the region matching module, step S203 is analyzed as follows:

[0117] The region matching unit 31 is used to traverse and match each second feature vector with multiple regions in the reference image to obtain a feature vector of the matching region.

[0118] The feature optimization unit 32 is used to optimize the second feature vectors using the feature vectors of the matching area to obtain a third feature vector corresponding to each second feature vector.

[0119] The output unit 33 is used to output a second vector matrix composed of all third eigenvectors.

[0120] In the embodiment of the present application, the region matching unit 31, the feature optimization unit 32 and the output unit 33 can introduce high-quality external images and optimize the local features of the noise image.

[0121] In summary, the first denoising process of the hybrid model is mainly to achieve the extraction and optimization of image features. The process mainly includes the following two aspects: one is to introduce a reference image, and the other is to use the regional matching module 300 to extract external similar structures from external images. In the feature optimization model in the preset hybrid model, the image feature extraction module 200 is used to extract local features of the noise image from the local structure of the original noise image through convolution operations; the image screening module 100 is used to use the image matching network framework to complete the search for similar images; the regional matching unit 31 matches the external similar structure in the similar image according to the local features extracted by the image feature extraction module 200; the feature optimization unit 32 uses enhancement methods such as weighted average algorithm to enhance the denoising effect of the local structure corresponding to the second feature vector, reduce the noise of the local structure, and obtain the third feature vector with enhanced denoising effect; finally, the optimized local features are vectorized and output through the output unit 33.

[0122] Embodiment 3:

[0123] Figure 4A flowchart of another image denoising method provided in an embodiment of the present application. Figure 1 Based on the illustrated embodiment, this embodiment describes the specific implementation of S30 in more detail.

[0124] In the embodiment of the present application, the sparse decomposition model in the preset hybrid model includes a cluster matching module and a sparse decomposition module connected in sequence. Figure 4 As shown, step S30, using the sparse decomposition model in the preset hybrid model to perform sparse decomposition processing on the second vector matrix to obtain a third vector matrix, includes the following steps:

[0125] S301. Clustering all third eigenvectors in the second vector matrix through a cluster matching module to obtain clustering results, and constructing a sparse transformation matrix based on the clustering results; wherein the rank of the sparse transformation matrix is ​​smaller than the rank of the second vector matrix.

[0126] In the embodiment of the present application, the embodiment of the present application does not specifically limit the type of clustering method used for clustering processing, and it can also be a K-means (K-means) clustering algorithm, a Gaussian mixture (Gaussian Mixture, GM) clustering algorithm, a density-based noise space (Density-Based Spatial Clustering of Applications with Noise, BDSCAN) clustering algorithm, etc. The sparse transformation matrix has a low-rank feature relative to the first vector matrix and the second vector matrix, so it is also called a low-rank matrix. In addition, the sparse transformation matrix is ​​also called a sparse matrix. The local structure corresponding to the sparse transformation matrix is ​​a clustering enhanced image structure.

[0127] In an embodiment of the present application, a cluster matching module is used to implement a rematching operation, which can enhance the image structure. Specifically: the i-th optimized image vector is correlated with other (N-n+1)×(N-n+1)-1 optimized vectors, the most relevant M optimized image vectors are selected for averaging, and the new image vector obtained after averaging replaces the i-th optimized image vector. The above process of obtaining similarity averaging can enhance structural features and suppress noise.

[0128] like Figure 5As shown in the figure, after the noise image is segmented, multiple local image blocks are obtained. The local image blocks are converted from two-dimensional vectors to one-dimensional vectors. The one-dimensional vectors corresponding to the local image blocks, after the first denoising, are clustered to obtain three clustering results (i.e., various types of image blocks), where each cluster contains image blocks as core features. Each local image block has a category to which it belongs, and then the internal similar structure (or similar block) is matched in the category to which it belongs; and then for each category, the internal similar structure is matched in the image blocks of the category to obtain a low-rank matrix.

[0129] Therefore, the purpose of clustering is to obtain the core features. In each category (the core features contain many image vectors belonging to this type of features), for each image vector, its similar blocks in a certain category need to be matched, and the degree of similarity varies. S similar image vectors at the head can be selected to form a low-rank matrix.

[0130] It should be understood that the similarity matching in the first denoising process is to search for external similar structures in similar images, so as to optimize the feature vector corresponding to the own image through the external similar structure, thereby realizing enhanced denoising of the own image; and the matching in the second denoising process is to search for internal similar structures in the own image within the clustering structure of the noise image, and then use low-rank optimization technology to complete the second denoising process, rather than simple averaging.

[0131] S302, performing singular value decomposition on the sparse transformation matrix through a sparse decomposition module to obtain singular values, performing threshold processing on the singular values ​​to obtain a singular value matrix, and outputting the singular value matrix as a third vector matrix.

[0132] The embodiment of the present application focuses on a detailed description of the second denoising process of the hybrid model: the second denoising process is mainly the optimization of the low-rank matrix. In the process of optimizing the low-rank matrix, the present embodiment performs singular value decomposition on the low-rank matrix to obtain P singular values. The singular values ​​represent the strength of the eigenvectors of the low-rank matrix. Strong eigenvectors are considered to have structural information, and weak eigenvectors are considered to be noise. Therefore, the embodiment of the present application can perform threshold processing on the singular values ​​corresponding to the eigenvectors in the low-rank matrix to achieve the second denoising and obtain the third vector matrix.

[0133] In a possible implementation, in step S302, threshold processing is performed on the singular values ​​to obtain a singular value matrix, including: retaining singular values ​​greater than or equal to a second preset threshold value through a singular value threshold algorithm, setting singular values ​​less than the second preset threshold value to preset values, and obtaining a singular value matrix after threshold processing.

[0134] It should be understood that the preset value can be zero or other values, and the embodiment of the present application does not specifically limit the range of the preset value. The singular value threshold algorithm is an algorithm commonly used in matrix nuclear norm minimization technology. The second preset threshold and the first preset threshold can both be customized values ​​according to actual needs, and the embodiment of the present application does not specifically limit these two thresholds. For example, the second preset threshold is set to threshold C, all singular values ​​less than threshold C are set to zero (denoising), and singular values ​​greater than C are retained.

[0135] In summary, the second denoising process of the hybrid model is mainly to achieve the optimization of the low-rank matrix. Specifically, after the output unit 33 vectorizes and outputs the optimized local features, clustering is performed through step S301, and in the clustering results of each category, the i-th third eigenvector (M×1) is similarly matched with other third eigenvectors contained in the clustering results of the category to obtain N internal similar structures, forming an MxN low-rank matrix, which has a low-rank characteristic. Then, through step S302, the singular value threshold algorithm included in the matrix nuclear norm minimization technology is used to perform threshold processing on the singular values ​​of the low-rank matrix (that is, the eigenvalues ​​below the threshold do not have feature information and are regarded as noise), and finally, image reconstruction is achieved through inverse transformation processing to obtain a denoised image.

[0136] In combination with Example 2 and Example 3, it can be seen that the image denoising method provided by the present application, in the first denoising process, completes the convergence of image features (i.e., external image matching and image feature extraction) through a neural network. The present application obtains high-quality image features from the external image during the convergence process, and uses the high-quality image features in the external image similar to the local image block as a one-dimensional variable, optimizes the second eigenvector corresponding to the local image block, and obtains the optimized eigenvector. Then in the second denoising process, the optimized eigenvectors corresponding to all local image blocks are used as one-dimensional variables to construct a two-dimensional sparse matrix (i.e., a low-rank matrix), and the nuclear norm minimization method is used to achieve the denoising purpose by reconstructing the low-rank matrix.

[0137] Embodiment 4:

[0138] Figure 6 A flowchart of another image denoising method provided in an embodiment of the present application is shown below. Figure 6 As shown, the image denoising method includes the following steps:

[0139] S61, constructing a vector matrix corresponding to the noise image; wherein the vector matrix is ​​composed of feature vectors corresponding to local image blocks after the noise image is divided into blocks.

[0140] It should be understood that the vector matrix corresponding to the noise image is also called the image vector matrix, and the eigenvector is also called the image vector. The construction process thereof is described in Example 1 and will not be repeated here. In step S61, the eigenvector refers to the first eigenvector in other embodiments.

[0141] S62, matching similar structures with respect to the local image blocks, and optimizing feature vectors corresponding to the local image blocks using the similar structures.

[0142] It should be understood that the similar structure refers to the external similar structure in other embodiments. Step S62 can optimize the similar structure of each image vector in the vector matrix. The process of similar structure optimization includes feature extraction and structure optimization. It can be seen from the description of steps S201 to S23 in the above embodiment 2 that the embodiment of the present application is based on the introduction of external high-quality similar images, matches the feature vector after feature extraction with the external similar structure, and then uses the external similar structure to optimize the feature vector corresponding to the local image block. In other words, since the feature vector corresponds to the local image block rather than the noise image, the above optimization is an optimization of the local structure.

[0143] S63. Perform a rematching operation on the vector matrix composed of all optimized eigenvectors to obtain a low-rank matrix.

[0144] In the embodiment of the present application, based on the number of local image blocks being (N-n+1)×(N-n+1), the size of the vector matrix formed by all optimized eigenvectors is (N-n+1)×(N-n+1). In step S63, all optimized eigenvectors are clustered and matched to achieve a rematching operation, thereby enhancing the image structure.

[0145] Among them, clustering is to cluster the image vectors using a set clustering method, and then use the clustering results as the target category (i.e., core features / structures); matching is based on the clustering results and matching within the range of the corresponding class, which can break through the range limitations of the traditional search window and obtain more matching images (i.e., internal similar structures) information to enhance the reconstruction performance of the image structure.

[0146] The purpose of step S63 is to further improve the quality of the image vector based on the optimized image vector to ensure the denoising effect of the low-rank matrix in the subsequent process. In the process of executing step S63, this embodiment can use a clustering method to obtain a low-rank matrix composed of similar blocks, and its low-rank characteristics are further enhanced.

[0147] S64. Perform singular value threshold processing on the low-rank matrix to obtain a singular value matrix.

[0148] It should be understood that the purpose of singular value threshold processing is to filter noise. The singular value matrix is ​​an optimized vector matrix. For low-rank matrices, this embodiment can use a matrix nuclear norm minimization method to filter noise information in the image.

[0149] S65, reconstructing the denoised image by performing an inverse transformation on the singular value matrix.

[0150] It should be understood that step S65 can achieve the conversion of image vectors into image blocks by using an inverse transformation process based on the singular value matrix. For pixel values ​​at overlapping positions, this embodiment can use an averaging method to determine the final pixel value.

[0151] In the above steps S61 to S65, the main features of the embodiment of the present application are: (1) introducing high-quality external images, and completing feature extraction and optimization of feature vectors corresponding to local image blocks through DCNN and RMM; (2) improving the image structure quality and matching accuracy through the two steps of re-matching and singular value threshold processing.

[0152] The image denoising method provided in the present application is a strong noise image denoising method based on a hybrid model. Since the hybrid model contains a dual noise filtering process (i.e., a dual noise suppression mechanism, the first layer is image feature extraction and optimization, and the second layer is low-rank matrix optimization), and in the dual noise filtering process, this embodiment combines the characteristics of the modeling method and the deep learning method, so the method can not only guarantee the denoising performance, but also significantly improve the generalization ability.

[0153] Figure 7 This is a schematic diagram of the results of the verification of the method provided by this application on the image processing standard data set. Among them, this embodiment selects the house and the butterfly as the effect demonstration, corresponding to Figure 7 The upper and lower figures of the first column in . Figure 7 The upper and lower images in the first column are both strong noise images. To obtain these two strong noise images, the embodiment of the present application artificially adds Gaussian noise of σ=100 to the standard image in advance (the image pixel is 8 bits, that is, its grayscale value range is 0-255).

[0154] from Figure 7 From the noise image given in the first column, it can be seen that the texture structure of the target object (i.e., the house, the butterfly) is almost completely submerged by the noise. In order to enhance the effect comparison of the embodiment of the present application, the existing weighted nuclear norm minimization (WNNM) method is introduced as a comparison scheme for denoising, where: Figure 7 The image in the second column is the denoised image obtained by the WNNM method; Figure 7The images in the third column are denoised images obtained using this scheme.

[0155] By comparison, it can be clearly seen that the WNNM method has two typical problems, the water ripples and structural distortion marked in the box, which have been well improved in the method of this application (hybrid). Moreover, by comparing the peak signal-to-noise ratio PSNR and the structural similarity factor SSIM through quantitative indicators, this application is 0.83dB higher (WNNM method: 27.67dB, this application: 28.5dB) and 0.012 higher (WNNM method: 0784, this application: 0.796), respectively, which can reflect that the denoising effect of this application is significantly improved.

[0156] In summary, the image denoising method based on the hybrid model of the present application has the characteristics of both the modeling method and the deep learning method. It can not only guarantee the denoising performance, but also significantly improve the generalization ability. It is very suitable for strong noise image denoising: on the one hand, the present application completes the screening and matching of external high-quality images through neural networks, and then uses the introduced external images to assist in improving the noisy images and provide more reference information; on the other hand, clustering is fused on the local image blocks corresponding to the optimized image vectors (or called the optimized local image structures) to obtain a low-rank matrix, and then optimize the "low-rank" characteristics of the low-rank matrix, enhance the low-rank denoising effect, and thus obtain a more accurate image denoising result (i.e., denoised image). Overall, the embodiment of the present application introduces an external image matching mechanism in the denoising process, which enhances the adaptability of the image denoising method to noisy images in different scenarios. At the same time, due to the dual noise suppression mechanism, it achieves a good "filtering" effect on strong noise, and also achieves high-fidelity restoration of the structural information of the image.

[0157] Embodiment 5:

[0158] Figure 8 Schematic diagram of the structure of an image denoising device provided in an embodiment of the present application. The device of this embodiment may be in the form of software and / or hardware. Figure 8 As shown, the image denoising device provided in this embodiment includes: an acquisition module 81, a feature optimization module 82, a sparse decomposition module 83 and an inverse transformation module 84. Among them:

[0159] The acquisition module 81 is used to acquire a noise image and construct a first vector matrix corresponding to the noise image.

[0160] The feature optimization module 82 is used to optimize the first vector matrix by using the feature optimization model in the preset hybrid model to obtain the second vector matrix.

[0161] The sparse decomposition module 83 is used to adopt a sparse decomposition model in a preset hybrid model to perform sparse decomposition processing on the second vector matrix to obtain a third vector matrix; wherein the rank of the third vector matrix is ​​smaller than the rank of the second vector matrix.

[0162] The inverse transformation module 84 is used to perform inverse transformation processing on the third vector matrix to obtain a denoised image.

[0163] In a possible implementation, the feature optimization model in the preset hybrid model includes: an image screening module, an image feature extraction module and a region matching module.

[0164] Among them, the output end of the image screening module and the output end of the image feature extraction module are both connected to the input end of the region matching module.

[0165] The feature optimization module 82 is further used for:

[0166] Through the image screening module, according to the first vector matrix, a reference image whose similarity with the noise image reaches a first preset threshold is obtained from a preset reference image library; wherein the variance of the reference image is smaller than the variance of the noise image.

[0167] Through the image feature extraction module, feature extraction is performed on each first eigenvector contained in the first vector matrix to obtain a second eigenvector corresponding to the first eigenvector; wherein the image feature extraction module is a deep convolutional neural network including multiple feature extraction layers.

[0168] Through the region matching module, for each second eigenvector, the second eigenvector is traversed and matched with multiple regions in the reference image to obtain the eigenvector of the matching region, and the second eigenvector is optimized using the eigenvector of the matching region to obtain the third eigenvector corresponding to each second eigenvector, and a second vector matrix composed of all the third eigenvectors is output.

[0169] In a possible implementation, the image screening module includes a denoising unit, a similarity calculation unit and a screening unit.

[0170] The feature optimization module 82 is further used for:

[0171] The first vector matrix is ​​denoised by a denoising unit to obtain a denoised first vector matrix.

[0172] The similarity calculation unit calculates the similarity between the first vector matrix after denoising and the vector matrices of multiple images in the preset reference image library to obtain the corresponding similarities.

[0173] The screening unit screens out reference images whose similarity to the noise image reaches a first preset threshold.

[0174] In a possible implementation, the sparse decomposition model in the preset hybrid model includes a cluster matching module and a sparse decomposition module.

[0175] The sparse decomposition module 83 is further used for:

[0176] All third eigenvectors in the second vector matrix are clustered through a cluster matching module to obtain a clustering result, and a sparse transformation matrix is ​​constructed based on the clustering result; wherein the rank of the sparse transformation matrix is ​​smaller than the rank of the second vector matrix.

[0177] The sparse transformation matrix is ​​subjected to singular value decomposition through the sparse decomposition module to obtain singular values, and the singular values ​​are subjected to threshold processing to obtain a singular value matrix, which is output as a third vector matrix.

[0178] In a possible implementation, the sparse decomposition module 83 is further used to:

[0179] Through the singular value threshold algorithm, singular values ​​greater than or equal to the second preset threshold are retained, and singular values ​​less than the second preset threshold are set to zero, and a singular value matrix is ​​obtained after threshold processing.

[0180] In a possible implementation, the image denoising device is further used to:

[0181] Calculate the index value of the denoised image under the evaluation index; wherein the evaluation index includes peak signal-to-noise ratio and / or structural similarity.

[0182] The image denoising device provided in this embodiment can be used to execute the image denoising method provided in any of the above method embodiments. The implementation principle and technical effect are similar and will not be described in detail here.

[0183] It should be noted that the user information and data involved in this application (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0184] That is to say, in the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the relevant laws and regulations and do not violate public order and good morals.

[0185] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0186] Fig. 9A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device includes a receiver 60, a transmitter 61, at least one processor 62 and a memory 63. The electronic device composed of the above components can be used to implement the above-mentioned several specific embodiments of the present application, which will not be described in detail here.

[0187] The embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, each step of the method in the above embodiment is implemented.

[0188] An embodiment of the present application also provides a computer program product, including a computer program, which implements each step of the method in the above embodiment when executed by a processor.

[0189] Various embodiments of the systems and techniques described above in the present application can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, which can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor, which can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0190] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or electronic device.

[0191] In the context of the present application, a computer-readable storage medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may be a machine-readable signal medium or a machine-readable storage medium. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a computer-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0192] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0193] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data electronic device), or a computing system that includes middleware components (e.g., an application electronic device), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0194] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps disclosed in this application can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in this application can be achieved, and this document does not limit this.

[0195] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of this application should be included in the protection scope of this application.

Claims

1. An image denoising method, characterized in that: include: Acquire a noise image, and construct a first vector matrix corresponding to the noise image; Using a feature optimization model in a preset hybrid model, the first vector matrix is ​​optimized to obtain a second vector matrix; Using a sparse decomposition model in a preset hybrid model, sparse decomposition processing is performed on the second vector matrix to obtain a third vector matrix; wherein the rank of the third vector matrix is ​​less than the rank of the second vector matrix; The third vector matrix is ​​subjected to inverse transformation processing to obtain a denoised image.

2. The method according to claim 1, characterized in that The feature optimization model in the preset hybrid model includes: an image screening module, an image feature extraction module and a region matching module; Wherein, the output end of the image screening module and the output end of the image feature extraction module are both connected to the input end of the region matching module; The method of using a feature optimization model in a preset hybrid model to optimize the first vector matrix to obtain a second vector matrix includes: By means of the image screening module, according to the first vector matrix, a reference image whose similarity with the noise image reaches a first preset threshold is obtained from a preset reference image library; wherein the variance of the reference image is smaller than the variance of the noise image; By means of the image feature extraction module, feature extraction is performed on each first feature vector contained in the first vector matrix to obtain a second feature vector corresponding to the first feature vector; Through the region matching module, for each of the second eigenvectors, the second eigenvector is traversally matched with multiple regions in the reference image to obtain the eigenvector of the matching region, and the second eigenvector is optimized using the eigenvector of the matching region to obtain the third eigenvector corresponding to each of the second eigenvectors, and a second vector matrix composed of all the third eigenvectors is output.

3. The method according to claim 2, characterized in that The image screening module comprises a denoising unit, a similarity calculation unit and a screening unit connected in sequence; The method further comprises: obtaining, by the image screening module, a reference image whose similarity with the noise image reaches a first preset threshold from a preset reference image library according to the first vector matrix, including: Performing denoising processing on the first vector matrix by the denoising unit to obtain a denoised first vector matrix; By means of the similarity calculation unit, similarity calculation is performed on the first vector matrix after denoising with vector matrices of multiple images in the preset reference image library to obtain respective corresponding similarities; The screening unit screens out reference images whose similarity with the noise image reaches a first preset threshold.

4. The method according to claim 1, characterized in that The sparse decomposition model in the preset hybrid model includes a cluster matching module and a sparse decomposition module connected in sequence; The method adopts a sparse decomposition model in a preset hybrid model to perform sparse decomposition processing on the second vector matrix to obtain a third vector matrix, including: By means of the cluster matching module, all the third eigenvectors in the second vector matrix are clustered to obtain a clustering result, and a sparse transformation matrix is ​​constructed based on the clustering result; wherein the rank of the sparse transformation matrix is ​​smaller than the rank of the second vector matrix; The sparse transformation matrix is ​​subjected to singular value decomposition by the sparse decomposition module to obtain singular values, and the singular values ​​are subjected to threshold processing to obtain a singular value matrix, which is output as a third vector matrix.

5. The method according to claim 4, characterized in that The step of performing threshold processing on the singular values ​​to obtain a singular value matrix includes: Through the singular value threshold algorithm, singular values ​​greater than or equal to the second preset threshold are retained, and singular values ​​less than the second preset threshold are set as preset values, and a singular value matrix is ​​obtained after threshold processing.

6. The method according to claim 1, characterized in that After performing inverse transformation processing on the third vector matrix to obtain a denoised image, the method further includes: Calculate the index value of the denoised image under the evaluation index; wherein the evaluation index includes peak signal-to-noise ratio and / or structural similarity.

7. The method according to claim 1, characterized in that The constructing a first vector matrix corresponding to the noise image includes: Dividing the noise image into a plurality of local image blocks by overlapping, wherein two adjacent local image blocks have overlapping areas; Performing vectorization processing on each of the local image blocks respectively to obtain a first eigenvector corresponding to each of the local image blocks; The first eigenvectors corresponding to all the local image blocks are concatenated to obtain a first vector matrix corresponding to the noise image; wherein each of the first eigenvectors is a row vector / column vector in the first vector matrix; and the number of columns / rows of the first vector matrix is ​​the number of the local image blocks contained in the noise image.

8. An image denoising device, characterized in that: include: An acquisition module, used for acquiring a noise image and constructing a first vector matrix corresponding to the noise image; A feature optimization module, used to optimize the first vector matrix by using a feature optimization model in a preset hybrid model to obtain a second vector matrix; A sparse decomposition module, used to use a sparse decomposition model in a preset hybrid model to perform sparse decomposition processing on the second vector matrix to obtain a third vector matrix; wherein the rank of the third vector matrix is ​​less than the rank of the second vector matrix; The inverse transformation module is used to perform inverse transformation processing on the third vector matrix to obtain a denoised image.

9. An electronic device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the image denoising method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the image denoising method according to any one of claims 1 to 7.

Images