An image processing method, apparatus, device and storage medium

Abstract

The application discloses an image processing method, device and equipment and a storage medium, and is applied to a denoising model. The denoising model comprises N denoising modules, wherein N is greater than or equal to 1, one of the N denoising modules comprises one filter layer, one Gaussian weight layer and one fusion layer, the method comprises the following steps: inputting a target image into the N denoising modules, performing denoising on the target image data through the N denoising modules, and obtaining denoised image data of the target image after denoising; if N is greater than 1, the N denoising modules are connected in series; and based on the denoised image data, the denoising model is trained to obtain a target denoising model for the target image. The scheme has the characteristics of small calculation amount, good interpretability, strong adjustability, strong adaptability, small dependence on training data, cold start, and small probability of information loss.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and relates to but is not limited to an image processing method and device, equipment and a storage medium. BACKGROUND

[0002] With the rapid development of computer technology, more and more technologies are applied in the financial field, and the traditional financial industry is gradually changing to Fintech. However, due to the security and real-time requirements of the financial industry, higher requirements are also put forward for technology.

[0003] For image noise repair, the related art processing can include a filter-based denoising method and a deep learning-based denoising method.

[0004] The filter-based denoising method mainly realizes denoising by manually adjusting the parameters of the filter. The deep learning-based denoising method mainly establishes an end-to-end neural convolutional network, and then trains a large number of images to obtain a black-box denoising model.

[0005] Analysis of the above denoising methods can obtain that the filter-based denoising method depends on the parameter adjustment and experience of algorithm experts, and has poor denoising effect and poor adaptability to the processing of different images. The deep learning-based denoising method seriously depends on training data, lacks cold start capability for types not included in the training data, has poor interpretability, requires high computational capacity and equipment, has high training difficulty, and has a high probability of information loss. SUMMARY

[0006] The present application provides an image processing method and device, equipment and a storage medium, which can obtain a target denoising model for a target image for different target images, and has the characteristics of small computational complexity, good interpretability, strong adjustability, strong adaptability, small dependence on training data, cold start, and small probability of information loss.

[0007] The technical solution of the present application is implemented as follows:

[0008] The present application provides an image processing method, which is applied to a denoising model, the denoising model includes N denoising modules, N is greater than or equal to 1, one of the N denoising modules includes a filter layer, a Gaussian weight layer and a fusion layer, and the method includes:

[0009] The target image is input into the N denoising modules, and the target image data is denoised by the N denoising modules to obtain denoised image data of the target image. If N is greater than 1, the N denoising modules are connected in series.

[0010] The first denoising module is any denoising module in the N denoising modules, and the target image data is denoised by the first denoising module, including:

[0011] The first image data of the target image with a pixel size of a x b is input to the filter layer of the first denoising module, the target image is subjected to first filtering processing by the filter layer, and second image data with a dimension of a x b is obtained; the weight values of each pixel in the Gaussian weight layer of the first denoising module are indexed based on the pixel coordinates in the a x b pixels of the target image, and a first Gaussian weight matrix with a dimension of a x b is obtained; the first image data, the second image data, and the first Gaussian weight matrix are input to the fusion layer of the first denoising module, and third image data with a dimension of a x b is obtained through the processing of the fusion layer.

[0012] The denoising model is trained based on the denoised image data, and a target denoising model for the target image is obtained.

[0013] The present application provides an image processing device, which is deployed in a denoising model, the denoising model includes N denoising modules, N is greater than or equal to 1, one of the N denoising modules includes a filter layer, a Gaussian weight layer and a fusion layer, and the device includes:

[0014] The denoising unit is configured to input a target image into the N denoising modules, denoise the target image data through the N denoising modules, and obtain denoised image data of the target image after denoising; if N is greater than 1, the N denoising modules are connected in series.

[0015] The denoising unit is specifically configured to:

[0016] The first image data of the target image with a pixel size of a x b is input to the filter layer of the first denoising module, the target image is subjected to first filtering processing by the filter layer, and second image data with a dimension of a x b is obtained; the weight values of each pixel in the Gaussian weight layer of the first denoising module are indexed based on the pixel coordinates in the a x b pixels of the target image, and a first Gaussian weight matrix with a dimension of a x b is obtained; the first image data, the second image data, and the first Gaussian weight matrix are input to the fusion layer of the first denoising module, and third image data with a dimension of a x b is obtained through the processing of the fusion layer.

[0017] The training unit is configured to train the denoising model based on the denoised image data, and obtain a target denoising model for the target image.

[0018] The application further provides an electronic device, comprising a memory and a processor, the memory storing a computer program capable of running on the processor, and the processor implements the image processing method when executing the program.

[0019] The application further provides a storage medium storing a computer program, which implements the image processing method when executed by a processor.

[0020] The image processing method, device, equipment and storage medium provided by the application are applied to a denoising model, the denoising model comprises N denoising modules, N is greater than or equal to 1, one of the N denoising modules comprises a filter layer, a Gaussian weight layer and a fusion layer, and the method comprises: inputting a target image into the N denoising modules, denoising the target image data through the N denoising modules to obtain denoised image data of the target image after denoising; if N is greater than 1, the N denoising modules are connected in series; for a first denoising module, the first denoising module is any denoising module in the N denoising modules, and the first denoising module is used to denoise the target image data, which comprises: inputting first image data of a target image with a pixel size of a x b into a filter layer of the first denoising module, performing first filter processing on the target image through the filter layer to obtain second image data with a dimension of a x b; indexing the weight value of each pixel in the Gaussian weight layer of the first denoising module based on the pixel coordinates in the a x b pixels of the target image to obtain a first Gaussian weight matrix with a dimension of a x b; inputting the first image data, the second image data and the first Gaussian weight matrix into a fusion layer of the first denoising module, and obtaining third image data with a dimension of a x b through processing of the fusion layer; and training the denoising model based on the denoised image data to obtain a target denoising model for the target image.

[0021] For the scheme of the application, first, a target denoising model for a target image can be obtained for different target images, and different images are automatically processed, so the degree of automation is high, and the adaptability is good; second, the filter layer of the application is implemented based on the principle of the filter, so it has high readability and interpretability; third, the denoising model can be trained to obtain a target denoising model with optimal parameters, and the denoising effect is good; fourth, the training process of the application does not involve the reconstruction process of the image, and less training data is required, so the data processing amount of the training process is small, and the implementation is simple; fifth, the fusion layer fuses multiple types of information, and tries to retain the original image information as much as possible, reducing the probability of information loss; sixth, the filter layer has certain filtering capability, so it can be started cold. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 An optional structural schematic diagram of the image processing system provided by the embodiment of the present application is provided.

[0023] Figure 2 An optional flow schematic diagram of the image processing method provided by the embodiment of the present application is provided.

[0024] Figure 3 An optional flow schematic diagram of the image processing method provided by the embodiment of the present application is provided.

[0025] Figure 4 An optional flow schematic diagram of the image processing method provided by the embodiment of the present application is provided.

[0026] Figure 5 An optional flow schematic diagram of the image processing method provided by the embodiment of the present application is provided.

[0027] Figure 6 An optional structural schematic diagram of the denoising scoring network provided by the embodiment of the present application is provided.

[0028] Figure 7 An optional logic schematic diagram of the image processing process provided by the embodiment of the present application is provided.

[0029] Figure 8 An optional logic schematic diagram of the image processing process provided by the embodiment of the present application is provided.

[0030] Figure 9 An optional structural schematic diagram of the image processing apparatus provided by the embodiment of the present application is provided.

[0031] Figure 10 An optional structural schematic diagram of the electronic device provided by the embodiment of the present application is provided. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the specific technical solutions of the present application will be further described in detail below with reference to the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but not to limit the scope of the present application.

[0033] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0034] In the following description, the terms "first," "second," and "third" are used only to distinguish different objects and do not represent a specific order of objects, nor are they constituting a chronological order. It is understood that "first," "second," and "third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0036] This application provides an image processing method, apparatus, device, and storage medium. In practical applications, the image processing method can be implemented by an image processing apparatus, and the functional entities in the image processing apparatus can be collaboratively implemented by the hardware resources of the electronic device, such as computing resources like processors and communication resources (such as those used to support various communication methods like optical fiber and cellular).

[0037] The image processing method provided in this application is applied to an image processing system, which includes an electronic device and a target image to be denoised.

[0038] An electronic device is configured to perform the following: inputting a target image into the N denoising modules, denoising the target image data through the N denoising modules, and obtaining denoised image data of the target image; wherein, if N is greater than 1, the N denoising modules are connected in series; wherein, for the first denoising module, the first denoising module is any one of the N denoising modules, and denoising the target image data through the first denoising module includes: inputting first image data of a target image with pixel size a×b into the filtering layer of the first denoising module, and denoising the target image data through the filtering layer. The target image undergoes a first filtering process to obtain a second image data of a×b dimensions. Based on the pixel coordinates of a×b pixels in the target image, the weight values ​​of each pixel in the Gaussian weight layer of the first denoising module are indexed to obtain a first Gaussian weight matrix of a×b dimensions. The first image data, the second image data, and the first Gaussian weight matrix are input to the fusion layer of the first denoising module, and the third image data of a×b dimensions is obtained through the processing of the fusion layer. Based on the denoised image data, the denoising model is trained to obtain a target denoising model for the target image.

[0039] Furthermore, the denoised target image is obtained based on the target denoising model.

[0040] As an example, the structure of an image processing system can be as follows: Figure 1 As shown, it includes: electronic device 10 and target image 20.

[0041] Here, the electronic device 10 is used to perform: inputting a target image to the N denoising modules, denoising the target image data through the N denoising modules, and obtaining denoised image data after denoising the target image; wherein, if N is greater than 1, the N denoising modules are connected in series; wherein, for the first denoising module, the first denoising module is any one of the N denoising modules, and denoising the target image data through the first denoising module includes: inputting the first image data of the target image with pixel size a×b to the filtering layer of the first denoising module, and denoising the target image data through the filtering layer... The first layer performs a first filtering process on the target image to obtain a second image data of a×b dimensions; based on the pixel coordinates of a×b pixels in the target image, the weight value of each pixel in the Gaussian weight layer of the first denoising module is indexed to obtain a first Gaussian weight matrix of a×b dimensions; the first image data, the second image data, and the first Gaussian weight matrix are input to the fusion layer of the first denoising module, and through the processing of the fusion layer, a third image data of a×b dimensions is obtained; based on the denoised image data, the denoising model is trained to obtain a target denoising model for the target image.

[0042] The electronic device 10 can be an electronic device with relevant data processing capabilities. For example, the electronic device 10 can be a mobile terminal device (such as a mobile phone, tablet computer, etc.) or a non-mobile terminal device (such as a desktop computer, server, etc.).

[0043] It should be noted that the specific number of target images 20 is not limited in the embodiments of this application. However, different target denoising models can be obtained by the image processing method provided in the embodiments of this application for different target images 20.

[0044] The image processing process provided in this application embodiment will be described below using the processing of a target image as an example.

[0045] Below, in conjunction with Figure 1 The schematic diagram of the image processing system shown illustrates various embodiments of the image processing methods, apparatus, devices, and storage media provided in this application.

[0046] In a first aspect, embodiments of this application provide an image processing method, which is applied to an image processing apparatus; wherein the image processing apparatus can be deployed in... Figure 1 The electronic device 10 is used as the execution subject. The image processing process provided in the embodiments of this application will now be described using the electronic device as the execution subject.

[0047] Figure 2 This illustrates a flowchart of one possible image processing method, for reference. Figure 2 The image processing method shown may include, but is not limited to, the following: Figure 2 S201 and S202 are shown.

[0048] The image processing method is applied to a denoising model, which includes N denoising modules, where N is greater than or equal to 1. One of the N denoising modules includes a filtering layer, a Gaussian weighting layer, and a fusion layer.

[0049] The embodiments of this application do not limit the number of denoising modules included in the denoising model, and can be configured according to actual conditions.

[0050] In one possible implementation, N equals 1, and the denoising model includes a denoising module.

[0051] In another possible implementation, N is greater than 1, and the denoising model includes multiple denoising modules. In this implementation, the N denoising modules are connected in series. That is, in the N denoising modules, the output of the previous module serves as the input of the next module.

[0052] It should be noted that the multiple noise reduction modules are not the same. For example, the first filtering module can be used to filter salt-and-pepper noise, and the second filtering module can be used to filter Gaussian noise.

[0053] Therefore, the more denoising modules a denoising model includes, the better the denoising effect.

[0054] S201. The electronic device inputs the target image to the N denoising modules, and the N denoising modules denoise the target image data to obtain the denoised image data after denoising the target image.

[0055] The target image refers to the noisy image to be denoised. For example, the target image could be a noisy image acquired due to insufficient lighting, or a noisy image obtained by re-photographing an old photograph, etc.

[0056] Wherein, if N equals 1, the output of the denoising module is used as the denoised image data; if N is greater than 1, the output of the last denoising module in the N cascaded denoising modules is used as the denoised image data.

[0057] In S201, the processing method of each denoising module is similar. The denoising process of the first denoising module will now be explained using the first denoising module as an example. For example... Figure 2 As shown, the process may include, but is not limited to, S2011 to S2013 below.

[0058] S2011. The electronic device inputs the first image data of a target image with a pixel size of a×b into the filtering layer of the first denoising module, and performs a first filtering process on the target image through the filtering layer to obtain the second image data of a×b dimension.

[0059] This application does not impose a specific limitation on the pixel size of the target image, which can be determined according to actual conditions. That is, the specific values ​​of a and b are not limited.

[0060] The first image data of the target image refers to the input data of the filtering layer. When the first filtering module is the first filtering module, the first image data can be the initial color data of the target image; when the first filtering module is the second filtering module, the first image data can be the data after denoising by the first filtering module.

[0061] The color data can be red, green, and blue (RGB) color data, or grayscale data of the target image, etc.

[0062] The filtering layer is used to denoise the target image. This application does not limit the implementation of the first filtering layer; it can be determined according to the actual situation.

[0063] In one possible implementation, the filtering layer can be implemented based on filters.

[0064] In another possible implementation, the filtering layer can also be implemented based on a convolutional network.

[0065] This application does not specifically limit the filtering method of the filtering layer, which can be determined according to the actual situation (e.g., noise type). For example, the filtering layer can be configured as an arithmetic mean filter or a geometric filter.

[0066] S2011 can be implemented as follows: The electronic device inputs the first image data of the target image with a pixel size of a×b into the filtering layer of the first filtering module, and performs a first filtering process on the first image data in the target image through the filtering layer to obtain the filtered second image data of a×b dimension.

[0067] It can be seen that the dimensions of the filtered second image data are the same as those of the first image data.

[0068] S2012. The electronic device indexes the weight values ​​of each pixel in the Gaussian weight layer of the first denoising module based on the pixel coordinates of the a×b pixels of the target image to obtain the first Gaussian weight matrix of the a×b dimension.

[0069] The Gaussian weighting layer corresponds one-to-one with the filtering layer and is used to perform weighted processing on the second image data obtained from the filtering layer.

[0070] The embodiments of this application do not limit the specific type of the Gaussian weight layer of the first filtering module, and can be determined according to the actual situation. For example, the Gaussian weight layer can be a two-dimensional normal distribution weight type.

[0071] The first Gaussian weight matrix output by the Gaussian weight layer also has a×b dimensions, which is consistent with the pixel size of the target image, that is, consistent with the dimensions of the first image data and the second image data.

[0072] For example, S2012 can be implemented as follows: the electronic device indexes the weight value of each pixel in the Gaussian weight layer of the first denoising module based on the pixel coordinates in the a×b pixels of the target image, for example, based on the two-dimensional Gaussian distribution function, to calculate the Gaussian weight of the pixel and obtain the first Gaussian weight matrix of the a×b dimension.

[0073] S2013. The electronic device inputs the first image data, the second image data, and the first Gaussian weight matrix into the fusion layer of the first denoising module, and obtains the third image data of dimension a×b through the processing of the fusion layer.

[0074] The fusion layer of the first denoising module is used to fuse the image data before and after filtering by the filtering layer with Gaussian weights.

[0075] The dimensions of the fused third image data are the same as those of the first and second image data, respectively.

[0076] S2013 can be implemented as follows: For each pixel in the a×b pixel array, the electronic device fuses its first image value in the first image with its second image value in the second image data, taking the first Gaussian weight value in the first Gaussian weight matrix as an example, to obtain the third image value of the pixel in the third image data. The electronic device iterates through all pixels in the a×b pixel array to obtain the third image data in the a×b dimension.

[0077] If N is greater than 1, the processing procedures for the other N-1 denoising modules are the same as those in S2011 to S2013 above, and will not be repeated here. It should be noted that the difference between each denoising module is that the input of each denoising module is different, and the output of the previous module is used as the input of the next module.

[0078] For example, the input to the first denoising module is the original color data of the target image, the input to the second denoising module is the output of the first denoising module, and the output of the last denoising module is used as the denoised image data.

[0079] S202. The electronic device trains the denoising model based on the denoised image data to obtain a target denoising model for the target image.

[0080] S202 can be implemented as follows: The electronic device reverses the parameters in the denoising model based on the denoised image data, and obtains new denoised image data through the adjusted parameters. This process is repeated to obtain the denoising parameters and denoising model with the best denoising effect. The denoising model with the best denoising effect is then determined as the target denoising model for the target image.

[0081] The image processing scheme provided in this application includes: applying a denoising model, wherein the denoising model includes N denoising modules, where N is greater than or equal to 1, and one of the N denoising modules includes a filtering layer, a Gaussian weighting layer, and a fusion layer, comprising: inputting a target image to the N denoising modules, and denoising the target image data through the N denoising modules to obtain denoised image data of the target image; wherein, if N is greater than 1, the N denoising modules are connected in series sequentially; wherein, for a first denoising module, the first denoising module is any one of the N denoising modules, and denoising the target image data through the first denoising module includes: denoising pixels... First image data of a target image with size a×b is input to the filtering layer of the first denoising module. The target image is subjected to a first filtering process by the filtering layer to obtain second image data of size a×b. Based on the pixel coordinates of a×b pixels in the target image, the weight value of each pixel in the Gaussian weight layer of the first denoising module is indexed to obtain a first Gaussian weight matrix of size a×b. The first image data, the second image data, and the first Gaussian weight matrix are input to the fusion layer of the first denoising module. The third image data of size a×b is obtained through the processing of the fusion layer. Based on the denoised image data, the denoising model is trained to obtain a target denoising model for the target image.

[0082] Regarding the solution proposed in this application, firstly, it can obtain target denoising models for different target images, and the processing is automatic for different images, resulting in a high degree of automation and good adaptability; secondly, the filtering layer of this application is implemented based on the principle of filters, thus possessing high readability and interpretability; thirdly, this application can train the denoising model to obtain a target denoising model with optimal parameters, resulting in good denoising performance; fourthly, the training process of this application does not involve image reconstruction and requires less training data, thus minimizing data processing volume and simplifying implementation; fifthly, the melting layer fuses multiple types of information, preserving as much original image information as possible and reducing the probability of information loss; and sixthly, due to the inherent filtering capability of the filtering layer, cold start is possible.

[0083] The following describes the process by which the electronic device in S2013 inputs the first image data, the second image data, and the first Gaussian weight matrix into the fusion layer of the denoising module, and obtains the third image data of dimension a×b through the processing of the fusion layer.

[0084] Specifically, for each pixel in the a×b pixels of the target image, the third image value corresponding to the pixel in the third image data is calculated based on the following first formula.

[0085] The first formula includes: final i =in i ×(1ω i )+out i ×ω i ;

[0086] Where i represents the pixel, and final i This represents the third image value corresponding to the pixel in the third image data, where in i The ω represents the first image value corresponding to the pixel in the first image data. i The out represents the first Gaussian weight value corresponding to the pixel in the first Gaussian weight matrix. i This represents the second image value corresponding to the pixel in the second image data.

[0087] It should be noted that ω in the first Gaussian weight matrix i The default value is used during the initial processing; during training, ω... i It will adjust as the parameters in the first Gaussian weight layer are adjusted.

[0088] In this way, the fused image data includes the first image data (such as the original data of the target image), so no information loss is caused while denoising, and the comprehensiveness of information can be guaranteed. On the other hand, the fusion ratio between the first image data and the second image data can be adjusted by the weight values ​​in the first weight matrix, which is highly flexible.

[0089] The following section provides a detailed explanation of the filtering types in the filtering layer.

[0090] The filtering layer includes any one of the following:

[0091] Arithmetic mean filters or geometric filters based on the spatial domain

[0092] Frequency domain-based small filter transform filters or notch filters

[0093] Maximum, minimum, or median filters based on statistical sorting.

[0094] It is understandable that the filtering layer can also include other filtering types, which will not be listed here.

[0095] Specifically, different filter types can be packaged into different program modules. In the actual implementation, different filter program modules can be directly called to configure the filtering layer according to the noise characteristics of the target image.

[0096] This provides a wide range of filter types to choose from, improving the flexibility and effectiveness of noise reduction.

[0097] The following describes the implementation methods of the filtering layer. Specifically, it may include, but is not limited to, method 1 or method 2 described below.

[0098] Method 1: Implementation of filter principle;

[0099] Method 2: The implementation method of convolutional network layer principle.

[0100] In Method 1, the principle of the filter is directly described through a program to obtain the filtering layer. This method is characterized by its simplicity and reliability in the noise reduction process.

[0101] In Method 2, the filter method is converted into a convolutional network layer method, and the filter layer is obtained by configuring the convolutional network layer. This method has the advantage of fast batch processing speed in the denoising process.

[0102] Below, taking an example with a denoising sliding window size of m×n, we will explain the process of implementing the filtering layer through a convolutional network layer. Figure 3 As shown, the process may include, but is not limited to, S301 and S302.

[0103] The sliding window is the smallest processing unit in the filtering process. This application does not limit the specific size of the sliding window; it can be configured according to actual needs.

[0104] S301. The electronic device configures the kernel size of the first convolutional network layer to be m×n.

[0105] Specifically, the electronic device invokes a convolutional network as the first convolutional network layer; the kernel size of the first convolutional network is configured to be m×n.

[0106] S302, The electronic device configures the convolutional weights of the first convolutional network layer as follows:

[0107] The electronic design configures the kernel size in the first convolutional network layer to be m×n.

[0108] It can be seen that the kernel size and convolution weights of the first convolutional network layer are related to the sliding window in the filtering process.

[0109] The following describes the process in S202 where the electronic device trains a denoising model based on the denoised image data to obtain a target denoising model for the target image. For example... Figure 4 As shown, the process may include, but is not limited to, S2021 to S2024 below.

[0110] S2021. The electronic device determines the denoising score corresponding to the denoising model based on the denoised image data.

[0111] The denoising score is used to characterize the denoising effect of the denoising model on the target image. A better denoising effect corresponds to a higher denoising score.

[0112] The embodiments of this application do not limit the specific method for determining the denoising score corresponding to the denoising model based on the denoised image, and can be determined according to the actual situation.

[0113] In one possible implementation, the electronic device can pre-train a denoising model scoring network and obtain a denoising score based on the scoring network.

[0114] In another possible implementation, the electronic device can measure the noise in the denoised image using a noise detection operator to obtain a denoising score. The higher the noise value detected by the noise detection operator, the lower the corresponding denoising score.

[0115] S2022. The electronic device determines a first loss value based at least on the denoising score.

[0116] The electronic device determines the loss function, inputs the denoising score into the loss function, and obtains the first loss value.

[0117] The embodiments of this application do not limit the specific type of loss function, which can be determined according to the actual situation.

[0118] S2023. The electronic device adjusts the parameters in the denoising model in reverse based on the first loss value to obtain a new denoising model, and denoises the target image again based on the new denoising model to obtain new denoised image data of the target image, and determines a new denoising score corresponding to the new denoising model based on the new denoised image data; and determines a new first loss value based at least on the new denoising score.

[0119] The parameters in the reverse-adjusted denoising model may include, but are not limited to: the filtering parameters of the filtering layer in each of the N filtering modules, and the weight adjustment parameters of the Gaussian weight layer in each of the filtering modules.

[0120] The filtering parameters in the filtering layer may differ or be the same depending on the filter type. For example, for an arithmetic mean filter, the corresponding filtering parameters are the sizes m and n of the denoising sliding window.

[0121] The weight adjustment parameters in the Gaussian weight layer are used to control the range and intensity of denoising. For example, the weight adjustment parameters may include μ1, μ2, σ1, and σ2, where μ1 and μ2 control the center coordinates of the denoising range, and σ1 and σ2 control the intensity of denoising.

[0122] The process of processing the new denoising model after parameter tuning to obtain the new first loss value is similar to the processing of the original denoising model and the process of determining the first loss value. The specific implementation process will not be described in detail.

[0123] S2024. If the electronic device does not converge with the new first loss value, it uses the new first loss value as the first loss value and re-executes: adjusting the parameters in the denoising model in reverse based on the first loss value until the new first loss value converges; if the new first loss value converges, the new denoising model corresponding to the converged new first loss value is determined as the target denoising model.

[0124] S2024 can be implemented as follows: If the electronic device does not converge with the new first loss value, it uses the new first loss value as the first loss value and re-executes: based on the first loss value, it reversely adjusts the parameters in the denoising model to obtain a new denoising model; it re-denoises the target image based on the new denoising model to obtain new denoised image data of the target image, and determines a new denoising score corresponding to the new denoising model based on the new denoised image data; it determines a new first loss value based at least on the new denoising score; until the new first loss value converges; if the new first loss value converges, it determines the new denoising model corresponding to the converged new first loss value as the target denoising model.

[0125] Thus, the loss value corresponding to the obtained denoising model is convergent, meaning the parameters of the denoising model are optimal, and the resulting target denoising model is the best denoising model for the target image. When denoising the target image using the target denoising model, the denoising effect is good.

[0126] The process by which the S2021 electronic device determines the denoising score corresponding to the denoising model based on the denoised image data will be described below. This process may include, but is not limited to, the following method A or method B.

[0127] Method A: Determine the denoising score based on a denoising network scoring model;

[0128] Method B: Determine the denoising score based on the noise points detected by the noise detection operator.

[0129] In Method A, a convolutional neural network is pre-trained to determine the denoising score of the image. Specifically, Method A involves inputting the denoised image data into a denoising scoring model, and obtaining the denoising score through the processing of the denoising scoring model.

[0130] Method A is characterized by its simplicity and high reliability.

[0131] The implementation of method B may include, but is not limited to, the following S20211 to S20213.

[0132] S20211, The electronic device determines at least one noise operator.

[0133] Each of the at least one noise detection operator corresponds to a noise detection direction.

[0134] Since a noise-free image can be considered a smooth image, an edge detection operator can be configured as a noise detection operator.

[0135] For example, the electronic device identifies four noise operators: noise operator 1, noise operator 2, noise operator 3, and noise operator 4. Noise operator 1 is... Corresponding to the horizontal detection direction; noise operator 2 is Corresponding to the vertical detection direction, noise operator 3 is For a detection direction of -45 degrees, noise operator 4 is The detection direction corresponds to 45 degrees.

[0136] S20212. The electronic device detects noise values ​​in at least one direction in the denoised image data using the at least one noise operator.

[0137] The electronic device multiplies the denoised image data with a noise operator to obtain the noise value in that direction. For example, this can be achieved using a convolutional network.

[0138] S20213. The electronic device determines the denoising score based on the noise value in the at least one direction.

[0139] The electronic device sums the noise values ​​from all directions to obtain the total noise value, and determines the denoising score based on the total noise value.

[0140] The lower the noise value, the higher the noise reduction score.

[0141] Method B is characterized by its simple principle.

[0142] The process by which the S2022 electronic device determines a first loss value based at least on the denoising score will be described below.

[0143] In one possible embodiment, the first loss value is determined based on the following second formula;

[0144] The second formula includes:

[0145] L = L2(1 / score) + λ × MSE(noisedimage, inputimage); where L represents the first loss value, L2 represents the second normal form loss function, score represents the denoising score, λ represents the adjustment coefficient, MSE represents the mean squared error loss function, noisedimage represents the denoised image data, and inputimage represents the first image data.

[0146] This loss function can maximize the score of the denoised image; and it can minimize the difference between the first image data and the denoised image data, resulting in better performance.

[0147] In another possible implementation, L 2 Where L represents the first loss value and r represents the denoising score.

[0148] The loss function is simple to implement.

[0149] The image processing method provided in this application embodiment can also adjust the parameters in the target denoising model. The specific adjustment process may include, but is not limited to, […]. Figure 5 S203 and S204 in the example.

[0150] S203, The electronic device receives the input value of the first parameter of the target denoising model.

[0151] The first parameter includes the filtering parameters of the first filtering layer or the weight adjustment parameters of the first Gaussian weight layer.

[0152] The user inputs the value of the first parameter of the target denoising model through the input interface of the electronic device, and the electronic device receives the input value of the first parameter of the target denoising model through the receiving interface.

[0153] S204. The electronic device adjusts the filtering parameters of the first filtering layer or the weight adjustment parameters of the first Gaussian weight layer in the target denoising model based on the value of the first parameter.

[0154] Thus, after obtaining the target denoising model, it can be assumed that adjusting the parameters in the target denoising model has good adaptability and high adjustability.

[0155] In some embodiments, the image processing method provided in this application may further include a process of denoising a target image based on a target denoising model. This process may include: an electronic device inputting the target image into the target denoising model; and the electronic device processing the target image through the target denoising model to obtain a target denoised image corresponding to the target image.

[0156] The image processing method provided in this application will now be described through an embodiment.

[0157] To facilitate understanding, some technical terms will be explained first.

[0158] Noisy images refer to images affected by noise. Noisy images are one of the main obstacles to image processing algorithms. The noise in noisy images comes from two main sources: damaged storage media or errors in the acquisition process. Factors that may lead to errors during acquisition can include: light intensity, scene changes, subject movement, camera performance, digital image information compression, and other reasons; it can also include information loss during secondary acquisition when re-examining old photographs. Noise in images not only degrades data quality but also affects the performance and accuracy of algorithms.

[0159] Noisy image inpainting operators: These are operators used to repair various types of noisy images. Examples include spatial domain-based mean filters and geometric mean filters; frequency domain-based wavelet transform filters and notch filters; and statistical sorting-based maximum, minimum, and median filters.

[0160] Noise probability density functions describe noise models that conform to various probability density functions, providing a useful tool for modeling a wide range of noise contamination scenarios. For example, Gaussian noise in images is caused by factors such as electronic circuit noise and sensor noise (due to insufficient lighting and / or high temperature). Rayleigh density helps characterize noise phenomena in distance imaging.

[0161] For image noise restoration, the relevant techniques can be divided into the following two main categories:

[0162] The first category consists of image restoration algorithms based on various filters. Examples include spatial domain-based mean filtering and geometric mean filtering; frequency domain-based wavelet transform filters and notch filtering; and statistical sorting-based maximum, minimum, and median filters.

[0163] The second category consists of denoising methods based on deep learning. Examples include denoising models based on encoder-decoder structures and denoising models based on U-net structures.

[0164] The first type of image noise restoration method has the following drawbacks: First, noise restoration through filters requires precise parameters, thus relying on the parameter tuning and experience of algorithm experts, making fully automated noise restoration impossible. Second, because filter-based noise restoration may require different filter parameters for different images, parameter adjustment depends on manual intervention, making it impossible to automatically adjust parameters for different images, resulting in poor adaptability and an inability to make targeted adjustments for specific images. Third, limited human experience makes it impossible to obtain optimal parameters, leading to poor denoising results.

[0165] The first type of image noise inpainting method has the following drawbacks: First, deep learning-based denoising methods heavily rely on training data and lack cold-start capability for types not included in the training data. Second, deep learning-based denoising methods are generally based on end-to-end neural network models, neglecting internal parameters and failing to estimate the probability model of noise, resulting in poor interpretability. Third, deep learning-based denoising methods, being based on end-to-end neural network models, do not focus on internal parameters and cannot adjust internal parameters or processes, making it difficult to adjust the denoising effect. Fourth, deep learning-based denoising methods require extracting features from the image, performing neural network calculations on these features, and finally reconstructing the image from the features, resulting in large data volumes, high computational demands, and high equipment requirements. Fifth, deep learning training is a process from scratch, requiring a large amount of training data and a long training period, making training difficult. Sixth, deep learning-based denoising methods often extract features from the image and then reconstruct the entire image. During reconstruction, some image details may be lost.

[0166] This application proposes a parameter-optimized noisy image inpainting technique. This technique combines traditional filter methods and deep learning methods, implementing the filter as a filtering layer in a denoising network. A Gaussian weight map is used to weight the output of the filtering layer, and then a gradient descent algorithm is employed to optimize the parameter sets of the filtering layer and the Gaussian weight map, thereby achieving parameter optimization. This application's embodiment performs denoising through the filtering layer; adjusts and controls the denoising range and intensity through the Gaussian weight map; and simultaneously trains the denoising model using a reverse gradient adjustment algorithm, ensuring that the parameters of the filtering layer and the Gaussian weight layer in the denoising model are optimized for each image, thus achieving optimal denoising results.

[0167] This embodiment of the application has the following technical effects:

[0168] First, the denoising model of this application is based on the design of a filter-based denoising operator, which has good interpretability and good parameter readability.

[0169] Second, this embodiment of the application implements the filter operator as part of the denoising network, so that it has a certain denoising effect even without training, reducing the dependence on training data.

[0170] Third, in this embodiment of the application, the parameters in the denoising model are mainly optimized based on the backpropagation algorithm when training the denoising model, which makes the training less difficult.

[0171] Fourth, after obtaining the optimal parameters of the denoising model, the operator can adjust or correct the parameters according to the actual effect and business needs.

[0172] The specific implementation process of this embodiment of the present application will be described below.

[0173] The input in this embodiment of the application is: a noisy image (an image can also be called a picture), for example, a picture with noise caused by problems in storage, acquisition, or transmission; or a noisy picture caused by camera shake; or a sensor noise picture caused by insufficient light or high temperature, etc.

[0174] The input in this embodiment of the application is: the denoised image, that is, the image after retaining the effective information of the image and repairing the noise of the image through the filtering layer.

[0175] Specifically, this embodiment may include, but is not limited to, steps 1 to 11 below.

[0176] Step 1: Prepare the image dataset.

[0177] The image samples in the image dataset can be collected from real-world use cases, such as facial images or identification documents in financial services scenarios.

[0178] Step 2: Determine the training data based on the image dataset.

[0179] For the image dataset from step 1, random noise such as salt-and-pepper noise, Gaussian noise, and uniform noise are added to generate a noisy image set. Then, the original image set and the noisy image set are used as positive and negative sample sets, respectively, to form the training data.

[0180] Step 3: Train the denoising and scoring network (equivalent to the denoising and scoring model).

[0181] Use the training data prepared in step 2 to train the denoising and scoring network (score_net).

[0182] The task of ScoreNet is to score the denoising effect of an image. This task has low complexity, so a complex network structure is not required. This embodiment uses a 3-layer convolutional neural network, trained using a binary classification loss function (cross-entropy).

[0183] like Figure 6 As shown, each image in the training data is input into the denoising scoring network. After calculation by the first layer convolutional neural network 601, the second layer convolutional neural network 602, and the third layer convolutional neural network 6703 in the denoising scoring network, the score corresponding to each image is obtained. Then, based on the score of the image and the label score corresponding to the image, the parameters of each layer convolutional neural network in the denoising scoring network are adjusted to obtain the trained denoising scoring network.

[0184] Step 4: Implement commonly used filters such as arithmetic mean filter and geometric mean filter as a filter layer (Denoise Layer).

[0185] Some filters can be implemented using convolution, such as the formula for the arithmetic average filter, as shown in formula (1) below.

[0186]

[0187] In formula (1), Let g(r, c) represent the filtered image data, and S represent the noisy image data. xy This represents a set of coordinates for a rectangular sub-image window (also known as a denoising sliding window) centered at (x, y) and of size m × n.

[0188] It can be seen that this filter can be implemented as a spatial convolution kernel of size m×n, where all coefficients are... That is, the following matrix A.

[0189]

[0190] Therefore, the arithmetic mean filter can be converted into a convolutional network, with a convolutional sum of size m×n and convolutional weights of...

[0191] The geometric mean filter can be implemented as shown in the following formula (2).

[0192]

[0193] In formula (2), Let g(r, c) represent the filtered image data, and S represent the noisy image data. xy This represents a set of coordinates for a rectangular sub-image window (also known as a denoising sliding window) centered at (x, y) and of size m × n.

[0194] The calculation process of the geometric mean filter involves multiplying the elements within the sub-region S and then raising the product to the power of 1 / (m×n). Unlike the arithmetic mean filter, the geometric mean filter cannot be transformed into a discrete convolution kernel. Instead, it performs forward propagation calculations using a sliding window according to the formula and backward propagation according to the derivative formula.

[0195] Step 5: Determine the Gaussian weight values ​​for the Gaussian weight layer.

[0196] This embodiment can use a weighted graph W that conforms to a Gaussian distribution. i The output of the i-th filter is weighted.

[0197] It can be known that W iThe dimension is the same as the output dimension of the i-th filter.

[0198] The Gaussian weights can satisfy the following formula (3).

[0199]

[0200] In formula (3), μ1, μ2, σ1, and σ2 are the parameters of the Gaussian weight layer, which are assumed to be two-dimensional normal distributions. x and y represent pixel coordinates, and G(x, y) is the weight value of that pixel.

[0201] The parameters of the Gaussian weights are adjusted during training to achieve optimal performance.

[0202] Each filtering layer corresponds to a Gaussian weighting layer.

[0203] Step 6: Determine the output.

[0204] For each filter layer, assume its input is in i The output is out. i .

[0205] Then use the Gaussian weight map W generated in step 5. i to in i and out i Perform a weighted summation and output "final". i Please refer to formula (3) below for details.

[0206] final i =in i ×(1ω i )+out i ×ω i Formula (4);

[0207] In formula (4), final i Indicates the final output data, in i ω represents the input data of the filter layer. i Indicates Gaussian weights, out i This represents the output data of the filter layer.

[0208] Step 7: Define the denoise network (denoise_net) (equivalent to the denoise model).

[0209] The network input consists of a noisy image (Inoise) and a parameter set (Params). The Inoise is the target image containing noise, and the Params set contains the parameters of each denoise layer and the parameters of the corresponding Gaussian weight map for each layer.

[0210] The parameters of the filtering layer may be the same or different depending on the filtering method. For example, for arithmetic mean filtering, the parameters of the filtering layer can be m and n as mentioned above; the parameters of the Gaussian weight layer include μ1, μ2, σ1, and σ2.

[0211] The above filtering layers and Gaussian weight layers are combined to form an end-to-end Denoise_net, and the Params in Denoise_net are optimized by score_net.

[0212] like Figure 7 As shown, the input is image 1. Image 1 is passed through the first filtering layer 701, the first Gaussian weight layer 702, the second filtering layer 703, the second Gaussian weight layer 704, the third filtering layer 705, and the third Gaussian weight layer 706 to obtain the denoised image 2. Image 2 is then input into the scoring network 707 for scoring.

[0213] Step 8: Define the loss function L for denoise_net.

[0214] L contains two optimization objectives: first, to maximize the score of the denoised image on score_net; second, to minimize the difference between the denoised image and the input image.

[0215] The loss function can be expressed as shown in the following formula (5).

[0216] L=L2(1score)+λ×MSE(noisedimage, inputimage) Formula (5);

[0217] In formula (5), L represents the loss value, L2 represents the second normal form loss function, score represents the denoising score, λ is the adjustment coefficient, which adjusts the weight ratio of the two optimization objectives; MSE represents the mean squared error loss function, noisedimage represents the denoised output data, and inputimage represents the input image data.

[0218] Step 9: Train denoise_net and optimize Params.

[0219] The input image is fed into the denoising network, and the optimal parameters Params are obtained by optimizing the loss function L.

[0220] It should be noted that during this process, the parameters of denoise_net are frozen, that is, the learning rate of denoise_net is set to 0; correspondingly, only Params are updated.

[0221] like Figure 8 As shown, the input is image 1. Image 1 is passed through the first filtering layer 701, the first Gaussian weight layer 702, the second filtering layer 703, the second Gaussian weight layer 704, the third filtering layer 705, and the third Gaussian weight layer 706 to obtain the denoised image 2. Image 2 is then input into the scoring network 707 for scoring. Params are adjusted and updated based on the scores. The parameters of denoise_net are then frozen.

[0222] Step 10: After iteration, training stops when the loss function L converges.

[0223] Thus, the optimal Params are obtained, resulting in the target denoising network.

[0224] Step 11: Input the image into the target denoising network again to obtain the target denoised image.

[0225] Since the parameters in the target denoising network are the optimal parameters obtained by training on image 1, inputting parameter 1 into the target denoising network can denoise image 1 to the greatest extent, resulting in a good target denoised image.

[0226] The process of determining the denoising score for the aforementioned denoising scoring network can be achieved through a noise detection network.

[0227] Based on noise detection filtering, a noise detection network is implemented.

[0228] This method employs a directional noise detection operator that assumes that noise-free images should have local smoothness.

[0229] The kernel of the noise operator is as follows:

[0230] Noise operator 1 is Corresponding to the horizontal detection direction; noise operator 2 is Corresponding to the vertical detection direction, noise operator 3 is For a detection direction of -45 degrees, noise operator 4 is The detection direction corresponds to 45 degrees.

[0231] The four directional nuclei are k1, k2, k3, and k4, respectively.

[0232] The input image is convolved with kernels k1-k4 in four directions to obtain k1(image), k2(image), k3(image), and k4(image).

[0233] The mean of the above results is calculated as r = mean(k1(image) + k2(image) + k3(image) + k4(image)).

[0234] The second normal form of the loss function is Loss = r 2 .

[0235] It should be noted that for this method, minimizing the loss function will eventually lead to the smoothing of the image information itself. Therefore, an early stop method needs to be used in the denoising process to control the number of iterations.

[0236] To implement the above image processing method, an image processing apparatus according to an embodiment of this application is described below in conjunction with... Figure 9 The schematic diagram of the image processing device shown is used to illustrate the structure.

[0237] The device is deployed on a denoising model, which includes N denoising modules, where N is greater than or equal to 1. One of the N denoising modules includes a filtering layer, a Gaussian weighting layer, and a fusion layer.

[0238] like Figure 9 As shown, the image processing apparatus 90 includes a denoising unit 901 and a training unit 902. Wherein:

[0239] The denoising unit 901 is used to input the target image into the N denoising modules, and to denoise the target image data through the N denoising modules to obtain denoised image data of the target image; wherein, if N is greater than 1, the N denoising modules are connected in series in sequence.

[0240] Specifically, the noise reduction unit is used for:

[0241] First image data of a target image with pixel size a×b is input to the filtering layer of the first denoising module. The target image is subjected to a first filtering process by the filtering layer to obtain second image data of dimension a×b. Based on the pixel coordinates of the a×b pixels of the target image, the weight value of each pixel in the Gaussian weight layer of the first denoising module is indexed to obtain a first Gaussian weight matrix of dimension a×b. The first image data, the second image data, and the first Gaussian weight matrix are input to the fusion layer of the first denoising module. The third image data of dimension a×b is obtained through the processing of the fusion layer.

[0242] Training unit 902 is used to train the denoising model based on the denoised image data to obtain a target denoising model for the target image.

[0243] In some embodiments, the noise reduction unit 901 is further configured to:

[0244] For each pixel in the a×b pixels of the target image, the third image value corresponding to the pixel in the third image data is calculated based on the following first formula.

[0245] The first formula includes: final i =in i ×(1ω i )+out i ×ω i ;

[0246] Where i represents the pixel, and final i This represents the third image value corresponding to the pixel in the third image data, where in i The ω represents the first image value corresponding to the pixel in the first image data. i The out represents the first Gaussian weight value corresponding to the pixel in the first Gaussian weight matrix. i This represents the second image value corresponding to the pixel in the second image data.

[0247] In some embodiments, the filtering layer includes any of the following:

[0248] Arithmetic mean filters or geometric filters based on the spatial domain

[0249] Frequency domain-based small filter transform filters or notch filters

[0250] Maximum, minimum, or median filters based on statistical sorting.

[0251] In some embodiments, when the size of the denoising sliding window is m×n, the filtering layer includes a first convolutional network layer; the image processing apparatus further includes a configuration unit, which is specifically used for:

[0252] Configure the kernel size of the first convolutional network layer to be m×n;

[0253] Configure the convolution weights of the first convolutional network layer as follows:

[0254] In some embodiments, the training unit 902 is specifically used for:

[0255] The denoising score corresponding to the denoising model is determined based on the denoised image data; the denoising score is used to characterize the denoising effect of the denoising model on the target image;

[0256] A first loss value is determined based at least on the denoised score;

[0257] Based on the first loss value, the parameters in the denoising model are adjusted in reverse to obtain a new denoising model. The target image is then denoised again based on the new denoising model to obtain new denoised image data of the target image. A new denoising score corresponding to the new denoising model is determined based on the new denoised image data. At least based on the new denoising score, a new first loss value is determined.

[0258] If the new first loss value does not converge, the new first loss value is used as the first loss value, and the following steps are repeated: adjusting the parameters in the denoising model in reverse based on the first loss value until the new first loss value converges; if the new first loss value converges, the new denoising model corresponding to the converged new first loss value is determined as the target denoising model.

[0259] The parameters in the denoising model include at least the filtering parameters of the first filtering layer and the weight adjustment parameters of the first Gaussian weight layer.

[0260] In some embodiments, the training unit 902 is further configured to:

[0261] The denoised image data is input into a denoising scoring model, and the denoising score is obtained through the processing of the denoising scoring model; the denoising scoring model is configured as a convolutional neural network.

[0262] or,

[0263] At least one noise operator is determined; each of the at least one noise detection operator corresponds to a noise detection direction;

[0264] The noise value in at least one direction in the denoised image data is detected by the at least one noise operator;

[0265] The denoising score is determined based on the noise value in at least one direction.

[0266] In some embodiments, the training unit 902 is further configured to:

[0267] The first loss value is determined based on the following second formula;

[0268] The second formula includes:

[0269] L = L2(1 / score) + λ × MSE(noisedimage, inputimage); where L represents the first loss value, L2 represents the second normal form loss function, score represents the denoising score, λ represents the adjustment coefficient, MSE represents the mean squared error loss function, noisedimage represents the denoised image data, and inputimage represents the first image data.

[0270] In some embodiments, the image processing apparatus further includes a parameter tuning unit, which is used to:

[0271] The system receives the numerical values ​​of the first parameter of the target denoising model as input; the first parameter includes the filtering parameters of the first filtering layer or the weight adjustment parameters of the first Gaussian weighting layer.

[0272] The filtering parameters of the first filtering layer or the weight adjustment parameters of the first Gaussian weight layer in the target denoising model are adjusted based on the numerical values ​​of the first parameter.

[0273] In some embodiments, the image processing apparatus further includes a processing unit, the processing unit being configured to:

[0274] The target image is input into the target denoising model;

[0275] The target image is processed by the target denoising model to obtain the target denoised image corresponding to the target image.

[0276] It should be noted that the image processing apparatus provided in this application embodiment includes all the units included, which can be implemented by a processor in an electronic device; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field-programmable gate array (FPGA), etc.

[0277] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0278] It should be noted that, in the embodiments of this application, if the above-described image processing method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0279] To implement the above image processing method, this application provides an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the steps of the image processing method provided in the above embodiments.

[0280] The following is combined Figure 10 The electronic device 100 shown is illustrated with a structural diagram.

[0281] In one example, electronic device 100 can be the aforementioned electronic device. For example... Figure 10 As shown, the electronic device 100 includes: a processor 1001, at least one communication bus 1002, a user interface 1003, at least one external communication interface 1004, and a memory 1005. The communication bus 1002 is configured to enable communication between these components. The user interface 1003 may include a display screen, and the external communication interface 1004 may include standard wired and wireless interfaces.

[0282] The memory 1005 is configured to store instructions and applications executable by the processor 1001, and can also cache data to be processed or already processed by the processor 1001 and various modules in the electronic device (e.g., image data, audio data, voice communication data and video communication data), which can be implemented by flash memory or random access memory (RAM).

[0283] Fourthly, embodiments of this application provide a storage medium, namely a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps in the image processing method provided in the above embodiments.

[0284] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0285] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0286] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0287] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0288] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0289] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0290] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0291] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0292] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, The method is applied to a denoising model, which includes N denoising modules, where N is greater than or equal to 1. One of the N denoising modules includes a filtering layer, a Gaussian weighting layer, and a fusion layer. The method includes: The target image is input to the N denoising modules, and the target image is denoised by the N denoising modules to obtain denoised image data of the target image; wherein, if N is greater than 1, the N denoising modules are connected in series. Specifically, the first denoising module, which is any one of the N denoising modules, denoises the target image data using the first denoising module, including: With pixel size The first image data of the target image is input to the filtering layer of the first denoising module, and the target image is subjected to a first filtering process by the filtering layer to obtain... Second-dimensional image data; based on the target image The pixel coordinates of each pixel are used to index the weight value of each pixel in the Gaussian weight layer of the first denoising module, thus obtaining... The first Gaussian weight matrix of dimension 1; the first image data, the second image data, and the first Gaussian weight matrix are input to the fusion layer of the first denoising module, and processed by the fusion layer to obtain... Third-dimensional image data; Based on the denoised image data, the denoising model is trained to obtain a target denoising model for the target image; The step of training a denoising model based on the denoised image data to obtain a target denoising model for the target image includes: The denoising score corresponding to the denoising model is determined based on the denoised image data; the denoising score is used to characterize the denoising effect of the denoising model on the target image; A first loss value is determined based at least on the denoised score; Based on the first loss value, the parameters in the denoising model are adjusted in reverse to obtain a new denoising model. The target image is then denoised again based on the new denoising model to obtain new denoised image data of the target image. A new denoising score corresponding to the new denoising model is determined based on the new denoised image data. At least based on the new denoising score, a new first loss value is determined. If the new first loss value does not converge, the new first loss value is used as the first loss value, and the following steps are repeated: adjusting the parameters in the denoising model in reverse based on the first loss value until the new first loss value converges; if the new first loss value converges, the new denoising model corresponding to the converged new first loss value is determined as the target denoising model. The parameters in the denoising model include at least the filtering parameters of the first filtering layer and the weight adjustment parameters of the first Gaussian weighting layer.

2. The method according to claim 1, characterized in that, The first image data, the second image data, and the first Gaussian weight matrix are input into the fusion layer of the first denoising module. Through processing by the fusion layer, the following is obtained: The third dimension of image data includes: For the target image Each of the pixels is calculated based on the following first formula to obtain the third image value corresponding to the pixel in the third image data; The first formula includes: ; Among them, the Represents the pixel, the This represents the third image value corresponding to the pixel in the third image data. This represents the first image value corresponding to the pixel in the first image data. This represents the first Gaussian weight value corresponding to the pixel in the first Gaussian weight matrix. This represents the second image value corresponding to the pixel in the second image data.

3. The method according to claim 1, characterized in that, The filtering layer includes any one of the following: Arithmetic mean filters or geometric filters based on the spatial domain Frequency domain-based small filter transform filters or notch filters Maximum, minimum, or median filters based on statistical sorting.

4. The method according to claim 1, characterized in that, The size of the noise reduction sliding window is In the case where the filtering layer includes a first convolutional network layer; the method further includes: Configure the kernel size of the first convolutional network layer as follows: ; Configure the convolution weights of the first convolutional network layer as follows: .

5. The method according to claim 1, characterized in that, Determining the denoising score corresponding to the denoising model based on the denoised image data includes: The denoised image data is input into a denoising scoring model, and the denoising score is obtained through the processing of the denoising scoring model; the denoising scoring model is configured as a convolutional neural network. or, At least one noise operator is determined; each of the at least one noise detection operator corresponds to a noise detection direction; The noise value in at least one direction in the denoised image data is detected by the at least one noise operator; The denoising score is determined based on the noise value in at least one direction.

6. The method according to claim 1, characterized in that, Determining the first loss value based at least on the denoised score includes: The first loss value is determined based on the following second formula; The second formula includes: ;in, Represents the first loss value, the Denotes the second normal form loss function, the Represents the denoising score, the Indicates the adjustment coefficient, the The mean squared error loss function is represented by the following. The denoised image data represents the data in question. This refers to the first image data.

7. The method according to claim 1, characterized in that, The method further includes: The system receives the numerical values ​​of the first parameter of the target denoising model as input; the first parameter includes the filtering parameters of the first filtering layer or the weight adjustment parameters of the first Gaussian weighting layer. The filtering parameters of the first filtering layer or the weight adjustment parameters of the first Gaussian weight layer in the target denoising model are adjusted based on the numerical values ​​of the first parameter.

8. The method according to claim 1, characterized in that, The method further includes: The target image is input into the target denoising model; The target image is processed by the target denoising model to obtain the target denoised image corresponding to the target image.

9. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor runs the computer program, the image processing method according to any one of claims 1 to 8 is implemented.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image processing method according to any one of claims 1 to 8.

Images