Image noise elimination method and device and storage medium

Abstract

The invention relates to an image noise elimination method and device and a storage medium. The image noise elimination method comprises the steps of obtaining a to-be-denoised image; calling a first model to denoise a flat region of the to-be-denoised image to obtain a first denoised image; reducing the receptive field of the to-be-denoised image to obtain a preprocessed image; calling a second model to de-noise the texture region of the preprocessed image to obtain a second de-noised image; and determining a target de-noised image based on the first de-noised image and the second de-noised image. According to the invention, the denoising effect of the image with complex noise can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image noise elimination method, device and storage medium. Background Art

[0002] Image denoising is an important research direction in the field of digital image processing.

[0003] In the related art, for noisy images, a denoising model is usually used to denoise the image. The denoising model used can be divided into a self-supervised denoising model and a non-self-supervised denoising model according to the training method.

[0004] Image noise in real life is relatively complex. In practical applications, the two current denoising models can only denoise images with relatively simple noise. When faced with images with relatively complex noise, neither of them can achieve a good denoising effect. Summary of the invention

[0005] In order to overcome the problems existing in the related art, the present disclosure provides an image noise elimination method, device and storage medium.

[0006] According to a first aspect of an embodiment of the present disclosure, there is provided an image noise elimination method, comprising: acquiring an image to be denoised; calling a first model to denoise a flat area of ​​the image to be denoised to obtain a first denoised image; reducing the receptive field of the image to be denoised to obtain a preprocessed image; calling a second model to denoise a texture area of ​​the preprocessed image to obtain a second denoised image; and obtaining a target denoised image based on the first denoised image and the second denoised image.

[0007] In one embodiment, the second model is determined in the following manner: determine a first training image and a first output image, wherein the first training image is used to train the first model, and the first output image is an image output by the first model based on the first training image; and train the second model based on the first training image and the first output image.

[0008] In another embodiment, the training of the second model based on the first training image and the first output image includes: reducing the receptive field of the first training image to obtain a third training image, extracting the texture area of ​​the first output image, and determining a texture mask of the texture area image; inputting the third training image into the second preset model for denoising to obtain a second output image; determining a second loss function based on the first output image, the second output image and the texture mask; and training constraining the second preset model based on the second loss function to obtain the second model.

[0009] In another embodiment, the first model is trained based on the following method, including: acquiring a first training image; determining a blind spot pixel area in the first training image, the blind spot pixel area including a central pixel point of the first training image and a target pixel area centered on the central pixel point; in the first training image, adjusting the pixel value of the blind spot pixel area to be less than a threshold to obtain a second training image; inputting the second training image into the first preset model for denoising to obtain a first output image; determining a first loss function based on the first image and the first training image; and training constraining the first preset model based on the first loss function to obtain the first model.

[0010] In another embodiment, determining the second loss function based on the first output image, the second output image and the texture mask includes: taking the second output image as a prediction result and taking the first output image as an actual result to construct a second sub-loss function; and weighting the second sub-loss function based on the texture mask to obtain the second loss function.

[0011] In yet another embodiment, obtaining a target denoised image based on the first denoised image and the second denoised image includes: performing weighted fusion on the first denoised image and the second denoised image, and determining the fused image as the target denoised image.

[0012] In another embodiment, it is characterized in that obtaining a target denoised image based on the first denoised image and the second denoised image includes: inputting the image to be denoised into a third model to obtain a third denoised image, wherein the third model is determined based on the first model and the second model; and weightedly fusing the first denoised image, the second denoised image and the third denoised image to obtain a target denoised image.

[0013] In another embodiment, the third model is determined based on the first model and the second model, including: inputting the first training image into the third preset model for denoising to obtain a third output image; determining a third loss function based on the first output image during the training process of the first model, the second output image during the training process of the second model, and the texture mask during the training process of the second model; determining the training constraints of the third preset model based on the third loss function to obtain the third model.

[0014] In another embodiment, the third loss function is determined based on the first output image, the second output image, the third output image and the texture mask in the following manner: the pixel value set of the first output image is determined as the first data set, the pixel value set of the second output image is determined as the second data set and the pixel value set of the third output image is determined as the third data set; the third data set is used as the prediction result and the first data set is used as the actual result to construct a third sub-loss function; the third data set is used as the prediction result and the second data set is used as the actual result to construct a fourth sub-loss function; and the third sub-loss function and the fourth sub-loss function are weightedly fused based on the texture mask to obtain a third loss function.

[0015] According to a second aspect of an embodiment of the present disclosure, there is provided an image noise elimination device, comprising: an acquisition unit, configured to acquire an image to be denoised;

[0016] The processing unit is used to call the first model to denoise the flat area of ​​the image to be denoised to obtain a first denoised image; reduce the receptive field of the image to be denoised to obtain a preprocessed image; call the second model to denoise the texture area of ​​the preprocessed image to obtain a second denoised image; and obtain a target denoised image based on the first denoised image and the second denoised image.

[0017] In one embodiment, the second model is determined based on the following method: determining a first training image and a first output image, wherein the first training image is used to train the first model, and the first output image is an image output by the first model based on the first training image training; and training the second model based on the first training image and the first output image.

[0018] In another embodiment, the second model is trained based on the first training image and the first output image in the following manner: the receptive field of the first training image is reduced to obtain a third training image, and the texture area of ​​the first output image is extracted, and a texture mask of the texture area image is determined; the third training image is input into the second preset model for denoising to obtain a second output image; a second loss function is determined based on the first output image, the second output image and the texture mask; and training constraints are imposed on the second preset model based on the second loss function to obtain the second model.

[0019] In another embodiment, the first model is trained based on the following method: obtaining a first training image; determining a blind spot pixel area in the first training image, wherein the blind spot pixel area includes a central pixel point of the first training image and a target pixel area centered on the central pixel point; in the first training image, adjusting the pixel value of the blind spot pixel area to be less than a threshold value to obtain a second training image; inputting the second training image into the first preset model for denoising to obtain a first output image; determining a first loss function based on the first image and the first training image; and training constraining the first preset model based on the first loss function to obtain the first model.

[0020] In another embodiment, a second loss function is determined based on the first output image, the second output image and the texture mask in the following manner: the second output image is used as a prediction result, and the first output image is used as an actual result to construct a second sub-loss function; the second sub-loss function is weighted based on the texture mask to obtain the second loss function.

[0021] In another embodiment, the processing unit is used to obtain a target denoised image based on the first denoised image and the second denoised image in the following manner: weighted fusion of the first denoised image and the second denoised image, and determining the fused image as the target denoised image.

[0022] In another embodiment, the processing unit obtains a target denoised image based on the first denoised image and the second denoised image in the following manner: inputting the image to be denoised into a third model to obtain a third denoised image, wherein the third model is determined based on the first model and the second model; and weightedly fusing the first denoised image, the second denoised image and the third denoised image to obtain a target denoised image.

[0023] In another embodiment, the third model is determined based on the first model and the second model in the following manner: the first training image is input into the third preset model for denoising to obtain a third output image; a third loss function is determined based on the first output image during the training process of the first model, the second output image during the training process of the second model, and the texture mask during the training process of the second model; the training constraints of the third preset model are determined based on the third loss function to obtain the third model.

[0024] In another embodiment, the third loss function is determined based on the first output image, the second output image, the third output image and the texture mask in the following manner: the pixel value set of the first output image is determined as the first data set, the pixel value set of the second output image is determined as the second data set and the pixel value set of the third output image is determined as the third data set; the third data set is used as the prediction result and the first data set is used as the actual result to construct a third sub-loss function; the third data set is used as the prediction result and the second data set is used as the actual result to construct a fourth sub-loss function; and the third sub-loss function and the fourth sub-loss function are weightedly fused based on the texture mask to obtain a third loss function.

[0025] According to a third aspect of the present disclosure, there is provided an image noise elimination device, comprising:

[0026] processor;

[0027] A memory for storing processor executable instructions; wherein the processor is configured to: execute the image noise elimination method described in the first aspect or any one of the embodiments of the first aspect.

[0028] According to a fourth aspect of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions in the storage medium are executed by a processor of a device, the device is enabled to execute the image noise elimination method described in the first aspect or any one of the embodiments of the first aspect.

[0029] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by calling the first model and the second model to perform denoising on the flat area and the texture area of ​​the denoised image respectively, and weighted fusion of the denoising results, a final denoised image is obtained. By distinguishing between the texture area and the flat area of ​​the denoised image, and performing targeted denoising on the texture area and the flat area, the denoising ability of the image with complex noise is improved compared to the denoising method that does not distinguish between regions in the related art, thereby improving the denoising effect of the actual image denoising.

[0030] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0032] Figure 1 The figure is a flow chart of a method for removing image noise according to an exemplary embodiment.

[0033] Figure 2 is a flow chart of a second model determination method according to an exemplary embodiment.

[0034] Figure 3 is a flow chart of yet another second model training method according to an exemplary embodiment.

[0035] Figure 4 It is a flow chart of a first model training method according to an exemplary embodiment.

[0036] Figure 5 The figure is a schematic diagram of a blind pixel area according to an exemplary embodiment.

[0037] Figure 6 The figure is a flow chart of a method for determining a second loss function according to an exemplary embodiment.

[0038] Figure 7 is a flowchart of yet another image processing method according to an exemplary embodiment.

[0039] Figure 8 is a flow chart of a third model training method according to an exemplary embodiment.

[0040] Fig. 9 The figure is a flow chart of a method for determining a third loss function according to an exemplary embodiment.

[0041] Fig.10 is a schematic diagram of a third model training scenario according to an exemplary embodiment.

[0042] Fig.11 The figure is a block diagram of a device for image noise elimination according to an exemplary embodiment.

[0043] Fig.12 The invention is a block diagram of a device for image processing according to an exemplary embodiment.

[0044] Fig.13 The invention is a block diagram of a device for image processing according to an exemplary embodiment. DETAILED DESCRIPTION

[0045] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0046] In the accompanying drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limitations on the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present disclosure. The embodiments of the present disclosure are described in detail below in conjunction with the accompanying drawings.

[0047] In recent years, research on computer vision, deep learning, machine learning, image processing, image recognition and other technologies based on artificial intelligence has made important progress. Artificial Intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies and application systems for simulating and extending human intelligence. Artificial intelligence is a comprehensive discipline involving many types of technologies such as chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, neural networks, etc. Computer vision, as an important branch of artificial intelligence, specifically allows machines to recognize the world. Computer vision technology usually includes face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, target detection, pedestrian recognition, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, behavior recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, robot navigation and positioning and other technologies. With the research and advancement of artificial intelligence technology, this technology has been applied in many fields, such as security, urban management, traffic management, building management, park management, facial access, facial attendance, logistics management, warehouse management, robots, intelligent marketing, computational photography, mobile phone imaging, cloud services, smart homes, wearable devices, unmanned driving, automatic driving, smart medical care, facial payment, facial unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile Internet, live streaming, beauty, makeup, medical beauty, and smart temperature measurement.

[0048] In the field of digital image processing, image denoising is an important research direction. When shooting or transmitting images, they may encounter various influencing factors, such as lens astigmatism or signal interference during image transmission. These influencing factors will bring interference signals to the original image, that is, image noise. For example, Gaussian noise, salt and pepper noise, etc. are all common analog noises.

[0049] In the field of machine learning, supervised learning or unsupervised learning is usually used to train a denoising model for image denoising, and the trained denoising model is used to implement denoising of the image. The specific implementation method is as follows:

[0050] To train a denoising model using supervised learning, we first need to model image noise (i.e., construct simulated noise) and describe the noise through the model. Then, we obtain a large number of noise-free images and use the noise model to sample the images to obtain simulated noise images, which together with the original noise-free images form a data set. Finally, we use a traditional or deep learning-based model to perform supervised training on the data of the constructed data set (noise-noise-free) to obtain a denoising model, and use this denoising model to remove noise.

[0051] The denoising model is trained in an unsupervised learning manner, usually taking a noisy image as input and then training a neural network to learn how to restore a clean image from the noisy image.

[0052] However, due to the particularity of their learning methods, the above two denoising models have limitations in actual image denoising scenarios. For example, the supervised learning method requires the construction of a data set (noise-free), which is very difficult to obtain data, so the actual training of the model in this learning method is difficult; the unsupervised learning method requires the assumption that the noise space is independent (for example: there is no superposition of multiple noises), so the trained model has poor denoising effect in scenes with multiple noise superposition. In real image noise, there are many cases of multiple noise superposition. Therefore, the denoising model trained by unsupervised learning cannot adapt to most image noise denoising scenarios.

[0053] In view of this, the embodiment of the present disclosure provides an image noise elimination method, in which the pixel values ​​corresponding to the center point of the image to be processed and the area around the center point are set to zero, the correlation between the noise in the image and the space is reduced, a first sub-image is obtained, the first sub-image is denoised, and the processed target image is determined based on the first output result obtained after denoising. By reducing the correlation between the noise in the image and the space, the processing capability of the unsupervised denoising model for complex noise is improved, thereby improving the denoising effect for actual image denoising.

[0054] Among them, the image processing method provided by the embodiment of the present disclosure can be executed by a device with image processing capabilities, such as a terminal, a computer, etc.

[0055] It should be noted that in the following embodiments, the image to be denoised can be understood as an image with noise, and the situation with noise includes one or more of the following examples, for example: the image to be denoised can be an image with a single noise, or an image with multiple noises superimposed. For another example, the image to be denoised can be an image that carries only Gaussian noise or only simulated noise such as salt and pepper noise; the image to be denoised can also be an image with multiple noises superimposed. For another example, the image to be denoised can be a complex noise image, such as a noise image with mixed flat areas and texture areas.

[0056] It should be noted that, in the following embodiments, image noise elimination and image denoising both refer to performing denoising processing on an image with noise. Therefore, for the convenience of description, image noise elimination in the following embodiments may be described as image denoising.

[0057] Figure 1 is a flow chart of a method for removing image noise according to an exemplary embodiment. Figure 1 As shown, the method includes the following steps.

[0058] In step S11, an image to be denoised is obtained.

[0059] In step S12, the first model is called to denoise the flat area of ​​the image to be denoised, so as to obtain a first denoised image.

[0060] In one implementation of the disclosed embodiment, the first model is a pre-trained model used for image noise removal. The first model is mainly used for noise removal in flat areas.

[0061] It should be understood that in the field of image processing, a flat area refers to a continuous area on an image where the pixel values ​​are very similar or exactly the same. It can be understood that in this area, multiple pixels on the image have the same or similar color, brightness or reflectivity and other characteristics. This area usually refers to an area without obvious changes or details. The first denoising model performs denoising on the flat area, which actually refers to denoising the continuous area with similar or identical features in the image.

[0062] It should be understood that the main structure of the first denoising model may be an existing model structure, or an updated structure based on the existing model structure, for example: a blind spot network or an updated model structure based on a blind spot network.

[0063] In step S13, the receptive field of the image to be denoised is reduced to obtain a preprocessed image.

[0064] In one implementation of the embodiments of the present disclosure, the denoised image may be processed by reducing the receptive field in at least one of the following ways, including: pooling, convolution, or image segmentation.

[0065] It should be understood that by reducing the receptive field of the image to be denoised, useless information in a larger receptive field can be eliminated, allowing the model to focus more on the features of the texture area.

[0066] It should be understood that reducing the receptive field can optimize the effect of subsequent model processing on the texture area. If the noise or related features of the texture area in the acquired image to be denoised are obvious enough, this process can be omitted.

[0067] In step S14, the second model is called to denoise the texture area of ​​the preprocessed image to obtain a second denoised image.

[0068] In one implementation of the disclosed embodiment, the second model is a pre-trained model used for image noise removal. The second model is mainly used for noise removal in texture areas.

[0069] It should be understood that in the field of image processing, a texture area generally refers to an area on an image having certain texture features, that is, the elements in the area such as pixels, lines, points, etc. are combined to form a certain regular and repetitive structure.

[0070] It should be understood that the main structure of the second denoising model can be a model structure in the related art, or an updated structure based on the model structure in the related art, such as a bilateral filtering model, a denoising model based on sparse coding, or a denoising model based on deep learning, or an updated model structure of the above models.

[0071] In step S15, a target denoised image is obtained based on the first denoised image and the second denoised image.

[0072] In the embodiment of the present disclosure, the first denoised image can represent the result of denoising the flat area of ​​the image to be denoised, and the second denoised image can represent the result of denoising the texture area of ​​the image to be denoised. Based on this, based on the first denoised image and the second denoised image, the denoising results of the texture area and the flat area of ​​the image to be denoised can be obtained, that is, the target denoised image can be obtained.

[0073] It should be understood that the weight setting for the first denoised image and the second denoised image can be understood as a hyperparameter setting.

[0074] The disclosed embodiment provides an image noise removal method, which performs denoising on the texture area and flat area of ​​the denoised image respectively, and performs weighted fusion on the texture area denoising result and the flat area denoising result to obtain the final denoised image result. By performing targeted processing on the flat area and the texture area together, the noise removal effect in the image with complex noise is improved.

[0075] In one example, a target denoised image is obtained based on the first denoised image and the second denoised image, which can be understood as: the first denoised image and the second denoised image are weighted fused, and the fused image is determined as the target denoised image.

[0076] In an implementation manner provided by an embodiment of the present disclosure, the first model can be obtained by training an initial model.

[0077] It should be understood that the initial model can be understood as the model after the neural network training is initialized, and the parameter iteration and update are performed based on this model. The initial model can also be understood as an initial parameter model or a parameter randomization model in some scenarios.

[0078] It should be understood that the preset models (first preset model, second preset model and third preset model) in the following embodiments of the present disclosure can also be understood as initial models for parameter iteration and updating. For ease of understanding, the following embodiments will be described with the concept of initial model.

[0079] In the disclosed embodiment, the second model may introduce some features of the first model for training supervision conditions.

[0080] Figure 2 is a flow chart of a second model determination method according to an exemplary embodiment. Figure 2 As shown, the second model is determined in the following manner, including:

[0081] In step S21, a first training image and a first output image are determined.

[0082] In step S22, a second model is trained based on the first training image and the first output image.

[0083] In one implementation of the embodiments of the present disclosure, the first training image is used to train the first model, and the first output image is an image output by the first model based on the training of the first training image.

[0084] In one example, Figure 3 FIG. 1 is a flow chart of a second model training method according to an exemplary embodiment. Figure 3 As shown, the second model is trained in the following manner, including:

[0085] In step S31, the receptive field of the first training image is reduced to obtain a third training image.

[0086] It should be understood that for the flat area, the pixel area associated with the central signal (or the noise associated with the central pixel point) is relatively large, and a larger receptive field can cover the area associated with the central signal as much as possible. However, for the texture area, the pixel area associated with the central signal is relatively small, so the receptive field needs to be reduced to eliminate irrelevant pixel areas.

[0087] It should be understood that, for the method of reducing the receptive field, the method described in the above embodiment is also applicable to this embodiment. For specific contents, please refer to the relevant description in the above embodiment, which will not be repeated here.

[0088] In step S32, a texture region image of the first output image is extracted.

[0089] In one implementation of the embodiments of the present disclosure, extracting the texture region image of the first output image can be understood as extracting texture features of the first output image.

[0090] In one implementation of the embodiments of the present disclosure, texture feature extraction of the first output image may be achieved by at least one of the following methods, including: local binary pattern, gray level co-occurrence matrix, Gaussian filtering and other methods.

[0091] It should be understood that for the local binary pattern method, specifically, the neighborhood of each pixel is compared with the pixel itself to extract texture features;

[0092] For the gray-level co-occurrence matrix, the texture features are extracted by calculating the gray-level co-occurrence matrix between pixel pairs;

[0093] For Gaussian filtering, the image is smoothed by a filter to extract texture features.

[0094] In step S33, a texture mask of the texture region image is determined.

[0095] It should be understood that texture masking is a technique used in image processing to enhance image texture, and can be used to enhance texture features in an image, thereby making the image easier to be processed by a processor.

[0096] In one implementation of the embodiment of the present disclosure, the four-neighborhood standard deviation method may be used for calculation of texture modeling.

[0097] For example, for each pixel point in the texture area, the standard deviation of the grayscale values ​​of the four surrounding pixels is calculated. The larger the standard deviation value of the four neighborhoods, the greater the difference in pixel values ​​around the pixel, that is, the more obvious the difference between the pixel and the surrounding pixels.

[0098] In step S34, the third training image is input into the second preset model for denoising to obtain a second output image.

[0099] In one implementation of the embodiment of the present disclosure, the second preset model may be an initial model with denoising capability, for example, a denoising model based on deep learning.

[0100] It should be understood that the image to be trained (ie, the third training image) with a reduced receptive field is input into the initial model for image denoising to obtain a denoising result (ie, the second output image).

[0101] In step S35, the second preset model is adjusted based on the second loss function to obtain a second model.

[0102] In one implementation of the embodiments of the present disclosure, adjusting the second preset model based on the second loss function can be understood as training constraints on the second preset model based on the second loss function.

[0103] In one implementation of the presently disclosed embodiment, the second loss function is determined based on the first output image, the second output image, and the texture mask.

[0104] It should be understood that determining the second loss function based on the first output image and the second output image, and using the output result of the first model as a constraint for training the second model, has better results than unconstrained training.

[0105] In the image noise elimination method provided by the embodiment of the present disclosure, the training samples with reduced receptive fields are used to train the second model, which can eliminate redundant and invalid information, thereby making the second model more focused on feature training in the texture area, thereby improving the denoising effect of the model for the texture area. By using the second output result and the first output result to establish a loss function, the result of the first output model can be used as supervision for the second model, which is better than unsupervised training. Further weighted processing using texture masks can amplify the texture features in the texture area and improve the model training effect.

[0106] In one example, Figure 4 is a flow chart of a first model training method according to an exemplary embodiment. Figure 2 As shown, the first model is trained based on the following method, including:

[0107] In step S41, a first training image is acquired.

[0108] In one implementation of the disclosed embodiment, the first training image can be understood as an image with noise. It can be understood that in some scenes, the first training image can be the image to be denoised in the aforementioned embodiment.

[0109] In step S42, a blind pixel area is determined in the first training image.

[0110] In one implementation of the embodiments of the present disclosure, the blind pixel area includes a central pixel point of the first training image and a target pixel area centered on the central pixel point.

[0111] It should be understood that the target pixel area can be understood as other pixel point areas determined as blind pixel areas in the first to-be-trained image except for the central pixel point.

[0112] It should be understood that each pixel point in the target pixel area can be understood as a target pixel point.

[0113] It should be understood that the determination of the target pixel area can be understood as a determination of a hyperparameter. The determination of the target pixel area can be determined according to the blind area setting requirements.

[0114] For example, Figure 5 FIG. 1 is a schematic diagram of a blind pixel region according to an exemplary embodiment. Figure 5 As shown, the blind pixel area can be understood as a pixel area composed of the center point of the first training image as the center pixel point and the target pixel points around the center pixel point. Figure 5 In the figure, each dot-shaded box represents a target pixel, the area formed by the dot-shaded boxes is the target pixel area, and the area formed by the target pixel area and the central pixel is the blind pixel area.

[0115] It should be understood that the receptive field of the blind pixel area is smaller than the receptive field of the first training image.

[0116] It should be understood that the determination of the blind spot pixel area here is only the selection and determination of the blind spot range, and the pixel values ​​within the range are not changed.

[0117] It should be understood that the definition of the blind area range can be understood as a process of setting hyperparameters. The blind area range can be a rectangle with a side length of a pixels, where a is an integer greater than 0.

[0118] It should be understood that the shape of the blind pixel area can be any shape within a plane, including closed figures (circle, triangle, trapezoid, etc.) and non-closed figures (spiral, etc.).

[0119] It should be understood that when defining the blind area, it is necessary to include the noise related to the space in the image as much as possible. In this way, setting it as the blind area that needs to be predicted can achieve the condition of being noise-independent.

[0120] In step S43, in the first training image, the pixel values ​​of the blind area pixel region are adjusted to be smaller than a threshold value, so as to obtain a second training image.

[0121] In one implementation of the present disclosure, Figure 3 As shown, the pixel values ​​of the blind pixel area can be adjusted to 0, and the obtained image is the second training image.

[0122] It should be understood that the center point also belongs to the pixel point in the blind pixel area. Therefore, when the pixel value of the blind pixel area is adjusted to 0, the pixel value of the center point is also adjusted to 0.

[0123] Exemplary, continued Figure 5 For example, if the pixel value of the target area is adjusted to be less than the threshold, we can get Figure 5 The dot-shaped blind pixel area in .

[0124] In step S44, the second training image is input into the first preset model for denoising to obtain a first output image.

[0125] In one implementation of the embodiment of the present disclosure, the first preset model may be an initial model with denoising capability, for example, a denoising model based on deep learning.

[0126] It should be understood that an image with pixel values ​​in the blind pixel area less than a threshold value is input into the initial model for image denoising to obtain a denoising result (ie, a first output image).

[0127] In step S45, the first preset model is adjusted based on the first loss function to obtain a first model.

[0128] In one implementation of the embodiments of the present disclosure, adjusting the first preset model based on the first loss function can be understood as determining the training constraints of the first preset model based on the first loss function.

[0129] In one implementation of the embodiments of the present disclosure, the first loss function is determined based on the first training image and the first output image.

[0130] Exemplarily, the first preset model is set as a blind neighborhood network (BNN), and the second training image input to the first preset model is input blind , the output result of the first preset model (the first output image) is output BNN , the first loss function is loss BNN , the main body of the loss function is L1, then the mathematical expression of the first preset model corrected based on the first loss function can be obtained:

[0131] output BNN =BNN(input blind )

[0132] lossBNN =L1(output BNN , input unblind )

[0133] Among them, L1 (output BNN , input unblind ) can be understood as the first sub-loss function in this embodiment.

[0134] It is understandable that the mean absolute error (MAE) function can be selected for the L1 function.

[0135] In the image noise elimination method provided by the embodiment of the present disclosure, the first model is trained using training samples with blind spots, which can eliminate the correlation between noise and space, thereby improving the denoising effect of the first model for flat areas. By using the model output results and the image before setting the blind spot to establish a loss function, the deviation between the output image and the actual input image in the actual image denoising scene is simulated, and the model parameters are adjusted based on this, so that the trained first model can be more suitable for the actual image denoising process.

[0136] In the embodiment of the present disclosure, the second model for image denoising in texture areas can be obtained through training.

[0137] In the disclosed embodiment, for the second loss function, the output result of the first model can be used as a training constraint, so that the second model can use the output result of the first model as supervision during training, thereby improving the model training effect.

[0138] Figure 6 The figure is a flowchart of a method for determining a second loss function according to an exemplary embodiment.

[0139] In an example, Figure 6 As shown, the second loss function is determined based on the first output image, the second output image and the texture mask, and includes the following steps:

[0140] In step S51, a pixel value set of a first output image is determined as a first data set, and a pixel value set of a second output image is determined as a second data set.

[0141] In step S52, the second data set is used as the prediction result, and the first data set is used as the actual result to construct a second sub-loss function.

[0142] In step S53, the second sub-loss function is weighted based on the texture mask to obtain a second loss function.

[0143] Exemplarily, the second preset model is set as a locally aware network (LAN), and the third training image input to the second preset model is input unblind , the output result (second output image or second data set) of the second preset model is output LAN , the second loss function is loss LAN , the main body of the loss function is L1 and the texture mask is mask, then the mathematical expression of the second preset model corrected based on the second loss function can be obtained:

[0144] output LAN =LAN(input unblind )

[0145] loss LAN =(1-mask)*L1(output LAN , ontput BNN )

[0146] Among them, L1 (output LAN , output BNN ) can be understood as the second sub-loss function in this embodiment.

[0147] It is understandable that the mean absolute error (MAE) function can be selected for the L1 function.

[0148] In the embodiment of the present disclosure, for the input image of the second model, that is, the third training image obtained by reducing the receptive field, there are relevant conditional restrictions on the reduction range of the receptive field in some scenes.

[0149] In one example, the third training image includes pixels in the blind pixel area.

[0150] In one implementation of the embodiment of the present disclosure, the third training image may include pixel points in part of the blind pixel area, or may include pixel points in all of the blind pixel area.

[0151] It should be understood that the blind spot pixels in the third training image can be understood as the pixels corresponding to the texture area. Therefore, in order to obtain a second model capable of identifying the features of the texture area, the third training image input to the second preset model must include some or all of the pixels in the blind spot pixel area.

[0152] In the disclosed embodiment, image noise elimination may also be performed using an independent model.

[0153] In one example, Figure 7FIG. 1 is a flowchart of another image processing method according to an exemplary embodiment. Figure 7 As shown, image noise elimination can also be achieved by the following methods, including:

[0154] In step S61, the image to be denoised is input into the third model to obtain a third denoised image.

[0155] In one implementation of the embodiment of the present disclosure, the third model is determined based on the first model and the second model.

[0156] In step S62, the first denoised image, the second denoised image and the third denoised image are weightedly fused to obtain a target denoised image.

[0157] In one implementation of the disclosed embodiment, the third model can perform denoising on the texture area and the flat area of ​​the image to be denoised separately, and perform weighted fusion on the results of the separate denoising to obtain the final target denoised image.

[0158] In the embodiment of the present disclosure, the third model can be obtained by training the initial model.

[0159] In one example, Figure 8 FIG. 1 is a flow chart of a third model training method according to an exemplary embodiment. Figure 8 As shown, the third model is trained in the following way, including:

[0160] In step S71, the first training image is input into the third preset model for denoising to obtain a third output image.

[0161] In one implementation of the embodiment of the present disclosure, the third preset model may be an initial model with denoising capability, for example, a denoising model based on deep learning.

[0162] In step S72, the third preset model is adjusted based on the third loss function to obtain a third model.

[0163] In one implementation of the embodiments of the present disclosure, adjusting the third preset model based on the third loss function can be understood as determining the training constraints of the third preset model based on the third loss function.

[0164] In one implementation of the disclosed embodiment, the third loss function is determined based on the first output image, the second output image, the third output image and the texture mask.

[0165] In one implementation of the disclosed embodiment, the third model can be understood as a fusion model of the first model and the second model in the aforementioned embodiment. That is, the third model can be a model that can perform denoising on the texture area and the flat area of ​​the noisy image respectively. Based on this, the output result of the texture area denoising model, the output result of the flat area denoising model and the prediction result of the third preset model can be used to establish a loss function, so that the trained third model can achieve denoising on the texture area and the flat area of ​​the image.

[0166] In the disclosed embodiment, for the third loss function, the training constraints can be determined based on the output results of the flat area denoising model and the results of the texture area denoising model, so that the third model can use the two as supervision during training to improve the model training effect.

[0167] Fig. 9 is a flow chart of a method for determining a third loss function according to an exemplary embodiment. Fig. 9 As shown, the third loss function is determined by the following method, including:

[0168] In step S81, a pixel value set of a third output image is determined as a third data set.

[0169] In one implementation of the embodiment of the present disclosure, while determining the third data set, the pixel value set of the first output image may be determined as the first data set and the pixel value set of the second output image may be determined as the second data set.

[0170] In step S82, the third data set is used as the prediction result, and the first data set is used as the actual result to construct a third sub-loss function.

[0171] In step S83, the third data set is used as the prediction result, and the second data set is used as the actual result to construct a fourth sub-loss function.

[0172] In step S84, the third sub-loss function and the fourth sub-loss function are weightedly fused based on the texture mask to obtain a third loss function.

[0173] Exemplarily, the third preset model is set as a denoising network (DN), the image to be trained input to the third preset model is inputunblind, the output result of the third preset model (the third output image or the third data set) is outputDN, and the output result of the first preset model (the first output image or the first data set) is output BNN , the output result (second output image or second data set) of the second preset model is output LAN, the third loss function is lossDN, the main body of the loss function is L1 and the texture mask is mask, then the mathematical expression of the third preset model modified based on the third loss function can be obtained:

[0174] output DN =DN(input unblind )

[0175] loss DN =(1-mask)*L1(output DN , output BNN )+mask

[0176] *L1(output DN , output LAN )

[0177] Among them, L1 (output DN , output BNN ) can be understood as the third sub-loss function in this embodiment, L1(output DN , output LAN ) can be understood as the fourth sub-loss function in this embodiment.

[0178] It can be understood that the MAE function can be selected as the function selection for L1.

[0179] To facilitate understanding of the third model, the embodiment of the present disclosure will be explained using an example in which the third model is a model trained based on the first model and the second model.

[0180] For ease of understanding, the following implementations involving features that are similar or identical to those in the above-mentioned embodiments will be referred back for explanation. For details, please refer to the corresponding introduction in the cited embodiments, and the detailed contents will not be repeated one by one.

[0181] Fig.10 FIG. 1 is a schematic diagram of a third model training scenario according to an exemplary embodiment. Fig.10 As shown, the training scene of the third model mainly includes three training scenes, a training scene 110 of the first model, a training scene 120 of the second model and a training scene 130 of the third model.

[0182] In the first model training scenario 110, the input image 111 can be understood as the first training image in the aforementioned embodiment. The input image 111 is subjected to feature extraction to obtain feature map 112, feature map 113 and feature map 114, which constitute the feature map of the input image 111 after blind spot processing (that is, the pixel value of the pixel point in the blind spot pixel area is adjusted to be less than the threshold), that is, the feature map of the second training image. Among them, feature map 112 and feature map 113 represent feature maps of the area that has not been processed by the blind spot. Feature map 114 represents the feature map of the area that has been processed by the blind spot. The feature map of the input image 111 after feature extraction is input into the first model (i.e., BNN), and after denoising, feature map 115 is obtained. Feature map 115 is the feature map after denoising by the BNN model, and the prediction result 116 is obtained based on feature map 115.

[0183] In the second model training scenario 120, the input image 121 can be understood as the third training image in the aforementioned embodiment. After feature extraction, the input image 121 obtains feature map 122, feature map 123 and feature map 124, which constitute the feature map of the input image 121, that is, the feature map of the third training image. Among them, feature map 122 and feature map 123 represent feature maps of non-texture areas. Feature map 124 represents the feature map of the texture area. The feature map of the input image 121 after feature extraction is input into the second model (i.e., LAN), and after denoising, feature map 125 is obtained. Feature map 125 is the feature map after denoising by the LAN model, and the prediction result 126 is obtained based on feature map 125.

[0184] In the second model training scenario, for the second model training constraints, the prediction result 116 of the first model and the texture mask 140 can be added as supervision. The texture mask 140 is obtained by extracting texture features from the prediction result of the first model.

[0185] In the third model training scenario, the input image 131 can be understood as the first training image in the aforementioned embodiment. After feature extraction, the input image 131 obtains a feature map 132. The feature map of the input image 131 after feature extraction is input into the third model (i.e., DN), and after noise reduction processing, a feature map 133 is obtained. The feature map 133 is the feature map after denoising by the DN model, and a prediction result 134 is obtained based on the feature map 133.

[0186] In the third model training scenario, for the third model training constraints, the prediction result 116 of the first model, the prediction result 126 of the second model, and the texture mask 140 may be added as supervision.

[0187] It is understandable that the embodiments of the present disclosure can be implemented separately or in combination with each other. For example, the embodiment of denoising by the third model can be implemented independently or together with the embodiments of denoising by the first model and the second model. The present disclosure only lists the independent implementation methods of each embodiment, and the implementation method of the combined application of each embodiment should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0188] Based on the same concept, an embodiment of the present disclosure also provides an image noise elimination device.

[0189] It is understandable that in order to achieve the above functions, the image noise elimination device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0190] Fig.11 FIG. 1 is a block diagram of an image noise elimination processing device according to an exemplary embodiment. Fig.11 The device 100 includes an acquisition unit 101 and a processing unit 102.

[0191] The acquisition unit 101 is used to acquire the image to be denoised.

[0192] The processing unit 102 is used to call the first model to denoise the flat area of ​​the image to be denoised to obtain a first denoised image; reduce the receptive field of the image to be denoised to obtain a preprocessed image; call the second model to denoise the texture area of ​​the preprocessed image to obtain a second denoised image; and obtain a target denoised image based on the first denoised image and the second denoised image.

[0193] In one example, the second model is determined based on the following method: determine a first training image and a first output image, the first training image is used to train the first model, and the first output image is an image output by the first model based on the first training image training; train the second model based on the first training image and the first output image.

[0194] In one example, the second model is trained based on the first training image and the first output image in the following manner: the receptive field of the first training image is reduced to obtain a third training image, and the texture area of ​​the first output image is extracted, and the texture mask of the texture area image is determined; the third training image is input into the second preset model for denoising to obtain a second output image; a second loss function is determined based on the first output image, the second output image and the texture mask; and the second preset model is trained and constrained based on the second loss function to obtain a second model.

[0195] In one example, a first model is trained based on the following method: obtaining a first training image; determining a blind spot pixel area in the first training image, the blind spot pixel area including a central pixel point of the first training image and a target pixel area centered on the central pixel point; in the first training image, adjusting the pixel value of the blind spot pixel area to be less than a threshold value to obtain a second training image; inputting the second training image into the first preset model for denoising to obtain a first output image; determining a first loss function based on the first image and the first training image; and training constraining the first preset model based on the first loss function to obtain a first model.

[0196] In one example, the third training image includes pixels in the blind pixel area.

[0197] In one example, the second loss function is determined based on the first output image, the second output image and the texture mask in the following manner: the second output image is used as the prediction result, and the first output image is used as the actual result to construct the second sub-loss function; the second sub-loss function is weighted based on the texture mask to obtain the second loss function.

[0198] In one example, the processing unit 102 obtains the target denoised image based on the first denoised image and the second denoised image in the following manner: weighted fusion is performed on the first denoised image and the second denoised image, and the fused image is determined as the target denoised image.

[0199] In one example, the processing unit 102 obtains a target denoised image based on the first denoised image and the second denoised image in the following manner: the image to be denoised is input into a third model to obtain a third denoised image, and the third model is determined based on the first model and the second model; the first denoised image, the second denoised image and the third denoised image are weightedly fused to obtain the target denoised image.

[0200] In one example, the third model is determined based on the first model and the second model in the following manner: the first training image is input into the third preset model for denoising to obtain a third output image; a third loss function is determined based on the first output image during the first model training process, the second output image during the second model training process, and the texture mask during the second model training process; the training constraints of the third preset model are determined based on the third loss function to obtain the third model.

[0201] In one example, the third loss function is determined based on the first output image, the second output image, the third output image and the texture mask in the following manner: the pixel value set of the first output image is determined as the first data set, the pixel value set of the second output image is determined as the second data set, and the pixel value set of the third output image is determined as the third data set; the third data set is used as the prediction result, and the first data set is used as the actual result to construct a third sub-loss function; the third data set is used as the prediction result, and the second data set is used as the actual result to construct a fourth sub-loss function; the third sub-loss function and the fourth sub-loss function are weightedly fused based on the texture mask to obtain the third loss function.

[0202] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0203] Fig.12 2 is a block diagram of a device 200 for screen display according to an exemplary embodiment. For example, the device 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0204] Reference Fig.12 , the device 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0205] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0206] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0207] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.

[0208] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0209] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC), and when the device 200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 204 or sent via the communication component 216. In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.

[0210] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0211] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200, the sensor assembly 214 can also detect the position change of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200 and the temperature change of the device 200. The sensor assembly 214 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0212] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0213] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0214] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the instructions can be executed by the processor 220 of the device 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0215] Fig.13 is a block diagram of a device 300 for screen display according to an exemplary embodiment. For example, the device 300 may be provided as a server. Fig.13The apparatus 300 includes a processing component 322, which further includes one or more processors, and a memory resource represented by a memory 332 for storing instructions executable by the processing component 322, such as an application. The application stored in the memory 332 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 322 is configured to execute instructions to perform the above-mentioned screen display method.

[0216] The device 300 may also include a power supply component 326 configured to perform power management of the device 300, a wired or wireless network interface 350 configured to connect the device 300 to a network, and an input / output (I / O) interface 358. The device 300 may operate based on an operating system stored in the memory 332, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0217] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.

[0218] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0219] It will be further understood that the terms “center”, “longitudinal”, “lateral”, “front”, “back”, “up”, “down”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation.

[0220] It can be further understood that, unless otherwise specified, “connection” includes a direct connection without other components between the two, and also includes an indirect connection with other components between the two.

[0221] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0222] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modifications, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0223] It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.

Claims

1. A method for eliminating image noise, characterized in that: include: Obtain an image to be denoised; Calling the first model to denoise the flat area of ​​the image to be denoised to obtain a first denoised image; Narrowing the receptive field of the image to be denoised to obtain a preprocessed image; Calling the second model to denoise the texture area of ​​the preprocessed image to obtain a second denoised image; A target denoised image is obtained based on the first denoised image and the second denoised image.

2. The image noise elimination method according to claim 1, characterized in that: The second model is determined in the following manner: Determine a first training image and a first output image, wherein the first training image is used to train a first model, and the first output image is an image output by the first model based on the training of the first training image; The second model is trained based on the first training image and the first output image.

3. The image noise elimination method according to claim 2, characterized in that: The step of training the second model based on the first training image and the first output image comprises: Reducing the receptive field of the first training image to obtain a third training image, extracting a texture area of ​​the first output image, and determining a texture mask of the texture area image; Inputting the third training image into the second preset model for denoising to obtain a second output image; determining a second loss function based on the first output image, the second output image, and the texture mask; The second preset model is trained and constrained based on the second loss function to obtain the second model.

4. The image noise elimination method according to claim 1, characterized in that: The first model is trained based on the following method, including: Acquire a first training image; Determine a blind pixel region in the first training image, where the blind pixel region includes a central pixel point of the first training image and a target pixel region centered on the central pixel point; In the first training image, adjusting the pixel value of the blind pixel area to be less than a threshold value, to obtain a second training image; Inputting the second training image into the first preset model for denoising to obtain a first output image; determining a first loss function based on the first image and the first training image; The first preset model is trained and constrained based on the first loss function to obtain the first model.

5. The image noise elimination method according to claim 3, characterized in that: The determining a second loss function based on the first output image, the second output image, and the texture mask comprises: Taking the second output image as a prediction result and taking the first output image as an actual result to construct a second sub-loss function; The second sub-loss function is weighted based on the texture mask to obtain the second loss function.

6. The image noise elimination method according to any one of claims 1 to 5, characterized in that: The step of obtaining a target denoised image based on the first denoised image and the second denoised image includes: The first denoised image and the second denoised image are weightedly fused, and the fused image is determined as a target denoised image.

7. The image noise elimination method according to any one of claims 1 to 5, characterized in that: The step of obtaining a target denoised image based on the first denoised image and the second denoised image includes: Inputting the image to be denoised into a third model to obtain a third denoised image, wherein the third model is determined based on the first model and the second model; The first denoised image, the second denoised image and the third denoised image are weightedly fused to obtain a target denoised image.

8. The image noise elimination method according to claim 7, characterized in that: The third model is determined based on the first model and the second model, including: Inputting the first training image into a third preset model for denoising to obtain a third output image; Determine a third loss function based on the first output image in the first model training process, the second output image in the second model training process, and the texture mask in the second model training process; Based on the third loss function, the training constraints of the third preset model are determined to obtain the third model.

9. The image noise elimination method according to claim 8, characterized in that: The third loss function is determined based on the first output image, the second output image, the third output image and the texture mask in the following manner: Determine a set of pixel values ​​of the first output image as a first data set, determine a set of pixel values ​​of the second output image as a second data set, and determine a set of pixel values ​​of the third output image as a third data set; Taking the third data set as the prediction result and taking the first data set as the actual result to construct a third sub-loss function; Using the third data set as a prediction result and the second data set as an actual result to construct a fourth sub-loss function; The third sub-loss function and the fourth sub-loss function are weightedly fused based on the texture mask to obtain a third loss function.

10. An image noise elimination device, characterized in that: include: An acquisition unit, used for acquiring an image to be denoised; A processing unit, configured to call a first model to denoise a flat area of ​​the image to be denoised, to obtain a first denoised image; Narrowing the receptive field of the image to be denoised to obtain a preprocessed image; Calling the second model to denoise the texture area of ​​the preprocessed image to obtain a second denoised image; A target denoised image is obtained based on the first denoised image and the second denoised image.

11. An image noise elimination device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: execute the image noise elimination method described in any one of claims 1 to 9.

12. A storage medium, characterized in that: The storage medium stores instructions, and when the instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the image noise elimination method according to any one of claims 1 to 9.

Images