An image processing method, system and storage medium

Abstract

Embodiments of the present specification provide an image processing method, system and storage medium. The method comprises: obtaining a to-be-processed image and a plurality of noise level images corresponding to a plurality of noise reduction degrees; determining a plurality of noise reduction images corresponding to the plurality of noise reduction degrees based on the to-be-processed image and the plurality of noise level images by using a noise reduction model; the noise reduction model is a machine learning model, comprising a base layer and a plurality of processing layers, the base layer generates a base image based on the to-be-processed image, and the plurality of processing layers comprise a plurality of target processing layers, each target processing layer generates one noise reduction image in the plurality of noise reduction images based on the base image and one noise level image in the plurality of noise level images.

Description

Technical Field

[0001] This specification relates to the field of image processing, and in particular to an image noise reduction method, system, and storage medium. Background Technology

[0002] Deep learning-based image denoising can remove complex noise from images and effectively improve the signal-to-noise ratio. However, it may risk weakening or erasing image details. Therefore, it is necessary to set an appropriate level of denoising to effectively reduce noise while preserving image details to the greatest extent. However, different images have different requirements for the level of denoising, and existing deep learning models can only output results with a single level of denoising.

[0003] Therefore, it is necessary to provide an image processing solution that can meet the different noise reduction requirements of different images. Summary of the Invention

[0004] This specification provides one or more embodiments of an image processing method, the method comprising: acquiring an image to be processed and multiple noise level images, the multiple noise level images corresponding to multiple noise reduction degrees; using a noise reduction model, determining multiple noise-reduced images corresponding to the multiple noise reduction degrees based on the image to be processed and the multiple noise level images; the noise reduction model is a machine learning model, including a base layer and multiple processing layers, the base layer generating a base image based on the image to be processed, the multiple processing layers including multiple target processing layers, each target processing layer generating one noise-reduced image from the multiple noise-reduced images based on the base image and one of the multiple noise level images.

[0005] This specification provides one or more embodiments of an image processing system, the system comprising: an acquisition module for acquiring an image to be processed and multiple noise level images, the multiple noise level images corresponding to multiple noise reduction degrees; and a noise reduction module for using a noise reduction model to determine multiple noise reduction images corresponding to multiple noise reduction degrees based on the image to be processed and the multiple noise level images; the noise reduction model is a machine learning model, including a base layer and multiple processing layers, the base layer generating a base image based on the image to be processed, and the multiple processing layers including multiple target processing layers, each target processing layer generating one noise reduction image from the multiple noise reduction images based on the base image and one of the multiple noise level images.

[0006] This specification provides one or more embodiments of an image processing apparatus, the apparatus including at least one storage medium storing computer instructions; and at least one processor executing the computer instructions to implement an image processing method.

[0007] This specification provides one or more embodiments of a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions from the storage medium, the computer executes an image processing method. Attached Figure Description

[0008] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0009] Figure 1 These are schematic diagrams illustrating application scenarios of the image processing system according to some embodiments of this specification;

[0010] Figure 2 These are exemplary block diagrams of an image processing system according to some embodiments of this specification;

[0011] Figure 3 This is an exemplary flowchart of an image processing method according to some embodiments of this specification;

[0012] Figure 4 This is a schematic diagram illustrating the generation of multiple denoised images using a denoising model according to some embodiments of this specification;

[0013] Figure 5 These are exemplary schematic diagrams of noise level images shown according to some embodiments of this specification;

[0014] Figure 6 This is an exemplary schematic diagram illustrating the process of training a noise reduction model according to some embodiments of this specification. Detailed Implementation

[0015] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0016] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0017] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0018] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0019] Figure 1 This is a schematic diagram illustrating an application scenario of an image processing system according to some embodiments of this specification.

[0020] In some embodiments, such as Figure 1 As shown, the image processing system 100 may include an imaging device 110, a processing device 120, a terminal device 130, a network 140, and a storage device 150. The components of the image processing system 100 may be connected in one or more ways. This is merely an example. Figure 1 As shown, imaging device 110 can be connected to processing device 120 via network 140. Alternatively, imaging device 110 can be directly connected to processing device 120 (as indicated by the dashed double-headed arrow connecting imaging device 110 and processing device 120). As a further example, storage device 150 can be connected to processing device 120 directly or via network 140. As a further example, terminal device 130 can be directly (as indicated by the dashed double-headed arrow connecting terminal device 130 and processing device 120) and / or via network 140 to processing device 120.

[0021] Imaging device 110 can acquire images of the object to be scanned. In some embodiments, the object to be scanned may include, but is not limited to, the human body, organs, organisms, damaged sites, tumors, objects, phantoms, etc. In some embodiments, imaging device 110 may include, but is not limited to, magnetic resonance imaging (MRI) equipment (also known as MR scanners), computed tomography (CT) equipment, ultrasound scanners, digital radiography (DR) scanners, digital subtraction angiography (DSA), positron emission tomography (PET) equipment, single-photon emission computed tomography (SPECT) equipment, etc., or any combination thereof.

[0022] Processing device 120 can process data and / or information acquired from imaging device 110, terminal device 130, and / or storage device 150. For example, processing device 120 can acquire an image to be processed from imaging device. As another example, processing device 120 can utilize a denoising model to generate multiple denoised images with different levels of denoising based on the acquired image to be processed. Yet another example, processing device 120 can train an initial denoising model based on sample images and a gold standard image to obtain a denoising model. In some embodiments, processing device 120 may include a central processing unit (CPU), digital signal processor (DSP), system-on-a-chip (SoC), microcontroller unit (MCU), and / or any combination thereof. In some embodiments, processing device 120 may include a computer, user console, a single server, or a group of servers. The server group may be centralized or distributed. In some embodiments, processing device 120 may be local or remote. For example, processing device 120 may access information and / or data stored in imaging device 110, terminal device 130, and / or storage device 150 via network 140. For example, processing device 120 can directly connect to imaging device 110, terminal device 130, and / or storage device 150 to access stored information and / or data. In some embodiments, processing device 120 can be implemented on a cloud platform. By way of example only, cloud platforms can include private clouds, public clouds, hybrid clouds, community clouds, distributed clouds, etc., or any combination thereof. In some embodiments, processing device 120 or a portion thereof can be integrated into imaging device 110.

[0023] Terminal device 130 enables interaction between a user and image processing system 100. For example, terminal device 130 can display images to the user, such as an image to be processed, a denoised image, and / or a displayed image. As another example, terminal device 130 can obtain the user-defined desired level of noise reduction. Terminal device 130 may include mobile device 131, tablet computer 132, laptop computer 133, etc., or any combination thereof. In some embodiments, terminal device 130 may be part of processing device 120.

[0024] Network 140 may include any suitable network that facilitates the exchange of information and / or data between the image processing system 100 and the image processing system 100. In some embodiments, one or more components of the image processing system 100 (e.g., imaging device 110, processing device 120, terminal device 130, storage device 150) may communicate information and / or data with one or more other components of the image processing system 100 via network 140. In some embodiments, network 140 may be and / or include public networks, private networks, wide area networks (WANs), wired networks, wireless networks, cellular networks, frame relay networks, virtual private networks, satellite networks, telephone networks, routers, hubs, switches, and any combination thereof. In some embodiments, network 140 may include one or more network access points. For example, network 140 may include wired and / or wireless network access points such as base stations and / or internet switching points, through which one or more components of the image processing system 100 may connect to network 140 to exchange data and / or information.

[0025] Storage device 150 can store data, instructions, and / or any other information. In some embodiments, storage device 150 can store data acquired from imaging device 110, terminal device 130, and / or processing device 120. In some embodiments, storage device 150 may include mass storage, removable storage, volatile read-write memory, read-only memory (ROM), and any combination thereof. In some embodiments, storage device 150 may be executed on a cloud platform. In some embodiments, storage device 150 may be connected to network 140 to communicate with one or more other components of image processing system 100 (e.g., imaging device 110, processing device 120, terminal device 130). One or more components of image processing system 100 may access data or instructions stored in storage device 150 via network 140. In some embodiments, storage device 150 may be directly connected to or communicate with one or more other components of image processing system 100 (e.g., imaging device 110, processing device 120, storage device 150, terminal device 130). In some embodiments, storage device 150 may be part of processing device 120.

[0026] It should be noted that the above description is provided for illustrative purposes only and is not intended to limit the scope of this specification. Various changes and modifications can be made by those skilled in the art based on the content of this specification. Features, structures, methods, and other features of the exemplary embodiments described herein can be combined in various ways to obtain other and / or alternative exemplary embodiments. However, these changes and modifications will not depart from the scope of this specification.

[0027] Figure 2These are exemplary block diagrams of an image processing system according to some embodiments of this specification. Figure 2 As shown, the image processing system 200 may include an acquisition module 210, a noise reduction module 220, a training module 230, and a display module 240.

[0028] The acquisition module 210 can be used to acquire the image to be processed and multiple noise level images. In some embodiments, the multiple noise level images may correspond to multiple noise reduction levels. In some embodiments, the acquisition module 210 can also acquire the expected noise reduction level set by the user via a user terminal.

[0029] The denoising module 220 can be used to determine multiple denoised images corresponding to various denoising levels based on the image to be processed and multiple noise level images using a denoising model. In some embodiments, the denoising model can be a machine learning model, including a base layer and multiple processing layers. In some embodiments, the base layer can generate a base map based on the image to be processed. In some embodiments, the multiple processing layers can include multiple target processing layers. In some embodiments, each target processing layer can generate one denoised image from the multiple denoised images based on the base map and one of the multiple noise level images. In some embodiments, the denoising module 220 can perform one or more of the following operations: process the image to be processed using the base layer to generate a base map; stitch the base map and each denoised level image in the multiple noise level images to obtain multiple stitched images; and process the multiple stitched images using the multiple target processing layers to determine multiple denoised images. In some embodiments, the denoising module 220 can perform one or more of the following operations: obtain denoising parameters and various denoising levels corresponding to multiple processing layers; and determine multiple target processing layers from the multiple processing layers for generating multiple denoised images based on the denoising parameters and various denoising levels.

[0030] Training module 230 can be used to train a denoising model. In some embodiments, the parameters of the denoising model can be determined through training. In some embodiments, the training loss function can include multiple loss terms corresponding to multiple processing layers of the initial denoising model. In some embodiments, at least one loss term can include an indirect loss term. In some embodiments, the indirect loss term can reflect feature differences between the model output image and the gold standard image. In some embodiments, the loss term can include a direct loss term. In some embodiments, the direct loss term can reflect image differences between the initial denoising model output image and the gold standard image. In some embodiments, the first weight of the indirect loss term and the second weight of the direct loss term can be determined based on the denoising degree of the corresponding initial denoising model processing layer.

[0031] The display module 240 can be used to present a display image. In some embodiments, the display module 240 can generate a display image based on a desired level of noise reduction and multiple noise-reduced images. In some embodiments, the display module 240 can present the display image via a user terminal.

[0032] In some embodiments, the acquisition module 210, the noise reduction module 220, the training module 230, and the display module 240 may be implemented on the same or different processing devices. For example, the acquisition module 210, the noise reduction module 220, and the display module 240 may be implemented on the application side of the noise reduction model, while the training module 230 may be implemented on the supply side and / or the design side of the noise reduction model.

[0033] Figure 3 This is an exemplary flowchart of an image processing method according to some embodiments of this specification.

[0034] In some embodiments, the processing device 120 and / or the image processing system 200 may execute process 300. For example... Figure 3 As shown, process 300 may include the following steps.

[0035] Step 310: Acquire the image to be processed and multiple noise level images. Specifically, step 310 can be performed by the acquisition module 210.

[0036] The image to be processed can be an image acquired by an imaging device that needs to undergo noise reduction processing. For example, the image to be processed can include, but is not limited to, magnetic resonance images, CT images, ultrasound images, DR images, DSA images, PET images and / or SPECT images.

[0037] In some embodiments, the format of the image to be processed may include, but is not limited to, Joint Photographic Experts Group (JPEG) format, Tagged Image File Format (TIFF) format, Graphics Interchange Format (GIF) format, Kodak Flash PiX (FPX) format, Digital Imaging and Communications in Medicine (DICOM) format, etc. In some embodiments, the image to be processed may be a two-dimensional (2D) image or a three-dimensional (3D) image.

[0038] The noise level of the image to be processed can be represented by noise features. In some embodiments, noise features may include, but are not limited to, at least one or a combination of signal-to-noise ratio (SNR), noise distribution, noise intensity, global noise intensity, and / or noise rate. In some embodiments, the acquisition module 210 can determine the noise features of the image to be processed based on the pixel values ​​of the image to be processed. For example, taking signal-to-noise ratio (SNR) as an example, the acquisition module 210 can calculate the local variance of all pixels in the image to be processed, and use the maximum and minimum values ​​of the local variance to represent the signal variance and noise variance, respectively, thereby correcting the ratio of signal variance to noise variance based on an empirical formula to obtain the SNR of the image to be processed. As another example, taking noise distribution as an example, the acquisition module 210 can use an image (such as a normal distribution map) to represent the local variance of all pixels in the image to be processed, thereby obtaining the noise distribution of the image to be processed. In some embodiments, the method for obtaining noise features may include, but is not limited to, image patch methods, filter methods, spatial domain sampling methods, Bayesian estimation algorithms, etc., which are not limited in this embodiment.

[0039] Noise level images can be used to represent the degree of noise reduction required for an image to be processed. Multiple noise level images correspond to different levels of noise reduction, used to perform different degrees of noise reduction on the image to be processed. In some embodiments, the noise level image may contain the noise reduction level corresponding to each element in the image to be processed. In some embodiments, the noise reduction level can be represented by text, letters, symbols, numbers, grades, etc. For example, level 1, level 2, and level 3 can be used to represent noise reduction levels from smallest to largest. Another example is that 10%, 20%, and 30% can be used to represent noise reduction levels from smallest to largest. In some embodiments, the noise level image can be an image of the same size as the image to be processed, and its element values ​​can be filled with values ​​between 1 and N. The value of an element can represent its noise reduction level for the corresponding element in the image to be processed. The larger the value, the stronger the noise reduction. Here, N can represent the maximum noise reduction intensity, and the value of N can be determined by the user or during the training process of the initial noise reduction model.

[0040] In some embodiments, the noise reduction level corresponding to multiple noise level images can be the system's default setting, or it can be manually set by the user, or it can be determined during the initial training of the noise reduction model. For example, the noise reduction level can be set by the user according to actual application requirements.

[0041] In some embodiments, the degree of denoising corresponding to multiple noise level images can be determined based on the noise characteristics of the image to be processed. For example, the denoising module 220 can determine the degree of denoising corresponding to multiple denoised images based on the signal-to-noise ratio (SNR) of the image to be processed. Specifically, if the SNR of the image to be processed is low, indicating severe noise, the degree of denoising corresponding to the multiple denoised images can be set to a higher value, and vice versa. For example, if the SNR is 120, the degree of denoising corresponding to the multiple denoised images can be set between 10% and 30%. If the SNR is 100, the degree of denoising corresponding to the multiple denoised images can be set between 20% and 40%.

[0042] Another example is that the noise reduction module 220 can determine the noise reduction level of multiple denoised images based on the noise distribution of the image to be processed. The more concentrated the noise distribution, the smaller the interval between the noise reduction levels of the multiple denoised images can be set. For example, when the noise distribution in the image to be processed is relatively concentrated, the interval between the noise reduction levels of the multiple denoised images can be set to an interval of 10%, i.e., 10%, 20%, and 30% respectively. When the noise distribution in the image to be processed is relatively dispersed, the interval between the noise reduction levels of the multiple denoised images can be set to 20%, i.e., 20% and 40% respectively.

[0043] In some embodiments, each element in the image to be processed can correspond to the same noise reduction level. That is, each element in the noise level image has the same value. For example, the value of each element in noise level image 1 is 1, and the value of each element in noise level image 2 is 2. The noise reduction level of noise level image 2 is higher than that of noise level image 1.

[0044] In some embodiments, different elements in the image to be processed can correspond to different noise reduction levels. That is, each element in the noise level image has multiple different values. For example, the noise level image can be divided into multiple sub-regions. Each sub-region can have the same noise reduction level, while different sub-regions have different noise reduction levels. For example, the region of interest (ROI) needs to retain more image details while denoising, and the corresponding noise reduction level can be Level II; the background region does not need to retain image details, but its noise reduction requirements are not high, and the corresponding noise reduction level can be Level I; the segmentation boundary between the ROI and the background region needs to retain a clear boundary line, and the corresponding noise reduction level can be Level III. As another example, ROIs corresponding to different organs can have different noise reduction levels, such as ROIs for "veins" and "heart" corresponding to noise reduction levels of 40% and 50%, respectively.

[0045] Figure 5 This is an exemplary schematic diagram of a noise level image according to some embodiments of this specification. Figure 5As shown, the elements of the image to be processed may include, but are not limited to, the region of interest (represented by a light gray ellipse), the background region (represented by white), and the segmentation boundary between the region of interest and the background region (represented by a solid black line). The noise level image can be divided into a first sub-region (represented by a light gray ellipse), a second sub-region (represented by dark gray), and a third sub-region (represented by white), corresponding to the region of interest, the segmentation boundary and its vicinity, and the background region, respectively. Different sub-regions in the noise level image correspond to different degrees of noise reduction. For example, in noise level image 1, the value of the elements in the first sub-region is 40%, the value of the elements in the second sub-region is 80%, and the value of the elements in the third sub-region is 20%; in noise level image 2, the value of the elements in the first sub-region is 50%, the value of the elements in the second sub-region is 70%, and the value of the elements in the third sub-region is 30%.

[0046] Step 320 involves using a denoising model to determine multiple denoised images corresponding to various denoising levels based on the image to be processed and multiple images with different noise levels. Specifically, step 320 can be executed by the denoising module 220.

[0047] In some embodiments, the multiple denoised images generated using the denoising model can have different degrees of denoising, which are determined by the noise level images in step 310. For example, if each element in noise level image 1 has a value of 1 and each element in noise level image 2 has a value of 2, then denoised image 1 corresponding to denoising level 1 and noise image 2 corresponding to denoising level 2 can be generated. In this case, denoised image 2 has a higher signal-to-noise ratio.

[0048] The denoising model can be a machine learning model for image denoising. In some embodiments, the denoising module 220 can input the image to be processed and the noise level image into the denoising model, which can generate multiple denoised images based on the processing of the image to be processed. In some embodiments, the image to be processed and the noise level image can be input into the denoising model together or separately. For example, the denoising model can include multiple layers, and the image to be processed and the noise level image can be input together into the same layer. Alternatively, the image to be processed and the noise level image can be input into different layers, for example, the image to be processed can be input into an input layer, while the noise level image can be input into an intermediate layer.

[0049] In some embodiments, before inputting the image to be processed into the denoising model, the denoising module 220 may preprocess the image to be processed and then input the preprocessed image into the denoising model. For example, preprocessing may include initial denoising of the image to be processed. In some embodiments, the initial denoising may have a lower degree of denoising on the image to be processed. In some embodiments, the initial denoising may have the same degree of denoising on all regions of the image to be processed. It can be understood that initial denoising performs indiscriminate denoising on the entire image to be processed with a lower degree of denoising, which can quickly denoise the image to be processed without weakening the details of the image to be processed, thereby improving the denoising efficiency of subsequent processing layers on the base image.

[0050] For example, preprocessing may include, but is not limited to, image normalization and / or image resampling. Image resampling can be used to adjust the resolution of the image to be processed to a preset size. Image normalization (or image standardization) can be a series of transformations that convert the image to be processed into a corresponding unique standard form. For example, image normalization can be to uniformly map the attributes (e.g., pixel values) of each pixel in the image to a specific interval (e.g., [-1, 1]) or a specific distribution (e.g., a distribution that conforms to a normal distribution with a mean of 0 and a variance of 1), and may include min-max standardization and z-score standardization, etc. The image to be processed after image normalization will be more suitable for processing by machine learning models.

[0051] For illustrative purposes, Figure 4 Schematic diagrams illustrating the generation of multiple denoised images using a denoising model according to some embodiments of this specification are provided. For example... Figure 4 As shown, the noise reduction model 400 may include a base layer and multiple processing layers, such as processing layer 1, processing layer 2, processing layer 3... processing layer N. N can be any positive integer greater than 1, for example, 3, 4, 5, etc.

[0052] In some embodiments, the base layer can be used to process the image to be processed and generate a base map. The base map may include feature information of the image to be processed, such as noise level, image scale, pixel value distribution range, depth features, etc. In some embodiments, the base layer may include, but is not limited to, a multi-layer perceptron (MLP) model, a deep neural network (DNN) model, and / or a convolutional neural network (CNN) model, or any combination thereof. In some embodiments, the base layer may also be referred to as the backbone layer.

[0053] The processing layer can be used to denoise its input to generate a denoised image. In some embodiments, the processing layer may include filters, such as median filters, mean filters, and / or sequential counting filters, or any combination thereof. In some embodiments, the processing layer may include a neural network model, such as a recurrent neural network (RNN) model, a long short-term memory network (LSTM) model, a Transformer model, and / or a BERT (Bidirectional Encoder Representation from Transformers) model, or any combination thereof.

[0054] In some embodiments, different processing layers have different noise reduction parameters. The noise reduction parameters of a processing layer can reflect the degree of noise reduction that the processing layer can achieve. For example, continuing with... Figure 4 For example, the denoising parameters corresponding to processing layers 1, 2, 3, ..., N can respectively achieve denoising levels of 10%, 20%, 30%, ..., 90% for the image to be processed. It can be understood that the denoising parameters corresponding to the processing layer reflect the overall denoising level of the image to be processed, such as the average denoising level. For example, if the processing layer includes a filter, the denoising parameters of the processing layer can include filter parameters. As an example only, the filter parameter set α1, α2, α3, ..., αN based on a median filter can respectively achieve denoising levels of 10%, 20%, 30%, ..., and 90% for the stitched image. If the processing layer includes a neural network model, the denoising parameters of the processing layer can include the weights corresponding to multiple neurons in the neural network. For example, the weight vector set β1, β2, β3, ..., βN corresponding to multiple neurons can respectively achieve denoising levels of 10%, 20%, 30%, ..., and 90% for the image to be processed.

[0055] In some embodiments, the multiple processing layers may include multiple target processing layers, each of which can generate one denoised image from a set of denoised images based on a base image and one of a set of images with varying noise levels. In some embodiments, each processing layer of the denoising model can be a target processing layer. That is, each processing layer is used to denoise the image to be processed and generate a corresponding denoised image. In some embodiments, some processing layers of the denoising model can serve as target processing layers to denoise the image to be processed. Figure 4 As shown, the noise reduction model has N processing layers, of which only M are used as target processing layers, and the remaining NM layers do not process the image to be processed.

[0056] In some embodiments, the input to each target processing layer may include a base image and one noise level image from a set of noise level images. In some embodiments, the denoising module 220 may stitch together the base image and each noise level image from the set of noise level images to obtain multiple stitched images. Specifically, the base image and each noise level image may be stitched along the channel dimension to generate corresponding stitched images. The denoising module 220 may further process the multiple stitched images using multiple target processing layers to generate multiple denoised images. Specifically, each stitched image may be input to one of the target processing layers to generate a denoised image with a corresponding degree of denoising.

[0057] For example, such as Figure 4 As shown, the noise reduction module 220 can stitch the base image with noise level image 1, noise level image 2, ..., noise level image M respectively to obtain the corresponding stitched image 1, stitched image 2, ..., stitched image M. M can be any positive integer greater than 1 and less than or equal to N. The stitched image 1, stitched image 2, ..., stitched image M can be input into processing layer 1, processing layer 2, ..., and processing layer M respectively to generate the corresponding noise-reduced image 1, noise-reduced image 2, ..., and noise-reduced image M.

[0058] In some embodiments, the denoising module 220 can acquire the denoising degree corresponding to multiple noise level images and the denoising parameters corresponding to multiple processing layers. The denoising module 220 can determine multiple target processing layers from multiple processing layers for generating the multiple denoised images based on the denoising parameters and the denoising degree. As mentioned above, the denoising parameters of a processing layer can reflect the denoising degree that the processing layer can achieve, and the denoising degree corresponding to the noise level image can reflect the denoising degree that the image to be processed needs to achieve. Therefore, the denoising module 220 can select a processing layer that can achieve the denoising degree corresponding to the noise level image as the matching target processing layer. For example, assuming the denoising degree corresponding to noise level image 1 is 10%, a processing layer with a denoising degree of 10% or closest to 10% can be selected as the target processing layer, and the stitched image 1 corresponding to noise level image 1 can be input into this target processing layer.

[0059] In some embodiments of this specification, selecting a matching target processing layer from multiple processing layers based on the noise reduction level corresponding to the noise reduction level image can ensure that the target processing layer meets the noise reduction level corresponding to the noise level image, thereby improving the noise reduction effect.

[0060] In some embodiments, the noise reduction module 220 can send multiple noise-reduced images to the terminal device 130, allowing the user to select the desired image from the multiple noise-reduced images through the terminal device 130.

[0061] In some embodiments of this specification, multiple processing layers (or target processing layers) of the denoising model are used to process the images to be processed separately based on images with different noise levels. This allows the multiple denoised images output by the denoising model to achieve different levels of noise reduction as a whole, meeting different application requirements. Simultaneously, in some embodiments, the noise level image can have multiple sub-regions corresponding to different levels of noise reduction, enabling different regions in the image to achieve different levels of noise reduction, improving the denoising effect and adaptability. In some embodiments, process 300 may further include steps 330-350.

[0062] Step 330: Obtain the expected noise reduction level set by the user via the user terminal. Specifically, step 330 can be executed by the acquisition module 210.

[0063] The expected noise reduction level can be the level of noise reduction that the user desires for the displayed image. In some embodiments, the user can set the expected noise reduction level through the display interface of the user terminal. For example, the user can input the expected noise reduction level in an input box on the display interface of the user terminal (such as terminal device 130). Alternatively, the user terminal can display a slider corresponding to a certain noise reduction range, and the user can slide on the slider to select the expected noise reduction level. Furthermore, the noise reduction module 220 can receive the expected noise reduction level set by the user sent by the user terminal.

[0064] Step 340: Based on the expected level of noise reduction and multiple denoised images, generate a display image. Specifically, step 340 can be performed by the display module 240.

[0065] The displayed image can be a denoised image that meets the user's denoising expectations, and the displayed image can correspond to the expected denoising level. In some embodiments, the display module 240 can select one denoised image from multiple denoised images based on the expected denoising level as the displayed image. For example, if the expected denoising level is 10%, the display module 240 can select the denoised image with a denoising level of 10% or the closest to 10% from multiple denoised images as the displayed image. In some embodiments, the display module 240 can generate the displayed image by merging at least two denoised images from multiple denoised images. For example, if the expected denoising level is 15%, and there are only denoised images with denoising levels of 10% and 20% in the denoised images, the display module 240 can generate the displayed image by merging the denoised images with denoising levels of 10% and 20%. In some embodiments, the user can set multiple expected denoising levels, and the display module 240 can generate multiple displayed images corresponding to these multiple expected denoising levels.

[0066] Step 350: Present the displayed image via the user terminal. Specifically, step 350 can be executed by the display module 240.

[0067] In some embodiments, the display module 240 may send the display image to a user terminal (such as terminal device 130) so that the user terminal can display the image.

[0068] In some embodiments of this specification, multiple noise-reduced images with different levels of noise reduction can be generated first, and then a display image that meets the user's expected noise reduction level can be generated, thereby satisfying the user's different noise reduction needs.

[0069] Figure 6 This is an exemplary schematic diagram illustrating the process of training a noise reduction model according to some embodiments of this specification. In some embodiments, the processing device 120, the image processing system 200, and / or the training module 230 execute flow 600.

[0070] The parameters of the denoising model can be determined through training. Specifically, a large number of labeled training samples are input into the initial denoising model, and the parameters of the initial denoising model are iteratively updated based on the value of the loss function to obtain a trained denoising model.

[0071] In some embodiments, training samples may include sample images. In some embodiments, sample images may be images with a low signal-to-noise ratio. In some embodiments, the training module 230 may perform preliminary processing operations on the sample images. For example, preliminary processing operations may include flipping (e.g., flipping vertically or horizontally), rotating (e.g., rotating 90°), data normalization, resampling, etc. It is understood that preliminary processing operations can prevent the trained denoising model from overfitting.

[0072] In some embodiments, the identifier can be a high signal-to-noise ratio (SNR) gold standard image corresponding to the sample image. In some embodiments, the sample image and its corresponding identifier can be obtained through various methods. For example, the training module 230 can obtain multiple sample images with different noise characteristics by adding different noise to the gold standard image. Alternatively, the sample object can be scanned under different scanning conditions (such as different durations) to obtain images with different SNRs, where low SNR images can be used as sample images and high SNR images can be used as gold standard images.

[0073] In some embodiments, the training loss function may include multiple loss terms corresponding to multiple processing layers.

[0074] In some embodiments, at least one loss term may include an indirect loss term. The indirect loss term may reflect the feature differences between the model output image and the gold standard image. The model output image may be a predicted image obtained after the processing layer of the initial denoising model in training denoises the sample image. The feature differences between the output image and the gold standard image refer to the differences between the features of the output image and the features of the gold standard image.

[0075] In some embodiments, the indirect loss can be constructed based on perceptual loss. Perceptual loss allows the user to measure the difference in depth features between the model output image and the gold standard image. For example, training module 230 can construct the perceptual loss based on a feature extraction model. The feature extraction model can be a neural network model. Specifically, the model output image and the gold standard image can be respectively input into the feature extraction model to extract depth features, and the perceptual loss can be used to measure the difference in depth features between the two images. In some embodiments, the feature extraction model can include, but is not limited to, a Convolutional Neural Network (CNN) model, a Deep Neural Network (DNN) model, and / or a Visual Geometry Group (VGG Net) model, or any combination thereof. For example, the feature extraction model can include a CNN model and / or a VGG Net model.

[0076] CNN models can extract initial features (such as S0) from the model output image and initial features (such as G0) from the gold standard image, respectively. In some embodiments, the initial features of the image may include, but are not limited to, Haar features, Histogram of Oriented Gridients (HOG) features, Edgelet features, Color-Self Similarity (CSS) features, and / or Census Transform Histogram (CENTRIST) features, etc.

[0077] The VGG NET model can include multiple layers of VGG networks (e.g., VGG-13, VGG-16, VGG-19, etc.). Each VGG network layer can further extract image features based on the image features output by the previous layer, and input the extracted image features into the next VGG network layer. Then, based on the difference between the model output image features extracted by at least one VGG network layer and the gold standard image features, a perceptual loss is constructed. For example, taking the VGG-16 model as an example, the VGG-16 model can include 16 layers of VGG networks, where: VGG1 can extract the first depth feature S1 of the model output image and the first depth feature G1 of the gold standard image based on S0 and G0 respectively; VGG2 can extract the second depth feature S2 of the model output image and the second depth feature G2 of the gold standard image based on S1 and G1 respectively; ...; VGG16 can extract the sixteenth depth feature S16 of the model output image and the sixteenth depth feature G16 of the gold standard image based on S15 and G15 respectively.

[0078] Furthermore, the feature extraction model can construct a corresponding perceptual loss based on the differences between the first and second deep features output from any of its layers, such as the differences between S1 and G1, S4 and G4, S12 and G12, and S16 and G16. For example, perceptual loss... Where j can represent the number of image feature output layers used to construct the perceptual loss, and Sj and Gj can represent the features of the model output image extracted by the j-th layer of the VGG network and the features of the gold standard image, respectively.

[0079] In some embodiments, the feature extraction model may be correlated with the noise features of the image set to be processed by the denoising model. For example, the number of neural network layers in the feature extraction model may be determined based on the signal-to-noise ratio (SNR) of the image set to be processed; a higher SNR results in a lower number of neural network layers. As another example, the spacing between neural network layers in the feature extraction model used to output feature differences may be determined based on the noise distribution of the image set to be processed; a more concentrated noise distribution results in a larger spacing.

[0080] In some embodiments, the differences between features (deep features and / or reconstructed features) can be represented by the distance between the vectors corresponding to the features. For example, Euclidean distance, Manhattan distance, etc.

[0081] In some embodiments, the loss term of the processing layer may include a direct loss term. The direct loss term can reflect the image difference between the model output image and the gold standard image. In some embodiments, the training module 230 can obtain the image difference between the model output image and the gold standard image based on the distance between the vector composed of pixel values ​​of the model output image and the vector composed of pixel values ​​of the gold standard image. For example, the direct loss may be... Where Si and Gi represent the vector composed of pixel values ​​of the model output image and the vector composed of pixel values ​​of the gold standard image, respectively.

[0082] In some embodiments, the loss term of the processing layer may include both a direct loss term and an indirect loss term, wherein the indirect loss term has a first weight and the direct loss term has a second weight. The first weight may represent the degree of influence of the indirect loss term, and the second loss term may represent the degree of influence of the direct loss term. In some embodiments, the first weight of the indirect loss term and the second weight of the direct loss term may be determined based on the noise reduction level of the corresponding processing layer. The greater the noise reduction level, the smaller the first weight and the larger the second weight. As an example, when the noise reduction level is 20%, the first weight is 0.2 and the second weight is 0.8; when the noise reduction level is 30%, the first weight is 0.1 and the second weight is 0.9. It is understandable that the direct loss term reflects the difference between the output image of the processing layer and the gold standard image, ensuring the denoising effect of the processing layer. The intensity of denoising can be adjusted by adjusting the size of the second weight. The larger the second weight, the stronger the denoising and the smoother the output image. The indirect loss term reflects the difference between the features extracted by the intermediate layers of the processing layer and the corresponding features of the gold standard image, to prevent the trained processing layer from erasing the image's detailed features. The degree of image detail preservation can be adjusted by adjusting the size of the first weight. The larger the first weight, the more details are preserved, and the richer and more natural the texture of the output image. Therefore, by adjusting the first and second weights, the degree of denoising and detail preservation of the output image by different processing layers can be adjusted.

[0083] The overall loss function of the initial denoising model can be determined based on the indirect loss term, direct loss term, first weight, and second weight corresponding to each processing layer.

[0084] For example, the indirect loss terms corresponding to processing layer 1, processing layer 2, processing layer 3, ... and processing layer N are Lj1, Lj2, Lj3, ... and LjN, respectively, and the direct loss terms are Lz1, Lz2, Lz3, ... and LzN, respectively. The first weights are a1, a2, a3, ... and aN, respectively, and the second weights are b1, b2, b3, ... and bN, respectively. Then, the loss term of processing layer 1 is a1*Lj1+b1*Lz1, the loss term of processing layer 2 is a2*Lj2+b2*Lz2, the loss term of processing layer 3 is a3*Lj3+b3*Lz3, and the loss term of processing layer N is aN*LjN+bN*LzN. The overall loss function of the initial denoising model is the sum of the loss terms of processing layers 1 to N, that is, the overall loss function is a1*Lj1+b1*Lz1+a2*Lj2+b2*Lz2+a3*Lj3+b3*Lz3+…+aN*LjN+bN*LzN.

[0085] In some embodiments of this specification, indirect loss terms and direct loss terms are used to construct the loss term corresponding to each processing layer in the loss function. By adjusting the first weight of the indirect loss term and the second weight of the direct loss term, different processing layers in the trained denoising model can achieve different degrees of denoising.

[0086] It should be noted that the above description of the process is provided for illustrative purposes only and is not intended to limit the scope of this specification. Various changes and modifications can be made by those skilled in the art based on the description in this specification. However, these changes and modifications do not depart from the scope of this specification. The operational diagrams of the processes presented above are illustrative. In some embodiments, the above processes may be accomplished using one or more additional operations not described and / or one or more operations not discussed. For example, the process may be stored in a storage device (e.g., storage device 150, the system's storage unit) in the form of a program or instructions, and the process can be implemented when the processing device 120 and / or the image processing system 200 executes the instructions. Furthermore, the order of operations of the processes shown in the figures and described above is not limiting.

[0087] The beneficial effects that the embodiments of this specification may bring include, but are not limited to: (1) using multiple processing layers (or target processing layers) of the denoising model, processing the images to be processed based on images with different noise levels, so that the multiple denoised images output by the denoising model can achieve different degrees of denoising as a whole, and each element of each denoised image can also achieve different degrees of denoising, thereby improving the denoising effect and adaptability; (2) determining the target processing layer in the denoising model based on the expected denoising degree set by the user, so that the display image output by the target processing layer can meet the user's needs, and the denoising model can achieve denoising based on different user needs; (3) using indirect loss terms and direct loss terms to construct the loss term corresponding to each processing layer in the loss function, and by adjusting the first weight of the indirect loss term and the second weight of the direct loss term, different processing layers in the trained denoising model can achieve different degrees of denoising.

[0088] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0089] Furthermore, this specification uses specific terms to describe embodiments thereof. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Moreover, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0090] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on existing servers or mobile devices.

[0091] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0092] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this specification are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0093] For each patent, patent application, patent application publication, and other material, such as articles, books, specifications, publications, and documents, referenced in this specification, the entire contents of which are incorporated herein by reference. This excludes historical application documents that are inconsistent with or conflict with the content of this specification, as well as documents that limit the broadest scope of the claims in this specification (currently or subsequently appended to this specification). It should be noted that in the event of any inconsistency or conflict between the descriptions, definitions, and / or terminology used in the supplementary materials to this specification and the content of this specification, the descriptions, definitions, and / or terminology used in this specification shall prevail.

[0094] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. An image processing method, characterized in that, include: The image to be processed and multiple noise level images are acquired, wherein the multiple noise level images correspond to various noise reduction degrees; Using a noise reduction model, based on the image to be processed and the multiple noise level images, multiple noise-reduced images corresponding to the various noise reduction levels are determined; wherein, the interval between the noise reduction levels corresponding to the multiple noise-reduced images is determined based on the concentration of the noise distribution in the image to be processed; each noise level image is divided into multiple sub-regions, and the multiple sub-regions have different noise reduction levels; The noise reduction model is a machine learning model, including a base layer and multiple processing layers. The base layer generates a base map based on the image to be processed. The multiple processing layers include multiple target processing layers. Each target processing layer generates a noise-reduced image from the multiple noise-level images based on the base map and one of the multiple noise-level images.

2. The method as described in claim 1, characterized in that, The step of using a noise reduction model to determine the noise-reduced images corresponding to the various noise levels, based on the image to be processed and the multiple noise-level images, includes: The base layer is used to process the image to be processed, generating the base map; The base image and each noise-reduced level image from the multiple noise-level images are stitched together to obtain multiple stitched images; and The multiple stitched images are processed using the multiple target processing layers to determine the multiple denoised images.

3. The method as described in claim 1, characterized in that, Also includes: Obtain the noise reduction parameters and various noise reduction levels corresponding to the multiple processing layers; Based on the noise reduction parameters and the various noise reduction levels, the plurality of target processing layers for generating the plurality of noise-reduced images are determined from the plurality of processing layers.

4. The method as described in claim 1, characterized in that, The parameters of the noise reduction model are determined through training, and the loss function of the training includes multiple loss terms corresponding to the multiple processing layers; At least one of the loss terms includes an indirect loss term that reflects the feature differences between the model output image and the gold standard image.

5. The method as described in claim 4, characterized in that, The indirect loss term is constructed based on perceived loss.

6. The method as described in claim 4, characterized in that, The loss term includes a direct loss term, which reflects the image difference between the model output image and the gold standard image; the first weight of the indirect loss term and the second weight of the direct loss term are determined based on the noise reduction degree of the corresponding processing layer.

7. The method as described in claim 1, characterized in that, The method further includes: Obtain the user's desired noise reduction level set via the user terminal; Based on the expected level of noise reduction and the multiple noise-reduced images, a display image is generated; The display image is presented via the user terminal.

8. An image processing system, characterized in that, include: The acquisition module is used to acquire the image to be processed and multiple noise level images, wherein the multiple noise level images correspond to multiple noise reduction degrees; A noise reduction module is used to determine multiple noise-reduced images corresponding to various noise reduction levels based on the image to be processed and the multiple noise level images using a noise reduction model; wherein, the interval between the noise reduction levels corresponding to the multiple noise-reduced images is determined based on the concentration of the noise distribution in the image to be processed; each noise level image is divided into multiple sub-regions, and the multiple sub-regions have different noise reduction levels; The noise reduction model is a machine learning model, including a base layer and multiple processing layers. The base layer generates a base map based on the image to be processed. The multiple processing layers include multiple target processing layers. Each target processing layer generates a noise-reduced image from the multiple noise-level images based on the base map and one of the multiple noise-level images.

9. An image processing apparatus, characterized in that, The device includes: At least one storage medium that stores computer instructions; At least one processor executes the computer instructions to implement the image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing computer instructions that, when read by a computer, execute the image processing method as described in any one of claims 1 to 7.

Images