Denoising super-division model training method and device, image processing method and device and computer equipment

Abstract

The invention relates to a denoising super-division model training and image processing method and device, computer equipment, a storage medium and a computer program product. The denoising super-division model training method comprises the following steps: acquiring a sample image, inputting the sample image into an initial model, and extracting input features of the sample image; processing the input features by using a target edge sensing convolution block of the initial model to obtain de-noising super-division features; the target edge sensing convolution block is obtained by reducing the number of channels of the initial edge sensing convolution block by adopting a re-parameterization method; obtaining an image prediction value according to the input features and the de-noising super-division features; obtaining target loss according to the image true value and the image predicted value of the sample image; and adjusting parameters of the initial model according to the target loss until a preset convergence condition is met, and obtaining a denoising super-division model. By adopting the method, both the image resolution and the detection speed can be considered.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, computer equipment, storage medium and computer program product for denoising super-resolution model training and image processing. Background Art

[0002] X-ray inspection equipment is an important non-destructive testing tool that can be widely used in materials science, biology, medicine and other fields. However, related X-ray inspection equipment has shortcomings such as low resolution and slow detection speed, which cannot meet the current demand for high-resolution and high-efficiency detection.

[0003] At present, there is a contradiction between the resolution and detection speed of X-ray detection equipment. If a solution to improve the resolution of X-ray detection equipment is used, it usually takes longer time and more computing resources, while a solution to increase the detection speed usually results in a decrease in resolution. Therefore, the solution for generating detection images by related X-ray detection equipment cannot take into account both resolution and detection speed. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for denoising super-resolution model training and image processing that can take into account both image resolution and detection speed in order to address the above-mentioned technical problems.

[0005] In a first aspect, the present application provides a denoising super-resolution model training method, comprising:

[0006] Acquire a sample image, input the sample image into an initial model, and extract input features of the sample image;

[0007] The input features are processed using the target edge-aware convolution block of the initial model to obtain denoised super-resolution features; the target edge-aware convolution block is obtained by reducing the number of channels of the initial edge-aware convolution block using a reparameterization method;

[0008] Obtaining an image prediction value according to the input feature and the denoising super-resolution feature;

[0009] Obtaining a target loss according to the true image value of the sample image and the image prediction value;

[0010] The parameters of the initial model are adjusted according to the target loss until a preset convergence condition is met to obtain a denoising super-resolution model.

[0011] In one embodiment, the processing of the input features using the target edge-aware convolution block of the initial model to obtain denoised super-resolution features comprises:

[0012] extracting the input features using the target edge-aware convolutional block;

[0013] Performing channel separation on the input features to obtain retained features and transferred features;

[0014] The transfer feature is iteratively processed for multiple times, and the input of each iterative process is the transfer feature output by the previous iterative process; each iterative process includes: using the target edge-aware convolution block to perform feature extraction and channel separation on the transfer feature to obtain a new retained feature and a new transfer feature;

[0015] According to the transfer features obtained through multiple channel separations, denoising super-resolution features are obtained.

[0016] In one embodiment, extracting the input features of the sample image includes:

[0017] Performing pixel expansion processing on the sample image to obtain a downsampled image;

[0018] extracting input features according to the downsampled image;

[0019] Accordingly, obtaining the image prediction value according to the input feature and the denoising super-resolution feature includes:

[0020] Aggregating the denoising super-resolution features to obtain aggregated features;

[0021] Obtaining a sub-pixel feature according to a residual between the aggregated feature and the input feature;

[0022] A pixel shrinkage process is performed on the sub-pixel feature to obtain an image prediction value.

[0023] In one embodiment, obtaining the target loss according to the true image value of the sample image and the image prediction value comprises:

[0024] Using an artifact discrimination loss function to calculate the true image value and the image prediction value of the sample image to obtain an artifact feature map;

[0025] A target loss is obtained according to the artifact feature map.

[0026] In one embodiment, obtaining the target loss according to the artifact feature map includes:

[0027] The artifact feature map is adjusted by using a sliding exponential average method to obtain a target feature map;

[0028] A target loss is obtained according to the target feature map.

[0029] In one embodiment, acquiring the sample image includes:

[0030] Get the original image;

[0031] Performing blur processing, adding noise and scaling processing on the original image to obtain a transition image;

[0032] Check whether the size of the transition image meets the preset conditions. If so, use the transition image as a sample image. If not, use the transition image as a new original image and return to execute the steps of blurring, adding noise and scaling the original image until the size of the transition image meets the preset conditions to obtain a sample image.

[0033] In one of the embodiments, the noise includes additive noise and multiplicative noise.

[0034] In a second aspect, the present application also provides an image processing method, comprising:

[0035] Get the image to be processed;

[0036] The image to be processed is processed using the denoising super-resolution model as described in the first aspect to obtain a target image.

[0037] In a third aspect, the present application also provides a denoising super-resolution model training device, comprising:

[0038] A sample acquisition module, used to acquire a sample image, input the sample image into an initial model, and extract input features of the sample image;

[0039] A feature extraction module, used to process the input features using the target edge-aware convolution block of the initial model to obtain denoised super-resolution features; the target edge-aware convolution block is obtained by reducing the number of channels of the initial edge-aware convolution block using a reparameterization method;

[0040] An image prediction module, used to obtain an image prediction value according to the input features and the denoising super-resolution features;

[0041] A loss calculation module, used for obtaining a target loss according to the true image value of the sample image and the image prediction value;

[0042] The model training module is used to adjust the parameters of the initial model according to the target loss until the preset convergence conditions are met to obtain a denoising super-resolution model.

[0043] In a fourth aspect, the present application further provides an image processing device, comprising:

[0044] An image acquisition module, used for acquiring an image to be processed;

[0045] The image processing module is used to process the image to be processed using the denoising super-resolution model as described in the first aspect to obtain a target image.

[0046] In a fifth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps described in the first aspect or the second aspect when executing the computer program.

[0047] In a sixth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps described in the first aspect or the second aspect.

[0048] In a seventh aspect, the present application further provides a computer program product, comprising a computer program, which implements the steps described in the first aspect or the second aspect when executed by a processor.

[0049] The above-mentioned denoising super-resolution model training method, device, computer equipment, storage medium and computer program product, by using the target edge-aware convolution block to process the input features in the sample image, obtain the denoising super-resolution features, wherein the edge-aware convolution block can better capture and respond to the edge structure in the image, thereby improving the sensitivity and recognition ability to the edge features and ensuring the resolution of the image. Further, the target edge-aware convolution block is obtained by reducing the number of channels of the initial edge-aware convolution block using the reparameterization method. Therefore, the reparameterization method is used to reduce the feature dimension in the model, and by compressing the number of feature channels, redundant information is reduced, and the computational burden is reduced, thereby improving the computational efficiency and output speed on the basis of ensuring the resolution. Further, the above-mentioned image processing method, device, computer equipment, storage medium and computer program product, using the above-mentioned denoising super-resolution model to process the image to be processed, can utilize the characteristics of the above-mentioned denoising super-resolution model to improve the image resolution while improving the image processing speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] FIG1 is a diagram showing an application environment of a denoising super-resolution model training method according to an embodiment;

[0052] FIG2 is a schematic diagram of a flow chart of a denoising super-resolution model training method in one embodiment;

[0053] FIG3 is a schematic flow chart of step S204 of a denoising super-resolution model training method in one embodiment;

[0054] FIG4 is a schematic flow chart of step S208 of a denoising super-resolution model training method in one embodiment;

[0055] FIG5 is a schematic flow chart of step S202 of a denoising super-resolution model training method in one embodiment;

[0056] FIG6 is a block diagram of a denoising super-resolution model training device in one embodiment;

[0057] FIG7 is a block diagram of an image processing device according to an embodiment;

[0058] FIG8 is a diagram showing the internal structure of a computer device in one embodiment;

[0059] FIG. 9 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] The denoising super-resolution model training method provided in the embodiment of the present application can be applied in the application environment as shown in Figure 1. Referring to Figure 1, the denoising super-resolution model training method is applied to a model training system. The denoising super-resolution model training system includes a terminal 102 and a server 104. The terminal 102 and the server 104 are connected via a network. The terminal 102 can specifically be a desktop terminal or a mobile terminal, and the mobile terminal can specifically be at least one of a mobile phone, a tablet computer, a laptop computer, etc., such as a display terminal of an X-ray detection device. The server 104 can be implemented with an independent server or a server cluster composed of multiple servers, and is connected to the data storage system network, wherein the data storage system stores transition data and model-related data during the model training process. Both the terminal 102 and the server 104 can be used alone to execute the denoising super-resolution model training method provided in the embodiment of the present application. The terminal 102 and the server 104 can also be used in collaboration to execute the denoising super-resolution model training method provided in the embodiment of the present application.

[0062] The solution provided in the embodiments of the present application involves artificial intelligence image processing technology, mainly super-resolution technology based on deep learning, image enhancement technology based on generative adversarial networks, and image reconstruction technology based on variational autoencoders. These technologies can effectively improve image resolution, enhance image details, restore image quality, and generate realistic high-definition images by learning the features and patterns of a large amount of image data.

[0063] Exemplarily, the solution provided in the embodiments of the present application can be used in an X-ray detection process to generate a high-resolution X-ray detection result image.

[0064] In an exemplary embodiment, as shown in FIG2 , a denoising super-resolution model training method is provided, and the method is applied to the server 104 in FIG1 as an example for explanation, including the following steps S202 to S210. Among them:

[0065] Step S202: Obtain a sample image, input the sample image into the initial model, and extract input features of the sample image.

[0066] The sample image refers to an image instance in an image dataset used to train and test a machine learning model. For example, the sample image can be an original X-ray detection image or an image that has been pre-processed by segmentation, cropping, etc.

[0067] The initial model refers to a denoising super-resolution model with basic functions before this training, which can be a neural network model for image processing.

[0068] Exemplarily, the initial model may have the following functions: obtaining raw image data, for example, by photographing with a camera, scanning with a scanner, or reading an image file from an image library; converting a color image into a grayscale image; applying basic filters to smooth the image and remove noise, such as a mean filter or a Gaussian filter; using an edge detection algorithm to identify edge information in the image, such as a Sobel or Canny operator; performing image segmentation according to a specific threshold to divide the image into different regions or objects; performing contrast enhancement, brightness adjustment, and histogram equalization on the image to improve the quality and clarity of the image.

[0069] The input feature refers to the sample feature required for performing the super-resolution denoising operation. Exemplarily, the input feature is the image feature extracted from the sample image after degradation processing.

[0070] Step S204, using the target edge-aware convolution block of the initial model to process the input features to obtain denoised super-resolution features.

[0071] The target edge-aware convolution block is obtained by reducing the number of channels of the initial edge-aware convolution block using a reparameterization method. The initial edge-aware convolution block is used to extract edge information from input features, thereby improving the model's ability to perceive image edge structures.

[0072] Exemplarily, the initial edge-aware convolution block can be an Edge-oriented Convolution Block (ECB), an edge-aware convolution block used in image processing tasks. It is mainly used to extract edge information in an image, and can effectively capture edge details in an image while maintaining the overall structural characteristics of the image. ECB consists of multiple convolutional layers and activation functions, in which the characteristics of edge information are particularly considered to improve the model's perception of the edge structure of the image.

[0073] Furthermore, the reparameterization method refers to a technique for reparameterizing model parameters during the neural network training process. By changing the representation of model parameters, the original model parameters can meet certain constraints or have certain specific properties, thereby improving the performance, stability or interpretability of the model.

[0074] Exemplarily, the reparameterization method can be one of binarization, pruning, quantization, grouped convolution and low-rank decomposition, or a combination of multiple methods. Among them, binarization refers to quantizing the parameters of the model into binary form, thereby reducing the computational complexity of the model and improving the operating efficiency of the model; pruning refers to removing redundant connections or parameters in the model, thereby reducing the number of parameters and the computational burden of the model, and improving the lightweight degree of the model; quantization refers to converting the floating-point parameters in the model into fixed-point numbers or low-precision values ​​to reduce the storage space and computational cost of the model and improve the reasoning speed of the model; grouped convolution refers to grouping the input channels in the convolution operation to reduce the computational complexity of the model and speed up the running speed of the model; low-rank decomposition refers to performing low-rank decomposition on the weight matrix of the model, thereby reducing the number of parameters and the amount of computation of the model, and improving the computational efficiency and generalization ability of the model.

[0075] Exemplarily, the server 104 first receives the image data of the input feature and applies the edge-aware convolution operation to capture the edge features in the image. Next, an activation function is used to introduce nonlinearity, thereby increasing the expressiveness of the model and capturing structural information and important features in the image. Finally, the output denoising super-resolution feature processed by the edge-aware convolution block is generated. In addition, the server 104 can also use pytorch to load the pre-trained file that has been trained, use the reparameterization method to make the ECB module equivalent to a 3×3 convolution, then save it as a new model file, and then export it as an onnx file, and use the onnxruntime tool to execute the onnx file.

[0076] Step S206, obtaining an image prediction value according to the input features and the denoising super-resolution features.

[0077] Exemplarily, after the server 104 generates the denoising super-resolution features, it performs an aggregate convolution operation on the denoising super-resolution features, and then connects them with the residual of the above input features, and obtains the image prediction value according to the image features after the residual connection.

[0078] Among them, residual connection refers to adding the output of a layer in the neural network directly to the input before the layer, and taking the residual of the two as the final output. This connection method can help information to be transmitted more smoothly, and helps to alleviate the gradient vanishing problem during training, thereby accelerating the convergence of the model. Therefore, when processing denoising super-resolution features, the use of residual connection methods can help retain the details and structural information of the input features, thereby improving the model's ability to retain details and improve the clarity of the image.

[0079] Step S208, obtaining the target loss according to the true image value and the image prediction value of the sample image.

[0080] Step S210, adjusting the parameters of the initial model according to the target loss until a preset convergence condition is met to obtain a denoising super-resolution model.

[0081] For the above steps S208 and S210, illustratively, the server 104 can use the Unet discriminator for adversarial training based on the Real ESRGAN network, and use the spectral normalization regularization term to constrain the Unet discriminator, so that the network parameters change within a range to stabilize the training of the discriminator. On this basis, the Gan loss function is used for training, and the parameters of the initial model are adjusted according to the target loss after the target loss is continuously obtained until the preset convergence condition is met to obtain a denoising super-resolution model.

[0082] Among them, the Unet discriminator is a discriminator model designed based on the Unet structure for adversarial training. The Unet discriminator usually adopts the encoder-decoder architecture of Unet to identify the authenticity of the input data and distinguish between generated data and real data. In adversarial training, the Unet discriminator is used together with the generator in the generative adversarial network to improve the generation effect and quality of the adversarial generative network through training, thereby generating more realistic and more real data samples.

[0083] Furthermore, the server 104 may also use a generative adversarial network (GAN), a machine learning model for unsupervised learning. It consists of two parts, a generator and a discriminator, and is trained by a game between the two models. The generator is responsible for generating data samples similar to real data, while the discriminator is responsible for determining whether the input data comes from the generator or a real data set. During the training process, the generator continuously generates samples to deceive the discriminator, while the discriminator continuously learns to improve its discrimination ability, so that the two continue to compete and eventually reach a dynamic balance, and the generator can generate more realistic samples.

[0084] The Real ESRGAN network is an improved version of the Generative Super-Resolution Network (ESRGAN). ESRGAN is a generative adversarial network for super-resolution reconstruction, which aims to improve the super-resolution of images through training and generate clearer and more realistic high-resolution images. Real ESRGAN further improves and optimizes ESRGAN, allowing it to reconstruct image details more realistically while maintaining high resolution.

[0085] Furthermore, the above spectral normalization is used to constrain the model weights, thereby improving the stability and robustness of the model. Using the spectral normalization regularization term in the Unet discriminator can control the norm of the weights and prevent the model from experiencing gradient vanishing or exploding, thereby making the model training process more stable and improving the generalization ability and performance of the model.

[0086] In the above-mentioned denoising super-resolution model training method, the input features in the sample image are processed by using the target edge-aware convolution block to obtain the denoising super-resolution features, wherein the edge-aware convolution block can better capture and respond to the edge structure in the image, thereby improving the sensitivity and recognition ability to the edge features and ensuring the resolution of the image. Furthermore, the target edge-aware convolution block is obtained by reducing the number of channels of the initial edge-aware convolution block by the reparameterization method. Therefore, the reparameterization method is used to reduce the feature dimension in the model, and by compressing the number of feature channels, redundant information is reduced, and the computational burden is reduced, thereby improving the computational efficiency and output speed while ensuring the resolution. Furthermore, the above-mentioned image processing method, device, computer equipment, storage medium and computer program product use the above-mentioned denoising super-resolution model to process the image to be processed, and can utilize the characteristics of the above-mentioned denoising super-resolution model to improve the image resolution while improving the image processing speed.

[0087] In an exemplary embodiment, a lightweight super-resolution structure is used to complete the super-resolution denoising operation, as shown in FIG3 , step S204 includes steps S302 to S308 . Among them:

[0088] Step S302, extracting input features using a target edge-aware convolutional block.

[0089] Exemplarily, the server 104 replaces the convolution in IMDM with the target edge-aware convolution block and uses the target edge-aware convolution block to extract input features. Among them, the lightweight super-resolution structure (Iterative Modified DeepMemory, IMDM) is a lightweight model structure designed for super-resolution reconstruction tasks, aiming to achieve efficient image super-resolution reconstruction. IMDM consists of multiple lightweight super-resolution blocks (Improved Memory Deep Block, IMDB), each of which contains a small number of convolutional layers and channel attention mechanisms (Channel-wise Context Aggregation, CCA) modules for extracting image features and enhancing the model's perception of image details.

[0090] Step S304, performing channel separation on the input features to obtain retained features and transferred features.

[0091] Among them, channel separation is an operation in image processing, which refers to dividing the input feature map into two parts according to the channel and processing the two parts of the feature map separately. In the deep learning model, channel separation is used to extract the information of a specific channel or reduce the correlation between channels.

[0092] Exemplarily, the server 104 performs a channel separation operation, so that the retained features account for 75% and the transferred features account for 25%.

[0093] Step S306, performing multiple iterations of processing on the transfer features.

[0094] Among them, the input of each iterative processing is the transfer characteristics of the output of the previous iterative processing, that is, the transfer characteristics of the previous iterative processing are used as the input characteristics of this iterative processing to perform channel separation again until the preset number of iterations is reached. For example, if the preset number of iterations is 3 times, then three channel separation operations are required.

[0095] Further, each iterative process includes: using the target edge-aware convolution block to perform feature extraction and channel separation on the transferred features to obtain new retained features and new transferred features. Exemplarily, each new retained feature accounts for 75% of the last transferred feature, and the new transferred feature accounts for 25% of the last transferred feature; or, each new retained feature accounts for 75% of the last retained feature, and the new transferred feature accounts for 25% of the last retained feature.

[0096] Step S308, obtaining denoising super-resolution features based on the transfer features obtained through multiple channel separations.

[0097] Exemplarily, the server 104 concatenates the transfer features of different stages obtained in the above steps, and then passes through the above CCA module. In the CCA module, the mean and standard deviation of each layer of feature maps are calculated, and the sum of the two is used as contrast information. After two layers of convolution and the above activation function (such as Sigmod function), basic feature information such as texture and edge is obtained, and then multiplied with the concatenated features of the transfer features of different stages, and finally a convolution is performed to obtain an IMDB output feature, which will pass through four IMDB refined features in IMDM to obtain the final denoising super-resolution feature.

[0098] In this embodiment, through repeated channel separation, multi-layer features in the image can be gradually extracted, so that the model can capture information and features at different levels in the image, thereby improving the accuracy and robustness of image processing, and enabling the model to perceive and process features and structures of different scales more finely, thereby improving the ability to understand image details and enhancing the model's ability to represent multi-scale features.

[0099] In an exemplary embodiment, step S202 includes: performing pixel expansion processing on the sample image to obtain a downsampled image; and extracting input features according to the downsampled image.

[0100] Exemplarily, the server 104 may use the Pixel Unshuffle layer to perform pixel expansion processing on the sample image to obtain a downsampled image. The Pixel Unshuffle layer is used to rearrange and reorganize the channels in the input image to implement an upsampling operation of the image, and can convert the input low-resolution image into a high-resolution image while retaining the details and features of the image. Exemplarily, the number of channels of the downsampled image is increased by 16 times, and the length and width are reduced by 4 times compared to the input sample image.

[0101] Accordingly, step S206 includes: performing aggregation processing on the denoising super-resolution features to obtain aggregated features; obtaining sub-pixel features according to the residual between the aggregated features and the input features; and performing pixel shrinkage processing on the sub-pixel features to obtain image prediction values.

[0102] Exemplarily, the server 104 performs an aggregate convolution operation on the denoised super-resolution features, and then connects the residual of the above-mentioned input features to obtain sub-pixel features, and then uses the Pixel shuffle layer to perform pixel shrinkage processing on the sub-pixel features to obtain image prediction values. Further, the server 104 can obtain first-level sub-pixel features by passing the sub-pixel features through a convolution layer, and the first-level sub-pixel features are upsampled through the Pixel shuffle layer to reduce the number of channels by four times and increase the length and width by two times, and then pass through another convolution layer to obtain second-level sub-pixel features, and the second-level sub-pixel features are upsampled through the Pixel shuffle layer to obtain image prediction values.

[0103] In the above embodiment, the Pixel Unshuffle layer is used to convert the high-resolution image to a low-resolution image, but the detail information is retained, which is similar to splitting a large pixel block into multiple small pixel blocks, which allows convolution operations on the low-resolution image while retaining the high-resolution features. Finally, the Pixel Shuffle layer can be used to convert the processed high-resolution image back to a low-resolution image while maintaining the main features of the image.

[0104] In an exemplary embodiment, as shown in FIG4 , step S208 includes steps S302 to S308 . Among them:

[0105] Step S402: using an artifact discrimination loss function to calculate the true image value and the image prediction value of the sample image to obtain an artifact feature map.

[0106] Among them, artifact discrimination is a technology used in image processing tasks to identify and analyze artifacts in images and take appropriate measures to reduce or eliminate these artifacts, thereby improving image quality, reducing visual interference, and making images more accurate and reliable.

[0107] For example, the server 104 may add artifact discrimination to the process of calculating the target loss to obtain an artifact feature map, and then obtain the target loss based on the artifact feature map. The artifact discrimination formula is:

[0108]

[0109]

[0110]

[0111]

[0112]

[0113] In the above formula, R represents the pixel of the high-resolution image and the pixels of the super-resolution image Subtract, initially characterize image artifacts, The local variance of the residual map R represents the artifact pixels, Represents computing a stable local variance from the residual graph.

[0114] Step S404: Use a sliding exponential average method to adjust the artifact feature map to obtain a target feature map.

[0115] The sliding exponential average method refers to weighting the moving average of the pixel values ​​in the image so that the recent pixel values ​​have a larger weight and the more distant pixel values ​​have a smaller weight, so as to eliminate the noise in the image, smooth the image texture, and retain the main features of the image.

[0116] For example, , the sliding exponential average method is used to compare the residual values ​​to obtain Fine-tune the artifact feature map and finally get the artifact discrimination loss:

[0117]

[0118] Step S406, obtaining the target loss according to the target feature map.

[0119] In the above embodiment, the use of the artifact discrimination method can reduce the texture artifacts generated by the GAN type super-resolution model, and the use of the sliding exponential averaging method can reduce the situation where the real details in the dense area are also classified as artifacts.

[0120] In an exemplary embodiment, as shown in FIG5 , step S202 further includes steps S502 to S506. Among them:

[0121] Step S502, obtaining an original image.

[0122] Exemplarily, the original image may be an acquired image that has not been preprocessed, such as an image directly generated after preliminary detection by an X-ray detection device.

[0123] Step S504, blurring, adding noise and scaling the original image to obtain a transition image.

[0124] Blurring refers to softening and blurring the outline and details of an image, thereby reducing the clarity and recognizability of the image. For example, blurring can be achieved by applying a blur filter, wherein the blur filter includes a mean filter, a Gaussian filter, a median filter, etc., which can blur and smooth the pixel values ​​of the image to a certain extent, thereby reducing the clarity of the image.

[0125] The noise includes additive noise and multiplicative noise.

[0126] For example, when the denoising super-resolution model is used in the image processing of X-ray detection equipment, due to the complexity of the imaging source of the X-ray instrument, the light tube excitation, multiplication, fluorescent screen, etc. in the imaging system will generate complex noise. No matter what type of noise it is considered, it is difficult to clearly use additive noise to describe it accurately. Therefore, in the image degradation process, the noise is considered as a combination of additive noise and multiplicative noise. The additive noise is considered to be the dark current fluctuation caused by the amplification of the thermal electrons by the multiplier, and the multiplicative noise is considered to be other possible noise in the imaging process. Therefore, the noise addition in the degradation process can be defined as:

[0127]

[0128] In the above formula, represents the noise degradation in the degradation process, M represents multiplicative noise, and N represents additive noise, both of which conform to Poisson distribution or Gaussian distribution. The overall degradation process is to first undergo blur degradation, noise degradation, and scaling degradation, and then repeat this process again, and finally scale the degradation result to the original image size to obtain a sample image.

[0129] The scaling process refers to changing the size of the image. For example, the scaling process may be performed in a random manner, such as nearest neighbor interpolation, bilinear interpolation, or bicubic interpolation.

[0130] Step S506, detect whether the size of the transition image meets the preset conditions. If it does, use the transition image as a sample image. If it does not, use the transition image as a new original image and return to execute the steps of blurring, adding noise and scaling the original image until the size of the transition image meets the preset conditions to obtain a sample image.

[0131] Exemplarily, the server 104 processes the original image through the above three independent degradation operations to complete the first stage degradation process, and then the second stage degradation can be performed, wherein the degradation process of the second stage is the same as that of the first stage, and the output result of the second stage degradation will be scaled again to fix the image size to the same size as required for the output image.

[0132] In another embodiment, the server 104 first uses an x-ray detection device to collect high-resolution and high-definition sample images as original images, and the size of the collected images is 2400×2900. Next, the server 104 divides the collected original images into a test set and a training set at a ratio of 1:4. In the training stage, the server 104 crops the original images of the training set into partial images with a size of 384×384 and a repetition rate of 0.5 for training.

[0133] Furthermore, in the above cropping process, the server 104 uses the degradation model to perform degradation processing on the original image, wherein the degradation processing includes three parts: blur processing, noise addition and scaling processing. The server 104 processes the original image through the above three independent degradation operations to complete the first stage of the degradation process, and then the second stage of degradation can be performed, wherein the degradation process of the second stage is the same as that of the first stage, and the output result of the second stage degradation will be scaled again to fix the image size to be the same as the size required for the output image.

[0134] Next, the server 104 normalizes the degraded image. The normalization method used can be expressed as:

[0135]

[0136] in, is the mean of the image, A tensor representing an image, represents the standard deviation, max represents the maximum value of the image tensor, and min represents the minimum value of the image tensor. Represents the normalized image tensor.

[0137] Subsequently, the server 104 first passes the normalized image through two consecutive Pixel unshuffle layers, outputs a downsampled image, the downsampled image channel increases 16 times, the length and width are reduced four times, and the downsampled image is subjected to a layer of convolution to obtain input features, and the input features are subjected to the IMDM module to obtain denoising features. In the IMDM module, features are first extracted through the ECB module, and then through a channel separation, 75% are retained as retained features, and 25% are separated as transfer features for backward transmission. The transfer features are passed through the ECB module again, and then channel separation is performed, and 75% are also retained as new retained features, and 25% are separated as new transfer features for backward transmission. The process is repeated twice, and finally the transfer features obtained at different stages are spliced. Subsequently, the server 104 performs an aggregate convolution operation on the denoised super-resolution features, and then connects them with the residual of the above-mentioned input features to obtain sub-pixel features, and then uses the Pixel shuffle layer to perform pixel shrinkage processing on the sub-pixel features to obtain image prediction values. Furthermore, the server 104 may pass the sub-pixel features through a convolution layer to obtain a first-level sub-pixel feature, and the first-level sub-pixel feature is upsampled through a pixel shuffle layer to reduce the number of channels by four times and increase the length and width by two times, and then pass it through another convolution layer to obtain a second-level sub-pixel feature, and the second-level sub-pixel feature is upsampled through a pixel shuffle layer to obtain an image prediction value.

[0138] Subsequently, the server 104 performs adversarial training using the Unet discriminator, and trains the adversarial training result using the VGG perceptual loss function and the L1 loss function, which is expressed as follows:

[0139]

[0140]

[0141] In the above formula, represents the size of the j-th layer feature map, represents the j-th layer loss network of VGG, y represents the label, The predicted value is refers to the predicted value of the model, Refers to the true value of the model. The VGG perceptual loss function measures the difference between the generated image and the target image by comparing the difference between the feature responses in the VGG network; the L1 loss function measures the difference between the generated image and the target image by calculating the absolute difference between the two.

[0142] Finally, artifact discrimination is added to the adversarial training, and the artifact discrimination loss function is corrected by the sliding exponential average method to obtain the artifact discrimination loss, thereby obtaining the target loss, and the initial model is trained according to the target loss until the preset convergence condition is reached, that is, the image quality meets the preset requirements. Among them, the server 104 uses the structure similarity metric (Structure Similarity-based Metric, SMD) function to evaluate the image quality. SMD is the sum function of the absolute value of grayscale difference. By taking the difference of adjacent pixels in the horizontal and vertical directions, taking the absolute value and then accumulating it, the accumulated value is used to represent the clarity of the image. The specific calculation method is as follows:

[0143]

[0144] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0145] The embodiment of the present application also provides an image processing method, which is mainly illustrated by applying the method to a computer device, and the computer device may specifically be the terminal 102 or the server 104 in FIG. 1 . The terminal 102 and the server 104 may be used alone to execute the image processing method provided in the embodiment of the present application. The terminal 102 and the server 104 may also be used in collaboration to execute the image processing method provided in the embodiment of the present application.

[0146] Exemplarily, the solution provided in the embodiment of the present application can be used in the process of X-ray detection to process and generate high-resolution X-ray detection result images. In an exemplary embodiment, an image processing method is provided, which is described by taking the application of the method to the server 104 in FIG. 1 as an example, including obtaining an image to be processed; and using the above-mentioned denoising super-resolution model to process the image to be processed to obtain a target image.

[0147] The denoising super-resolution model consists of two pixel unshuffle layers, one input convolution Conv1, one IMDM, one aggregation layer convolution Conv2, two upsampling convolutions Conv3 and Conv4, and two pixel shuffle layers. Furthermore, the IMDM module consists of four IMDB blocks, and the convolution in each IMDB block is replaced by a re-parameterized edge-aware convolution block (ECB).

[0148] In an exemplary embodiment, after obtaining the image to be processed, the server 104 may also pre-process the image, which includes: obtaining an initial image; performing blur processing, adding noise, and scaling processing on the initial image to obtain a transition image; detecting whether the size of the transition image meets a preset condition, if it meets the preset condition, using the transition image as the image to be processed; if it does not meet the preset condition, using the transition image as a new initial image, and returning to perform the steps of blurring, adding noise, and scaling processing on the initial image until the size of the transition image meets the preset condition to obtain the image to be processed.

[0149] Blurring refers to softening and blurring the outline and details of an image, thereby reducing the clarity and recognizability of the image. For example, blurring can be achieved by applying a blur filter, wherein the blur filter includes a mean filter, a Gaussian filter, a median filter, etc., which can blur and smooth the pixel values ​​of the image to a certain extent, thereby reducing the clarity of the image.

[0150] Furthermore, the above noise includes additive noise and multiplicative noise.

[0151] For example, when the denoising super-resolution model is used in the image processing of X-ray detection equipment, due to the complexity of the imaging source of the X-ray instrument, the light tube excitation, multiplication, fluorescent screen, etc. in the imaging system will generate complex noise. No matter what type of noise it is considered, it is difficult to clearly use additive noise to describe it accurately. Therefore, in the image degradation process, the noise is considered as a combination of additive noise and multiplicative noise. The additive noise is considered to be the dark current fluctuation caused by the amplification of the thermal electrons by the multiplier, and the multiplicative noise is considered to be other possible noise in the imaging process. Therefore, the noise addition in the degradation process can be defined as:

[0152]

[0153] In the above formula, Indicates the noise degradation in the degradation process, M indicates multiplicative noise, and N indicates additive noise, both of which conform to Poisson distribution or Gaussian distribution. The overall degradation process is to first undergo blur degradation, noise degradation, and scaling degradation, and then repeat this process again, and finally scale the degradation result to the original image size to obtain the image to be processed.

[0154] The scaling process refers to changing the size of the image. For example, the scaling process may be performed in a random manner, such as nearest neighbor interpolation, bilinear interpolation, or bicubic interpolation.

[0155] In this embodiment, the denoising super-resolution model is used to process the image to be processed, and the characteristics of the denoising super-resolution model can be utilized to improve the image resolution and the speed of image processing.

[0156] Based on the same inventive concept, the embodiment of the present application also provides a denoising super-resolution model training device for implementing the denoising super-resolution model training method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more denoising super-resolution model training device embodiments provided below can refer to the limitations of the denoising super-resolution model training method above, and will not be repeated here.

[0157] In an exemplary embodiment, as shown in FIG6 , a denoising super-resolution model training device is provided, comprising: a sample acquisition module 602, a feature extraction module 604, an image prediction module 606, a loss calculation module 608 and a model training module 610, wherein:

[0158] The sample acquisition module 602 is used to acquire a sample image, input the sample image into the initial model, and extract input features of the sample image;

[0159] A feature extraction module 604 is used to process the input features using the target edge-aware convolution block of the initial model to obtain denoised super-resolution features; the target edge-aware convolution block is obtained by reducing the number of channels of the initial edge-aware convolution block using a re-parameterization method;

[0160] An image prediction module 606 is used to obtain an image prediction value according to the input features and the denoising super-resolution features;

[0161] A loss calculation module 608, used to obtain a target loss according to the true image value and the image prediction value of the sample image;

[0162] The model training module 610 is used to adjust the parameters of the initial model according to the target loss until the preset convergence conditions are met to obtain a denoising super-resolution model.

[0163] In one embodiment, the feature extraction module 604 includes:

[0164] A feature extraction submodule for extracting input features using target edge-aware convolutional blocks;

[0165] The channel separation submodule is used to perform channel separation on the input features to obtain the retained features and the transferred features;

[0166] The iterative processing submodule is used to perform multiple iterative processing on the transfer features, and the input of each iterative processing is the transfer features output by the previous iterative processing; each iterative processing includes: using the target edge-aware convolution block to perform feature extraction and channel separation processing on the transfer features to obtain new retained features and new transfer features;

[0167] The feature processing submodule is used to obtain denoising super-resolution features based on the transfer features obtained through multiple channel separations.

[0168] In one embodiment, the sample acquisition module 602 includes:

[0169] A pixel expansion submodule is used to perform pixel expansion processing on the sample image to obtain a downsampled image;

[0170] A downsampling submodule, used to extract input features based on the downsampled image;

[0171] Accordingly, the image prediction module 606 includes:

[0172] The aggregation processing submodule is used to aggregate the denoising super-resolution features to obtain aggregate features;

[0173] The residual connection submodule is used to obtain sub-pixel features based on the residual of the aggregated features and the input features;

[0174] The pixel shrinkage submodule is used to perform pixel shrinkage processing on sub-pixel features to obtain image prediction values.

[0175] In one embodiment, the loss calculation module 608 includes:

[0176] An artifact discrimination submodule is used to calculate the true image value and the image prediction value of the sample image using an artifact discrimination loss function to obtain an artifact feature map;

[0177] The loss acquisition submodule is used to obtain the target loss based on the artifact feature map.

[0178] In one embodiment, the loss acquisition submodule includes:

[0179] A feature averaging unit, used to adjust the artifact feature map by using a sliding exponential averaging method to obtain a target feature map;

[0180] The loss generation unit is used to obtain the target loss according to the target feature map.

[0181] In one embodiment, the sample acquisition module 602 further includes:

[0182] An image acquisition submodule, used to acquire the original image;

[0183] The degradation processing submodule is used to perform blurring, noise addition and scaling on the original image to obtain a transition image;

[0184] The image transition submodule is used to detect whether the size of the transition image meets the preset conditions. If it meets the preset conditions, the transition image is used as a sample image. If it does not meet the preset conditions, the transition image is used as a new original image and returns to execute the steps of blurring, adding noise and scaling the original image until the size of the transition image meets the preset conditions to obtain a sample image.

[0185] In one embodiment, the noise includes additive noise and multiplicative noise.

[0186] Each module in the above-mentioned denoising super-resolution model training device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0187] Based on the same inventive concept, the embodiment of the present application also provides an image processing device for implementing the above-mentioned image processing method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more image processing device embodiments provided below can refer to the limitations on the image processing method above, and will not be repeated here.

[0188] In an exemplary embodiment, as shown in FIG7 , an image processing device is provided, including: an image acquisition module 702 and an image processing module 704, wherein:

[0189] An image acquisition module 702 is used to acquire an image to be processed;

[0190] The image processing module 704 is used to process the image to be processed using the denoising super-resolution model of the first aspect to obtain a target image.

[0191] Each module in the above image processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0192] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG8. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a denoising super-resolution model training method or an image processing method is implemented.

[0193] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG9. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a denoising super-resolution model training method or an image processing method is implemented.

[0194] Those skilled in the art will understand that the structure shown in FIG. 9 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0195] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the following steps when executing the computer program: obtaining a sample image, and inputting the sample image into an initial model to extract input features of the sample image; using a target edge-aware convolution block of the initial model to process the input features to obtain denoising super-resolution features; the target edge-aware convolution block is obtained by reducing the number of channels of the initial edge-aware convolution block by a reparameterization method; obtaining an image prediction value based on the input features and the denoising super-resolution features; obtaining a target loss based on the true image value and the image prediction value of the sample image; adjusting the parameters of the initial model according to the target loss until a preset convergence condition is met to obtain a denoising super-resolution model.

[0196] In one embodiment, when the processor executes the computer program, the following steps are also implemented: using the target edge-aware convolution block to extract input features; performing channel separation on the input features to obtain retained features and transfer features; performing multiple iterative processes on the transfer features, wherein the input of each iterative process is the transfer features output by the previous iterative process; each iterative process includes: using the target edge-aware convolution block to perform feature extraction and channel separation on the transfer features to obtain new retained features and new transfer features; and obtaining denoising super-resolution features based on the transfer features obtained by multiple channel separations.

[0197] In one embodiment, when the processor executes the computer program, the following steps are also implemented: performing pixel expansion processing on the sample image to obtain a downsampled image; extracting input features based on the downsampled image; performing aggregation processing on the denoising super-resolution features to obtain aggregated features; obtaining sub-pixel features based on the residual between the aggregated features and the input features; performing pixel contraction processing on the sub-pixel features to obtain image prediction values.

[0198] In one embodiment, when the processor executes the computer program, the following steps are also implemented: using the artifact discrimination loss function to calculate the true image value and the image prediction value of the sample image to obtain an artifact feature map; and obtaining the target loss according to the artifact feature map.

[0199] In one embodiment, when the processor executes the computer program, the following steps are also implemented: using a sliding exponential average method to adjust the artifact feature map to obtain a target feature map; and obtaining a target loss based on the target feature map.

[0200] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining an original image; blurring, adding noise, and scaling the original image to obtain a transition image; detecting whether the size of the transition image meets preset conditions, and if so, using the transition image as a sample image; if not, using the transition image as a new original image, and returning to execute the steps of blurring, adding noise, and scaling the original image until the size of the transition image meets the preset conditions to obtain a sample image.

[0201] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the following steps when executing the computer program: acquiring an image to be processed; processing the image to be processed using the above-mentioned denoising super-resolution model to obtain a target image.

[0202] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0203] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0204] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0205] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0206] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A denoising super-resolution model training method, characterized in that: The method comprises: Acquire a sample image, input the sample image into an initial model, and extract input features of the sample image; The input features are processed using the target edge-aware convolution block of the initial model to obtain denoised super-resolution features; the target edge-aware convolution block is obtained by reducing the number of channels of the initial edge-aware convolution block using a reparameterization method; Obtaining an image prediction value according to the input feature and the denoising super-resolution feature; Obtaining a target loss according to the true image value of the sample image and the image prediction value; The parameters of the initial model are adjusted according to the target loss until a preset convergence condition is met to obtain a denoising super-resolution model.

2. The method according to claim 1, characterized in that The process of processing the input features using the target edge-aware convolution block of the initial model to obtain denoising super-resolution features includes: extracting the input features using the target edge-aware convolutional block; Performing channel separation on the input features to obtain retained features and transferred features; The transfer feature is iteratively processed for multiple times, and the input of each iterative process is the transfer feature output by the previous iterative process; each iterative process includes: using the target edge-aware convolution block to perform feature extraction and channel separation on the transfer feature to obtain a new retained feature and a new transfer feature; According to the transfer features obtained through multiple channel separations, denoising super-resolution features are obtained.

3. The method according to claim 2, characterized in that The step of extracting the input features of the sample image comprises: Performing pixel expansion processing on the sample image to obtain a downsampled image; extracting input features according to the downsampled image; Accordingly, obtaining the image prediction value according to the input feature and the denoising super-resolution feature includes: Aggregating the denoising super-resolution features to obtain aggregated features; Obtaining a sub-pixel feature according to a residual between the aggregated feature and the input feature; A pixel shrinkage process is performed on the sub-pixel feature to obtain an image prediction value.

4. The method according to claim 1, characterized in that: The obtaining of the target loss according to the true image value of the sample image and the image prediction value comprises: Using an artifact discrimination loss function to calculate the true image value and the image prediction value of the sample image to obtain an artifact feature map; A target loss is obtained according to the artifact feature map.

5. The method according to claim 4, characterized in that The obtaining of the target loss according to the artifact feature map comprises: The artifact feature map is adjusted by using a sliding exponential average method to obtain a target feature map; A target loss is obtained according to the target feature map.

6. The method according to any one of claims 1 to 5, characterized in that: The acquiring of the sample image comprises: Get the original image; Performing blur processing, adding noise and scaling processing on the original image to obtain a transition image; Check whether the size of the transition image meets the preset conditions. If so, use the transition image as a sample image. If not, use the transition image as a new original image and return to execute the steps of blurring, adding noise and scaling the original image until the size of the transition image meets the preset conditions to obtain a sample image.

7. The method according to claim 6, characterized in that The noise includes additive noise and multiplicative noise.

8. An image processing method, characterized in that: The method comprises: Get the image to be processed; The image to be processed is processed using the denoising super-resolution model as described in any one of claims 1 to 7 to obtain a target image.

9. A denoising super-resolution model training device, characterized in that: The device comprises: A sample acquisition module, used to acquire a sample image, input the sample image into an initial model, and extract input features of the sample image; A feature extraction module, used to process the input features using the target edge-aware convolution block of the initial model to obtain denoised super-resolution features; the target edge-aware convolution block is obtained by reducing the number of channels of the initial edge-aware convolution block using a reparameterization method; An image prediction module, used to obtain an image prediction value according to the input features and the denoising super-resolution features; A loss calculation module, used for obtaining a target loss according to the true image value of the sample image and the image prediction value; The model training module is used to adjust the parameters of the initial model according to the target loss until the preset convergence conditions are met to obtain a denoising super-resolution model.

10. An image processing device, characterized in that: The device comprises: An image acquisition module, used for acquiring an image to be processed; An image processing module is used to process the image to be processed using the denoising super-resolution model as described in any one of claims 1 to 7 to obtain a target image.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images