Image processing method and device, equipment, storage medium and product

Abstract

The invention relates to the technical field of image processing, and discloses an image processing method, device and equipment, a storage medium and a computer program product.The method comprises the steps that in response to an image processing request, an input image is input into a preset image processing model, and a compression quality factor of the input image is predicted through the preset image processing model; performing image restoration on the input image according to the compression quality factor to obtain an output image; the compression quality factor of the input image is predicted through the preset image processing model, and image restoration is performed on the input image according to the compression quality factor, so that the compression quality factor of the image can be decoupled from the input image, images with different compression degrees can be processed in a single model, and the image restoration efficiency is improved. And the application range and flexibility of the model are improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, storage medium and computer program product. Background Art

[0002] JPEG (Joint Photographic Experts Group) is a widely used image compression algorithm and format. Almost all image browsers, editors, and cameras support the JPEG format because it can greatly compress image data, reduce file size, and is suitable for storage and transmission. Users can also adjust the compression quality factor (QF) to balance file size and image quality as needed. However, JPEG is a lossy compression algorithm. The degree of image compression is controlled by QF (0-100). The lower the QF, the higher the degree of compression and the smaller the storage size. Some image information will be lost during the compression process, especially when the compression degree is high, compression artifacts such as block effects and blurring may appear.

[0003] Existing image artifact removal methods require training different models for image processing for images with different compression levels, which results in a lack of versatility and flexibility in the models. Summary of the invention

[0004] The main purpose of this application is to provide an image processing method, apparatus, device, storage medium and computer program product, aiming to solve the technical problem that the existing image artifact removal method requires training different models for image processing for images with different compression levels, resulting in a lack of versatility and flexibility of the model.

[0005] To achieve the above object, the present application provides an image processing method, which includes:

[0006] In response to an image processing request, inputting an input image into a preset image processing model;

[0007] The compression quality factor of the input image is predicted by the preset image processing model, and image restoration is performed on the input image according to the compression quality factor to obtain an output image.

[0008] Optionally, the preset image processing model includes a feature extractor, a quality factor predictor, a flexible controller and an image restorer;

[0009] The method of predicting the compression quality factor of the input image by using the preset image processing model and performing image restoration on the input image according to the compression quality factor to obtain an output image includes:

[0010] Extracting image features and compression quality factor features from the input image by the feature extractor;

[0011] predicting a compression quality factor of the input image by the quality factor predictor according to the quality factor feature;

[0012] Searching, by the flexible controller, for a quality factor parameter pair corresponding to the compression quality factor in a preset mapping function;

[0013] The image restorer performs image restoration on the input image based on the image feature and the quality factor parameter to obtain an output image.

[0014] Optionally, before extracting the image features and the compression quality factor features from the input image by the feature extractor, the method further includes:

[0015] Responding to an interactive adjustment request for the compression quality factor;

[0016] Adjust the compression quality factor according to the interactive request to obtain an adjusted compression quality factor;

[0017] The extracting the image feature and the compression quality factor feature from the input image by the feature extractor comprises:

[0018] The flexible controller searches for a quality factor parameter pair corresponding to the adjusted compression quality factor in a preset mapping function.

[0019] Optionally, the feature extractor includes multiple stages, each stage uses a residual block, and the residual block includes multiple convolutional layers and non-linear activation functions;

[0020] The extracting the image feature and the compression quality factor feature from the input image by the feature extractor comprises:

[0021] Image features and compression quality factor features are extracted from the input image through the multiple convolutional layers and the nonlinear activation function, wherein the image feature extraction process includes residual learning, channel adjustment and downsampling scaling.

[0022] Optionally, the network structure used by the feature extractor corresponds to the network structure used by the image restorer, there is a jump connection between the feature extractor and the image restorer at each stage, and the image restorer includes a QF attention block;

[0023] The performing image restoration on the input image by the image restorer based on the image feature and the quality factor parameter to obtain an output image comprises:

[0024] Performing bilinear interpolation upsampling on the image features to obtain a high-level feature map;

[0025] Obtaining the output feature map of the feature extractor at each stage through the jump connection;

[0026] An output image is obtained by spatially applying an affine transformation to the feature map within the QF attention block based on the quality factor parameter, wherein the feature map includes the high-level feature map and the output feature map.

[0027] Optionally, the preset mapping function is obtained by mapping the compressed quality factors into a low-dimensional continuous vector space during the model training process and analyzing the semantic relationship and context information between the quality factors.

[0028] Optionally, the quality factor predictor comprises a plurality of multilayer perceptrons, and nodes of hidden layers of the plurality of multilayer perceptrons have the same dimension as the compressed quality factor feature.

[0029] Optionally, before inputting the input image into a preset image processing model in response to the image processing request, the method further includes:

[0030] Acquire multiple original images through multiple image sources, use the multiple original images as a data set, and divide the data set into a training set and a test set, wherein the original images are high-resolution images;

[0031] Training the initial image processing model according to the training set to obtain a trained image processing model;

[0032] A performance test is performed on the trained image processing model according to the test set, and the trained image processing model is adjusted according to the performance test result to obtain a preset image processing model.

[0033] Optionally, the training the initial image processing model according to the training set to obtain a trained image processing model includes:

[0034] Randomly generate compression quality factor;

[0035] Compressing the original images in the training set into a specified format according to the randomly generated compression quality factor to obtain a target training set;

[0036] The initial image processing model is trained according to the target training set to obtain a trained image processing model, wherein the true value label of the target training set is the original image.

[0037] Optionally, the training the initial image processing model according to the target training set to obtain a trained image processing model includes:

[0038] Training the initial image processing model according to the target training set, and obtaining the prediction loss and the recovery loss in the training process, wherein the prediction loss is the prediction loss of the quality factor in each batch, and the recovery loss is the loss between the restored image in each batch and the original image;

[0039] The initial image processing model is adjusted according to the prediction loss and the recovery loss to obtain a trained image processing model.

[0040] Optionally, performing a performance test on the trained image processing model according to the test set, and adjusting the trained image processing model according to the performance test result to obtain a preset image processing model includes:

[0041] Calculating a peak signal-to-noise ratio, a structural similarity index, and a peak signal-to-noise ratio sensitive to blockiness of the trained image processing model according to the test set;

[0042] Generating a performance test result of the trained image processing model according to the peak signal-to-noise ratio, the structural similarity index, and the peak signal-to-noise ratio sensitive to the blocking effect;

[0043] The trained image processing model is adjusted according to the performance test results to obtain a preset image processing model.

[0044] In addition, to achieve the above-mentioned purpose, the present application also proposes an image processing device, the image processing device comprising:

[0045] A model input module, used for inputting an input image into a preset image processing model in response to an image processing request;

[0046] The model processing module is used to predict the compression quality factor of the input image through the preset image processing model, and perform image restoration on the input image according to the compression quality factor to obtain an output image.

[0047] Optionally, the preset image processing model includes a feature extractor, a quality factor predictor, a flexible controller and an image restorer;

[0048] The model processing module is also used to extract image features and compression quality factor features from the input image through the feature extractor; predict the compression quality factor of the input image through the quality factor predictor according to the quality factor features; search for the quality factor parameter pair corresponding to the compression quality factor in a preset mapping function through the flexible controller; and perform image restoration on the input image based on the image features and the quality factor parameters through the image restorer to obtain an output image.

[0049] Optionally, the image processing device further includes:

[0050] An interactive module, configured to respond to an interactive adjustment request for the compression quality factor; adjust the compression quality factor according to the interactive request to obtain an adjusted compression quality factor;

[0051] The model processing module is further used to search for the quality factor parameter pair corresponding to the adjusted compression quality factor in a preset mapping function through the flexible controller.

[0052] Optionally, the feature extractor includes multiple stages, each stage uses a residual block, and the residual block includes multiple convolutional layers and non-linear activation functions;

[0053] The model processing module is also used to extract image features and compression quality factor features from the input image through the multiple convolutional layers and the nonlinear activation function, wherein the image feature extraction process includes residual learning, channel adjustment and downsampling scaling.

[0054] Optionally, the network structure used by the feature extractor corresponds to the network structure used by the image restorer, there is a jump connection between the feature extractor and the image restorer at each stage, and the image restorer includes a QF attention block;

[0055] The model processing module is also used to perform bilinear interpolation upsampling on the image features to obtain a high-level feature map; obtain the output feature map of the feature extractor at each stage through the jump connection; and apply affine transformation to the feature map in the QF attention block spatially based on the quality factor parameter to obtain an output image, wherein the feature map includes the high-level feature map and the output feature map.

[0056] Optionally, the preset mapping function is obtained by mapping the compressed quality factors into a low-dimensional continuous vector space during the model training process and analyzing the semantic relationship and context information between the quality factors.

[0057] In addition, to achieve the above-mentioned purpose, the present application also proposes an image processing device, which includes a memory, a processor, and an image processing program stored in the memory and executable on the processor, wherein the image processing program is configured to implement the image processing method described above.

[0058] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, on which an image processing program is stored, and when the image processing program is executed by a processor, the image processing method as described above is implemented.

[0059] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes an image processing program, and when the image processing program is executed by a processor, it implements the image processing method described above.

[0060] One or more technical solutions proposed in this application have at least the following technical effects:

[0061] In the present application, it is disclosed that in response to an image processing request, an input image is input into a preset image processing model, a compression quality factor of the input image is predicted by the preset image processing model, and image restoration is performed on the input image according to the compression quality factor to obtain an output image; since the present application predicts the compression quality factor of the input image by the preset image processing model and performs image restoration on the input image according to the compression quality factor, the compression quality factor of the image can be decoupled from the input image, and then images of different compression degrees can be processed in a single model, thereby improving the scope of application and flexibility of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0063] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0064] Figure 1 This is a schematic diagram of the flow chart of the first embodiment of the image processing method of the present application;

[0065] Figure 2 A schematic diagram of a flow chart of a second embodiment of the image processing method of the present application;

[0066] Figure 3 This is the overall architecture diagram of the ICDNet model of an embodiment of the image processing method of this application;

[0067] Figure 4 This is a schematic flow chart of a third embodiment of the image processing method of the present application;

[0068] Figure 5 This is a schematic diagram of the module structure of the image processing device according to an embodiment of the present application;

[0069] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the image processing method in the embodiment of the present application.

[0070] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0071] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0072] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0073] JPEG (Joint Photographic Experts Group) is a widely used image compression algorithm and format. Almost all image browsers, editors, and cameras support JPEG format because it can greatly compress image data, reduce file size, and is suitable for storage and transmission. Users can also balance file size and image quality by adjusting the compression quality factor (QF) as needed. However, JPEG is a lossy compression algorithm. The degree of image compression is controlled by QF (0-100). The lower the QF, the higher the degree of compression and the smaller the storage size. Some image information will be lost during the compression process, especially when the compression degree is high, compression artifacts such as block effects and blurring may occur. Therefore, how to remove artifact blurring caused by image quality compression is of great significance. At present, inspired by the success of convolutional neural networks (CNN) in image classification, deep convolutional networks can be used to remove JPEG artifact blurring.

[0074] However, in practical applications, existing JPEG artifact removal methods require training different models for image processing for images with different compression levels, which results in a lack of versatility and flexibility in the models.

[0075] Therefore, in order to overcome the above-mentioned defects, the present application provides a solution, which includes: in response to an image processing request, inputting an input image into a preset image processing model, predicting the compression quality factor of the input image through the preset image processing model, and performing image restoration on the input image according to the compression quality factor to obtain an output image; since the present application predicts the compression quality factor of the input image through a preset image processing model and performs image restoration on the input image according to the compression quality factor, it is possible to decouple the compression quality factor of the image from the input image, and then it is possible to process images of different compression degrees in a single model, thereby improving the scope of application and flexibility of the model.

[0076] It should be noted that the execution subject of this embodiment can be an image processing device with data processing, network communication and program running functions, such as a computer, or other electronic devices that can achieve the same or similar functions, and this embodiment does not impose any restrictions on this.

[0077] Based on this, the present application embodiment provides an image processing method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the image processing method of the present application.

[0078] In this embodiment, the image processing method includes:

[0079] Step S10: In response to the image processing request, input the input image into a preset image processing model.

[0080] It should be understood that the application scenario of the embodiment of the present application may be to remove the artifact blur caused by image quality compression. In a specific implementation, it may be the artifact blur in a JPEG image, and this embodiment is not limited to this.

[0081] It is understandable that the user may initiate an image processing request by inputting an image, or may first initiate an image processing request and then input the input image into a preset image processing model, and this embodiment does not impose any limitation on this.

[0082] Step S20: predicting the compression quality factor of the input image by using the preset image processing model, and performing image restoration on the input image according to the compression quality factor to obtain an output image.

[0083] It should be noted that JPEG is a lossy compression algorithm. The degree of image compression is controlled by the compression quality factor (QF) (0-100). The lower the QF, the higher the compression degree and the smaller the storage size.

[0084] It should be understood that the preset image processing model can be a pre-trained deep convolutional network. The preset image processing model can include convolution layers, activation functions, and pooling layers, etc., which are used to extract image features and predict the compression quality factor of the input image based on the features; the preset image processing model can also include specific image algorithms (such as deblurring algorithms, etc.), and the input image can be restored by a specific image algorithm according to the compression quality factor to obtain an output image. This embodiment does not limit this.

[0085] In a specific implementation, the preset image processing model may be an Image Compression Deblur Network (ICDNet) model. The ICDNet model may include a feature extractor, a quality factor predictor, a flexible controller, and an image restorer for restoring compressed images. This embodiment does not impose any restrictions on this.

[0086] For ease of understanding, the following examples are given, but are not intended to limit the present application. As an example, assume that a JPEG image is input into the ICDNet model, the compression quality factor of the image is predicted by the ICDNet model, and the JPEG image is restored according to the compression quality factor to obtain an output image.

[0087] This embodiment predicts the compression quality factor of the input image through a preset image processing model, and performs image restoration on the input image based on the compression quality factor, thereby being able to decouple the compression quality factor of the image from the input image, and then being able to process images of different compression degrees in a single model, thereby improving the scope of application and flexibility of the model.

[0088] Reference Figure 2 , Figure 2 This is a flow chart of the second embodiment of the image processing method of the present application. Figure 1 The first embodiment shown provides a second embodiment of the image processing method of the present application.

[0089] In the second embodiment, the preset image processing model includes a feature extractor, a quality factor predictor, a flexible controller and an image restorer; the step S20 includes:

[0090] Step S201: extracting image features and compression quality factor features from the input image through the feature extractor.

[0091] It should be understood that in order to improve the image restoration effect, in this embodiment, the preset image processing model includes a feature extractor, a quality factor predictor, a flexible controller and an image restorer, and image restoration is performed through the feature extractor, the quality factor predictor, the flexible controller and the image restorer.

[0092] For ease of understanding, refer to Figure 3 This invention is provided for illustration, but is not intended to limit the present application. Figure 3 This is an overall architecture diagram of the ICDNet model of an embodiment of the image processing method of the present application. As an example, in the figure, ICDNet consists of four parts: a feature extractor, a quality factor predictor, a flexible controller and an image restorer.

[0093] It is understandable that a deep convolutional network architecture (such as ResNet series, UNet series, and Xception architectures) may be provided in the feature extractor to extract image features and compression quality factor features from the input image through the deep convolutional network architecture.

[0094] Further, in order to improve, the step S201 includes: extracting image features and compression quality factor features from the input image through the multiple convolutional layers and the nonlinear activation function, wherein the image feature extraction process includes residual learning, channel adjustment and downsampling scaling.

[0095] For ease of understanding, refer to Figure 3 This invention is provided for illustration, but is not intended to limit the present application. Figure 3 The overall architecture diagram of the ICDNet model of an embodiment of the image processing method of the present application is shown as an example. In the figure, the feature extractor is used to extract the deep features of the image and decouple the potential compression quality factor from the input image. In the feature extractor, you can choose to use a deep convolutional network architecture, such as (ResNet series, UNet series, and Xception architectures) to extract the deep features of the image. But the premise is that the network structure used here needs to correspond to the network structure used in the image restorer, because there is a jump connection with the restorer at each stage, which can effectively alleviate the gradient vanishing problem during the model training process, accelerate model convergence, prevent network degradation, improve information flow, simplify model design, and realize feature reuse. The deep convolutional network used in this application involves four stages, each stage uses a residual block, each residual block consists of two 3*3 convolutional layers, batch normalization (Batch Normalization) and ReLU activation are used in the middle, and 3 1*1 convolutional layers are used to adjust channel alignment, and the downsampling scaling operation uses a 2*2 step length convolution. The number of output channels of each layer from the first layer to the fourth layer is set to (64, 128, 256, 512) respectively, which can be dynamically adjusted according to the different deep convolutional networks used. The parameters output by the feature extractor are shared by the image restorer and the QF predictor. The input image is extracted by the feature extractor and the deep image features are output to the image restorer. The predicted quality factor branch is decoupled and further processed to extract higher-level information, thereby predicting the compression quality factor of the image. Since the quality factor is a single value for each image, a global average pooling layer is used to obtain global information from the image features.

[0096] This embodiment adopts a deep convolutional neural network (CNN) in the network structure design to perform feature extraction and deblurring processing. Through the combination of multiple convolutional layers and non-linear activation functions, the model can extract the deep features of the image and effectively remove the block effect and blurring artifacts caused by JPEG compression during the deblurring process. The feature extractor includes multiple stages, each of which uses a residual block, and the residual block includes multiple convolutional layers and non-linear activation functions.

[0097] Step S202: predicting the compression quality factor of the input image through the quality factor predictor according to the quality factor feature.

[0098] It should be understood that in order to better predict the compression quality factor of the input image, in this embodiment, the quality factor predictor includes multiple multilayer perceptrons, and the nodes of the hidden layers of the multiple multilayer perceptrons have the same dimension as the compression quality factor feature.

[0099] For ease of understanding, refer to Figure 3 This invention is provided for illustration, but is not intended to limit the present application. Figure 3 This is an overall architecture diagram of the ICDNet model of an embodiment of the image processing method of the present application. As an example, in the figure, the quality factor predictor is composed of 3 layers of multilayer perceptron (MLP), which shares the output parameters of the feature extractor, takes the high-order features of the quality factor as input (the dimension of the quality factor here is determined by the deep feature extraction network adopted by the feature extractor), and sets the number of nodes in each hidden layer to be the same as the dimension of the quality factor feature in order to better predict and output the predicted quality factor of the compressed image.

[0100] It is understandable that during the training of the ICDNet model, different small-sized patches cropped from the same image may contain only limited information, resulting in multiple quality factors being predicted, making it impossible to accurately estimate the compression quality factor of the original image, which will lead to instability in the training process. Therefore, the L1 loss function is used to avoid excessive penalties for such outliers. The prediction loss of the quality factor in each batch (Loss QF ) is calculated as follows:

[0101]

[0102] Where N represents the batch size during training. represents the quality factor predicted for each image, Represents the true quality factor of each image.

[0103] Step S203: searching, by the flexible controller, for a quality factor parameter pair corresponding to the compression quality factor in a preset mapping function.

[0104] It should be understood that the preset mapping function is obtained by mapping the compressed quality factors into a low-dimensional continuous vector space during the model training process and analyzing the semantic relationship and context information between the quality factors.

[0105] For ease of understanding, refer to Figure 3 This invention is provided for illustration, but is not intended to limit the present application. Figure 3 This is the overall architecture diagram of the ICDNet model of an embodiment of the image processing method of the present application. As an example, in the figure, the flexible controller flexibly controls the strategy of the image restorer when restoring the image according to the input of different quality factors, so as to achieve a balance between artifact removal and detail retention, and realize interactive selection and adjustment to meet the preferences of different users. The flexible controller is also a 3-layer MLP, which takes the quality factor as input to represent the degree of compression of the target image. Inspired by the Transformer architecture of the large language model, the controller realizes flexible control by learning the Embedding of the quality factor and integrating it into the image restorer. An embedding matrix is ​​used to map the quality factor to a low-dimensional continuous vector space, so that similar quality factors are close in this vector space, which can effectively capture the semantic relationship and contextual information between the quality factors. Specifically, the controller learns this mapping function, which outputs a modulation parameter pair (α, β) embedded in a given quality factor. The three layers of the MLP correspond to the three scales in the image restorer, respectively. Different parameter pairs of different scales are learned in the restorer. The QF attention block enables the restorer to produce different results according to different QF embeddings, thereby achieving the balance between controlling artifact removal and detail retention.

[0106] Furthermore, considering that the existing methods of training a single model for different quality factors can only provide a deterministic reconstruction result for each image, while ignoring the user's preference requirements between artifact removal effect and detail retention. Therefore, in order to meet the preference requirements of different users, before step S203, it also includes: responding to an interactive adjustment request for the compression quality factor; adjusting the compression quality factor according to the interactive request to obtain an adjusted compression quality factor; accordingly, step S203 also includes: searching for a quality factor parameter pair corresponding to the adjusted compression quality factor in a preset mapping function through the flexible controller.

[0107] It should be understood that by introducing a controllable variable with clear physical meaning (predicted compression quality factor), the user is allowed to adjust the balance between artifact removal effect and detail retention through interactive selection, thereby meeting the preferences of different users. This allows the model to not only provide high-quality artifact removal effects, but also optimize detail retention according to the specific needs of users, thereby improving the user experience and the practicality of the model.

[0108] Step S204: performing image restoration on the input image through the image restorer based on the image features and the quality factor parameters to obtain an output image.

[0109] Furthermore, the network structure used by the feature extractor corresponds to the network structure used by the image restorer, and there is a jump connection between the feature extractor and the image restorer at each stage, and the image restorer includes a QF attention block; the step S204 includes: performing bilinear interpolation upsampling on the image features to obtain a high-level feature map; obtaining the output feature map of the feature extractor at each stage through the jump connection; and applying an affine transformation to the feature map in the QF attention block spatially based on the quality factor parameter to obtain an output image, wherein the feature map includes the high-level feature map and the output feature map.

[0110] For ease of understanding, refer to Figure 3 This invention is provided for illustration, but is not intended to limit the present application. Figure 3 This is the overall architecture diagram of the ICDNet model of an embodiment of the image processing method of the present application. As an example, in the figure, the image restorer is responsible for generating a clear image after restoration. It has two inputs, namely the high-level feature map output by the feature extractor and the quality factor parameter pair (α, β) output by the flexible controller. The restorer has four stages, each of which receives a jump connection from the corresponding stage of the feature extractor. The input low-size high-level feature map is upsampled using bilinear interpolation to restore the image size. In the last three stages, each layer has 4 QF attention blocks to learn the quality factor parameter pair (α, β) output by the flexible controller. The parameter pair adaptively affects the output by applying an affine transformation to each feature map in the QF attention block of each stage in space. The parameter pair (α, β) transforms the input image through feature mapping, and the calculation formula is as follows:

[0111]

[0112] In the formula, F out and F in represents the feature map before and after the affine transformation, ⊙ represents element multiplication, Represents element-wise addition.

[0113] It is understandable that in the process of training the ICDNet model, in a batch of N training samples, the goal of the image restorer is to minimize the restored image I res and the original high-resolution image I gt The L1 loss between:

[0114]

[0115] Where: N represents the batch size during training, represents the restored image, Represents the original high-resolution image.

[0116] In this embodiment, the preset image processing model includes a feature extractor, a quality factor predictor, a flexible controller and an image restorer, and image restoration is performed through the feature extractor, the quality factor predictor, the flexible controller and the image restorer, thereby improving the image restoration effect.

[0117] Reference Figure 4 , Figure 4 This is a flow chart of the third embodiment of the image processing method of the present application. Based on the above embodiments, the third embodiment of the image processing method of the present application is proposed.

[0118] In the third embodiment, before step S10, the method further includes:

[0119] Step S01: obtaining a plurality of original images through a plurality of image sources, taking the plurality of original images as a data set, and dividing the data set into a training set and a test set, wherein the original images are high-resolution images.

[0120] It should be understood that in order to improve the performance of the preset image processing model, in this embodiment, multiple original images are obtained in advance through multiple image sources as data sets to train the image processing model.

[0121] For ease of understanding, the following examples are given, but are not intended to limit the present application. As an example, assume that a high-quality dataset generated by DIV2K, Flickr2K, and 360 Image Search is fused as a dataset, wherein the training set has 810,000 and the test set has 200,000.

[0122] Step S02: training the initial image processing model according to the training set to obtain a trained image processing model.

[0123] It is understandable that, considering that most existing methods use generated images for model training, low-quality images are assumed to be compressed only once. However, most images from the Internet are often compressed multiple times. Therefore, in order to overcome the above-mentioned defects, this embodiment processes training images by randomly generating different compression quality factors, which can effectively simulate the situation where real images are compressed multiple times during Internet transmission. The model trained with such data has stronger generalization ability and is more flexible in practical applications. The step S02 includes: randomly generating a compression quality factor; compressing the original image in the training set into a specified format according to the randomly generated compression quality factor to obtain a target training set; training the initial image processing model according to the target training set to obtain a trained image processing model, wherein the true value label of the target training set is the original image.

[0124] For ease of understanding, the following examples are given, but they are not intended to limit this application. As an example, the selection of data sets during model training is very flexible. This application can effectively remove the blur caused by quality compression of images. Therefore, the true value label of the training set is the original high-resolution image. The training image is compressed into JPEG format using the imencode() and imdecode() functions in the openCV library. The quality factor used for compression is randomly selected from the range of [5,96] by the random generation function random.randint(), thereby simulating the blurred images caused by various compression levels in real scenes. The main code is as follows:

[0125] quality_factor = random.randint(5,96) # Randomly generate random values ​​in [5,96]

[0126] img_H = cv2.cvtColor(img_H,cv2.COLOR_RGB2BGR) #Convert image img_H from RGB color space to BGR color space

[0127] result,encimg=cv2.imencode('.jpg',img_H,[int(cv2.IMWRITE_JPEG_QUALITY),quality_factor])#Encode the image into JPEG format using the specified JPEG quality factor quality_factor

[0128] img_L = cv2.imdecode(encimg, 1) #Decode the encoded image data back to the image format img_L = cv2.cvtColor(img_L, cv2.COLOR_BGR2RGB) #Convert the image img_L from BGR color space back to RGB color space

[0129] The high-quality datasets generated by DIV2K, Flickr2K and 360 Image Search were fused as the dataset, with 810,000 training sets and 200,000 test sets. During training, image patches of size 64×64 were randomly extracted, and the quality factor was randomly sampled from 5 to 96. The batch_size was set to 1024, and the Adam optimizer was used. The learning rate started from 1.6e-4, decayed by 0.5 times every 40,000 iterations, and finally ended at 5e-6. λ was set to 0.1, and the model was trained using PyTorch on 8 GPUs.

[0130] It can be understood that training the initial image processing model according to the target training set to obtain the trained image processing model includes: training the initial image processing model according to the target training set, and obtaining the prediction loss and recovery loss in the training process, wherein the prediction loss is the prediction loss of the quality factor in each batch, and the recovery loss is the loss between the restored image and the original image in each batch; adjusting the initial image processing model according to the prediction loss and the recovery loss to obtain the trained image processing model.

[0131] For ease of understanding, the following examples are provided, but are not intended to limit the present application. As an example, during the training of the ICDNet model, different small-sized patches cropped from the same image may contain only limited information, resulting in the prediction of multiple quality factors, making it impossible to accurately estimate the compression quality factor of the original image, which will lead to instability in the training process. Therefore, the L1 loss function is used to avoid excessive penalties for such outliers. The prediction loss of the quality factor in each batch (Loss QF ) is calculated as follows:

[0132]

[0133] Where N represents the batch size during training. represents the quality factor predicted for each image, Represents the true quality factor of each image.

[0134] In a batch of N training samples, the goal of the image restorer is to minimize the restored image I res and the original high-resolution image I gt The L1 loss between:

[0135]

[0136] Where: N represents the batch size during training, represents the restored image, Represents the original high-resolution image.

[0137] Finally, the loss function used in the training process is calculated as follows:

[0138] Loss all =Loss res +λLoss QF

[0139] Where λ controls the weight balance between image restoration and QF prediction, and the initial value of λ can be set to [0.1, 0.01, 0.001], etc.

[0140] Step S03: Perform a performance test on the trained image processing model according to the test set, and adjust the trained image processing model according to the performance test result to obtain a preset image processing model.

[0141] It should be understood that the performance test of the trained image processing model is performed according to the test set. The performance test of the trained image processing model is performed according to the test set by using a preset performance indicator. The preset performance indicator can be preset, and this embodiment does not limit this. Adjusting the trained image processing model according to the performance test results can be to optimize the trained image processing model according to the performance test results to obtain a preset image processing model.

[0142] This embodiment obtains multiple original images from multiple image sources in advance as data sets to train the image processing model, thereby improving the performance of the preset image processing model.

[0143] Furthermore, in order to measure the degree of distortion and perceived quality of the image, the step S03 includes: calculating the peak signal-to-noise ratio, the structural similarity index, and the peak signal-to-noise ratio sensitive to blockiness of the trained image processing model according to the test set; generating a performance test result of the trained image processing model according to the peak signal-to-noise ratio, the structural similarity index, and the peak signal-to-noise ratio sensitive to blockiness; and adjusting the trained image processing model according to the performance test result to obtain a preset image processing model.

[0144] For ease of understanding, the following examples are given, but are not intended to limit the present application. As an example, the Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM) and Blockiness-sensitive Peak Signal-to-Noise Ratio (PSNR-b) indicators are used on the test set to evaluate the quality of the restored image, which are used to measure the degree of image distortion and perceived quality, respectively.

[0145] PSNR is an objective standard for measuring the quality of reconstructed images. It is mainly used to evaluate the effect of compression algorithms and measure the quality of deblurred images compared with original images. The unit is decibel (dB). The larger the PSNR value, the smaller the difference between the reconstructed image and the original image, and the higher the image quality. PSNR is calculated based on the mean square error (MSE) of the image. The formula is as follows:

[0146]

[0147] Where MAX is the maximum possible pixel value in the image. For an 8-bit image, MAX = 255. MSE is the mean square error between the original image and the restored image. The calculation formula is:

[0148]

[0149] Where I and K represent the pixel values ​​of the original image and the reconstructed image respectively.

[0150] SSIM is an image quality assessment method that is more in line with the human visual system. SSIM takes into account the brightness, contrast, and structural information of the image. The value range is between -1 and 1. The closer the value is to 1, the more similar the two images are. The calculation formula is as follows:

[0151]

[0152] In the formula, u x and u y is the mean of images x and y, and is the variance of images x and y, σ xy is the covariance of images x and y.

[0153] PSNR-B is an improved PSNR for blockiness, which is common in compressed images, especially JPEG compression. PSNR-B takes special care of blockiness during calculation, and detects and adjusts block boundaries in the image, so that the calculated PSNR better reflects the real perceived quality of the image.

[0154] We randomly selected images from high-quality datasets generated by DIV2K, Flickr2K, and 360 Image Search to form a test set to evaluate the performance of the ICDNet model in processing JPEG compressed image restoration. We calculated PSNR, SSIM, and PSNR-B for quantitative evaluation. The quantitative results are shown in Table 1:

[0155] Table 1

[0156]

[0157]

[0158] The beneficial effects brought by this application are as follows:

[0159] 1. Enhanced flexibility and versatility: Traditional CNN-based JPEG artifact removal methods usually require training a specific model for each compression quality factor (QF), which results in a lack of versatility and flexibility in the model. The Image Compression Deblurring Network (ICDNet) proposed in this application can process JPEG images with different compression quality factors in a single model by decoupling the compression quality factor of the image from the input features. This design greatly improves the applicability and flexibility of the model and reduces the time and resource consumption of training a separate model for each QF.

[0160] 2. Controllability and user customization: This application introduces a controllable variable with clear physical meaning - the predicted quality factor, allowing users to adjust the balance between artifact removal effect and detail retention through interactive selection. This design meets the preferences of different users, so that the model can not only provide high-quality artifact removal effects, but also optimize detail retention according to the specific needs of users, improving the user experience and the practicality of the model.

[0161] 3. Enhanced generalization ability: Traditional methods mostly use generated low-quality images for model training. These images are usually assumed to be compressed only once, which fails to effectively simulate the situation where real Internet images may be compressed multiple times. In order to improve the performance of the model in practical applications, this application randomly generates training images with different compression quality factors to simulate the real situation where images are compressed multiple times during Internet transmission. This data enhancement strategy enables the model to effectively remove artifacts when facing multiple compressed images, significantly enhancing the generalization ability of the model.

[0162] 4. Improve artifact removal and restoration quality: The ICDNet of this application can effectively remove the block effect and blur artifacts generated during JPEG compression while retaining the image details during the prediction and adjustment of the image compression quality factor. Through the design of the deep feature extraction stage, the model can more accurately restore high-quality images and improve the overall visual effect of the image. This efficient artifact removal method is of great significance for improving image quality in image processing, storage and transmission.

[0163] 5. Advantages in practical applications: The ICDNet of this application not only performs well in laboratory environments, but its design concept and technical solutions also have significant advantages in practical applications. By taking into account the processing of different compression quality factors and the adjustment of user preferences, ICDNet can be applied in various practical scenarios, such as image editing, online image platforms, social media, etc., to provide stable and high-quality image artifact removal and restoration effects.

[0164] In summary, this application achieves effective processing of JPEG images with different compression quality factors through innovative technical design and solutions, meets users' diverse needs for image quality and detail retention, enhances the generalization ability of the model and its flexibility in practical applications, significantly improves the artifact removal effect and image restoration quality, and has important application value and broad prospects.

[0165] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the image processing method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0166] This application also provides an image processing device, please refer to Figure 5 , the image processing device comprises:

[0167] A model input module 10, for inputting an input image into a preset image processing model in response to an image processing request;

[0168] The model processing module 20 is used to predict the compression quality factor of the input image through the preset image processing model, and perform image restoration on the input image according to the compression quality factor to obtain an output image.

[0169] The image processing device provided by the present application adopts the image processing method in the above embodiment, which can solve the technical problem that the existing image artifact removal method needs to train different models for different compression levels for image processing, resulting in the lack of versatility and flexibility of the model. Compared with the prior art, the beneficial effects of the image processing device provided by the present application are the same as the beneficial effects of the image processing method provided by the above embodiment, and the other technical features in the image processing device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0170] The present application provides an image processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image processing method in the above-mentioned embodiment one.

[0171] Reference below Figure 6 , which shows a schematic diagram of the structure of an image processing device suitable for implementing the embodiment of the present application. The image processing device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The image processing device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0172] like Figure 6As shown, the image processing device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a ROM (Read Only Memory) 1002 or a program loaded from a storage device 1003 to a RAM (Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the image processing device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, an LCD (Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the image processing device to communicate with other devices wirelessly or by wire to exchange data. Although the image processing device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0173] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0174] The image processing device provided by the present application adopts the image processing method in the above embodiment, which can solve the technical problem that the existing image artifact removal method needs to train different models for different compression levels for image processing, resulting in the lack of versatility and flexibility of the model. Compared with the prior art, the beneficial effects of the image processing device provided by the present application are the same as the beneficial effects of the image processing method provided by the above embodiment, and the other technical features in the image processing device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0175] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0176] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0177] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the image processing method in the above-mentioned embodiment.

[0178] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory) or flash memory, optical fiber, CD-ROM (CD-Read Only Memory, portable compact disk read-only memory), optical storage device, magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0179] The computer-readable storage medium may be included in the image processing device; or may exist independently without being assembled into the image processing device.

[0180] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the image processing device, the image processing device: responds to the image processing request, inputs the web page data to be cleaned into the noise text classification model, the noise text classification model is pre-trained based on a preset data optimization training method, and the noise text is cleaned on the web page data to be cleaned through the noise text classification model to obtain the cleaned web page data.

[0181] The computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a LAN (Local Area Network) or a WAN (Wide Area Network), or it can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0182] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0183] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0184] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned image processing method, and can solve the technical problem that the existing image artifact removal method needs to train different models for different compression levels for image processing, resulting in the lack of versatility and flexibility of the model. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the image processing method provided by the above-mentioned embodiment, and will not be repeated here.

[0185] The present application also provides a computer program product, including a computer program, which implements the above-mentioned image processing method when executed by a processor.

[0186] The computer program product provided by the present application can solve the technical problem that the existing image artifact removal method needs to train different models for different compression levels of images for image processing, resulting in the lack of versatility and flexibility of the model. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the image processing method provided by the above embodiment, and will not be repeated here.

[0187] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

[0188] The present application discloses A1, an image processing method, the image processing method comprising:

[0189] In response to an image processing request, inputting an input image into a preset image processing model;

[0190] The compression quality factor of the input image is predicted by the preset image processing model, and image restoration is performed on the input image according to the compression quality factor to obtain an output image.

[0191] A2. The image processing method as described in A1, wherein the preset image processing model includes a feature extractor, a quality factor predictor, a flexible controller and an image restorer;

[0192] The method of predicting the compression quality factor of the input image by using the preset image processing model and performing image restoration on the input image according to the compression quality factor to obtain an output image includes:

[0193] Extracting image features and compression quality factor features from the input image by the feature extractor;

[0194] predicting a compression quality factor of the input image by the quality factor predictor according to the quality factor feature;

[0195] Searching, by the flexible controller, for a quality factor parameter pair corresponding to the compression quality factor in a preset mapping function;

[0196] The image restorer performs image restoration on the input image based on the image feature and the quality factor parameter to obtain an output image.

[0197] A3. The image processing method as described in A2, before extracting the image features and compression quality factor features from the input image by the feature extractor, further comprising:

[0198] Responding to an interactive adjustment request for the compression quality factor;

[0199] Adjust the compression quality factor according to the interactive request to obtain an adjusted compression quality factor;

[0200] The extracting the image feature and the compression quality factor feature from the input image by the feature extractor comprises:

[0201] The flexible controller searches for a quality factor parameter pair corresponding to the adjusted compression quality factor in a preset mapping function.

[0202] A4. The image processing method as described in A2, wherein the feature extractor includes multiple stages, each stage uses a residual block, and the residual block includes multiple convolutional layers and a nonlinear activation function;

[0203] The extracting the image feature and the compression quality factor feature from the input image by the feature extractor comprises:

[0204] Image features and compression quality factor features are extracted from the input image through the multiple convolutional layers and the nonlinear activation function, wherein the image feature extraction process includes residual learning, channel adjustment and downsampling scaling.

[0205] A5. The image processing method as described in A4, wherein the network structure used by the feature extractor corresponds to the network structure used by the image restorer, there is a jump connection between the feature extractor and the image restorer at each stage, and the image restorer includes a QF attention block;

[0206] The performing image restoration on the input image by the image restorer based on the image feature and the quality factor parameter to obtain an output image comprises:

[0207] Performing bilinear interpolation upsampling on the image features to obtain a high-level feature map;

[0208] Obtaining the output feature map of the feature extractor at each stage through the jump connection;

[0209] An output image is obtained by spatially applying an affine transformation to the feature map within the QF attention block based on the quality factor parameter, wherein the feature map includes the high-level feature map and the output feature map.

[0210] A6. In the image processing method as described in A2, the preset mapping function is obtained by mapping the compressed quality factors into a low-dimensional continuous vector space during the model training process and analyzing the semantic relationship and context information between the quality factors.

[0211] A7. In the image processing method described in A2, the quality factor predictor comprises a plurality of multilayer perceptrons, and the nodes of the hidden layers of the plurality of multilayer perceptrons have the same dimension as the compression quality factor feature.

[0212] A8. The image processing method as described in any one of A1 to A7, wherein before inputting the input image into the preset image processing model in response to the image processing request, the method further comprises:

[0213] Acquire multiple original images through multiple image sources, use the multiple original images as a data set, and divide the data set into a training set and a test set, wherein the original images are high-resolution images;

[0214] Training the initial image processing model according to the training set to obtain a trained image processing model;

[0215] A performance test is performed on the trained image processing model according to the test set, and the trained image processing model is adjusted according to the performance test result to obtain a preset image processing model.

[0216] A9. The image processing method as described in A8, wherein the initial image processing model is trained according to the training set to obtain a trained image processing model, comprising:

[0217] Randomly generate compression quality factor;

[0218] Compressing the original images in the training set into a specified format according to the randomly generated compression quality factor to obtain a target training set;

[0219] The initial image processing model is trained according to the target training set to obtain a trained image processing model, wherein the true value label of the target training set is the original image.

[0220] A10. The image processing method as described in A9, wherein the initial image processing model is trained according to the target training set to obtain a trained image processing model, comprising:

[0221] Training the initial image processing model according to the target training set, and obtaining the prediction loss and the recovery loss in the training process, wherein the prediction loss is the prediction loss of the quality factor in each batch, and the recovery loss is the loss between the restored image in each batch and the original image;

[0222] The initial image processing model is adjusted according to the prediction loss and the recovery loss to obtain a trained image processing model.

[0223] A11. The image processing method as described in A8, wherein the performance test is performed on the trained image processing model according to the test set, and the trained image processing model is adjusted according to the performance test result to obtain a preset image processing model, including:

[0224] Calculating a peak signal-to-noise ratio, a structural similarity index, and a peak signal-to-noise ratio sensitive to blockiness of the trained image processing model according to the test set;

[0225] Generating a performance test result of the trained image processing model according to the peak signal-to-noise ratio, the structural similarity index, and the peak signal-to-noise ratio sensitive to the blocking effect;

[0226] The trained image processing model is adjusted according to the performance test results to obtain a preset image processing model.

[0227] The present application also discloses B12, an image processing device, the image processing device comprising:

[0228] A model input module, used for inputting an input image into a preset image processing model in response to an image processing request;

[0229] The model processing module is used to predict the compression quality factor of the input image through the preset image processing model, and perform image restoration on the input image according to the compression quality factor to obtain an output image.

[0230] B13. The image processing device as described in B12, wherein the preset image processing model includes a feature extractor, a quality factor predictor, a flexible controller and an image restorer;

[0231] The model processing module is also used to extract image features and compression quality factor features from the input image through the feature extractor; predict the compression quality factor of the input image through the quality factor predictor according to the quality factor features; search for the quality factor parameter pair corresponding to the compression quality factor in a preset mapping function through the flexible controller; and perform image restoration on the input image based on the image features and the quality factor parameters through the image restorer to obtain an output image.

[0232] B14. The image processing device as described in B13, further comprising:

[0233] An interactive module, configured to respond to an interactive adjustment request for the compression quality factor; adjust the compression quality factor according to the interactive request to obtain an adjusted compression quality factor;

[0234] The model processing module is further used to search for the quality factor parameter pair corresponding to the adjusted compression quality factor in a preset mapping function through the flexible controller.

[0235] B15. The image processing device as described in B13, wherein the feature extractor includes multiple stages, each stage uses a residual block, and the residual block includes multiple convolutional layers and a nonlinear activation function;

[0236] The model processing module is also used to extract image features and compression quality factor features from the input image through the multiple convolutional layers and the nonlinear activation function, wherein the image feature extraction process includes residual learning, channel adjustment and downsampling scaling.

[0237] B16, the image processing device as described in B15, the network structure used by the feature extractor corresponds to the network structure used by the image restorer, there is a jump connection between the feature extractor and the image restorer at each stage, and the image restorer includes a QF attention block;

[0238] The model processing module is also used to perform bilinear interpolation upsampling on the image features to obtain a high-level feature map; obtain the output feature map of the feature extractor at each stage through the jump connection; and apply affine transformation to the feature map in the QF attention block spatially based on the quality factor parameter to obtain an output image, wherein the feature map includes the high-level feature map and the output feature map.

[0239] B17. An image processing device as described in B13, wherein the preset mapping function is obtained by mapping the compression quality factors into a low-dimensional continuous vector space during the model training process and analyzing the semantic relationship and context information between the quality factors.

[0240] The present application also discloses C18, an image processing device, which includes: a memory, a processor, and an image processing program stored in the memory and executable on the processor, wherein the image processing program implements the image processing method described above when executed by the processor.

[0241] The present application also discloses D19, a storage medium, on which an image processing program is stored, and when the image processing program is executed by a processor, the image processing method as described above is implemented.

[0242] The present application also discloses E20, a computer program product, which includes an image processing program, and when the image processing program is executed by a processor, the image processing method as described above is implemented.

Claims

1. An image processing method, characterized in that: The image processing method comprises: In response to an image processing request, inputting an input image into a preset image processing model; The compression quality factor of the input image is predicted by the preset image processing model, and image restoration is performed on the input image according to the compression quality factor to obtain an output image.

2. The image processing method according to claim 1, characterized in that: The preset image processing model includes a feature extractor, a quality factor predictor, a flexible controller and an image restorer; The method of predicting the compression quality factor of the input image by using the preset image processing model and performing image restoration on the input image according to the compression quality factor to obtain an output image includes: Extracting image features and compression quality factor features from the input image by the feature extractor; predicting a compression quality factor of the input image by the quality factor predictor according to the quality factor feature; Searching, by the flexible controller, for a quality factor parameter pair corresponding to the compression quality factor in a preset mapping function; The image restorer performs image restoration on the input image based on the image features and the quality factor parameters to obtain an output image.

3. The image processing method according to claim 2, characterized in that: Before extracting the image features and the compression quality factor features from the input image by the feature extractor, the method further includes: Responding to an interactive adjustment request for the compression quality factor; Adjust the compression quality factor according to the interactive request to obtain an adjusted compression quality factor; The extracting image features and compression quality factor features from the input image by the feature extractor comprises: The flexible controller searches for a quality factor parameter pair corresponding to the adjusted compression quality factor in a preset mapping function.

4. The image processing method according to claim 2, wherein: The feature extractor includes multiple stages, each stage uses a residual block, and the residual block includes multiple convolutional layers and non-linear activation functions; The extracting image features and compression quality factor features from the input image by the feature extractor comprises: Image features and compression quality factor features are extracted from the input image through the multiple convolutional layers and the nonlinear activation function, wherein the image feature extraction process includes residual learning, channel adjustment and downsampling scaling.

5. The image processing method according to claim 4, characterized in that: The network structure used by the feature extractor corresponds to the network structure used by the image restorer, there is a jump connection between the feature extractor and the image restorer at each stage, and the image restorer includes a QF attention block; The performing image restoration on the input image by the image restorer based on the image feature and the quality factor parameter to obtain an output image comprises: Performing bilinear interpolation upsampling on the image features to obtain a high-level feature map; Obtaining the output feature map of the feature extractor at each stage through the jump connection; Based on the quality factor parameter, an affine transformation is spatially applied to the feature map within the QF attention block to obtain an output image, wherein the feature map includes the high-level feature map and the output feature map.

6. The image processing method according to claim 2, characterized in that: The preset mapping function is obtained by mapping the compressed quality factors into a low-dimensional continuous vector space during the model training process and analyzing the semantic relationship and context information between the quality factors.

7. An image processing device, characterized in that: The image processing device comprises: A model input module, used for inputting an input image into a preset image processing model in response to an image processing request; The model processing module is used to predict the compression quality factor of the input image through the preset image processing model, and perform image restoration on the input image according to the compression quality factor to obtain an output image.

8. An image processing device, characterized in that: The image processing device comprises: a memory, a processor, and an image processing program stored in the memory and executable on the processor, wherein the image processing program implements the image processing method according to any one of claims 1 to 6 when executed by the processor.

9. A storage medium, characterized in that: The storage medium stores an image processing program, and when the image processing program is executed by the processor, the image processing method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product comprises an image processing program, and when the image processing program is executed by a processor, the image processing method according to any one of claims 1 to 6 is implemented.

Images