Image reconstruction methods, apparatus, devices and storage media

Abstract

This invention provides an image reconstruction method, apparatus, device, and storage medium, comprising: acquiring an image to be processed; inputting the image to be processed into an image reconstruction model to obtain a target reconstructed image output by the image reconstruction model; the image reconstruction model is trained based on an original image set and its corresponding compressed images, the compressed images being obtained by quantizing and compressing the original images based on several quality factor parameters; the image reconstruction model includes a meta-convolutional module, the training parameters of which are obtained through meta-learning based on the quantization coefficient table information of the compressed images and quality factor parameters. This invention, by designing an adaptive meta-convolutional module based on meta-learning and introducing quantization coefficient table information, can generate the convolution parameters of the meta-convolutional module from the quantization coefficient table information of different quality factor parameters, achieving defect repair of compressed images under any QF quality factor parameter, effectively improving the robustness and practicality of the image reconstruction model.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image reconstruction method, apparatus, device, and storage medium. Background Technology

[0002] With the development of modern information technology and network communication technology, image compression is almost a necessary step in image transmission or preview due to limitations in network bandwidth and physical storage space. The JPEG compression algorithm, due to its high compression ratio and ease of use, has become one of the most widely used image compression algorithms. The JPEG compression algorithm mainly consists of image segmentation, color space conversion, discrete cosine transform, quantization, and encoding. Among these, the quantization process is the cause of high-frequency detail loss and image artifacts during compression. Furthermore, the JPEG compression algorithm requires specifying a quality factor (QF) before compression to determine the quantization matrix. A smaller QF value results in a higher compression ratio and a smaller image size, but also more image artifacts.

[0003] Currently, deep learning-based image defect removal networks can be used to reconstruct images to address defects in JPEG format image compression. However, deep learning-based image defect removal networks are usually trained on compressed images generated with a fixed quality factor (QF), and the trained models are only applicable to defect removal on compressed images with a fixed quality factor (QF), resulting in low robustness and practicality. Summary of the Invention

[0004] This invention provides an image reconstruction method, apparatus, device, and storage medium, aiming to solve the technical problem that models trained on compressed images generated based on a fixed quality factor (QF) can only be applied to the removal of defects on compressed images with a fixed quality factor (QF), resulting in low robustness and practicality of the model application.

[0005] This invention provides an image reconstruction method, comprising:

[0006] Obtain the image to be processed;

[0007] The image to be processed is input into the image reconstruction model to obtain the target reconstructed image output by the image reconstruction model;

[0008] The image reconstruction model is trained based on the original image set and the compressed image corresponding to each original image in the original image set. The compressed image is obtained by quantizing and compressing the original image based on several quality factor parameters.

[0009] The image reconstruction model includes a meta-convolution module, whose training parameters are obtained through meta-learning based on the compressed image and the quantization coefficient table information corresponding to the quality factor parameter.

[0010] According to an image reconstruction method provided by the present invention, the image reconstruction model further includes a feature extraction module; the step of inputting the image to be processed into the image reconstruction model to obtain the target reconstructed image output by the image reconstruction model includes:

[0011] The image to be processed is input into the feature extraction module to obtain the feature extraction result output by the feature extraction module;

[0012] The feature extraction results are input into the meta-convolution module to obtain the target reconstructed image output by the meta-convolution module.

[0013] According to an image reconstruction method provided by the present invention, the image reconstruction model is trained based on the following steps:

[0014] Acquire several original images;

[0015] Based on different quality factor parameters, each of the original images is quantized and compressed to obtain the compressed image corresponding to each of the original images.

[0016] Based on each compressed image and its corresponding original image, the image reconstruction model is iteratively trained to obtain the image reconstruction model.

[0017] According to an image reconstruction method provided by the present invention, before iteratively training the reconstruction model to be trained based on the compressed image set and its corresponding original images to obtain the image reconstruction model, the method includes:

[0018] Select each target compressed image corresponding to any quality factor parameter from each of the compressed images;

[0019] Based on the target compressed images and their corresponding original images, the initial model to be trained is iteratively trained to obtain the target initial model.

[0020] The parameters of the initial feature extraction module in the target initial model are transferred to the feature extraction module in the model to be trained and reconstructed, wherein the network structure of the initial feature extraction module and the feature extraction module is the same.

[0021] According to an image reconstruction method provided by the present invention, the reconstruction model to be trained further includes a fully connected layer; the step of iteratively training the reconstruction model to be trained based on each compressed image and its corresponding original image to obtain the image reconstruction model includes:

[0022] For any compressed image, the compressed image is input into the feature extraction module to obtain the training image features output by the feature extraction module;

[0023] The coordinates of each pixel in the compressed image and the quantization coefficient table information of the quality factor parameter corresponding to the compressed image are input into the fully connected layer to obtain the convolution parameters output by the fully connected layer. The convolution parameters are used to update the training parameters of the meta-convolution module.

[0024] The training image features are input into the updated meta-convolutional module to obtain the first training reconstructed image output by the meta-convolutional module.

[0025] The first model loss value is calculated based on the original image corresponding to the first training reconstructed image and the compressed image;

[0026] Based on the first model loss value obtained in each iteration, the parameters of the reconstructed model to be trained are updated to obtain the image reconstruction model.

[0027] According to an image reconstruction method provided by the present invention, the feature extraction module includes a first convolutional layer and at least one convolutional network;

[0028] The at least one convolutional network is connected sequentially;

[0029] The output of the first convolutional layer is connected to the first convolutional network in the at least one convolutional network.

[0030] Each of the convolutional networks includes a second convolutional layer, an activation function, a third convolutional layer, and a feature fusion layer connected in sequence;

[0031] The feature fusion layer in the convolutional network is used to fuse the output signals of the first convolutional layer and the third convolutional layer.

[0032] According to an image reconstruction method provided by the present invention, the initial model to be trained further includes an image reconstruction module; the step of iteratively training the initial model to be trained based on each of the target compressed images and their corresponding original images to obtain the target initial model includes:

[0033] For any target compressed image, the target compressed image is preprocessed to obtain a training compressed image;

[0034] The training compressed image is input into the initial feature extraction module to obtain the image features output by the initial feature extraction module;

[0035] The image features are input into the image reconstruction module to obtain the second training reconstructed image output by the image reconstruction module;

[0036] The second model loss value is calculated based on the original image corresponding to the second training reconstructed image and the target compressed image;

[0037] Based on the second model loss value obtained in each iteration, the parameters of the initial model to be trained are updated to obtain the target initial model.

[0038] The present invention also provides an image reconstruction apparatus, comprising:

[0039] The acquisition module is used to acquire the image to be processed;

[0040] The reconstruction module is used to input the image to be processed into the image reconstruction model to obtain the target reconstructed image output by the image reconstruction model.

[0041] The image reconstruction model is trained based on the original image set and the compressed image corresponding to each original image in the original image set. The compressed image is obtained by quantizing and compressing the original image based on several quality factor parameters.

[0042] The image reconstruction model includes a meta-convolution module, whose training parameters are obtained through meta-learning based on the compressed image and the quantization coefficient table information corresponding to the quality factor parameter.

[0043] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image reconstruction method as described above.

[0044] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image reconstruction method as described above.

[0045] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the image reconstruction method as described above.

[0046] The image reconstruction method, apparatus, device, and storage medium provided by this invention design an adaptive meta-convolution module based on meta-learning and introduce quantization coefficient table information corresponding to quality factor parameters. This allows the convolution parameters of the meta-convolution module to be self-learned from the quantization coefficient table information corresponding to different quality factor parameters, thereby achieving the repair of defects in compressed images under any QF quality factor parameter and effectively improving the robustness and practicality of the image reconstruction model. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating the image reconstruction method provided by the present invention;

[0049] Figure 2 This is a schematic diagram of the training process of the image reconstruction model provided in an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the feature extraction module provided in an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of the image reconstruction device provided by the present invention;

[0052] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0054] The terminology used in one or more embodiments of the present invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of the invention. The singular forms “a,” “the,” and “the” used in one or more embodiments of the invention are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” used in one or more embodiments of the invention refers to and includes any or all possible combinations of one or more associated listed items.

[0055] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of the present invention, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of the present invention, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0056] The following is combined with Figures 1-3 The exemplary embodiments of the present invention will be described in detail below.

[0057] Figure 1 This is a schematic flowchart of the image reconstruction method provided by the present invention. Figure 1 As shown, the image reconstruction method includes:

[0058] Step 11, Obtain the image to be processed;

[0059] It should be noted that the image to be processed is an image compressed by the quality factor parameter QF. The smaller the value of QF, the greater the compression ratio of the image, the smaller the image size, and the more image defects are caused.

[0060] Step 12: Input the image to be processed into the image reconstruction model to obtain the target reconstructed image output by the image reconstruction model;

[0061] The image reconstruction model is trained based on the original image set and the compressed image corresponding to each original image in the original image set. The compressed image is obtained by quantizing and compressing the original image based on several quality factor parameters.

[0062] The image reconstruction model includes a meta-convolution module, whose training parameters are obtained through meta-learning based on the compressed image and the quantization coefficient table information corresponding to the quality factor parameter.

[0063] Specifically, the image to be processed is input into an image reconstruction model, and the image segmentation result is obtained based on the output of the image reconstruction model. The image reconstruction model is trained on an original image set and a compressed image corresponding to each original image in the original image set. The compressed image is obtained by quantizing and compressing the original image based on several quality factor parameters (QF). These quality factor parameters (QF) are selected from a pre-set range, for example, a range of 1 to 50. It can be understood that after training, the image reconstruction model can effectively remove defects from the compressed image to obtain the target reconstructed image corresponding to the image to be processed.

[0064] It should be noted that the image reconstruction model includes a meta-convolution module and a feature extraction module. During the training phase, the initial parameters of the feature extraction module are obtained by transferring the parameters of the initial feature extraction module in the pre-trained target initial model. The target initial model is obtained by iterative training of compressed images generated by quantization and compression of multiple original images with a fixed quality factor parameter QF, and the original images. The fixed quality factor parameter QF is a certain quality factor parameter QF selected from the range of quality factor parameters. The training parameters of the meta-convolution module are obtained by meta-learning based on the compressed image and the quantization coefficient table information corresponding to the quality factor parameter QF.

[0065] Specifically, the image to be processed is input to the feature extraction module to obtain the feature extraction result output by the feature extraction module, and then the feature extraction result is input to the meta-convolution module to obtain the target reconstructed image output by the meta-convolution module.

[0066] This invention, through the above-described scheme, namely, acquiring an image to be processed; inputting the image to be processed into an image reconstruction model to obtain a target reconstructed image output by the image reconstruction model; wherein, the image reconstruction model is trained based on an original image set and a compressed image corresponding to each original image in the original image set, and the compressed image is obtained by quantizing and compressing the original image based on several quality factor parameters; the image reconstruction model includes a meta-convolution module, the training parameters of which are obtained through meta-learning based on the compressed image and the quantization coefficient table information corresponding to the quality factor parameters, thereby realizing the design of an adaptive meta-convolution module based on meta-learning, and introducing the quantization coefficient table information corresponding to the quality factor parameters, enabling the generation of convolution parameters of the meta-convolution module from the quantization coefficient table information corresponding to different quality factor parameters, thereby realizing the repair of defects in compressed images under any QF quality factor parameter, effectively improving the robustness and practicality of the image reconstruction model.

[0067] In one embodiment, the image reconstruction model is trained based on the following steps:

[0068] A number of original images are acquired; each original image is quantized and compressed based on different quality factor parameters to obtain a compressed image corresponding to each original image; the image reconstruction model is iteratively trained based on each compressed image and its corresponding original image to obtain the image reconstruction model.

[0069] Before iteratively training the reconstruction model based on each compressed image and its corresponding original image to obtain the image reconstruction model, the process includes:

[0070] Select target compressed images corresponding to any quality factor parameter from each of the compressed images; based on each target compressed image and its corresponding original image, iteratively train the initial model to be trained to obtain the target initial model; transfer the parameters of the initial feature extraction module in the target initial model to the feature extraction module in the reconstructed model to be trained, wherein the network structure of the initial feature extraction module and the feature extraction module is the same.

[0071] Specifically, several original images are acquired, and then different quality factor parameters QF are selected within a pre-set quality factor parameter range. Each of the original images is quantized and compressed according to the different quality factor parameters QF to obtain each compressed image. Preferably, different quality factor parameters are selected evenly within the quality factor parameter range so that the number of compressed images corresponding to each quality factor parameter is evenly distributed.

[0072] Furthermore, such as Figure 2 As shown, Figure 2 This is a schematic diagram of the training process of the image reconstruction model provided in this embodiment of the invention. In each of the compressed images, a compressed image corresponding to any quality factor parameter QF is selected as the target compressed image. For example, a compressed image with a quality factor parameter QF of 10 is selected. Then, for any target compressed image, the target compressed image is input into the initial model to be trained, resulting in a training reconstructed image output by the initial model. Further, based on the training reconstructed image and the original image corresponding to the target compressed image, the loss value of the initial model to be trained is calculated. After calculating the loss value, the model parameters in the initial model to be trained are updated using the backpropagation algorithm. This training process ends, and then the next training is performed. During the training process, it is determined whether the updated initial model to be trained meets the preset training termination conditions. If it does, the updated initial model to be trained is used as the target initial model; otherwise, the model training continues. The preset training termination conditions include loss convergence and reaching the maximum iteration threshold, etc.

[0073] Furthermore, after training the initial target model, the model parameters of the initial target model are saved. Then, the parameters of the initial feature extraction module in the initial target model are transferred to the feature extraction module in the model to be trained for reconstruction. The initial feature extraction module and the feature extraction module have the same network structure. After feature transfer, for any compressed image, the compressed image is input to the feature extraction module in the model to be trained for reconstruction to obtain the training image features output by the feature extraction module. The coordinates of each pixel in the compressed image and the quantization coefficient table information of the quality factor parameter corresponding to the compressed image during the compression process are fully connected through multiple connection layers in the model to be trained for reconstruction to generate convolution parameters. The parameters of the convolutional layers in the meta-convolution module are updated using the convolution parameters. Then, the training image features are input to the meta-convolution module after parameter update to obtain the training reconstructed image output by the meta-convolution module. Finally, based on the training reconstructed image and the original image corresponding to the compressed image, the model to be trained for reconstruction is iteratively optimized to obtain the image reconstruction model.

[0074] The embodiments of the present invention, through the above-described scheme, achieve the transfer of parameters from the original feature extraction module in the target initial model with a fixed quality factor parameter QF to the feature extraction module in the model to be reconstructed. Then, the feature extraction module with transferred parameters is jointly trained with the meta-convolution module to obtain a complete adaptive image reconstruction model. This enables the image reconstruction model to repair defects in compressed images generated with any quality factor parameter QF, effectively improving the robustness and practicality of the image reconstruction model.

[0075] In one embodiment, the step of iteratively training the reconstruction model to be trained based on the compressed image set and its corresponding original images to obtain the image reconstruction model includes:

[0076] For any compressed image, the compressed image is input to the feature extraction module to obtain the training image features output by the feature extraction module; the coordinates of each pixel in the compressed image and the quantization coefficient table information of the quality factor parameter corresponding to the compressed image are input to the fully connected layer to obtain the convolution parameters output by the fully connected layer, wherein the convolution parameters are used to update the training parameters of the meta-convolution module; the training image features are input to the parameter-updated meta-convolution module to obtain the first training reconstructed image output by the meta-convolution module; based on the first training reconstructed image and the original image corresponding to the compressed image, a first model loss value is calculated; based on the first model loss value obtained in each iteration, the parameters of the reconstructed model to be trained are updated to obtain the image reconstruction model.

[0077] It should be noted that during the image compression process, each quality factor parameter (QF) has its corresponding quantization coefficient table information. In this embodiment, the QF quantization coefficient table information for each compressed image can be represented as follows: Among them, (x k y k ) represents the coordinates of the k-th pixel in the compressed image, and q represents (x k y k The pixel coordinates correspond to the quantization coefficients in the quantization coefficient table information. H represents the height of the compressed image, and W represents the width of the compressed image.

[0078] It should be further explained that, Figure 3 This is a schematic diagram of the feature extraction module provided in an embodiment of the present invention, as shown below. Figure 3 The feature extraction module includes a first convolutional layer and at least one convolutional network; the at least one convolutional network is connected in sequence; the output of the first convolutional layer is connected to the first convolutional network in the at least one convolutional network; each convolutional network includes a second convolutional layer, an activation function, a third convolutional layer, and a feature fusion layer connected in sequence; the feature fusion layer in the convolutional network is used to fuse the output signal of the first convolutional layer and the output signal of the third convolutional layer in the convolutional network. The convolutional kernel of the first convolutional layer is 3×3, the convolutional kernel of the second convolutional layer is 1×1, and the convolutional kernel of the third convolutional layer is 3×3.

[0079] Specifically, for any compressed image, the compressed image is input into the first convolutional layer of the feature extraction module to obtain the first convolutional feature output by the first convolutional layer. Then, the first convolutional feature is input into the convolutional network to perform convolution processing on the first convolutional feature using the second convolutional layer in the convolutional network to obtain the second convolutional feature output by the second convolutional layer. Then, the second convolutional feature is input into the third convolutional layer to obtain the third convolutional feature output by the third convolutional layer. Then, the third convolutional feature and the first convolutional feature are fused through the feature fusion layer to obtain the fused feature. The fused feature is used as the input of the second convolutional layer in the next convolutional network until the output of the feature fusion layer in the last convolutional network is obtained, thereby obtaining the training image features corresponding to the feature extraction module.

[0080] Additionally, the meta-convolution module includes fully connected layers and convolutional layers. During the model training phase, the training parameters of the convolutional layers in the meta-convolution module are obtained through meta-learning based on the compressed image and the quantization coefficient table information corresponding to the quality factor parameter. Specifically, the coordinates of each pixel in the compressed image and the quantization coefficient table information of the quality factor parameter corresponding to the compressed image are input to the fully connected layer. The quantization coefficient table information contains the quantization coefficients corresponding to each pixel coordinate, thereby obtaining the meta-convolution parameters output by the fully connected layer. The meta-convolution parameters are then used to update the training parameters of the convolutional layers in the meta-convolution model. Furthermore, the training image features output by the feature extraction module are convolved based on the parameter-updated convolutional layers to obtain the first training reconstructed image.

[0081] Furthermore, based on the original image corresponding to the first training reconstructed image and the compressed image, a first model loss value is calculated, wherein the formula for calculating the first model loss value is as follows:

[0082]

[0083]

[0084] Where Q(Ic, θ) represents the reconstructed model to be trained, Ic represents the compressed image input to the reconstructed model to be trained, θ represents the model parameters of the reconstructed model to be trained, I represents the original image corresponding to the compressed image, L1(Q(Ic, θ), I) represents the L1 loss function for calculating the first model loss value, w and h represent the width and height of the compressed image, respectively, u and v represent the horizontal and vertical coordinates of each pixel in the compressed image, respectively, and I′ k I represents the first training reconstructed image. k Represents the original image, ||I′ k (u,v)-I k (u,v)‖1 represents the calculation of the first model loss value corresponding to the first reconstructed image and the original image. In other embodiments, the loss function can be set according to actual needs, and no specific limitation is made here. After calculating and obtaining the first model loss value, the current training process ends. Then, the model parameters in the reconstructed model to be trained are updated using the error backpropagation algorithm, and then the next training is performed until the reconstructed model to be trained meets the preset training termination condition, thus obtaining the image reconstruction model.

[0085] This invention, through the above-described scheme, achieves the self-learning of the convolution parameters of the meta-convolution module by introducing the quantization coefficient table information corresponding to the quality factor parameters, thereby realizing the defect repair of compressed images generated by arbitrary quality factor parameters QF, effectively improving the robustness and practicality of the image reconstruction model.

[0086] In one embodiment, the initial model to be trained further includes an image reconstruction module; the step of iteratively training the initial model to be trained based on each of the target compressed images and their corresponding original images to obtain the target initial model includes:

[0087] For any target compressed image, the target compressed image is preprocessed to obtain a training compressed image; the training compressed image is input to the initial feature extraction module to obtain image features output by the initial feature extraction module; the image features are input to the image reconstruction module to obtain a second training reconstructed image output by the image reconstruction module; based on the second training reconstructed image and the original image corresponding to the target compressed image, a second model loss value is calculated; based on the second model loss value obtained in each iteration, the parameters of the initial model to be trained are updated to obtain the target initial model.

[0088] Specifically, the target compressed image is preprocessed to obtain a training compressed image. This preprocessing includes image cropping, flipping, and other processing methods. Preferably, the target compressed image is cropped to 128×128 pixels. The preprocessed target compressed image is then input into the initial feature extraction module. It should be noted that since the initial feature extraction module has the same network structure as the feature extraction module, its extraction process is essentially the same as the feature extraction process described above, and will not be repeated here. This yields the image features output by the initial feature extraction module. Further, these image features are input into the image reprocessing module. The image reconstruction module includes a fourth convolutional layer to obtain a second training reconstructed image output by the image reconstruction module. Then, based on the second training reconstructed image and the original image corresponding to the target compressed image, a second model loss value is calculated. The calculation process of the second model loss value is basically the same as the calculation process of the first model loss value, and will not be repeated here. After the second model loss value is calculated, the current training process ends. Then, the model parameters in the initial model to be trained are updated using the error backpropagation algorithm, and then the next training is performed until the initial model to be trained meets the preset training termination condition, and the target initial model is obtained.

[0089] The present invention, through the above scheme, enables the model training of the target compressed image obtained by compression with a fixed quality factor parameter, thereby obtaining the target initial model. This lays the foundation for transferring the parameters of the initial feature extraction module in the target initial model, thereby improving the generalization of the image reconstruction model by reusing the parameters of the mature and efficient initial feature extraction module.

[0090] The image reconstruction apparatus provided by the present invention will be described below. The image reconstruction apparatus described below and the image reconstruction method described above can be referred to in correspondence.

[0091] Figure 4 This is a schematic diagram of the image reconstruction device provided by the present invention, as shown below. Figure 4 As shown, an image reconstruction apparatus according to an embodiment of the present invention includes:

[0092] Acquisition module 41 is used to acquire the image to be processed;

[0093] Reconstruction module 42 is used to input the image to be processed into the image reconstruction model to obtain the target reconstructed image output by the image reconstruction model;

[0094] The image reconstruction model is trained based on the original image set and the compressed image corresponding to each original image in the original image set. The compressed image is obtained by quantizing and compressing the original image based on several quality factor parameters.

[0095] The image reconstruction model includes a meta-convolution module, whose training parameters are obtained through meta-learning based on the compressed image and the quantization coefficient table information corresponding to the quality factor parameter.

[0096] The image reconstruction apparatus further includes:

[0097] The image reconstruction model also includes a feature extraction module.

[0098] The reconstruction module 42 is also used for:

[0099] The image to be processed is input into the feature extraction module to obtain the feature extraction result output by the feature extraction module;

[0100] The feature extraction results are input into the meta-convolution module to obtain the target reconstructed image output by the meta-convolution module.

[0101] The image reconstruction apparatus further includes:

[0102] Acquire several original images;

[0103] Based on different quality factor parameters, each of the original images is quantized and compressed to obtain the compressed image corresponding to each of the original images.

[0104] Based on each compressed image and its corresponding original image, the image reconstruction model is iteratively trained to obtain the image reconstruction model.

[0105] The image reconstruction apparatus further includes:

[0106] Select each target compressed image corresponding to any quality factor parameter from each of the compressed images;

[0107] Based on the target compressed images and their corresponding original images, the initial model to be trained is iteratively trained to obtain the target initial model.

[0108] The parameters of the initial feature extraction module in the target initial model are transferred to the feature extraction module in the model to be trained and reconstructed, wherein the network structure of the initial feature extraction module and the feature extraction module is the same.

[0109] The image reconstruction apparatus further includes:

[0110] The reconstructed model to be trained also includes a fully connected layer;

[0111] For any compressed image, the compressed image is input into the feature extraction module to obtain the training image features output by the feature extraction module;

[0112] The coordinates of each pixel in the compressed image and the quantization coefficient table information of the quality factor parameter corresponding to the compressed image are input into the fully connected layer to obtain the convolution parameters output by the fully connected layer. The convolution parameters are used to update the training parameters of the meta-convolution module.

[0113] The training image features are input into the updated meta-convolutional module to obtain the first training reconstructed image output by the meta-convolutional module.

[0114] The first model loss value is calculated based on the original image corresponding to the first training reconstructed image and the compressed image;

[0115] Based on the first model loss value obtained in each iteration, the parameters of the reconstructed model to be trained are updated to obtain the image reconstruction model.

[0116] The image reconstruction apparatus further includes:

[0117] The feature extraction module includes a first convolutional layer and at least one convolutional network;

[0118] The at least one convolutional network is connected sequentially;

[0119] The output of the first convolutional layer is connected to the first convolutional network in the at least one convolutional network.

[0120] Each of the convolutional networks includes a second convolutional layer, an activation function, a third convolutional layer, and a feature fusion layer connected in sequence;

[0121] The feature fusion layer in the convolutional network is used to fuse the output signals of the first convolutional layer and the third convolutional layer.

[0122] The image reconstruction apparatus further includes:

[0123] The initial model to be trained also includes an image reconstruction module;

[0124] For any target compressed image, the target compressed image is preprocessed to obtain a training compressed image;

[0125] The training compressed image is input into the initial feature extraction module to obtain the image features output by the initial feature extraction module;

[0126] The image features are input into the image reconstruction module to obtain the second training reconstructed image output by the image reconstruction module;

[0127] The second model loss value is calculated based on the original image corresponding to the second training reconstructed image and the target compressed image;

[0128] Based on the second model loss value obtained in each iteration, the parameters of the initial model to be trained are updated to obtain the target initial model.

[0129] It should be noted that the apparatus provided in this embodiment of the invention can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Therefore, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail here.

[0130] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 5As shown, the electronic device may include a processor 610, a memory 520, a communication interface 530, and a communication bus 540. The processor 510, memory 520, and communication interface 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 520 to execute an image reconstruction method. This method includes: acquiring an image to be processed; inputting the image to be processed into an image reconstruction model to obtain a target reconstructed image output by the image reconstruction model; wherein the image reconstruction model is trained based on an original image set and compressed images corresponding to each original image in the original image set, and the compressed images are obtained by quantizing and compressing the original images based on several quality factor parameters; the image reconstruction model includes a meta-convolution module, and the training parameters of the meta-convolution module are obtained through meta-learning based on the compressed images and the quantization coefficient table information corresponding to the quality factor parameters.

[0131] Furthermore, the logical instructions in the aforementioned memory 520 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0132] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the image reconstruction method provided by the above methods. The method includes: acquiring an image to be processed; inputting the image to be processed into an image reconstruction model to obtain a target reconstructed image output by the image reconstruction model; wherein the image reconstruction model is trained based on an original image set and a compressed image corresponding to each original image in the original image set, and the compressed image is obtained by quantizing and compressing the original image based on several quality factor parameters; the image reconstruction model includes a meta-convolutional module, and the training parameters of the meta-convolutional module are obtained through meta-learning based on the compressed image and the quantization coefficient table information corresponding to the quality factor parameters.

[0133] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image reconstruction method provided by the above methods. The method includes: acquiring an image to be processed; inputting the image to be processed into an image reconstruction model to obtain a target reconstructed image output by the image reconstruction model; wherein the image reconstruction model is trained based on an original image set and a compressed image corresponding to each original image in the original image set, and the compressed image is obtained by quantizing and compressing the original image based on several quality factor parameters; the image reconstruction model includes a meta-convolution module, and the training parameters of the meta-convolution module are obtained by meta-learning based on the compressed image and the quantization coefficient table information corresponding to the quality factor parameters.

[0134] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0135] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image reconstruction method, characterized in that, include: Obtain the image to be processed; The image to be processed is input into the image reconstruction model to obtain the target reconstructed image output by the image reconstruction model; The image reconstruction model is trained based on the original image set and the compressed image corresponding to each original image in the original image set. The compressed image is obtained by quantizing and compressing the original image based on several quality factor parameters. The image reconstruction model includes a meta-convolution module, and the training parameters of the meta-convolution module are obtained by meta-learning based on the compressed image and the quantization coefficient table information corresponding to the quality factor parameter. The image reconstruction model further includes a feature extraction module; the step of inputting the image to be processed into the image reconstruction model to obtain the target reconstructed image output by the image reconstruction model includes: The image to be processed is input into the feature extraction module to obtain the feature extraction result output by the feature extraction module; The feature extraction results are input into the meta-convolution module to obtain the target reconstructed image output by the meta-convolution module; The image reconstruction model is trained based on the following steps: Acquire several original images; Based on different quality factor parameters, each of the original images is quantized and compressed to obtain the compressed image corresponding to each of the original images. Based on each compressed image and its corresponding original image, the image reconstruction model to be trained is iteratively trained to obtain the image reconstruction model. The reconstructed model to be trained further includes a fully connected layer; the iterative training of the reconstructed model to be trained based on each compressed image and its corresponding original image to obtain the image reconstruction model includes: For any compressed image, the compressed image is input into the feature extraction module to obtain the training image features output by the feature extraction module; The coordinates of each pixel in the compressed image and the quantization coefficient table information of the quality factor parameter corresponding to the compressed image are input into the fully connected layer to obtain the convolution parameters output by the fully connected layer. The convolution parameters are used to update the training parameters of the meta-convolution module. The training image features are input into the updated meta-convolutional module to obtain the first training reconstructed image output by the meta-convolutional module. The first model loss value is calculated based on the original image corresponding to the first training reconstructed image and the compressed image; Based on the first model loss value obtained in each iteration, the parameters of the reconstructed model to be trained are updated to obtain the image reconstruction model.

2. The image reconstruction method according to claim 1, characterized in that, Before iteratively training the reconstruction model based on the compressed image set and its corresponding original images to obtain the image reconstruction model, the process includes: Select each target compressed image corresponding to any quality factor parameter from each of the compressed images; Based on the target compressed images and their corresponding original images, the initial model to be trained is iteratively trained to obtain the target initial model. The parameters of the initial feature extraction module in the target initial model are transferred to the feature extraction module in the model to be trained and reconstructed, wherein the network structure of the initial feature extraction module and the feature extraction module is the same.

3. The image reconstruction method according to claim 1, characterized in that, The feature extraction module includes a first convolutional layer and at least one convolutional network; The at least one convolutional network is connected sequentially; The output of the first convolutional layer is connected to the first convolutional network in the at least one convolutional network. Each of the convolutional networks includes a second convolutional layer, an activation function, a third convolutional layer, and a feature fusion layer connected in sequence; The feature fusion layer in the convolutional network is used to fuse the output signals of the first convolutional layer and the third convolutional layer.

4. The image reconstruction method according to claim 2, characterized in that, The initial model to be trained further includes an image reconstruction module; the iterative training of the initial model to be trained based on each of the target compressed images and their corresponding original images to obtain the target initial model includes: For any target compressed image, the target compressed image is preprocessed to obtain a training compressed image; The training compressed image is input into the initial feature extraction module to obtain the image features output by the initial feature extraction module; The image features are input into the image reconstruction module to obtain the second training reconstructed image output by the image reconstruction module; The second model loss value is calculated based on the original image corresponding to the second training reconstructed image and the target compressed image; Based on the second model loss value obtained in each iteration, the parameters of the initial model to be trained are updated to obtain the target initial model.

5. An image reconstruction apparatus, characterized in that, include: The acquisition module is used to acquire the image to be processed; The reconstruction module is used to input the image to be processed into the image reconstruction model to obtain the target reconstructed image output by the image reconstruction model. The image reconstruction model is trained based on the original image set and the compressed image corresponding to each original image in the original image set. The compressed image is obtained by quantizing and compressing the original image based on several quality factor parameters. The image reconstruction model includes a meta-convolution module, and the training parameters of the meta-convolution module are obtained by meta-learning based on the compressed image and the quantization coefficient table information corresponding to the quality factor parameter. The image reconstruction model further includes a feature extraction module; the reconstruction module is also used for: The image to be processed is input into the feature extraction module to obtain the feature extraction result output by the feature extraction module; The feature extraction results are input into the meta-convolution module to obtain the target reconstructed image output by the meta-convolution module; The image reconstruction apparatus further includes: Acquire several original images; Based on different quality factor parameters, each of the original images is quantized and compressed to obtain the compressed image corresponding to each of the original images. Based on each compressed image and its corresponding original image, the image reconstruction model to be trained is iteratively trained to obtain the image reconstruction model. The reconstructed model to be trained further includes a fully connected layer; the image reconstruction device further includes: For any compressed image, the compressed image is input into the feature extraction module to obtain the training image features output by the feature extraction module; The coordinates of each pixel in the compressed image and the quantization coefficient table information of the quality factor parameter corresponding to the compressed image are input into the fully connected layer to obtain the convolution parameters output by the fully connected layer. The convolution parameters are used to update the training parameters of the meta-convolution module. The training image features are input into the updated meta-convolutional module to obtain the first training reconstructed image output by the meta-convolutional module. The first model loss value is calculated based on the original image corresponding to the first training reconstructed image and the compressed image; Based on the first model loss value obtained in each iteration, the parameters of the reconstructed model to be trained are updated to obtain the image reconstruction model.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the image reconstruction method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the image reconstruction method as described in any one of claims 1 to 4.

Images