Method and device for restoring compressed image, electronic equipment and computer program product

Abstract

The invention discloses a method and a device for restoring a compressed image, electronic equipment and a computer program product. Relates to the audio and video field, and the method comprises the following steps: obtaining a target compressed image, decoding the target compressed image according to a preset decoding mode to obtain an initial restored image, the preset decoding mode being a decoding mode corresponding to a compression mode of the target compressed image; the initial restored image is input into a target model, a target restored image is obtained, and the target model is composed of N shallow feature extraction modules, M deep feature extraction modules and an up-sampling module. Through the method and the device, the problem of high computing resource consumption in a compressed image restoration process in the related art is solved.

Description

Technical Field

[0001] This application relates to the field of audio and video, and more specifically, to a method, apparatus, electronic device, and computer program product for restoring compressed images. Background Technology

[0002] With the rapid development of technology, high-resolution images and videos have become commonplace in daily life and work. High-resolution image files are large in size and occupy a lot of storage space, which undoubtedly increases the cost and power consumption of storage devices. This problem is particularly prominent on mobile terminals with limited storage resources, such as smartphones and tablets.

[0003] On mobile devices, the limitations of storage space and the efficiency of network transmission are even more pressing issues. The portability and ubiquity of mobile devices such as smartphones and tablets make them the primary tools for acquiring and sharing image data. However, these devices have relatively fixed storage capacities, and network conditions can be unstable. This necessitates that image processing technologies not only compress file sizes but also ensure that image quality and transmission efficiency are not affected. While existing image compression technologies have addressed storage and transmission issues to some extent, they often involve trade-offs between compression ratio and image restoration quality. This is especially true when processing high-resolution images, where compressed image quality can significantly degrade, impacting user experience. Furthermore, compression processes consume substantial computational resources.

[0004] There is currently no effective solution to the problem of high computational resource consumption in the process of image compression restoration in related technologies. Summary of the Invention

[0005] The main objective of this application is to provide a method, apparatus, electronic device, and computer program product for restoring compressed images, so as to solve the problem of high computational resource consumption in the process of restoring compressed images in related technologies.

[0006] To achieve the above objectives, according to one aspect of this application, a method for restoring a compressed image is provided. The method includes: acquiring a target compressed image; decoding the target compressed image according to a preset decoding method to obtain an initial restored image, wherein the preset decoding method is the decoding method corresponding to the compression method of the target compressed image; and inputting the initial restored image into a target model to obtain a target restored image, wherein the target model consists of N shallow feature extraction modules, M deep feature extraction modules, and an upsampling module.

[0007] Optionally, the shallow feature extraction module includes: a first convolutional layer with P channels, used to extract features from the initial reconstructed image and output a feature map with P channels; a first instance normalization layer, used to receive the feature map with P channels, normalize the feature map with P channels, and output a normalized feature map; and a first activation layer, used to receive the normalized feature map, process it through a preset activation function, and obtain a shallow feature map.

[0008] Optionally, the deep feature extraction module includes: a second convolutional layer for performing convolution operations on the shallow feature map to obtain an output feature map; a second activation layer for processing the output feature map to obtain an activated feature map; a depthwise separable convolutional layer for performing depthwise convolution on the activated feature map to extract features, obtaining multiple depthwise convolutional feature maps, and performing pointwise convolution operations on the multiple depthwise convolutional feature maps to obtain a fused feature map; a third activation layer for performing nonlinear transformation on the fused feature map to obtain a transformed feature map; and a normalization layer for performing normalization operations on the transformed feature map to obtain a target feature map.

[0009] Optionally, the upsampling module includes: a transposed convolutional layer for upsampling the target feature map to obtain a feature map with increased image resolution; a second instance normalization layer for normalizing the feature map with increased image resolution to obtain a normalized upsampled feature map; and a fourth activation layer for performing a nonlinear transformation on the normalized upsampled feature map to obtain the target restored image.

[0010] Optionally, the target model is trained as follows: an image sample set is obtained; for each image sample in the image sample set, the image sample is blurred and downsampled to obtain an input image; the input image is input into the target model to obtain an output image; the output image and the image sample are input into a preset loss function; the value of the preset loss function is minimized by the stochastic gradient descent algorithm to obtain the target model parameters corresponding to the minimum loss value; and the neural network model corresponding to the target model parameters is determined as the target model.

[0011] Optionally, obtaining the target compressed image includes: obtaining the original image; compressing the original image according to a preset compression method to obtain a compressed image; performing downsampling processing on the compressed image to obtain a compressed image with reduced image resolution; and encoding the compressed image into a string format to obtain the target compressed image.

[0012] Optionally, after obtaining the target reconstructed image, the method further includes: determining the image resolution of the target reconstructed image, and determining whether the image resolution is greater than or equal to a resolution threshold; if the image resolution is greater than or equal to the resolution threshold, determining that the target compressed image reconstruction is complete; if the image resolution is less than the resolution threshold, repeating the step of inputting the initial reconstructed image into the target model until the image resolution is greater than or equal to the resolution threshold.

[0013] To achieve the above objectives, according to another aspect of this application, a device for restoring a compressed image is provided. The device includes: an acquisition unit for acquiring a target compressed image and decoding the target compressed image according to a preset decoding method to obtain an initial restored image, wherein the preset decoding method is the decoding method corresponding to the compression method of the target compressed image; and an input unit for inputting the initial restored image into a target model to obtain a target restored image, wherein the target model consists of N shallow feature extraction modules, M deep feature extraction modules, and an upsampling module.

[0014] In this embodiment, a target compressed image is acquired and decoded according to a preset decoding method to obtain an initial restored image. The preset decoding method is the decoding method corresponding to the compression method of the target compressed image. The initial restored image is input into a target model to obtain the target restored image. The target model consists of N shallow feature extraction modules, M deep feature extraction modules, and an upsampling module. By constructing a lightweight target model composed of shallow feature extraction modules, deep feature extraction modules, and upsampling modules, the image resolution is improved while maintaining the image quality. This achieves the goal of reducing model parameters during the image restoration process, thereby reducing the computational resource consumption during the compressed image restoration process and solving the technical problem of high computational resource consumption during the compressed image restoration process. Attached Figure Description

[0015] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0016] Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a method for restoring compressed images is shown.

[0017] Figure 2 This is a flowchart of a method for restoring a compressed image according to an embodiment of this application;

[0018] Figure 3 This is a schematic diagram of the structure of the target model provided in the embodiments of this application;

[0019] Figure 4 This is a schematic diagram of the shallow feature extraction module provided in the embodiments of this application;

[0020] Figure 5 This is a schematic diagram of the deep feature extraction module provided in the embodiments of this application;

[0021] Figure 6 This is a schematic diagram of the upsampling module provided according to an embodiment of this application;

[0022] Figure 7 This is a schematic diagram of the target model training process provided according to the embodiments of this application;

[0023] Figure 8 This is a schematic diagram of a compressed image restoration system provided according to an embodiment of this application;

[0024] Figure 9 This is a schematic diagram of a compressed image restoration apparatus provided according to an embodiment of this application;

[0025] Figure 10 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] It should be noted that the information collected in this application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding access points are provided for users to choose to authorize or refuse. For example, interfaces are set up between this system and relevant users or organizations, providing users with corresponding access points to choose to agree to or refuse automated decision-making results; if the user chooses to refuse, the process proceeds to the expert decision-making stage.

[0029] Example 1

[0030] According to an embodiment of this application, a method embodiment for restoring a compressed image is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a method for restoring compressed images is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, processing devices such as MCU (Microcontroller Unit) or FPGA (Field-Programmable Gate Array), memory 104 for storing data, and transmission device 106 for communication functions. In addition, it may also include: a display, input / output interfaces (I / O interfaces), a USB (Universal Serial Bus) port (which may be included as one of the ports of a BUS (Business Bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0032] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0033] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the compressed image restoration method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the compressed image restoration method described above. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0034] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0035] The display may be, for example, a touchscreen LCD display that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0036] Under the above operating environment, this application provides a method for restoring compressed images. Figure 2 This is a flowchart of a method for restoring compressed images according to embodiments of this application, such as... Figure 2 As shown, the method includes:

[0037] Step S201: Obtain the target compressed image, decode the target compressed image according to the preset decoding method, and obtain the initial restored image. The preset decoding method is the decoding method corresponding to the compression method of the target compressed image.

[0038] In step S201, the target compressed image is obtained from a storage medium or network transmission. This image may have been compressed using formats such as JPEG to save storage space and optimize transmission efficiency. A preset decoding method matching the compression method is used to restore the compressed image. For example, if the image is compressed using the JPEG standard, then the corresponding JPEG decoder is used to decode it, restoring the image's RGB color information and spatial resolution. The decoder reads the binary stream of the compressed image bit by bit, parsing the image's size, color depth, and pixel value information, gradually reconstructing the original image data. After decoding, the resulting initial restored image may differ from the original image in resolution and detail, but it restores the image's basic structure and color information, providing a foundational image for subsequent deep learning super-resolution processing.

[0039] Step S202: Input the initial restored image into the target model to obtain the target restored image. The target model consists of N shallow feature extraction modules, M deep feature extraction modules, and an upsampling module.

[0040] In step S202, the target model can be an image super-resolution model. Figure 3 This is a schematic diagram of the structure of the target model provided in the embodiments of this application, such as... Figure 3 As shown, the system consists of one shallow feature extraction module 100, five deep feature extraction modules 200, and one upsampling module 300. The shallow feature extraction modules first process the initial reconstructed image, extracting basic features such as edges and textures. The deep feature extraction modules further mine deeper features, such as object contours and complex textures, capturing the image's inherent structure and complex patterns. The upsampling module is responsible for upscaling the extracted feature maps to the target resolution, converting low-resolution feature maps into high-resolution images through transpose convolution and other methods.

[0041] The decoded initial reconstructed image is input into the target model. The shallow and deep feature extraction modules in the model process the image sequentially, extracting and fusing features at multiple levels. Finally, the upsampling module enlarges the processed feature map, restoring image details and generating the target reconstructed image.

[0042] The compressed image restoration method provided in this application obtains a target compressed image, decodes it according to a preset decoding method to obtain an initial restored image, wherein the preset decoding method is the decoding method corresponding to the compression method of the target compressed image; the initial restored image is input into a target model to obtain the target restored image, wherein the target model consists of N shallow feature extraction modules, M deep feature extraction modules, and an upsampling module, solving the problem of high computational resource consumption in the compressed image restoration process in related technologies. By constructing a lightweight target model composed of shallow feature extraction modules, deep feature extraction modules, and upsampling modules, image resolution is improved while maintaining image quality, achieving the goal of reducing model parameters in the image restoration process, thereby achieving the technical effect of reducing computational resource consumption in the compressed image restoration process, and thus solving the technical problem of high computational resource consumption in the compressed image restoration process.

[0043] Optionally, in the compressed image restoration method provided in this application embodiment, the shallow feature extraction module includes: a first convolutional layer with P channels, used to extract features of the initial restored image and output a feature map with P channels; a first instance normalization layer, used to receive the feature map with P channels, normalize the feature map with P channels, and output a normalized feature map; and a first activation layer, used to receive the normalized feature map, process it through a preset activation function, and obtain a shallow feature map.

[0044] For example, Figure 4 This is a schematic diagram of the shallow feature extraction module provided in the embodiments of this application, as shown below. Figure 4 As shown, the shallow feature extraction module consists of a convolutional kernel size of 3. 3. The first convolutional layer 101 is composed of a stride of 1, h k This represents the input feature map of the convolutional layer, and the convolutional kernel W of the convolutional layer. k The bias term is b k The feature map with P output channels of the first convolutional layer 101 is represented by the following formula:

[0045] ;

[0046] in This is a convolution operation. Different feature maps are obtained depending on the number of output channels of the convolutional layer. For example, if P is 32, and the input image is of size N×M×3, after convolution, the output feature map size is N×M×32. To accelerate the training of deep neural networks and improve network performance, a first instance normalization layer (IN) of size 102 is added after the first convolutional layer 101. The feature map x of the input instance normalization layer has a size of H×W×C, where C is the number of channels. The normalized feature map is then represented by the following formula:

[0047] ;

[0048] in and For trainable variables, and This calculates the mean and standard deviation of each individual channel in each sample:

[0049] ;

[0050] ;

[0051] in This is the minimum value. A 10³ ReLU (Rectified Linear Unit) is chosen as the activation function for this module. The normalized result, after being calculated using the activation function, is used as the output of the entire module.

[0052] ;

[0053] Here, f(x) is the shallow feature map, and x represents the initial reconstructed image.

[0054] This embodiment extracts low-level features from the initial reconstructed image using a shallow feature extraction module, while ensuring the stability and diversity of these features through normalization and nonlinear transformation. These processed shallow feature maps then become the input for subsequent deep feature extraction and upsampling modules, enabling the learning of more complex features and the enhancement of image resolution.

[0055] Optionally, in the compressed image restoration method provided in this application embodiment, the deep feature extraction module includes: a second convolutional layer for performing convolution operations on the shallow feature map to obtain an output feature map; a second activation layer for processing the output feature map to obtain an activated feature map; a depthwise separable convolutional layer for performing depthwise convolution on the activated feature map to extract features, obtaining multiple depthwise convolutional feature maps, and performing pointwise convolution operations on the multiple depthwise convolutional feature maps to obtain a fused feature map; a third activation layer for performing nonlinear transformation on the fused feature map to obtain a transformed feature map; and a normalization layer for performing normalization operations on the transformed feature map to obtain a target feature map.

[0056] For example, Figure 5 This is a schematic diagram of the deep feature extraction module provided in the embodiments of this application, such as... Figure 5 As shown, the deep feature extraction module uses a combination of standard convolution and depthwise separable convolution structures. Compared with traditional convolution, this significantly reduces the number of model parameters and improves the model's computation speed while ensuring feature extraction performance. The output of the deep feature extraction module 200 is expressed by the following formula:

[0057] ;

[0058] Where x represents the shallow feature map and y represents the target feature map. The module consists of second convolutional layers 201 and 205, a depthwise separable convolutional layer 203, a second activation layer 202, a third activation layer 204, and a normalization layer 206. The number of input and output feature map channels remains constant at 32. The operation process of the depthwise separable convolutional layer is as follows: Figure 5 As shown on the right. The deep feature extraction module 200 reduces the number of parameters by 4 times compared to a single standard convolutional layer.

[0059] This embodiment utilizes the multi-layer processing of the deep feature extraction module, enabling the model to extract and fuse more layers of detailed features from the input image, thereby improving the detail and clarity of the super-resolution image.

[0060] Optionally, in the compressed image restoration method provided in this application embodiment, the upsampling module includes: a transposed convolutional layer, used to perform upsampling operation on the target feature map to obtain a feature map with increased image resolution; a second instance normalization layer, used to normalize the feature map with increased image resolution to obtain a normalized upsampled feature map; and a fourth activation layer, used to perform nonlinear transformation on the normalized upsampled feature map to obtain the restored target image.

[0061] For example, Figure 6 This is a schematic diagram of the upsampling module provided in the embodiments of this application, as shown below. Figure 6As shown, the upsampling module 300 consists of a transposed convolution 301 with a stride of 1 / 2. The operation process of the transposed convolution 301 is as follows: Figure 6 As shown on the right. The 301 convolution kernel size is 3×3, the input feature map size is 2×2, and the output feature map size is 4×4. The entire process is equivalent to padding the input feature map with 0 values, followed by regular convolution, achieving an output size twice that of the input, thus achieving upsampling. This is followed by the second instance normalization (302) and the fourth activation layer (303) to calculate the output of the entire module. Each upsampling module can increase image resolution by a factor of 2. Multiple modules can be placed according to actual needs. If a downsampling module is used to reduce the resolution by a factor of 4 during image compression, and to restore the image resolution, two upsampling modules are needed, with 32 and 3 output channels respectively. The final output layer replaces ReLU with a tanh activation layer, and after calculation, the output value is controlled to be between 0 and 255, meeting the requirements of image numerical values.

[0062] The upsampling module in this embodiment improves image resolution by combining a transposed convolutional layer, a second instance normalization layer, and a fourth activation layer, while preserving image detail and sharpness through normalization and nonlinear transformation. This improves the visual effect of low-resolution images after magnification, enabling the model to provide high-quality image compression and restoration when deployed on mobile devices.

[0063] To improve the image quality of the restored compressed image and reduce computational resource consumption, a target model needs to be trained first. Optionally, in the compressed image restoration method provided in this application embodiment, the target model is trained in the following manner: obtaining an image sample set; for each image sample in the image sample set, blurring and downsampling are performed on the image sample to obtain an input image; inputting the input image into the target model to obtain an output image; inputting the output image and the image sample into a preset loss function; minimizing the value of the preset loss function through the stochastic gradient descent algorithm to obtain the target model parameters corresponding to the minimum loss value; and determining the neural network model corresponding to the target model parameters as the target model.

[0064] In some embodiments, Figure 7 This is a schematic diagram of the target model training process provided in the embodiments of this application, such as... Figure 7 As shown, the entire training system consists of two parts: a 402 image super-resolution network and a 402 image super-resolution network. The other part is the 405 loss network I used to define the preset loss function. The loss network can be a pre-trained VGG16 network (a deep convolutional neural network) used in image classification. The image super-resolution network is a deep neural network parameterized by weights W, which maps... Transform a low-resolution image x (401 resolution) into a high-resolution image y (404 resolution). Loss function. Used to measure the output image y and the 403 target high-resolution image y t The difference lies in the fact that the 402 image super-resolution network uses stochastic gradient descent to minimize the value of the loss function.

[0065] ;

[0066] The purpose of the loss function is to make the 404 output image y of the image super-resolution network match the 403 target image y. t Lost network at 405 They have similar feature representations. Using... Indicates 405 network When processing images, the feature map of the ReLU2_2 layer, i.e., the output feature map of the fourth convolutional layer after the ReLU activation function, has a size of [size missing]. The loss function value is 405. The loss network processes the output image y (404) and the target image y (403). t Euclidean distance between temporal feature maps:

[0067] ;

[0068] For the images in the training set, divide them into image blocks of size 288×288 as the 403 target images y. t The standard deviation of the segmented image is used as follows: The image is blurred using a Gaussian kernel, and then the blurred image is downsampled by 4 times to obtain a low-resolution image, which is used as the 401 input image x of the image super-resolution network.

[0069] This embodiment trains a target model to learn how to recover details and improve resolution from low-quality images after downsampling and blurring, thereby achieving efficient decoding and high-quality restoration of compressed images in practical applications.

[0070] Image compression can be achieved by combining JPEG (Joint Photographic Experts Group) with image downsampling. Optionally, in the method for restoring compressed images provided in this application embodiment, obtaining the target compressed image includes: obtaining the original image; compressing the original image according to a preset compression method to obtain the compressed image; performing downsampling processing on the compressed image to obtain a compressed image with reduced image resolution; and encoding the compressed image into a string to obtain the target compressed image.

[0071] In some embodiments, high-resolution raw images without any processing are obtained from a data source. These images may come from camera sensors, network downloads, local storage, etc., and have high resolution and rich detail, making them suitable for deep learning processing and analysis. The default compression method is JPEG, which compresses the raw images using the JPEG standard. JPEG can remove redundant information in the image, significantly reducing the image file size while maintaining good visual quality. The compression ratio can reach 1:8 or higher, depending on the compression settings, including quality factor, chroma subsampling, etc., which affect the detail retention and file size of the compressed image. After JPEG compression, the compressed image is further downsampled, i.e., the image resolution is reduced. For example, the image resolution can be reduced to approximately 1 / 16 or 1 / 4 of the original. Downsampling helps reduce the computational load and storage requirements of mobile devices.

[0072] Huffman coding is used to encode downsampled compressed images into string format. Huffman coding is a lossless compression algorithm based on the statistical distribution of image data. It assigns variable-length prefix codes based on the frequency of data occurrence; frequently occurring data is assigned shorter codes, while infrequent data is assigned longer codes, thus reducing the overall size of the encoded data. The resulting string format is convenient for storage in common relational databases and consumes less bandwidth during transmission, making it ideal for data management and exchange in mobile environments.

[0073] This embodiment combines JPEG with image downsampling to reduce the resolution of high-definition images by approximately 1 / 16. The compressed image is then compressed and encoded using Huffman coding, transmitted as a string, and stored in a database. When needed, it can be quickly decoded and restored to its high-resolution state using Huffman coding without excessively consuming device resources.

[0074] If the image resolution of the target restored image does not meet the requirements, the image is restored again through the target model. Optionally, in the method for restoring compressed images provided in this application embodiment, after obtaining the target restored image, the method further includes: determining the image resolution of the target restored image and judging whether the image resolution is greater than or equal to a resolution threshold; if the image resolution is greater than or equal to the resolution threshold, determining that the restoration of the target compressed image is complete; if the image resolution is less than the resolution threshold, repeating the step of inputting the initial restored image into the target model until the image resolution is greater than or equal to the resolution threshold.

[0075] In some embodiments, after each model processing step, the resolution of the currently generated target restored image is acquired and measured. Resolution refers to the number of pixels in the width and height of the image, and is an important indicator for evaluating image sharpness and detail. Higher resolution results in better image quality, but also greater computational and storage requirements. The resolution of the current target restored image is compared with a preset resolution threshold. If the resolution of the current image is greater than or equal to the resolution threshold, it indicates that the image has been restored to the desired size and sharpness level, and the restoration process of the target compressed image can be considered complete. If the image resolution is lower than the resolution threshold, it indicates that the current image is still not sharp enough or insufficient in size to meet the needs of subsequent applications. In this case, the current preliminary restoration result (i.e., the initial restored image) needs to be input into the target model again for processing to further improve the image resolution. The model processing steps are executed iteratively, with the image resolution being lower than the resolution threshold, until the resolution of the generated image reaches or exceeds the preset threshold.

[0076] This embodiment provides a method for automatically adjusting the image restoration process, ensuring that image decompression and restoration on mobile devices achieves both minimal computational and storage efficiency while meeting high visual quality standards. By setting a resolution threshold as the exit condition, precise control over the image restoration process is achieved, avoiding unnecessary computational waste while guaranteeing the final output image quality.

[0077] According to another embodiment of this application, a system for restoring compressed images is also provided. Figure 8 This is a schematic diagram of a compressed image restoration system provided according to an embodiment of this application, such as... Figure 8 As shown, the system includes: an image JPEG compression module 801, an image downsampling module 802, a Huffman coding module 803, a data storage module 804, a Huffman decoding module 805, and an image super-resolution module 806.

[0078] Specifically, the image JPEG compression module 801 receives the original image and, for non-JPG format images, compresses them according to the JPEG standard, removing non-essential information, achieving a compression ratio of up to 1 / 8. The image downsampling module 802 performs downsampling processing on the compressed image, reducing the image resolution to 1 / 4 of the original. The Huffman coding module 803 uses the Huffman algorithm to encode the compressed image into a string format for convenient subsequent transmission and storage. The data storage module 804 stores the encoded string, which can be stored in common relational databases, etc. The Huffman decoding module 805 reads the encoded string from the data storage module when using the image and decodes it using the Huffman algorithm to restore the image. The image super-resolution module 806, based on deep learning technology, constructs a lightweight convolutional neural network model, uses the trained classification network VGG16 to extract image features, constructs a loss function between features, and achieves image resolution improvement while maintaining the subjective quality of the image, achieving image restoration. The model has few parameters and supports deployment on mobile devices. This module receives the restored low-resolution image and outputs a high-quality image with a resolution comparable to the original.

[0079] This embodiment combines JPEG (Joint Photographic Experts Group) with image downsampling to reduce the resolution of high-definition images by approximately 1 / 16. The compressed image is then encoded using Huffman coding, transmitted as a string, and stored in a database. Simultaneously, a lightweight deep learning image super-resolution model is constructed to upscale and restore the resolution of the image restored using Huffman coding, resulting in a high-quality image. This significantly reduces the required storage space, and the image's existence as an encoded string facilitates transmission and storage. By leveraging deep learning technology, image resolution is improved while maintaining subjective image quality, achieving accurate image restoration. Furthermore, the model has a small number of parameters, supporting deployment and operation on mobile devices.

[0080] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0081] Example 2

[0082] This application also provides an image restoration apparatus. It should be noted that the image restoration apparatus of this application can be used to execute the image restoration method provided in this application. The image restoration apparatus provided in this application will be described below.

[0083] According to an embodiment of this application, an apparatus for implementing the above-described method for restoring compressed images is also provided. Figure 9 This is a schematic diagram of a compressed image restoration apparatus provided according to an embodiment of this application, such as... Figure 9 As shown, the device includes:

[0084] The acquisition unit 901 is used to acquire the target compressed image, decode the target compressed image according to a preset decoding method, and obtain an initial restored image. The preset decoding method is the decoding method corresponding to the compression method of the target compressed image.

[0085] The input unit 902 is used to input the initial restored image into the target model to obtain the target restored image. The target model consists of N shallow feature extraction modules, M deep feature extraction modules, and an upsampling module.

[0086] The compressed image restoration apparatus provided in this application embodiment acquires a target compressed image through an acquisition unit 901, decodes the target compressed image according to a preset decoding method, and obtains an initial restored image. The preset decoding method is the decoding method corresponding to the compression method of the target compressed image. The input unit 902 inputs the initial restored image into a target model to obtain a target restored image. The target model consists of N shallow feature extraction modules, M deep feature extraction modules, and an upsampling module. By constructing a lightweight target model composed of shallow feature extraction modules, deep feature extraction modules, and upsampling modules, the image resolution is improved while maintaining the image quality. This achieves the goal of reducing model parameters during the image restoration process, thereby reducing the computational resource consumption during the compressed image restoration process and solving the technical problem of high computational resource consumption during the compressed image restoration process.

[0087] Optionally, in the compressed image restoration apparatus provided in this application embodiment, the shallow feature extraction module includes: a first convolutional layer with P channels, used to extract features of the initial restored image and output a feature map with P channels; a first instance normalization layer, used to receive the feature map with P channels, normalize the feature map with P channels, and output a normalized feature map; and a first activation layer, used to receive the normalized feature map, process it through a preset activation function, and obtain a shallow feature map.

[0088] Optionally, in the compressed image restoration apparatus provided in this application embodiment, the deep feature extraction module includes: a second convolutional layer for performing convolution operations on the shallow feature map to obtain an output feature map; a second activation layer for processing the output feature map to obtain an activated feature map; a depthwise separable convolutional layer for performing depthwise convolution on the activated feature map to extract features, obtaining multiple depthwise convolutional feature maps, and performing pointwise convolution operations on the multiple depthwise convolutional feature maps to obtain a fused feature map; a third activation layer for performing nonlinear transformation on the fused feature map to obtain a transformed feature map; and a normalization layer for performing normalization operations on the transformed feature map to obtain a target feature map.

[0089] Optionally, in the compressed image restoration apparatus provided in this application embodiment, the upsampling module includes: a transposed convolutional layer, used to perform upsampling operation on the target feature map to obtain a feature map with increased image resolution; a second instance normalization layer, used to normalize the feature map with increased image resolution to obtain a normalized upsampled feature map; and a fourth activation layer, used to perform nonlinear transformation on the normalized upsampled feature map to obtain the target restored image.

[0090] Optionally, in the compressed image restoration apparatus provided in this application embodiment, the target model is trained in the following manner: acquiring an image sample set; for each image sample in the image sample set, performing blurring and downsampling processing on the image sample to obtain an input image; inputting the input image into the target model to obtain an output image; inputting the output image and the image sample into a preset loss function; minimizing the value of the preset loss function through a stochastic gradient descent algorithm to obtain the target model parameters corresponding to the minimum loss value; and determining the neural network model corresponding to the target model parameters as the target model.

[0091] Optionally, in the compressed image restoration apparatus provided in this application embodiment, the acquisition unit 901 includes: an acquisition module, used to acquire the original image, compress the original image according to a preset compression method to obtain a compressed image; a downsampling module, used to perform downsampling processing on the compressed image to obtain a compressed image with reduced image resolution; and to encode the compressed image into a string format to obtain a target compressed image.

[0092] Optionally, in the compressed image restoration apparatus provided in the embodiments of this application, the apparatus further includes: a first determining unit, configured to determine the image resolution of the target restored image and determine whether the image resolution is greater than or equal to a resolution threshold; a second determining unit, configured to determine that the restoration of the target compressed image is complete when the image resolution is greater than or equal to the resolution threshold; and an execution unit, configured to repeatedly execute the step of inputting the initial restored image into the target model when the image resolution is less than the resolution threshold, until the image resolution is greater than or equal to the resolution threshold.

[0093] It should be noted that the acquisition unit 901 and input unit 902 mentioned above correspond to steps S201 to S202 in Embodiment 1. The two units and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules or units can also be part of a device and run in the computer terminal 10 provided in Embodiment 1.

[0094] Example 3

[0095] Embodiments of this application may provide an electronic device. Figure 10 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 10 As shown, the electronic device may include: one or more ( Figure 10 (Only one is shown) processor 1002, memory 1004, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0096] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0097] The processor can call the information and application program stored in the memory through the transmission device to perform the following steps: acquire the target compressed image, decode the target compressed image according to the preset decoding method to obtain the initial restored image, wherein the preset decoding method is the decoding method corresponding to the compression method of the target compressed image; input the initial restored image into the target model to obtain the target restored image, wherein the target model consists of N shallow feature extraction modules, M deep feature extraction modules and upsampling modules.

[0098] The processor can also call the information and application program stored in the memory through the transmission device to perform the following steps: a first convolutional layer with P channels, used to extract features of the initial restored image and output a feature map with P channels; a first instance normalization layer, used to receive the feature map with P channels, normalize the feature map with P channels, and output the normalized feature map; a first activation layer, used to receive the normalized feature map, process it through a preset activation function, and obtain a shallow feature map.

[0099] The processor can also access information and applications stored in the memory via a transmission device to execute the following steps: a second convolutional layer for performing convolution operations on the shallow feature map to obtain an output feature map; a second activation layer for processing the output feature map to obtain an activated feature map; a depthwise separable convolutional layer for performing depthwise convolution on the activated feature map to extract features, obtaining multiple depthwise convolutional feature maps, and performing pointwise convolution operations on the multiple depthwise convolutional feature maps to obtain a fused feature map; a third activation layer for performing nonlinear transformation on the fused feature map to obtain a transformed feature map; and a normalization layer for performing normalization operations on the transformed feature map to obtain a target feature map.

[0100] The processor can also call the information and application program stored in the memory through the transmission device to perform the following steps: a transposed convolutional layer, used to upsample the target feature map to obtain a feature map with improved image resolution; a second instance normalization layer, used to normalize the feature map with improved image resolution to obtain a normalized upsampled feature map; and a fourth activation layer, used to perform a nonlinear transformation on the normalized upsampled feature map to obtain the target restored image.

[0101] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: acquire an image sample set; for each image sample in the image sample set, perform blurring and downsampling processing on the image sample to obtain an input image; input the input image into the target model to obtain an output image; input the output image and the image sample into a preset loss function; minimize the value of the preset loss function through the stochastic gradient descent algorithm to obtain the target model parameters corresponding to the minimum loss value; and determine the neural network model corresponding to the target model parameters as the target model.

[0102] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: acquire the original image, compress the original image according to a preset compression method to obtain a compressed image; perform downsampling processing on the compressed image to obtain a compressed image with reduced image resolution; encode the compressed image into a string to obtain the target compressed image.

[0103] The processor can also call the information and application program stored in the memory through the transmission device to perform the following steps: determine the image resolution of the target restored image, and determine whether the image resolution is greater than or equal to the resolution threshold; if the image resolution is greater than or equal to the resolution threshold, determine that the target compressed image restoration is complete; if the image resolution is less than the resolution threshold, repeat the step of inputting the initial restored image into the target model until the image resolution is greater than or equal to the resolution threshold.

[0104] This application provides a method for acquiring a target compressed image, decoding the target compressed image according to a preset decoding method to obtain an initial restored image, wherein the preset decoding method is the decoding method corresponding to the compression method of the target compressed image; inputting the initial restored image into a target model to obtain the target restored image, wherein the target model consists of N shallow feature extraction modules, M deep feature extraction modules, and an upsampling module. By constructing a lightweight target model composed of shallow feature extraction modules, deep feature extraction modules, and upsampling modules, image resolution is improved while maintaining image quality, achieving the goal of reducing model parameters during image restoration, thereby reducing the technical effect of computational resource consumption in the compressed image restoration process, and thus solving the technical problem of high computational resource consumption in the compressed image restoration process.

[0105] Those skilled in the art will understand that Figure 10 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 10 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 10 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 10 The different configurations shown.

[0106] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0107] Example 4

[0108] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the compressed image restoration method provided in Embodiment 1.

[0109] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0110] This application also provides a computer program product, which, when executed on a data processing device, is adapted to perform the steps of a method for restoring a compressed image.

[0111] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0112] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0113] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0114] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0115] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0116] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0117] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for restoring a compressed image, characterized in that, include: A target compressed image is acquired, and the target compressed image is decoded according to a preset decoding method to obtain an initial restored image, wherein the preset decoding method is the decoding method corresponding to the compression method of the target compressed image; The initial restored image is input into the target model to obtain the target restored image. The target model consists of N shallow feature extraction modules, M deep feature extraction modules, and an upsampling module.

2. The method according to claim 1, characterized in that, The shallow feature extraction module includes: The first convolutional layer, with P channels, is used to extract features from the initial restored image and outputs a feature map with P channels. The first instance normalization layer is used to receive the feature map with P channels, perform normalization processing on the feature map with P channels, and output the normalized feature map. The first activation layer is used to receive the normalized feature map and process it through a preset activation function to obtain a shallow feature map.

3. The method according to claim 2, characterized in that, The deep feature extraction module includes: The second convolutional layer is used to perform convolution operations on the shallow feature map to obtain the output feature map; The second activation layer is used to process the output feature map to obtain the activated feature map. A depthwise separable convolutional layer is used to perform depthwise convolution on the activated feature map to extract features, resulting in multiple depthwise convolutional feature maps. Then, a pointwise convolution operation is performed on the multiple depthwise convolutional features to obtain a fused feature map. The third activation layer is used to perform a nonlinear transformation on the fused feature map to obtain the transformed feature map. The normalization layer is used to normalize the transformed feature map to obtain the target feature map.

4. The method according to claim 3, characterized in that, The upsampling module includes: A transposed convolutional layer is used to upsample the target feature map to obtain a feature map with improved image resolution. The second instance normalization layer is used to normalize the feature map after the image resolution is increased, so as to obtain the normalized upsampled feature map. The fourth activation layer is used to perform a nonlinear transformation on the normalized upsampled feature map to obtain the target restored image.

5. The method according to claim 1, characterized in that, The target model is trained in the following manner: Obtain an image sample set; for each image sample in the image sample set, perform blurring and downsampling processing on the image sample to obtain the input image. The input image is input into the target model to obtain the output image, and the output image and the image sample are input into a preset loss function; The value of the preset loss function is minimized by the stochastic gradient descent algorithm to obtain the target model parameters corresponding to the minimum loss value, and the neural network model corresponding to the target model parameters is determined as the target model.

6. The method according to claim 1, characterized in that, Obtaining the target compressed image includes: Obtain the original image, compress the original image according to a preset compression method, and obtain the compressed image; The compressed image is downsampled to obtain a compressed image with reduced resolution; The compressed image is encoded into a string to obtain the target compressed image.

7. The method according to claim 1, characterized in that, After obtaining the target reconstructed image, the method further includes: Determine the image resolution of the target restored image, and determine whether the image resolution is greater than or equal to a resolution threshold; If the image resolution is greater than or equal to the resolution threshold, it is determined that the target compressed image restoration is complete; If the image resolution is less than the resolution threshold, the step of inputting the initial restored image into the target model is repeated until the image resolution is greater than or equal to the resolution threshold.

8. A device for restoring compressed images, characterized in that, include: An acquisition unit is used to acquire a target compressed image, decode the target compressed image according to a preset decoding method, and obtain an initial restored image, wherein the preset decoding method is the decoding method corresponding to the compression method of the target compressed image; The input unit is used to input the initial restored image into the target model to obtain the target restored image, wherein the target model consists of N shallow feature extraction modules, M deep feature extraction modules and an upsampling module.

9. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method for restoring a compressed image as described in any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the method for restoring a compressed image as described in any one of claims 1 to 7.

Images