Image compression method and device, electronic equipment and readable medium

Abstract

The invention provides an image compression method and device, electronic equipment and a readable medium, and relates to the technical field of image processing, and the image compression method comprises the steps: obtaining an input image and a target code rate; inputting the input image into the content feature analysis model to input a feature vector of the image, the feature vector being capable of reflecting a perception feature of the input image; and inputting the target code rate and the feature vector into a joint parameter decision model to generate a spatial scaling parameter and a quantization parameter adaptive to the input image and the target code rate. According to the embodiment of the invention, under the limitation of the target code rate, the details of the visual salient region are reserved preferentially according to the adaptive control of the image prediction optimal compression parameter, and the method is widely applied to application scenes such as extremely-low bandwidth image transmission, terminal front-end compression and Internet of Things image perception.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more specifically, to an image compression method, apparatus, electronic device, and readable medium. Background Technology

[0002] Image compression is the cornerstone of digital image processing. Its core lies in carrying the maximum visual information with the minimum data volume, resolving the fundamental contradiction between massive data storage and limited transmission bandwidth. In modern applications, it is a key supporting technology for the widespread adoption of high-definition streaming media, real-time communication, cloud storage, and mobile computing. The development trend is evolving from the traditional "fidelity reconstruction" to "intelligence-oriented": on the one hand, coding standards (such as H.266 / VVC) are continuously optimizing rate-distortion performance; on the other hand, end-to-end compression technologies based on deep learning are emerging and deeply integrated with computer vision tasks, achieving efficient semantic compression designed for machine understanding.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this disclosure is to provide an image compression method, apparatus, electronic device, and readable medium for overcoming, at least to some extent, image compression problems caused by limitations and defects in related technologies.

[0005] According to a first aspect of the present disclosure, an image compression method is provided, comprising: Obtain the input image and target bitrate; The input image is input into a content feature analysis model, and the feature vector of the input image is used to reflect the perceptual characteristics of the input image. The target bitrate and the feature vector are input into a joint parameter decision model to generate spatial scaling parameters and quantization parameters that are adapted to the input image and the target bitrate. The joint parameter decision model is trained with the joint loss function as the objective. The joint loss function is determined based on the bit rate deviation and perceptual distortion loss function. The spatial scaling parameter is a parameter used to spatially scale the input image. The quantization parameter is used to encode and compress the spatially scaled image.

[0006] In one exemplary embodiment of this disclosure, the content feature analysis model is configured with multi-scale convolutional layers and deep convolutional layers. The multi-scale convolutional layer is used to capture the local edge texture and contrast information of the input image. The deep convolutional layer is used to extract high-level semantic features and information about salient regions of the input image; The outputs of the multi-scale convolutional layer and the deep convolutional layer are fused and / or nonlinearly activated to generate the feature vector.

[0007] In one exemplary embodiment of this disclosure, the training process of the joint parameter decision model includes: Acquire training samples, which include sample images and randomly sampled target bitrates. The sample images are labeled, and the labels include a combination of spatial scaling parameters and quantization parameters that optimize image quality under the constraints of the randomly sampled target bitrates. Using the sample image and the randomly sampled target bitrate as input to the joint parameter decision model, the spatial scaling parameter and quantization parameter predicted by the joint parameter decision model are obtained; The training process is performed using a loss function constructed from the predicted spatial scaling parameters and quantization parameters, and the regression error between the labels.

[0008] In one exemplary embodiment of this disclosure, the label is determined through a grid search. The constraints of the grid search include, in response to a smaller spatial scaling parameter, matching a smaller quantization parameter to the spatial scaling parameter. Alternatively, in response to a large spatial scaling parameter, a larger quantization parameter may be matched to the spatial scaling parameter.

[0009] In one exemplary embodiment of this disclosure, the perceptual distortion loss function is configured to be determined based on a weighted loss of salient and non-critical regions of the compressed image.

[0010] In one exemplary embodiment of this disclosure, the expression for the perceived distortion loss function includes: L perceptual =w s ×(1-SSIM s )+w n ×(1-SSIM n ), Wherein, the L perceptual The SSIM is the result of calculating the perceptual distortion loss function. s The SSIM is the structural similarity value of the salient region. n The structural similarity value of the non-critical region is w. s With the w nThe weighting coefficients are preset, and w s >the w n .

[0011] In one exemplary embodiment of this disclosure, the joint loss function of the joint parameter decision model is configured to be determined based on a weighted sum of the bit rate deviation and the perceptual distortion loss function.

[0012] In one exemplary embodiment of this disclosure, the expression for the joint loss function includes: L total =λ×L compare +(1-λ)×L perceptual , Wherein, the L total The L is the result of the calculation of the target loss function. compare The L is the result of the calculation of the bit rate deviation. perceptual The result of the perceptual distortion loss function is given by λ, which is a preset hyperparameter used to balance the bitrate control and perceptual quality after image compression.

[0013] In one exemplary embodiment of this disclosure, the expression for the bit rate deviation includes: Lcompare=λ1×MSE(α,α ) +λ2×MSE(QP, QP) ), The α The QP is the predicted spatial scaling parameter. The quantization parameters obtained from the prediction are: α is the spatial scaling parameter in the label, QP is the quantization parameter in the label, and λ1 and λ2 are preset weight coefficients.

[0014] According to a second aspect of the present disclosure, an image compression apparatus is provided, comprising: The parameter acquisition module is configured to acquire the input image and target bitrate. The feature extraction module is configured to input the input image into the content feature analysis model, using the feature vector of the input image, which can reflect the perceptual characteristics of the input image; The image compression module is configured to input the target bitrate and the feature vector into a joint parameter decision model to generate spatial scaling parameters and quantization parameters that are adapted to the input image and the target bitrate. The joint parameter decision model is trained with the joint loss function as the objective. The joint loss function is determined based on the bit rate deviation and perceptual distortion loss function. The spatial scaling parameter is a parameter used to spatially scale the input image. The quantization parameter is used to encode and compress the spatially scaled image.

[0015] According to a third aspect of this disclosure, an electronic device is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to perform an image compression method as described in any of the preceding claims based on instructions stored in the memory.

[0016] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a program stored thereon that, when executed by a processor, implements the image compression method as described in any of the preceding claims.

[0017] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the image compression method as described in any of the above technical solutions.

[0018] This embodiment of the disclosure explicitly fuses the target bitrate and image content features through a model, adaptively outputting the optimal strategy, replacing traditional methods that rely on empirical rules or repeated trial and error. A single forward inference can obtain the parameter combination suitable for the target bitrate, avoiding iterative recompression and significantly improving real-time performance.

[0019] Furthermore, in embodiments of this disclosure, the spatial scaling parameter (α) and compression quality parameter (QP) of the image can be predicted simultaneously through an end-to-end model to achieve joint optimization of the two parameters. The image compression framework based on embodiments of this disclosure exhibits superior bitrate control accuracy and quality balance capabilities in extremely low bandwidth scenarios. It can suppress block artifacts and blurring caused by adjusting only QP, and also mitigate content loss caused by simple downsampling. The image compression system does not rely on the encoder's internal implementation, supports seamless integration with mainstream standards (such as HEVC, AV1, etc.), and possesses strong engineering deployability.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0022] Figure 1A schematic diagram of an exemplary system architecture to which an image compression scheme according to embodiments of the present disclosure can be applied is shown.

[0023] Figure 2 This is a flowchart of an image compression method according to an exemplary embodiment of the present disclosure.

[0024] Figure 3 This is a flowchart of another image compression method in an exemplary embodiment of this disclosure.

[0025] Figure 4 This is a schematic diagram of the architecture of an image compression scheme in an exemplary embodiment of this disclosure.

[0026] Figure 5 This is a block diagram of an image compression apparatus according to an exemplary embodiment of the present disclosure.

[0027] Figure 6 This is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0028] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0029] Furthermore, the accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different models and / or processor devices and / or microcontroller devices.

[0030] In related technologies, the development trend of image technology solutions is evolving from the traditional "fidelity reconstruction" to "intelligence-oriented": on the one hand, coding standards (such as H.266 / VVC) continue to optimize rate-distortion performance; on the other hand, end-to-end compression technology based on deep learning has emerged and is deeply integrated with computer vision tasks to achieve efficient semantic compression for machine understanding.

[0031] However, in extremely low bandwidth scenarios, adjusting QP or image scaling alone is insufficient to balance bitrate and quality. Increasing QP alone can cause significant blockiness and blurring, while downsampling alone can irreversibly lose details, especially high-frequency information and small targets. These bitrate and quality effects are non-linearly coupled and become more sensitive when bandwidth is limited. Existing solutions typically employ fixed parameter combinations or frame-by-frame parameter tuning, lacking the ability to jointly model image content, target bitrate, and visual quality, resulting in insufficient control precision and stability.

[0032] Therefore, there is an urgent need for a new image compression scheme that can balance image content, target bit rate, and visual quality to solve the above-mentioned technical problems.

[0033] Before describing the image compression scheme in conjunction with the embodiments of this disclosure, the key terms are explained as follows: Bitrate: The amount of data transmitted per unit of time, used to measure compression efficiency.

[0034] Quantization parameter (QP): A quantization control parameter in image compression. The larger the value, the higher the compression ratio and the lower the quality.

[0035] Spatial scaling parameter: also known as scaling factor α (Alpha), is a parameter that controls the scaling of image size. The smaller α is, the lower the resolution and the lower the bit rate.

[0036] Figure 1 A schematic diagram of an exemplary system architecture to which an image compression scheme according to embodiments of the present disclosure can be applied is shown.

[0037] like Figure 1 As shown, system architecture 100 may include one or more of terminal devices 101, 102, and 103, model 104, and server 105. Model 104 is used as a medium to provide a communication link between terminal devices 101, 102, and 103 and server 105. Model 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0038] It should be understood that Figure 1The number of terminal devices, models, and servers shown is merely illustrative. Depending on implementation needs, there can be any number of terminal devices, models, and servers. For example, server 105 could be a server cluster composed of multiple servers.

[0039] Users can use terminal devices 101, 102, and 103 to interact with server 105 through model 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with displays, including but not limited to smartphones, tablets, laptops, and desktop computers, etc.

[0040] In some embodiments, the image compression method provided in this disclosure is generally executed by server 105, and correspondingly, the image compression device is generally disposed in terminal device 103 (or terminal device 101 or 102). In other embodiments, some terminals may have similar functions to the server device to execute this method.

[0041] The exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0042] Figure 2 This is a flowchart of an image compression method in an exemplary embodiment of this disclosure.

[0043] refer to Figure 2 Image compression methods may include: Step S202: Obtain the input image and target bitrate; Step S204: Input the input image into the content feature analysis model, using the feature vector of the input image, which can reflect the perceptual characteristics of the input image; Step S206: Input the target bitrate and the feature vector into the joint parameter decision model to generate spatial scaling parameters and quantization parameters that are adapted to the input image and the target bitrate; The joint parameter decision model is trained with the joint loss function as the objective. The joint loss function is determined based on the bit rate deviation and perceptual distortion loss function. The spatial scaling parameter is a parameter used to spatially scale the input image. The quantization parameter is used to encode and compress the spatially scaled image.

[0044] This embodiment of the disclosure breaks through the limitations of fixed parameters or empirical adjustments in traditional encoding. It understands the perceptual characteristics of images through a content feature analysis model and, in conjunction with the target bitrate, dynamically generates optimal spatial scaling and quantization parameters through a decision model. The joint loss function simultaneously optimizes bitrate and perceptual quality, enabling the compression system to automatically allocate appropriate compression resources for images with different content at a specified bitrate, thereby achieving better rate-distortion performance and visual quality.

[0045] The following section provides a detailed explanation of each step in the image compression method.

[0046] In one exemplary embodiment of this disclosure, the content feature analysis model is configured with multi-scale convolutional layers and deep convolutional layers. The multi-scale convolutional layer is used to capture the local edge texture and contrast information of the input image. The deep convolutional layer is used to extract high-level semantic features and information about salient regions of the input image; The outputs of the multi-scale convolutional layer and the deep convolutional layer are fused and / or nonlinearly activated to generate the feature vector.

[0047] This disclosure improves the comprehensiveness and accuracy of content feature extraction by designing a dedicated network with multi-scale and deep convolutions. Multi-scale layers effectively capture fundamental image details (such as texture and edges), while the deep network understands high-level semantics and salient regions. The feature vector generated by the fusion of these two layers accurately reflects the differences in the importance of the image to human visual perception. This provides a reliable input rich in content-aware information for subsequent parameter decisions, and is a key foundation for adaptive compression to be "localized."

[0048] In one exemplary embodiment of this disclosure, such as Figure 3 As shown, the training process of the joint parameter decision model includes: Step S302: Obtain training samples, the training samples include sample images and randomly sampled target bitrates, the sample images are configured with labels, the labels include a combination of spatial scaling parameters and quantization parameters that optimize image quality under the constraints of the randomly sampled target bitrates; Step S304: Using the sample image and the randomly sampled target bitrate as input to the joint parameter decision model, obtain the spatial scaling parameter and quantization parameter predicted by the joint parameter decision model; Step S306: Training is performed on a loss function constructed based on the predicted spatial scaling parameters and quantization parameters and the regression error between the labels.

[0049] In this embodiment, by using the optimal parameter combination obtained through grid search or similar methods as labels for regression training, the model can learn the complex mapping relationship between image content, target bitrate, and optimal compression parameters. This enables the trained model to accurately predict decisions that approximate the theoretical optimum, rather than relying on random or heuristic rules, thus ensuring the high performance and stability of the compression system in practical applications.

[0050] In one exemplary embodiment of this disclosure, the label is determined through a grid search. The constraints of the grid search include, in response to a smaller spatial scaling parameter, matching a smaller quantization parameter to the spatial scaling parameter. Alternatively, in response to a large spatial scaling parameter, a larger quantization parameter may be matched to the spatial scaling parameter.

[0051] In this embodiment, the label generation rules are optimized, making the training data more consistent with the inherent laws of encoding efficiency. By constraining the reasonable matching relationship between scaling parameters and quantization parameters (small scaling with small quantization, large scaling with large quantization), unreasonable parameter combinations are avoided as training targets. This guides the model to learn decision strategies that are more in line with compression principles, resulting in theoretically better parameter combinations, thereby improving the final model's decision quality and compression efficiency.

[0052] In one exemplary embodiment of this disclosure, the perceptual distortion loss function is configured to be determined based on a weighted loss of salient and non-critical regions of the compressed image.

[0053] In this embodiment of the disclosure, salient regions and non-critical regions, these two features together define a distortion assessment standard guided by human visual perception. By distinguishing between salient regions and non-critical regions, and assigning a higher distortion weight to salient regions (w... s >w n This allows the optimization of compression quality to focus on areas sensitive to the human eye. This loss function guides the entire system to prioritize the preservation of details and structural information in important areas during the compression process, thereby obtaining reconstructed images with better subjective perceived quality at the same bitrate.

[0054] In one exemplary embodiment of this disclosure, the expression for the perceived distortion loss function includes: L perceptual =w s ×(1-SSIM s )+w n ×(1-SSIM n ), Wherein, the L perceptual The SSIM is the result of calculating the perceptual distortion loss function. sThe SSIM is the structural similarity value of the salient region. n The structural similarity value of the non-critical region is w. s With the w n The weighting coefficients are preset, and w s >the w n .

[0055] In one exemplary embodiment of this disclosure, the joint loss function of the joint parameter decision model is configured to be determined based on a weighted sum of the bit rate deviation and the perceptual distortion loss function.

[0056] In one exemplary embodiment of this disclosure, the expression for the joint loss function includes: L total =λ×L compare +(1-λ)×L perceptual , Wherein, the L total The L is the result of the calculation of the target loss function. compare The L is the result of the calculation of the bit rate deviation. perceptual The result of the perceptual distortion loss function is given by λ, which is a preset hyperparameter used to balance the bitrate control and perceptual quality after image compression.

[0057] This disclosure proposes a new final optimization objective for training a joint parameter decision model, which is achieved by controlling the bitrate deviation (L... compare ) and perceptual distortion (L perceptual By using adjustable hyperparameter λ for weighted combination, a unified optimization objective L was constructed. total This allows the model to simultaneously consider the sometimes conflicting goals of "meeting the target bit rate requirement" and "improving perceptual quality" during training, and achieves the best trade-off for different application scenarios (such as storage saving or visual fidelity) by adjusting λ.

[0058] In one exemplary embodiment of this disclosure, the expression for the bit rate deviation includes: L compare =λ1×MSE(α,α ) +λ2×MSE(QP, QP) ), The α The QP is the predicted spatial scaling parameter. The quantization parameters obtained from the prediction are: α is the spatial scaling parameter in the label, QP is the quantization parameter in the label, and λ1 and λ2 are preset weight coefficients.

[0059] This disclosure not only specifies the calculation method for the bit rate deviation loss, transforming it into minimizing the mean squared error (MSE) between the predicted parameters and the optimal label parameters, but also provides a clear and differentiable optimization objective that directly drives the model output air conditioning scaling parameter (α) and quantization parameter (QP) to be as close as possible to the theoretical optimal values ​​(α, QP). Furthermore, the importance of the prediction accuracy of the two parameters can be flexibly adjusted through λ1 and λ2, which is the foundation for ensuring accurate model decisions and achieving efficient distortion control.

[0060] In one exemplary embodiment of this disclosure, such as Figure 4 As shown, the overall architecture of the image compression scheme 400 includes a content feature analysis network, a joint parameter decision module, a perceptual quality assessment module, and an end-to-end training framework. The image compression process includes: The input image is first preprocessed to be normalized and size aligned, and then fed into the content feature analysis network to extract multidimensional perceptual features; The extracted feature vector and the target bit rate parameter are input into the joint parameter decision module to obtain the optimal combination of spatial scaling factor and quantization parameter; The perception quality assessment module evaluates the generated parameter combinations based on structural similarity or pre-trained perception models to ensure that visually salient regions are given priority protection. The entire system is optimized through an end-to-end training framework, thereby enabling it to adapt to different image content and bitrate requirements.

[0061] based on Figure 4 The overall architecture of the image compression scheme shown is 400. Taking natural scene image compression as the application scenario, the specific implementation stages are as follows: 1. Preprocessing module: Normalizes the input image (resolution 1920×1080) (adjusts the pixel value range to [0,1]) and unifies the size to 1024×576 through bilinear interpolation.

[0062] 2. Content Feature Analysis Network: A 3-layer convolutional structure is adopted. The first layer uses a 3×3 convolutional kernel (64 channels) to extract edge texture. The second layer uses a 5×5 convolutional kernel (128 channels) to capture contrast distribution. The third layer uses the SENet channel attention module (256 channels) to mine salient region features such as face and text. Finally, the output is a feature vector F with dimension D=256.

[0063] 3. Joint Parameter Decision Module: F is concatenated with the normalized target bitrate R (e.g., target bitrate 500kbps, normalized to the [0,1] interval) to form a 513-dimensional input vector. This vector is then processed by three fully connected layers (256→128→64 neurons) and an LSTM gated unit to generate a combination of α∈{0.75,0.5,0.3} and QP∈{22,28,34}.

[0064] When α=0.75 (resolution 768×432), QP can be set to 22 to preserve detail.

[0065] When α=0.3 (resolution 307×173), QP can be set to 34 to enhance compression.

[0066] 4. Perceptual Quality Assessment: Based on SSIM calculation, the distortion of high-perceptual-area (face, 15%) and non-critical area (background, 85%) is set, and w is... s =2.0、w n =0.5, perceptual loss L perceptual =2.0×(1-SSIM s )+0.5×(1-SSIM n ).

[0067] 5. End-to-end training: Using the COCO Natural Image Dataset (100,000 images) and the Medical Image Dataset (20,000 images), the target bitrate (300-1000kbps) and the parameter tuning results of traditional methods were labeled. During training, λ=0.6, batch size 16, learning rate 1e-4, and 100 iterations were performed. The final model achieved a target bitrate error ≤3% and a SSIM ≥0.92 in high-sensitivity regions on the test set.

[0068] Corresponding to the above method embodiments, this disclosure also provides an image compression apparatus that can be used to execute the above method embodiments.

[0069] Figure 5 This is a block diagram of an image compression apparatus according to an exemplary embodiment of the present disclosure.

[0070] refer to Figure 5 The image compression device 500 may include: The parameter acquisition module 502 is configured to acquire the input image and the target bitrate. The feature extraction module 504 is configured to input the input image into the content feature analysis model, using the feature vector of the input image, wherein the feature vector can reflect the perceptual characteristics of the input image; The image compression module 506 is configured to input the target bitrate and the feature vector into a joint parameter decision model to generate spatial scaling parameters and quantization parameters that are adapted to the input image and the target bitrate. The joint parameter decision model is trained with the joint loss function as the objective. The joint loss function is determined based on the bit rate deviation and perceptual distortion loss function. The spatial scaling parameter is a parameter used to spatially scale the input image. The quantization parameter is used to encode and compress the spatially scaled image.

[0071] In one exemplary embodiment of this disclosure, the content feature analysis model is configured with multi-scale convolutional layers and deep convolutional layers. The multi-scale convolutional layer is used to capture the local edge texture and contrast information of the input image. The deep convolutional layer is used to extract high-level semantic features and information about salient regions of the input image; The outputs of the multi-scale convolutional layer and the deep convolutional layer are fused and / or nonlinearly activated to generate the feature vector.

[0072] In one exemplary embodiment of this disclosure, the training process of the joint parameter decision model includes: Acquire training samples, which include sample images and randomly sampled target bitrates. The sample images are labeled, and the labels include a combination of spatial scaling parameters and quantization parameters that optimize image quality under the constraints of the randomly sampled target bitrates. Using the sample image and the randomly sampled target bitrate as input to the joint parameter decision model, the spatial scaling parameter and quantization parameter predicted by the joint parameter decision model are obtained; The training process is performed using a loss function constructed from the predicted spatial scaling parameters and quantization parameters, and the regression error between the labels.

[0073] In one exemplary embodiment of this disclosure, the label is determined through a grid search. The constraints of the grid search include, in response to a smaller spatial scaling parameter, matching a smaller quantization parameter to the spatial scaling parameter. Alternatively, in response to a large spatial scaling parameter, a larger quantization parameter may be matched to the spatial scaling parameter.

[0074] In one exemplary embodiment of this disclosure, the perceptual distortion loss function is configured to be determined based on a weighted loss of salient and non-critical regions of the compressed image.

[0075] In one exemplary embodiment of this disclosure, the expression for the perceived distortion loss function includes: L perceptual =w s ×(1-SSIM s )+w n ×(1-SSIM n ), Wherein, the L perceptual The SSIM is the result of calculating the perceptual distortion loss function. s The SSIM is the structural similarity value of the salient region. n The structural similarity value of the non-critical region is w. s With the w n The weighting coefficients are preset, and w s >the w n .

[0076] In one exemplary embodiment of this disclosure, the joint loss function of the joint parameter decision model is configured to be determined based on a weighted sum of the bit rate deviation and the perceptual distortion loss function.

[0077] In one exemplary embodiment of this disclosure, the expression for the joint loss function includes: L total =λ×L compare +(1-λ)×L perceptual , Wherein, the L total The L is the result of the calculation of the target loss function. compare The L is the result of the calculation of the bit rate deviation. perceptual The result of the perceptual distortion loss function is given by λ, which is a preset hyperparameter used to balance the bitrate control and perceptual quality after image compression.

[0078] In one exemplary embodiment of this disclosure, the expression for the bit rate deviation includes: Lcompare=λ1×MSE(α,α ) +λ2×MSE(QP, QP) ), The α The QP is the predicted spatial scaling parameter. The quantization parameters obtained from the prediction are: α is the spatial scaling parameter in the label, QP is the quantization parameter in the label, and λ1 and λ2 are preset weight coefficients.

[0079] Since the functions of the device 500 have been described in detail in their respective method embodiments, they will not be repeated here.

[0080] It should be noted that although several modules or units of the device for performing actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0081] In an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided.

[0082] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely in hardware, entirely in software (including firmware, microcode, etc.), or in a combination of hardware and software, collectively referred to herein as “circuit,” “module,” or “system.”

[0083] The following reference Figure 6 To describe an electronic device 600 according to this embodiment of the present invention. Figure 6 The electronic device 600 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0084] like Figure 6 As shown, the electronic device 600 is manifested in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, and a bus 630 connecting different system components (including storage unit 620 and processing unit 610).

[0085] The storage unit stores program code that can be executed by the processing unit 610, causing the processing unit 610 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 610 can perform the method shown in the embodiments of this disclosure.

[0086] Storage unit 620 may include readable media in the form of volatile storage units, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include read-only memory (ROM) 6203.

[0087] Storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of the model environment.

[0088] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0089] Electronic device 600 can also communicate with one or more external devices 640 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more models (e.g., local area network (LAN), wide area network (WAN), and / or public models, such as the Internet) via model adapter 660. As shown, model adapter 660 communicates with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0090] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a model, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or model device, etc.) to execute the method according to the embodiments of this disclosure.

[0091] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section of this specification.

[0092] The program product for implementing the above-described method according to embodiments of the present invention may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0093] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0094] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0095] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0096] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of model, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0097] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0098] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and concept of this disclosure are indicated by the claims.

Claims

1. An image compression method characterized by, The method comprises: obtaining an input image and a target code rate; inputting the input image into a content feature analysis model to obtain a feature vector of the input image, the feature vector being capable of reflecting a perceptual characteristic of the input image; inputting the target code rate and the feature vector into a joint parameter decision model to generate a spatial scaling parameter and a quantization parameter adapted to the input image and the target code rate; wherein the joint parameter decision model is trained with a joint loss function as a target, the joint loss function is determined based on a code rate deviation and a perceptual distortion loss function, the spatial scaling parameter is a parameter for spatial scaling of the input image, and the quantization parameter is a parameter for encoding and compressing the spatially scaled image.

2. The image compression method of claim 1, wherein the content feature analysis model is configured with a multi-scale convolution layer and a deep convolution layer, the multi-scale convolution layer is configured to capture local edge texture and contrast information of the input image, the deep convolution layer is configured to extract high-level semantic features and information of salient regions of the input image; and the outputs of the multi-scale convolution layer and the deep convolution layer are fused and / or subjected to nonlinear activation to generate the feature vector.

3. The image compression method of claim 1, wherein, The training process of the joint parameter decision model comprises: obtaining a training sample, the training sample comprising a sample image and a randomly sampled target code rate, the sample image being configured with a label, the label comprising a combination of a spatial scaling parameter and a quantization parameter that optimizes image quality under the constraint of the randomly sampled target code rate; inputting the sample image and the randomly sampled target code rate into the joint parameter decision model as inputs of the joint parameter decision model to obtain a predicted spatial scaling parameter and a predicted quantization parameter of the joint parameter decision model; training based on a loss function constructed based on a regression error between the predicted spatial scaling parameter and the predicted quantization parameter and the label.

4. The image compression method of claim 3, wherein the label is determined by grid search, the constraint conditions of the grid search comprise matching a smaller quantization parameter to the spatial scaling parameter in response to the spatial scaling parameter being smaller, or matching a larger quantization parameter to the spatial scaling parameter in response to the spatial scaling parameter being larger.

5. The image compression method of claim 3, wherein the perceptual distortion loss function is configured to be determined based on a weighted loss of salient regions and non-key regions after image compression.

6. The image compression method of claim 5, wherein the expression of the perceptual distortion loss function comprises: L perceptual = w s × SSIM s + w n × (1 - SSIM n ), wherein the L perceptual is a calculation result of the perceptual distortion loss function, the SSIM s is a structural similarity value of the salient region, the SSIM n is a structural similarity value of the non-key region, the w s is the w n is a preset weight coefficient, and the w s > the w n .

7. The image compression method of claim 3, wherein the joint loss function of the joint parameter decision model is configured to be determined based on a weighted sum of a code rate deviation and a perceptual distortion loss function.

8. The image compression method of claim 7, wherein the expression of the joint loss function comprises: L total = λ x L compare + (1 - λ) x Lperceptual, Wherein, the L total is the calculation result of the target loss function, the L compare is the calculation result of the code rate bias, the L perceptual is the calculation result of the perceptual distortion loss function, and the λ is a preset hyperparameter for balancing the code rate control and the perceptual quality after image compression.

9. The image compression method of claim 7, wherein the expression of the code rate deviation comprises: L compare = λ1 x MSE (a, a ) + λ2 x MSE (QP, QP ), The α is a predicted spatial scaling parameter, the QP is a predicted quantization parameter, the α is a spatial scaling parameter in the label, the QP is a quantization parameter in the label, and the λ1 and the λ2 are preset weight coefficients.

10. An image compression apparatus characterized by comprising: ​ a parameter obtaining module configured to obtain an input image and a target code rate; a feature extraction module configured to input the input image into a content feature analysis model to obtain a feature vector of the input image, the feature vector being capable of reflecting perceptual characteristics of the input image; an image compression module configured to input the target code rate and the feature vector into a joint parameter decision model to generate spatial scaling parameters and quantization parameters adapted to the input image and the target code rate; wherein the joint parameter decision model is trained with a joint loss function as a target, the joint loss function is determined based on a code rate bias and a perceptual distortion loss function, the spatial scaling parameters are parameters for spatial scaling of the input image, the quantization parameters are parameters for encoding and compression of the spatially scaled image.

11. An electronic device, comprising: comprising: a memory; a processor coupled to the memory, the processor configured to execute instructions stored in the memory to perform the image compression method of any one of claims 1-9.

12. A computer readable storage medium having stored thereon a program which, when executed by a processor, performs the image compression method of any one of claims 1-9. comprising a computer program which, when executed by a processor, performs the image compression method of any one of claims 1-9.

13. A computer program product, characterised in that ​

Images