A multi-scale semantically guided image compression method, system and storage medium

Abstract

This invention discloses a multi-scale semantically guided image compression method, system, and storage medium, comprising the following steps: acquiring input image data; preprocessing the input image to obtain standardized image data; inputting the standardized image data into a pre-trained semantic segmentation network to generate multi-scale semantic feature maps and their corresponding semantic weight maps; constructing a three-level pyramid encoder to perform the following steps on the standardized image data: depthwise separable convolutional downsampling to generate multi-scale features; performing nonlinear transformation on the multi-scale features using an invertible neural network; decomposing the nonlinearly transformed multi-scale features into low-frequency and high-frequency sub-bands using an adaptive discrete wavelet transform; performing dynamic selective state-space modeling on the high-frequency sub-bands based on the semantic weight maps to generate a compressed bitstream; inputting the compressed bitstream into a decoder; decoding based on a lightweight Mamba module; and reconstructing the image by combining inverse wavelet transform and the semantic weight maps.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a multi-scale semantically guided image compression method, system, and storage medium. Background Technology

[0002] In recent years, deep learning-based image compression methods have gradually replaced traditional hand-designed compression algorithms. However, existing compression models based on attention mechanisms or convolutional structures still face challenges in handling long-range dependencies and preserving semantic regions. Especially in low bit-rate scenarios, the perceptual reconstruction quality of important regions (such as faces and text) is difficult to guarantee, leading to blurring of key regions and semantic distortion. Existing deep learning-based models struggle to balance long-range dependency modeling with computational efficiency. Summary of the Invention

[0003] This invention aims to address at least one of the technical problems existing in the prior art. To this end, this invention proposes a multi-scale semantically guided image compression method, system, and storage medium, which can solve the problems of low modeling efficiency and severe compression distortion faced by current image compression algorithms in efficiently modeling long-distance dependencies, maintaining image quality in key semantic regions, and achieving high-resolution image compression.

[0004] A multi-scale semantically guided image compression method according to a first aspect of the present invention includes the following steps:

[0005] S100: Acquire input image data, perform preprocessing operations on the input image to obtain standardized image data. ;

[0006] S200, the standardized image data Input the pre-trained semantic segmentation network to generate multi-scale semantic feature maps. and its corresponding semantic weight graph ;

[0007] S300. Construct a three-level pyramid encoder for the standardized image data. Execution step by step:

[0008] S301, Depthwise separable convolutional downsampling, generating multi-scale features. ;

[0009] S302, Reversible neural network for the multi-scale features Perform nonlinear transformation;

[0010] S303, Adaptive discrete wavelet transform transforms the multi-scale features after nonlinear transformation. Decomposed into low-frequency subbands With high-frequency subband ;

[0011] S400, Based on the semantic weight graph For the high-frequency subband Perform dynamic selective state-space modeling, including:

[0012] The bidirectional scanning mechanism updates the status;

[0013] Dynamic convolution kernel generation;

[0014] Channel-space dual-path attention gating enhancement feature;

[0015] S500: Perform non-uniform quantization and entropy coding on the features after modeling each layer in step S400 to generate a compressed bitstream;

[0016] S600. Input the compressed bitstream into the decoder, decode it based on the lightweight Mamba module, and combine inverse wavelet transform and semantic weight graph. Reconstruct the image.

[0017] According to some embodiments of the present invention, in step S200, the standardized image data is... Input the pre-trained semantic segmentation network to generate multi-scale semantic feature maps. and its corresponding semantic weight graph include:

[0018] Multi-scale features were extracted using the Xception65 backbone of the DeepLabv3+ network, and then fused using a dilated spatial pyramid pooling module to generate a semantic feature map. The semantic weight map is generated by aligning the encoding levels using bilinear interpolation and taking the maximum value along the channel dimension. .

[0019] According to some embodiments of the present invention, in step S300, the depth-separable convolutional downsampling generates multi-scale features. include:

[0020] Original image scale characteristics Half-scale features Quarter-scale features ;

[0021] Where: R is the real number space; H is the feature map height; W is the feature map width.

[0022] According to some embodiments of the present invention, in step S300, the reversible neural network processes the multi-scale features. Nonlinear transformations include:

[0023] The reversible neural network performs a forward transformation:

[0024] ;

[0025] ;

[0026] ;

[0027] The reversible neural network performs the inverse transformation:

[0028] ;

[0029] ;

[0030] ;

[0031] in:

[0032] , Multi-scale features Two parts divided along the channel dimension;

[0033] For output features;

[0034] F and G are three-layer convolutional residual blocks;

[0035] T This represents a forward mapping function;

[0036] This represents the inverse mapping function.

[0037] According to some embodiments of the present invention, in step S400, the bidirectional scanning mechanism includes:

[0038] Forward state update equation:

[0039] ;

[0040] Backward state update equation:

[0041] ;

[0042] in:

[0043] 、 These represent the forward and backward states, respectively.

[0044] ∈R C×1 For state memory weights;

[0045] ∈R C×C The input is the gating weight matrix;

[0046] Depth-separable convolution operations are used to dynamically generate convolution kernels;

[0047] It is an element-wise product.

[0048] According to some embodiments of the present invention, in step S400, the dynamic convolution kernel generation includes:

[0049] Based on the semantic weight graph With the high-frequency subband Generate query Q, key K, and value V, and compute the convolution kernel parameter matrix using multi-head attention. :

[0050] ;

[0051] Where: d is the dimension of the attention head; Softmax is the normalized exponential function; T is the matrix transpose.

[0052] According to some embodiments of the present invention, in S400, the channel-space dual-path attention gating enhancement feature includes:

[0053] Channel attention:

[0054] ;

[0055] Spatial attention:

[0056] ;

[0057] Final output features:

[0058] ;

[0059] in:

[0060] X represents the input feature;

[0061] GAP stands for Global Average Pooling;

[0062] MLP stands for Multilayer Perceptron;

[0063] for Activation function;

[0064] ⊙ represents element-wise product.

[0065] According to some embodiments of the present invention, in S500, the non-uniform quantization measurement includes:

[0066] The quantization step size is adaptively adjusted based on the semantic weight graph:

[0067] ;

[0068] in:

[0069] Based on the quantization step size;

[0070] Position in the semantic weight graph The response intensity;

[0071] λ is the adjustment coefficient;

[0072] Quantization operation is defined as: ;

[0073] in: Indicates location , Eigenvalues; Z i，j The feature values ​​to be quantized are from the output of dynamic SSM modeling; This is the rounding function; The adaptive quantization step size is dynamically adjusted by semantic weights.

[0074] According to some embodiments of the present invention, in S500, the entropy encoding includes:

[0075] Based on the aforementioned state-space modeled hyperprior network, a joint probability model is constructed for the latent variables:

[0076] ;

[0077] in:

[0078] N represents a Gaussian distribution;

[0079] They represent the feature dimensions respectively. , The predicted mean and standard deviation of dimension are generated by the super-prior network:

[0080] ; ;

[0081] in:

[0082] This is a lightweight neural network module based on state-space modeling, used to estimate parameters at each feature location. ;

[0083] This is a lightweight neural network module based on state-space modeling, used to estimate parameters at each feature location. ;

[0084] Indicates a channel splicing operation;

[0085] These are decoded features.

[0086] According to some embodiments of the present invention, in step S600, the compressed bitstream is input to the decoder for decoding based on a lightweight Mamba module, combined with inverse wavelet transform and semantic weight graph. Reconstructing the image involves the following steps:

[0087] Mamba module decoding status update:

[0088] ;

[0089] in:

[0090] This is the current state vector;

[0091] A B State transition coefficients generated for semantic guidance;

[0092] For gated functions;

[0093] Inverse wavelet transform reconstructed features:

[0094] ;

[0095] in:

[0096] For the first k Layer reconstruction feature map;

[0097] For learnable inverse wavelet transform operators;

[0098] Multi-scale semantic fusion output image:

[0099] ;

[0100] in:

[0101] For learnable fusion weights.

[0102] According to some embodiments of the present invention, S500 further includes median deviation mapping quantization encoding, comprising the following steps:

[0103] latent eigenvalues Map to the median reference coordinate system and calculate its corresponding interval. median And calculate the deviation value:

[0104] ;

[0105] Symmetric discretization quantization of the deviation:

[0106] ;

[0107] Where: γ is the quantization step size;

[0108] Generating ternary Compress the representation;

[0109] in: For spatial location index, This is for quantification bias.

[0110] A multi-scale semantically guided image compression system according to a second aspect of the present invention includes a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the multi-scale semantically guided image compression method.

[0111] According to a third aspect of the present invention, a storage medium stores a determination program for a multi-scale semantically guided image compression method, wherein the determination program, when executed by a processor, implements the multi-scale semantically guided image compression method.

[0112] According to an embodiment of the present invention, a multi-scale semantically guided image compression method, system, and storage medium have at least the following beneficial effects: The present invention introduces a dynamic selective state-space modeling mechanism, combined with bidirectional scanning and semantically sensitive attention gating, which effectively reduces computational complexity while enhancing the ability to capture details and understand the global context of key image regions such as faces and text, thus alleviating the image detail blurring problem caused by high-frequency information loss in traditional compression methods. Through a multi-scale-wavelet joint coding architecture constructed by fusing reversible neural networks and adaptive wavelet transform, lossless compression of low-frequency subbands is effectively achieved, avoiding low-frequency distortion; simultaneously, high-frequency subbands retain texture details through lightweight dynamic convolutional coding, meeting the real-time processing requirements of edge devices. At the decoding end, a selective state-space activation mechanism based on the Mamba module decoding structure is introduced, retaining only key channels for image reconstruction, significantly reducing the decoding computation, and further reducing computational complexity through inverse wavelet transform and semantic weight graphs. The output image is then fused and reconstructed.

[0113] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0114] The present invention will be further described below with reference to the accompanying drawings and embodiments, wherein:

[0115] Figure 1 This is a schematic diagram of the computer device structure of the hardware operating environment according to an embodiment of the present invention;

[0116] Figure 2 This is a flowchart of a multi-scale semantically guided image compression method according to an embodiment of the present invention;

[0117] Figure 3 This is a schematic diagram of the overall network structure of a multi-scale semantically guided image compression method according to an embodiment of the present invention;

[0118] Figure 4 This is a structural diagram of the ASPP module of a multi-scale semantically guided image compression method according to an embodiment of the present invention;

[0119] Figure 5 This is a structural diagram of the multi-scale wavelet coding module of a multi-scale semantic-guided image compression method according to an embodiment of the present invention;

[0120] Figure 6 This is a structural diagram of the dynamic SSM module of a multi-scale semantically guided image compression method according to an embodiment of the present invention;

[0121] Figure 7 This is a median deviation mapping quantization coding strategy diagram of a multi-scale semantically guided image compression method according to an embodiment of the present invention;

[0122] Figure 8 This is a structural diagram of the decoding module of a multi-scale semantically guided image compression method according to an embodiment of the present invention;

[0123] Figure 9 This is a structural block diagram of a multi-scale semantically guided image compression system according to an embodiment of the present invention.

[0124] Figure Number:

[0125] Processor 1001; Communication bus 1002; User interface 1003; Network interface 1004; Memory 1005. Detailed Implementation

[0126] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0127] In the description of the present invention, it should be understood that descriptions involving orientation, such as the orientation or positional relationship indicated by up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0128] In the description of this invention, "multiple" refers to two or more. The use of "first" and "second" is for distinguishing technical features only and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features or their sequential relationship.

[0129] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0130] Reference Figure 1 , Figure 1 This is a schematic diagram of the computer device structure of the hardware operating environment involved in the embodiments of this application.

[0131] like Figure 1 As shown, the computer device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0132] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0133] like Figure 1As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a face detection program for edge computing.

[0134] exist Figure 1 In the computer device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in this application can be set in the computer device. The computer device calls the multi-scale semantic guided image compression program based on dynamic state space modeling stored in the memory 1005 through the processor 1001, and executes the multi-scale semantic guided image compression method based on dynamic state space modeling provided in the embodiment of this application.

[0135] Reference Figure 2 As shown, this invention discloses a multi-scale semantically guided image compression method, comprising the following steps:

[0136] S100. Acquire input image data, perform preprocessing operations on the input image to obtain standardized image data. ;

[0137] S200, Standardize image data Input the pre-trained semantic segmentation network to generate multi-scale semantic feature maps. and its corresponding semantic weight graph ;

[0138] S300, constructing a three-level pyramid encoder for standardized image data. Execution step by step:

[0139] S301, Depthwise separable convolutional downsampling, generates multi-scale features. ;

[0140] S302, Reversible Neural Networks for Multi-Scale Features Perform nonlinear transformation;

[0141] S303, Adaptive Discrete Wavelet Transform transforms multi-scale features after nonlinear transformation. Decomposed into low-frequency subbands With high-frequency subband ;

[0142] S400, based on semantic weight graph For high-frequency subband Perform dynamic selective state-space modeling, including:

[0143] The bidirectional scanning mechanism updates the status;

[0144] Dynamic convolution kernel generation;

[0145] Channel-space dual-path attention gating enhancement feature;

[0146] S500: Perform non-uniform quantization and entropy coding on the features after modeling each layer in step S400 to generate a compressed bitstream;

[0147] S600 inputs the compressed bitstream into the decoder, decodes it based on the lightweight Mamba module, and combines inverse wavelet transform and semantic weight graph. Reconstruct the image.

[0148] In this embodiment, the input RGB image is converted to YUV color space, normalized ([0, 255] → [-1, 1]), and anomaly filtered to generate a standardized image. Semantic segmentation network (DeepLabv3+) extracts multi-scale feature maps. A semantic weight graph is generated by taking the maximum value along the channel. The encoder employs a three-level pyramid structure:

[0149] Depth-separable convolution downsampling (stride=2);

[0150] INN blocks perform coupling transformations: , ;

[0151] Adaptive discrete wavelet transform converts multi-scale features that have undergone nonlinear transformation. Decomposed into low-frequency subbands With high-frequency subband .

[0152] By employing a dynamic SSM coding module, dynamic convolution and attention mechanisms are integrated into the state-space model to perform state-space modeling and dynamic encoding of image data, enhancing the ability to model key regions. The dynamic SSM coding module also supports high-frequency subbands. Semantic-guided state-space modeling is performed, entropy coding employs a non-uniform quantization strategy, and the decoding end reconstructs the image using a lightweight Mamba module and inverse wavelet transform (IDWT). Semantic weight graph. As a spatial importance prior, the dynamic SSM encoding module models, allocates quantization step size, and activates decoding paths to achieve on-demand resource allocation.

[0153] In some embodiments of the present invention, in step S100, input image data is acquired through an image acquisition module, and color space conversion, normalization, and abnormal image filtering are performed to generate standardized image data. The steps for generating standardized image data include: converting the RGB image to the YUV color space using a standard transformation matrix, separating the luminance (Y) and chrominance (UV) components, retaining the Y channel for subsequent compression, and resampling the UV components to reduce data volume; linearly normalizing the pixel values ​​of each YUV channel while retaining the sign bit to support subsequent wavelet transform negative value calculations; calculating the image sharpness score using the Laplacian operator, statistically analyzing the proportion of pixel values ​​exceeding a threshold in the luminance channel, identifying and filtering abnormal images; and finally outputting the preprocessed standardized image data. ∈R H×W×3 Its dimensions are consistent with the original input.

[0154] In step S200, the standardized image data is... Input the pre-trained semantic segmentation network to generate multi-scale semantic feature maps. and its corresponding semantic weight graph include:

[0155] Multi-scale features were extracted using the Xception65 backbone of the DeepLabv3+ network, and then fused using a dilated spatial pyramid pooling module to generate a semantic feature map. Furthermore, the encoding levels are aligned using bilinear interpolation, and the semantic weight map is generated by taking the maximum value along the channel dimension. .

[0156] In this embodiment, a pre-trained network is used to extract multi-scale semantic features, and a semantic weight map is generated through dilated spatial pyramid pooling for semantic guidance in subsequent encoding.

[0157] Specifically, a pre-trained network is used to extract multi-scale semantic features, and a semantic weight map is generated through dilated spatial pyramid pooling.

[0158] It should be noted that a pre-trained DeepLabv3+ model was used, with its backbone network being Xception65. Standardized image data. Low-level features are obtained by sequentially extracting features through convolutional layers in DeepLabv3+. Intermediate features and advanced features , where C1=256, C2=512, C3=1024.

[0159] It should be noted that multi-scale contextual modeling incorporates high-level features. Enter the ASPP module, such as Figure 4 As shown, the process is handled through five parallel branches:

[0160] Branch 1: 1×1 standard convolution, preserving the spatial dimensions;

[0161] Branches 2-4: 3×3 dilated convolutions with dilation rates of 6, 12, and 18, respectively;

[0162] Branch 5: After global average pooling, upsample to the original spatial dimension, and restore the channel dimension through 1×1 convolution.

[0163] The outputs of the five branches are concatenated along the channel dimension and then fused using a 1×1 convolution to obtain a multi-scale feature fusion map. .

[0164] Semantic segmentation graph generation: F ASPP Input 3×3 convolutions and Softmax layers to generate semantic prediction maps , where Cs represents the number of semantic categories.

[0165] S is upsampled to align with the input resolution of each level of the encoder by bilinear interpolation, generating a three-level semantic feature map {S0, S1, S2}, which corresponds to the original resolution, 1 / 2 resolution, and 1 / 4 resolution, respectively.

[0166] Subsequently, multi-scale semantic feature maps are generated for each. The weighted graph is obtained by taking the maximum value along the channel dimension:

[0167] ;

[0168] The semantic weight graph It will be used for dynamic attention generation and important region guidance during the encoding and compression stages.

[0169] In some embodiments of the present invention, in S300, depthwise separable convolutional downsampling generates multi-scale features. include:

[0170] Original image scale characteristics Half-scale features Quarter-scale features .

[0171] In this embodiment, a three-level pyramid structure is constructed by using a multi-scale-wavelet joint coding module and employing depthwise separable convolution with a stride of 2 for stepwise downsampling. Then, adaptive wavelet transform is performed on the feature maps at each scale to decompose the image features into low-frequency subbands. With high-frequency subband And lossless compression is achieved through a reversible neural network.

[0172] Specifically, the pyramid structure encoder performs the following operations at each level: Figure 5As shown:

[0173] Initial downsampling and feature extraction: standardizing image data Inputting into a three-layer separable convolutional network generates three levels of feature representations:

[0174] Level 1: Input standardized image data Output ;

[0175] Second level, input Output ;

[0176] Level 3, Input Output ;

[0177] The depthwise separable convolution operation is defined as follows:

[0178] ;

[0179] in, Depend on Convolution and Convolutional concatenation can significantly reduce computational overhead and is suitable for edge computing devices.

[0180] Reversible Neural Network Transformation (INN): For each level of features Decompose it into and Two parts, performing the following forward mapping of the coupling layer:

[0181] ;

[0182] ;

[0183] ;

[0184] Output = ( , ), where F and G are three-layer convolutional residual blocks, each containing BatchNorm (batch normalization), LeakyReLU (linear unit with leakage correction) and 3×3 convolution.

[0185] Adaptive wavelet decomposition: INN output features Input a custom wavelet decomposition module and execute a Learnable Discrete Wavelet Transform (DWT):

[0186] ;

[0187] in:

[0188] Low-frequency subband, preserving the main structure information of the image;

[0189] High-frequency subbands retain texture and edge details and are further processed by the state space modeling module.

[0190] The high-frequency subband size and number of channels are: The three channels correspond to the horizontal, vertical, and diagonal detail directions in the wavelet transform.

[0191] It should be noted that the invertible neural network INN supports exact inverse transformation, and its inverse function is defined as:

[0192] ;

[0193] ;

[0194] ;

[0195] This represents the reverse mapping function, ensuring the output of the encoder. It can be recovered through INN during the decoding stage, meeting the lossless compression requirements of low-frequency subbands.

[0196] In some embodiments of the present invention, in S400, the bidirectional scanning mechanism includes:

[0197] Forward state update equation:

[0198] ;

[0199] Backward state update equation:

[0200] ;

[0201] in:

[0202] 、 These represent the forward and backward states, respectively.

[0203] ∈R C×1 For state memory weights;

[0204] ∈R C×C The input is the gating weight matrix;

[0205] Depth-separable convolution operations are used to dynamically generate convolution kernels;

[0206] It is an element-wise product.

[0207] In this embodiment, a dynamic SSM encoding module is used to integrate dynamic convolution and attention mechanisms into the state space model to perform state space modeling and dynamic encoding of image data, thereby enhancing the ability to model key regions.

[0208] A bidirectional scanning mechanism and a semantic weight graph-driven dynamic convolutional kernel are introduced, and key region modeling is enhanced through channel-space dual-path attention gating, reducing computational complexity. Specifically, this includes:

[0209] like Figure 6 As shown, the implementation of the dynamic SSM module includes:

[0210] Receive high-frequency subband features from step S300 and the multi-scale semantic weight map generated in step S200 A Dynamic Selective State Space Modeling (SSM) module is constructed to achieve dynamic modeling and compressed representation of features in high semantic regions.

[0211] It should be noted that semantic alignment processing includes: first, using the bilinear interpolation function resize. semantic weight graph Spatial dimensions adjusted to correspond to high-frequency characteristics at the corresponding scale Consistency:

[0212] ;

[0213] in: The aligned semantic weight graph; resize This is a function for adjusting spatial dimensions.

[0214] This operation ensures that the semantic guidance corresponds one-to-one with the feature space, thus enhancing the guidance accuracy.

[0215] The two-way state update mechanism includes:

[0216] For feature modeling of time series data, a bidirectional state space structure is introduced, including forward state updates and backward state updates. The specific update formula is as follows:

[0217] The forward state update equation is:

[0218] ;

[0219] The backward state update equation is:

[0220] ;

[0221] in:

[0222] 、 These represent the forward and backward states, respectively. ∈R C×1 The state memory weights are influenced by the multi-scale semantic weight graph. guide, ∈R C×C For the input gate weight matrix, Dynamically generated convolution kernel depth can separate convolution operations. It is an element-wise product.

[0223] In some embodiments of the present invention, in S400, the dynamic convolution kernel generation includes:

[0224] Based on semantic weight graph With high-frequency subband Generate query Q, key K, and value V, and compute the convolution kernel parameter matrix using multi-head attention. :

[0225] ;

[0226] Where d is the dimension of the attention head; Softmax is the normalized exponential function; T is the matrix transpose.

[0227] In this embodiment, the dynamic convolution kernel generation mechanism is as follows:

[0228] The weights of the dynamic convolution kernel are generated based on an attention mechanism. First, a 1×1 convolution is performed on the semantic map and high-frequency features respectively to obtain the query Q, key K, and value V:

[0229] ;

[0230] ;

[0231] ;

[0232] The dynamic convolution kernel parameter matrix is ​​calculated based on the standard multi-head attention mechanism. :

[0233] ;

[0234] Where d is the dimension of the attention head, used for normalization to prevent gradient explosion.

[0235] In some embodiments of the present invention, in step S400, the channel-space dual-path attention gating enhancement feature includes:

[0236] Channel attention:

[0237] ;

[0238] Spatial attention:

[0239] ;

[0240] Final output features:

[0241] ;

[0242] in:

[0243] X represents the input feature;

[0244] GAP stands for Global Average Pooling;

[0245] MLP stands for Multilayer Perceptron;

[0246] for Activation function;

[0247] ⊙ represents element-wise product.

[0248] Specifically, the channel-space dual attention gating mechanism includes:

[0249] Attention gating enhancement is applied to the features X generated by the dynamic convolution kernel, which consists of two stages: channel attention (CA) and spatial attention (SA).

[0250] The formula for calculating channel attention is: ;

[0251] The formula for calculating spatial attention is: ;

[0252] The final output features are dual-path enhanced features:

[0253] ;

[0254] In some embodiments of the present invention, in step S500, the non-uniform quantization measurement includes:

[0255] The quantization step size is adaptively adjusted based on the semantic weight graph:

[0256] ;

[0257] in:

[0258] Based on the quantization step size;

[0259] Position in the semantic weight graph The response intensity;

[0260] λ is an adjustment coefficient used to enhance the resolution of high semantic regions;

[0261] Quantization operation is defined as: ;

[0262] in: Indicates location , Eigenvalues; Z i，j The feature values ​​to be quantized are from the output of dynamic SSM modeling; This is the rounding function; The adaptive quantization step size is dynamically adjusted by semantic weights.

[0263] In this embodiment, the semantically guided non-uniform quantization strategy and joint probability modeling process include: receiving dynamic features Z= from the S400 output. Quantization and modeling are performed to achieve a balance between compression rate and fidelity.

[0264] The adaptive quantization step size at each position is calculated using the following formula. :

[0265] ;

[0266] Quantization operation is defined as:

[0267] ;

[0268] Quantized features It is an entropy-encoded compressed representation. This strategy ensures that semantically important regions (such as faces and text) are encoded more finely, while background regions can be processed more coarsely to save bit rate.

[0269] In some embodiments of the present invention, in S500, entropy coding includes:

[0270] Based on a state-space modeling-based super-prior network, a joint probability model is constructed for the latent variables:

[0271] ;

[0272] in:

[0273] N represents a Gaussian distribution;

[0274] They represent the feature dimensions respectively. , The predicted mean and standard deviation of dimension are generated by the super-prior network:

[0275] ; ;

[0276] in:

[0277] This is a lightweight neural network module based on state-space modeling, used to estimate parameters at each feature location. ;

[0278] This is a lightweight neural network module based on state-space modeling, used to estimate parameters at each feature location. ;

[0279] Indicates a channel splicing operation;

[0280] These are decoded features.

[0281] In this embodiment, to improve compression efficiency, the joint probability distribution of the features is modeled. The construction format is as follows:

[0282] ;

[0283] The advanced prior network employs a multi-scale feature fusion approach, integrating decoded features... semantic feature maps at corresponding scales After concatenation, the data is input into the state-space modeling module (Mamba) for parameter prediction, specifically:

[0284] ;

[0285] ⊕ indicates channel splicing operation, and the exponential function is used to ensure that the standard deviation of the prediction is positive.

[0286] In some embodiments of the present invention, in S600, the compressed bitstream is input to the decoder, decoded based on the lightweight Mamba module, and combined with inverse wavelet transform and semantic weight graph. Reconstructing the image involves the following steps:

[0287] Mamba module decoding status update:

[0288] ;

[0289] in:

[0290] This is the current state vector;

[0291] A B State transition coefficients generated for semantic guidance;

[0292] For gated functions;

[0293] Inverse wavelet transform reconstructed features:

[0294] ;

[0295] in:

[0296] For learnable inverse wavelet transform operators;

[0297] Multi-scale semantic fusion output image:

[0298] ;

[0299] in:

[0300] For learnable fusion weights.

[0301] In specific implementation, such as Figure 8 As shown, the semantically guided image reconstruction process based on the Mamba decoder includes:

[0302] The bitstream representation output during the receive encoding stage Combined with semantic weight graph and wavelet subband ( The original image is reconstructed using the Mamba decoder and inverse wavelet transform module.

[0303] The Mamba decoder is based on state-space modeling. It uses a lightweight state-space mechanism and semantically guided selective activation channels for state updates. The calculation process is as follows: The state update equation is:

[0304] ;

[0305] in: ∈ R C This indicates the current channel status. State transition coefficients generated for semantic guidance;

[0306] This is a gating function that suppresses invalid feature channels.

[0307] The inverse wavelet reconstruction module receives the output features of the Mamba decoder. Combined with low-frequency subband and high frequency sub-band Perform inverse wavelet transform:

[0308] ;

[0309] in The learnable inverse wavelet transform operator is used to reconstruct the feature map of the current scale image.

[0310] Multi-scale fusion and semantic-guided reconstruction: The decoding results of three scales are weighted and fused with the semantic map to generate the final reconstructed image. :

[0311] ;

[0312] in: ∈R represents the fusion weight; This is the semantic weight graph at the corresponding scale.

[0313] The fusion strategy gives greater influence to semantically high-weight regions, ensuring that key structures (such as facial contours and text edges) are more completely preserved in image reconstruction.

[0314] In practice, the loss function used is as follows:

[0315] To effectively improve the semantic region restoration quality, structural fidelity, and visual consistency during image compression, the reconstructed image is subjected to [further processing / training] during the model training phase. With the original image Multiple loss functions are introduced to construct a composite loss objective function. The loss function includes semantic edge loss, structure preservation loss, and a traditional reconstruction loss term. Specifically:

[0316] Pixel-level reconstruction loss (MSE):

[0317] This loss is used to measure the pixel-level error between the reconstructed image and the original image, and is defined as:

[0318] ;

[0319] Where N is the total number of pixels in the image. and Pixels in the reconstructed image and the original image, respectively. The value of .

[0320] Structural Preservation Loss (SSIM Loss):

[0321] The Structural Similarity Index (SSIM) is used to measure image structural consistency, and is defined as follows:

[0322] ;

[0323] This loss term primarily constrains the brightness, contrast, and structural information of the image, ensuring that the reconstructed image remains perceptually similar to the original image.

[0324] Considering that the edges of semantic regions often carry important structural information, a semantic edge-guided loss is introduced:

[0325] ;

[0326] in:

[0327] M Normalization factor;

[0328] k For pyramid-level indexing;

[0329] These are pixel space coordinates;

[0330] The gradient values ​​for reconstructing the image;

[0331] The gradient value of the original image;

[0332] This represents the high-response semantic region at level k;

[0333] ∈[0,1] represents the guiding strength of this position in the semantic weight graph;

[0334] This loss term emphasizes the fidelity of semantic region edge reconstruction.

[0335] Compression rate targets are considered concurrently during training, and bitrate control terms are added:

[0336] ;

[0337] in:

[0338] For quantitative characteristics Expected value;

[0339] For quantification features The probability value;

[0340] This is the information content function.

[0341] This loss term measures the average number of bits in the compressed code after encoding and is derived from the joint probability model established in step S500.

[0342] Understandably, the total loss function can be combined in the following form:

[0343] ;

[0344] in:

[0345] λ 1~ λ4 is the loss term balancing coefficient, which is tuned during training using cross-validation. Typically, λ3 is set to be greater than λ1 to highlight the reconstruction accuracy of semantic edge regions. For real-time compression applications, λ4 can be appropriately increased to control the overall bitrate.

[0346] By employing the aforementioned multi-loss joint optimization strategy, the structural preservation capability and reconstruction quality of the image compression system in semantically salient regions (such as faces, text, and object contours) can be effectively improved, while also taking into account the overall image compression efficiency. This approach is suitable for various scenarios requiring "high compression ratio and high-fidelity reconstruction".

[0347] In some embodiments of the present invention, step S500 further includes median deviation mapping quantization encoding, comprising the following steps:

[0348] latent eigenvalues Map to the median reference coordinate system and calculate its corresponding interval. median And calculate the deviation value:

[0349] ;

[0350] in: These are the original eigenvalues;

[0351] Symmetric discretization quantization of the deviation:

[0352] ;

[0353] Where: γ is the quantization step size;

[0354] Generating ternary Compress the representation;

[0355] in For spatial location index, This is for quantification bias.

[0356] This distributed model is used in the encoder to optimize bit rate allocation and in the decoder to provide prior guidance for reconstruction.

[0357] In some embodiments of the present invention, a discrete coding mechanism based on deviation representation is further introduced on the basis of the non-uniform quantization module, such as... Figure 7 As shown. First, the potential feature value or pixel value range [0, 255] is divided into N intervals, each interval is defined as:

[0358] ;

[0359] Each interval is assigned a median value. As a reference value for reconstruction. For each encoded feature point By finding its corresponding interval And calculate its relative median deviation:

[0360] ;

[0361] Symmetric discretization quantization of the deviation:

[0362] ;

[0363] Ultimately The triples are used for encoding, where For spatial location index, To quantize the bias, entropy coding or variable-length coding can be used to compress the triplet data to achieve a more efficient bitstream representation.

[0364] This invention introduces a dynamic selective state-space modeling mechanism, combined with bidirectional scanning and semantically sensitive attention gating, to effectively reduce computational complexity while enhancing the ability to capture details and understand the global context of key image regions such as faces and text. This alleviates the image detail blurring problem caused by the loss of high-frequency information in traditional compression methods. Furthermore, based on a semantically guided non-uniform quantization strategy and a median deviation mapping quantization coding mechanism, the quantization step size can be dynamically adjusted according to the semantic weight map, and three-dimensional coordinate encoding can be performed using spatial location and deviation information. This allows for the allocation of finer quantization levels to high semantic value regions, optimizing the balance between compression efficiency and reconstruction quality. A multi-scale-wavelet joint coding architecture, constructed by fusing reversible neural networks and adaptive wavelet transform, effectively achieves lossless compression of low-frequency subbands, avoiding low-frequency distortion. Simultaneously, high-frequency subbands retain texture details through lightweight dynamic convolutional coding, meeting the real-time processing requirements of edge devices. At the decoding end, a selective state-space activation mechanism based on the Mamba structure is introduced, retaining only key channels for image reconstruction, significantly reducing decoding computation. The output image is reconstructed through inverse wavelet transform and multi-scale semantic fusion. These technological breakthroughs make this application of significant practical value and broad application prospects in application scenarios that require a balance between compression rate and visual fidelity, such as security monitoring, mobile communication, and medical imaging.

[0365] refer to Figure 9 As shown, the present invention also discloses a multi-scale semantically guided image compression system, including a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the multi-scale semantically guided image compression method.

[0366] Furthermore, to achieve end-to-end deployment of the image compression system, at the system architecture level, the image compression system proposed in this invention also includes the following modules:

[0367] Image preprocessing module: acquires the original image, performs color space conversion, normalization, and outlier removal, and outputs a standardized image.

[0368] Semantic guidance module: Utilizes the DeepLabv3+ semantic segmentation model to extract semantic features and obtains a multi-scale semantic weight map through bilinear interpolation to guide subsequent encoding.

[0369] Multi-scale wavelet coding module: Constructs a three-level coding pyramid, each level consisting of depthwise separable convolution, invertible neural network and adaptive wavelet transform, outputting low-frequency subband and high-frequency subband.

[0370] Dynamic SSM modeling module: It integrates semantic graph guidance and bidirectional scanning mechanism to realize state space modeling and output dynamic convolutional encoded features.

[0371] Entropy coding module: It uses a super-prior network to extract contextual features, and combines the semantic weight graph to estimate the probability distribution, and performs non-uniform quantization and median deviation mapping quantization coding strategies to generate the bitstream.

[0372] Decoding and Reconstruction Module: Combining Mamba state updates and semantic gating mechanisms, the module generates a reconstructed image through inverse wavelet transform and semantic weighted fusion.

[0373] The present invention also discloses a storage medium storing a determination program for a multi-scale semantically guided image compression method, which, when executed by a processor, implements the multi-scale semantically guided image compression method.

[0374] Since the multi-scale semantic-guided image compression system and storage medium adopt all the technical solutions of the multi-scale semantic-guided image compression method in the above embodiments, they have at least all the beneficial effects brought about by the technical solutions in the above embodiments, which will not be repeated here.

[0375] The embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in the relevant technical field without departing from the scope of the present invention.

Claims

1. A multi-scale semantically guided image compression method, characterized in that, Includes the following steps: S100: Acquire input image data, perform preprocessing operations on the input image to obtain standardized image data. I ; S200, the standardized image data I Input the pre-trained semantic segmentation network to generate multi-scale semantic feature maps. and its corresponding semantic weight graph ; S300. Construct a three-level pyramid encoder for the standardized image data. I Execution step by step: S301, Depthwise separable convolutional downsampling, generating multi-scale features. x k ; S302, Reversible neural network for the multi-scale features x k Perform nonlinear transformation; S303, Adaptive discrete wavelet transform transforms the multi-scale features after nonlinear transformation. x k Decomposed into low-frequency subbands With high-frequency subband ; S400, Based on the semantic weight graph For the high-frequency subband Perform dynamic selective state-space modeling, including: The bidirectional scanning mechanism updates the status; Dynamic convolution kernel generation; Channel-space dual-path attention gating enhancement feature; S500: Perform non-uniform quantization and entropy coding on the features after modeling each layer in step S400 to generate a compressed bitstream; S600. Input the compressed bitstream into the decoder, decode it based on the lightweight Mamba module, and combine inverse wavelet transform and semantic weight graph. Reconstruct the image; In step S200, the standardized image data is... I Input the pre-trained semantic segmentation network to generate multi-scale semantic feature maps. and its corresponding semantic weight graph include: Multi-scale features were extracted using the Xception65 backbone of the DeepLabv3+ network, and then fused using a dilated spatial pyramid pooling module to generate a semantic feature map. The semantic weight map is generated by aligning the encoding levels using bilinear interpolation and taking the maximum value along the channel dimension. .

2. The multi-scale semantically guided image compression method according to claim 1, characterized in that, In step S300, the depth-separable convolutional downsampling generates multi-scale features. x k include: Original image scale characteristics Half-scale features Quarter-scale features ; Where: R is the real number space; H is the feature map height; W is the feature map width.

3. The multi-scale semantically guided image compression method according to claim 1, characterized in that, In step S300, the reversible neural network applies the multi-scale features. x k Nonlinear transformations include: The reversible neural network performs a forward transformation: The reversible neural network performs the inverse transformation: in: 、 Multi-scale features x k Two parts divided along the channel dimension; For output features; F and G are three-layer convolutional residual blocks; T(⋅) represents the forward mapping function; This represents the inverse mapping function.

4. The multi-scale semantically guided image compression method according to claim 1, characterized in that, In S400, the bidirectional scanning mechanism includes: Forward state update equation: Backward state update equation: in: 、 These represent the forward and backward states, respectively. ∈R C×1 For state memory weights; ∈R C×C The input is the gating weight matrix; Depth-separable convolution operations are used to dynamically generate convolution kernels; It is an element-wise product.

5. The multi-scale semantically guided image compression method according to claim 1, characterized in that, In step S400, the dynamic convolution kernel generation includes: Based on the semantic weight graph With the high-frequency subband Generate query Q, key K, and value V, and compute the convolution kernel parameter matrix using multi-head attention. : Where: d is the dimension of the attention head; Softmax is the normalized exponential function; T is the matrix transpose.

6. The multi-scale semantically guided image compression method according to claim 1, characterized in that, In S400, the channel-space dual-path attention gating enhancement feature includes: Channel attention: Spatial attention: Final output features: in: X represents the input feature; GAP stands for Global Average Pooling; MLP stands for Multilayer Perceptron; for Activation function; ⊙ represents element-wise product.

7. The multi-scale semantically guided image compression method according to claim 1, characterized in that, In S500, non-uniform quantization measurement includes: The quantization step size is adaptively adjusted based on the semantic weight graph: in: Based on the quantization step size; Position in the semantic weight graph ( i , j The response intensity of ) λ is the adjustment coefficient; Quantization operation is defined as: in: Indicates location , Eigenvalues; Z i，j These are the feature values ​​to be quantized; This is the rounding function; For adaptive quantization step size.

8. The multi-scale semantically guided image compression method according to claim 7, characterized in that, In S500, the entropy encoding includes: Based on the aforementioned state-space modeled hyperprior network, a joint probability model is constructed for the latent variables: in: N represents a Gaussian distribution; They represent the feature dimensions respectively. , The predicted mean and standard deviation of dimension are generated by the super-prior network: ； ； in: This is a lightweight neural network module based on state-space modeling, used to estimate parameters at each feature location. ; This is a lightweight neural network module based on state-space modeling, used to estimate parameters at each feature location. ; This indicates a channel splicing operation; These are decoded features.

9. The multi-scale semantically guided image compression method according to claim 1, characterized in that, In step S600, the compressed bitstream is input into the decoder, decoded based on the lightweight Mamba module, and combined with inverse wavelet transform and semantic weight graph. Reconstructing the image involves the following steps: Mamba module decoding status update: in: This is the current state vector; A B State transition coefficients generated for semantic guidance; For gated functions; Inverse wavelet transform reconstructed features: in: For the k Layer reconstruction feature map; For learnable inverse wavelet transform operators; Multi-scale semantic fusion output image: in: For learnable fusion weights.

10. The multi-scale semantically guided image compression method according to claim 1, characterized in that, The S500 also includes median deviation mapping quantization encoding, comprising the following steps: latent eigenvalues Map to the median reference coordinate system and calculate its corresponding interval. median And calculate the deviation value: Symmetric discretization quantization of the deviation: in: γ is the floor function; γ is the quantization step size; Generating ternary Compress the representation; in: For spatial location index, This is for quantification bias.

11. A multi-scale semantically guided image compression system, characterized in that, The system includes a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the multi-scale semantically guided image compression method as described in any one of claims 1 to 10.

12. A storage medium, characterized in that, The storage medium stores a determination program for a multi-scale semantically guided image compression method, which, when executed by a processor, implements the multi-scale semantically guided image compression method as described in any one of claims 1-10.

Images