Multi-scale semantic guidance image compression method and system and storage medium

Abstract

The invention discloses a multi-scale semantic guidance image compression method and system and a storage medium, and the method comprises the following steps: obtaining input image data, carrying out the preprocessing of an input image, and obtaining standardized image data; inputting the standardized image data into a pre-trained semantic segmentation network to generate a multi-scale semantic feature map and a semantic weight map corresponding to the multi-scale semantic feature map; a three-stage pyramid encoder is constructed, and the standardized image data is subjected to the following steps of: sampling under depth separable convolution to generate multi-scale features; the reversible neural network carries out nonlinear transformation on the multi-scale features; the multi-scale feature subjected to nonlinear transformation is decomposed into a low-frequency sub-band and a high-frequency sub-band through adaptive discrete wavelet transformation, dynamic selective state space modeling is executed on the high-frequency sub-band based on a semantic weight map, and a compressed code stream is generated; and inputting the compressed code stream into a decoder, decoding based on a lightweight Mama module, and reconstructing an image in combination with inverse wavelet transform and a semantic weight map.

Description

Technical Field

[0001] The present invention relates to the field of image data processing technology, and in particular to a multi-scale semantically guided image compression method, system and storage medium. Background Art

[0002] In recent years, deep learning-based image compression methods have gradually replaced traditional hand-crafted compression algorithms. However, existing compression models based on attention mechanisms or convolutional architectures still face challenges in handling long-range dependencies and preserving semantic regions. Especially in low-bitrate scenarios, the perceptual restoration quality of important regions (such as faces and text) is difficult to ensure, resulting in blurring of key areas and semantic distortion. Existing deep learning-based models struggle to balance long-range dependency modeling with computational efficiency. Summary of the Invention

[0003] The present invention aims to address at least one of the technical problems existing in the prior art. To this end, the present invention proposes a multi-scale semantically guided image compression method, system, and storage medium. These methods address the low modeling efficiency and severe compression distortion issues faced by current image compression algorithms in efficiently modeling long-range dependencies, maintaining image quality in key semantic regions, and achieving high-resolution image compression.

[0004] A multi-scale semantically guided image compression method according to an embodiment of the first aspect of the present invention comprises the following steps: S100: Obtain input image data, perform preprocessing on the input image, and obtain standardized image data. ; S200, the standardized image data Input the pre-trained semantic segmentation network to generate multi-scale semantic feature maps and its corresponding semantic weight graph ; S300, constructing a three-level pyramid encoder to decode the standardized image data Execute step by step: S301, depth-separable convolution downsampling, generating multi-scale features ; S302, reversible neural network for the multi-scale features Perform nonlinear transformations; S303, adaptive discrete wavelet transform the multi-scale features after nonlinear transformation Decompose into low-frequency subbands With high frequency sub-band ; S400, based on the semantic weight map For the high frequency sub-band Perform dynamic selective state-space modeling, including: Bidirectional scanning mechanism updates status; Dynamic convolution kernel generation; Channel-spatial dual-path attention gating enhancement features; S500, performing non-uniform quantization and entropy coding on the features after modeling at each layer in step S400 to generate a compressed bit stream; S600: Input the compressed code stream into the decoder, decode it based on the lightweight Mamba module, and combine the inverse wavelet transform with the semantic weight map. Reconstruct the image.

[0005] According to some embodiments of the present invention, in S200, the standardized image data Input the pre-trained semantic segmentation network to generate multi-scale semantic feature maps and its corresponding semantic weight graph include: The Xception65 backbone of the DeepLabv3+ network is used to extract multi-scale features, which are then fused through the dilated spatial pyramid pooling module to generate a semantic feature map. And align the encoding levels through bilinear interpolation, and take the maximum value along the channel dimension to generate the semantic weight map .

[0006] According to some embodiments of the present invention, in S300, the depthwise separable convolution downsampling generates multi-scale features. include: Original image scale characteristics , half-scale features , quarter-scale features ; Where: R is the real number space; H is the feature map height; W is the feature map width.

[0007] According to some embodiments of the present invention, in S300, the reversible neural network processes the multi-scale features. Performing nonlinear transformations includes: The reversible neural network performs the forward transformation: ; ; ; The reversible neural network performs the inverse transformation: ; ; ; in: 、 Multi-scale features Two parts divided along the channel dimension; is the output feature; F and G are three-layer convolution residual blocks; T represents the forward mapping function; Represents the inverse mapping function.

[0008] According to some embodiments of the present invention, in S400, the bidirectional scanning mechanism includes: Forward state update equation: ; Backward state update equation: ; in: 、 Represent the forward and backward states respectively; ∈R C×1 is the state memory weight; ∈R C×C is the input gating weight matrix; Depthwise separable convolution operation for dynamically generated convolution kernels; is the element-wise product.

[0009] According to some embodiments of the present invention, in S400, generating the dynamic convolution kernel includes: Based on the semantic weight map With the high frequency sub-band Generate query Q, key K, value V, and calculate the convolution kernel parameter matrix through multi-head attention : ; Where: d is the dimension of the attention head; Softmax is the normalized exponential function; T is the matrix transpose.

[0010] According to some embodiments of the present invention, in S400, the channel-space dual-path attention gating enhancement feature includes: Channel Attention: ; Spatial Attention: ; Final output features: ; in: X is the input feature; GAP is global average pooling; MLP is a multi-layer perceptron; for The activation function of ⊙ is the element-wise product.

[0011] According to some embodiments of the present invention, in S500, the non-uniform quantization measurement includes: The quantization step size is adaptively adjusted according to the semantic weight map: ; in: is the basic quantization step size; is the position in the semantic weight graph The intensity of the response; λ is the adjustment coefficient; The quantization operation is defined as: ; in: Indicates location , The eigenvalue at Z i，j is the eigenvalue to be quantified, which is the output from dynamic SSM modeling; is the rounding function; It is an adaptive quantization step, which is dynamically adjusted by the semantic weight.

[0012] According to some embodiments of the present invention, in S500, the entropy coding includes: Based on the hyper-prior network of state space modeling, a joint probability model is constructed for the latent variables: ; in: N represents Gaussian distribution; Represents the feature dimension , The predicted mean and standard deviation of the dimension, generated by the hyperprior network: ; ; in: It is a lightweight neural network module based on state space modeling, used to estimate the parameters of each feature position ; It is a lightweight neural network module based on state space modeling, used to estimate the parameters of each feature position ; Represents channel splicing operation; Decoded features.

[0013] According to some embodiments of the present invention, in S600, the compressed code stream is input into the decoder, and the code stream is decoded based on the lightweight Mamba module, combined with the inverse wavelet transform and the semantic weight map. Reconstructing an image involves the following steps: Mamba module decoding status update: ; in: is the state vector at the current moment; A 、B State transition coefficients generated for semantic guidance; is the gating function; Inverse wavelet transform to reconstruct features: ; in: For the k Layer reconstruction feature map; is a learnable inverse wavelet transform operator; Multi-scale semantic fusion output image: ; in: is the learnable fusion weight.

[0014] According to some embodiments of the present invention, the step S500 further includes median deviation mapping quantization encoding, including the following steps: The potential feature value Map to the median reference coordinate system and calculate the interval to which it belongs median , and calculate the deviation value: ; Symmetrically discretize the deviations: ; Where: γ is the quantization step size; Generate ternary Perform compressed representation; in: is the spatial position index, To quantify the deviation.

[0015] According to a second aspect of an embodiment of the present invention, a multi-scale semantically guided image compression system includes a memory and a processor. When the processor executes a computer program stored in the memory, the multi-scale semantically guided image compression method is implemented.

[0016] According to a third aspect of an embodiment of the present invention, a storage medium stores a determination program for a multi-scale semantically guided image compression method, and when the determination program is executed by a processor, the multi-scale semantically guided image compression method is implemented.

[0017] A multi-scale semantically guided image compression method, system and storage medium according to an embodiment of the present invention have at least the following beneficial effects: the present invention introduces a dynamic selective state space modeling mechanism, combined with bidirectional scanning and semantically sensitive attention gating, while effectively reducing computational complexity, it enhances the ability to capture details of key image areas such as faces and text and understand the global context, and alleviates the problem of blurred image details caused by the loss of high-frequency information in traditional compression methods. By integrating a reversible neural network and an adaptive wavelet transform to construct a multi-scale-wavelet joint coding architecture, lossless compression of low-frequency subbands is effectively achieved, avoiding low-frequency distortion; at the same time, high-frequency subbands retain texture details with the help of lightweight dynamic convolution coding, meeting the real-time processing requirements of edge devices. At the decoding end, a selective state space activation mechanism based on the Mamba module decoding structure is introduced, and only key channels are retained to participate in image reconstruction, which significantly reduces the amount of decoding calculations, and through inverse wavelet transform and semantic weight map The fused reconstructed output image.

[0018] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which: Figure 1 A schematic diagram of the computer device structure of the hardware operating environment of an embodiment of the present invention; Figure 2 A flowchart of a multi-scale semantic-guided image compression method according to an embodiment of the present invention; Figure 3 A schematic diagram of the overall network structure of a multi-scale semantically guided image compression method according to an embodiment of the present invention; Figure 4This is a structural diagram of an ASPP module of a multi-scale semantically guided image compression method according to an embodiment of the present invention; Figure 5 This is a structural diagram of a multi-scale-wavelet coding module of a multi-scale semantic-guided image compression method according to an embodiment of the present invention; Figure 6 This is a structural diagram of a dynamic SSM module of a multi-scale semantically guided image compression method according to an embodiment of the present invention; Figure 7 A diagram of a median deviation mapping quantization coding strategy for a multi-scale semantically guided image compression method according to an embodiment of the present invention; Figure 8 This is a structural diagram of a decoding module of a multi-scale semantically guided image compression method according to an embodiment of the present invention; Figure 9 This is a structural block diagram of a multi-scale semantically guided image compression system according to an embodiment of the present invention.

[0020] Figure Number: Processor 1001 ; communication bus 1002 ; user interface 1003 ; network interface 1004 ; memory 1005 . DETAILED DESCRIPTION

[0021] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0022] In the description of the present invention, it should be understood that descriptions involving orientation, such as the orientation or positional relationship indicated by up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0023] In the description of the present invention, "a plurality" refers to more than two. The use of "first" or "second" is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of the indicated technical features, or implicitly indicating the order of the indicated technical features.

[0024] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0025] Reference Figure 1 , Figure 1 This is a schematic diagram of the computer device structure of the hardware operating environment involved in the embodiment of the present application.

[0026] like Figure 1 As shown, the computer device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display and an input unit, such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a wireless fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also be a storage device independent of the processor 1001.

[0027] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0028] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a face detection program for edge computing.

[0029] exist Figure 1 In the computer device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in this application can be set in the computer device, and the computer device calls the multi-scale semantically guided image compression program based on dynamic state space modeling stored in the memory 1005 through the processor 1001, and executes the multi-scale semantically guided image compression method based on dynamic state space modeling provided in the embodiment of the present application.

[0030] Reference Figure 2 As shown, the present invention discloses a multi-scale semantic guided image compression method, comprising the following steps: S100: Obtain input image data, perform preprocessing on the input image, and obtain standardized image data. ; S200, standardize the image data Input the pre-trained semantic segmentation network to generate multi-scale semantic feature maps and its corresponding semantic weight graph ; S300, build a three-level pyramid encoder to standardize image data Execute step by step: S301, depth-separable convolution downsampling, generating multi-scale features ; S302, Reversible Neural Network for Multi-Scale Features Perform nonlinear transformations; S303, adaptive discrete wavelet transform transforms the multi-scale features after nonlinear transformation Decompose into low-frequency subbands With high frequency sub-band ; S400, based on semantic weight graph For high frequency sub-band Perform dynamic selective state-space modeling, including: Bidirectional scanning mechanism updates status; Dynamic convolution kernel generation; Channel-spatial dual-path attention gating enhancement features; S500, performing non-uniform quantization and entropy coding on the features after modeling at each layer in step S400 to generate a compressed bit stream; S600, input the compressed code stream into the decoder, decode it based on the lightweight Mamba module, and combine the inverse wavelet transform with the semantic weight map Reconstruct the image.

[0031] In this embodiment, the input RGB image is converted to a color space (RGB→YUV), normalized ([0, 255]→[-1, 1]), and filtered to generate a standardized image. . Semantic segmentation network (DeepLabv3+) extracts multi-scale feature maps , and take the maximum value along the channel to generate a semantic weight map The encoder adopts a three-level pyramid structure: Depthwise separable convolution downsampling (stride=2); The INN block performs a coupled transformation: , ; Adaptive discrete wavelet transform transforms the multi-scale features after nonlinear transformation Decompose into low-frequency subbands With high frequency sub-band .

[0032] Through the dynamic SSM coding module, dynamic convolution and attention mechanism are integrated into the state space model to perform state space modeling and dynamic coding on image data, thus enhancing the modeling capability of key areas. Semantic-guided state space modeling is performed, entropy coding adopts a non-uniform quantization strategy, and the decoding end uses a lightweight Mamba module and inverse wavelet transform (IDWT) to reconstruct the image. Semantic weight map As the spatial importance prior, dynamic SSM coding module modeling, quantization step size allocation and decoding path activation are implemented to achieve on-demand resource allocation.

[0033] In some embodiments of the present invention, in step S100, the input image data is acquired through the image acquisition module, and color space conversion, normalization and abnormal image filtering are performed to generate standardized image data. The steps of generating standardized image data include: using a standard conversion matrix to convert the RGB image into a YUV color space, separating the brightness (Y) and chrominance (UV) components, retaining the Y channel for subsequent compression, and subsampling the UV component to reduce the amount of data; linearly normalizing the pixel values ​​of each YUV channel, while retaining the sign bit to support the negative value calculation of the subsequent wavelet transform; calculating the image clarity score through the Laplace operator, counting the proportion of pixel values ​​in the brightness channel that exceed the threshold, determining abnormal images and filtering; and finally outputting the pre-processed standardized image data ∈R H×W×3 , whose dimensions are consistent with the original input.

[0034] In step S200, the normalized image data Input the pre-trained semantic segmentation network to generate multi-scale semantic feature maps and its corresponding semantic weight graph include: The Xception65 backbone of the DeepLabv3+ network is used to extract multi-scale features, which are then fused through the dilated spatial pyramid pooling module to generate a semantic feature map. And align the encoding levels through bilinear interpolation, and take the maximum value along the channel dimension to generate a semantic weight map .

[0035] In this embodiment, a pre-trained network is used to extract multi-scale semantic features, and a semantic weight map is generated through dilated spatial pyramid pooling for semantic guidance in subsequent encoding.

[0036] Specifically, a pre-trained network is used to extract multi-scale semantic features, and a semantic weight map is generated through dilated spatial pyramid pooling; It should be noted that the pre-trained DeepLabv3+ model is used, and its backbone network is Xception65. The convolutional layers of DeepLabv3+ are used to extract features and obtain low-level features. , intermediate features and advanced features , where C1=256, C2=512, and C3=1024.

[0037] It should be noted that multi-scale context modeling combines high-level features Enter the ASPP module, such as Figure 4 As shown, it is processed through five parallel branches: Branch 1: 1×1 standard convolution, keeping the spatial dimension unchanged; Branches 2 to 4: 3×3 dilated convolutions with dilation rates of 6, 12, and 18, respectively; Branch 5: After global average pooling, upsample to the original spatial dimension and restore the channel dimension through 1×1 convolution.

[0038] The five branch outputs are spliced ​​in the channel dimension and fused through 1×1 convolution to obtain a multi-scale feature fusion map .

[0039] Semantic segmentation map generation: F ASPP Input 3×3 convolution and Softmax layer to generate semantic prediction map , where Cs represents the number of semantic categories.

[0040] S is upsampled to align with the input resolution of each level of the encoder through bilinear interpolation, generating three-level semantic feature maps {S0, S1, S2}, corresponding to the original resolution, 1 / 2 resolution, and 1 / 4 resolution respectively.

[0041] Then a multi-scale semantic feature map is generated for each Taking the maximum value along the channel dimension to get the weight map: ; The semantic weight map It will be used for dynamic attention generation and important area guidance in the encoding and compression stages.

[0042] In some embodiments of the present invention, in S300, depthwise separable convolution downsampling is performed to generate multi-scale features. include: Original image scale characteristics , half-scale features , quarter-scale features .

[0043] In this embodiment, a multi-scale wavelet joint coding module is used to construct a three-level pyramid structure by using depth-separable convolution with a step size of 2 and then adaptive wavelet transform is performed on the feature map of each scale to decompose the image features into low-frequency sub-bands. With high frequency sub-band , and achieve lossless compression through reversible neural networks.

[0044] Specifically, each level of the pyramid structure encoder performs the following operations, such as Figure 5 As shown: Initial downsampling and feature extraction: normalize image data Input into a three-layer depth-wise separable convolutional network to generate three levels of feature representation: First level, input standardized image data , output ; Level 2, input , output ; Level 3, Input , output ; The depth-wise separable convolution operation is defined as: ; in, Depend on Convolution and The convolutional layers are connected in series, which can significantly reduce computational overhead and is suitable for edge computing devices.

[0045] Reversible Neural Network Transformation (INN): For each level of features , decompose it into and In two parts, the following coupled layer forward mapping is performed: ; ; ; Output = ( , ), where F and G are three-layer convolutional residual blocks, each layer contains BatchNorm (batch normalization), LeakyReLU (with leakage correction linear unit) and 3×3 convolution.

[0046] Adaptive wavelet decomposition: INN output features Enter the custom wavelet decomposition module and perform the learnable discrete wavelet transform (Learnable DWT): ; in: : low-frequency subband, retaining the main structural information of the image; : High-frequency subband, retaining texture and edge details, is further sent to the state space modeling module for processing.

[0047] The high-frequency sub-band size and number of channels are: , where the triple channels correspond to the horizontal, vertical and diagonal detail directions in the wavelet transform.

[0048] It should be noted that the reversible neural network INN supports exact inverse transformation, and its inverse function is defined as: ; ; ; Represents the inverse mapping function to ensure that the output of the encoder It can be restored through INN in the decoding stage to meet the lossless compression requirements of the low-frequency sub-band.

[0049] In some embodiments of the present invention, in S400, the bidirectional scanning mechanism includes: Forward state update equation: ; Backward state update equation: ; in: 、 Represent the forward and backward states respectively; ∈R C×1 is the state memory weight; ∈R C×C is the input gating weight matrix; Depthwise separable convolution operation for dynamically generated convolution kernels; is the element-wise product.

[0050] In this embodiment, the dynamic SSM encoding module is used to integrate dynamic convolution and attention mechanisms in the state-space model to perform state-space modeling and dynamic encoding on image data, thereby enhancing the key area modeling capability.

[0051] The bidirectional scanning mechanism and dynamic convolution kernel driven by semantic weight map are introduced to enhance key area modeling through channel-spatial dual-path attention gating, thus reducing computational complexity. Specifically, like Figure 6 As shown, the implementation of the dynamic SSM module includes: Receive the high frequency sub-band features from step S300 and the multi-scale semantic weight map generated in step S200 , construct a dynamic selective state space modeling (Dynamic Selective SSM) module to achieve dynamic modeling and compressed representation of high semantic region features.

[0052] It should be noted that the semantic alignment process includes: first, using the bilinear interpolation function resize Semantic Weight Graph The spatial size is adjusted to the high-frequency features of the corresponding scale Consistent: ; in: is the aligned semantic weight map; resize is the spatial size adjustment function.

[0053] This operation ensures that the semantic guidance effect corresponds one-to-one with the feature space, enhancing the guidance accuracy.

[0054] The two-way status update mechanism includes: For time series feature modeling, a bidirectional state space structure is introduced, including forward state update and backward state update. The specific update formula is as follows: The forward state update equation is: ; The backward state update equation is: ; in: 、 Represent the forward and backward states respectively, ∈R C×1 is the state memory weight, which is affected by the multi-scale semantic weight map guide, ∈R C×C is the input gating weight matrix, : Dynamically generate depth-wise separable convolution operations with convolution kernels, is the element-wise product.

[0055] In some embodiments of the present invention, in S400, generating a dynamic convolution kernel includes: Based on semantic weight graph With high frequency sub-band Generate query Q, key K, value V, and calculate the convolution kernel parameter matrix through multi-head attention : ; Where d is the dimension of the attention head; Softmax is the normalized exponential function; T is the matrix transpose.

[0056] In this embodiment, the dynamic convolution kernel generation mechanism is: The weights of the dynamic convolution kernel are generated based on the attention mechanism. First, a 1×1 convolution is performed on the semantic graph and high-frequency features to obtain the query Q, key K and value V: ; ; ; Based on the standard multi-head attention mechanism, calculate the dynamic convolution kernel parameter matrix : ; Where d is the dimension of the attention head, which is used for normalization to prevent gradient explosion.

[0057] In some embodiments of the present invention, in step S400, the channel-spatial dual-path attention gating enhancement feature includes: Channel Attention: ; Spatial Attention: ; Final output features: ; in: X is the input feature; GAP is global average pooling; MLP is a multi-layer perceptron; for The activation function of ⊙ is the element-wise product.

[0058] Specifically, the channel-spatial dual attention gating mechanism includes: The feature X generated by the dynamic convolution kernel is enhanced by attention gating, which is divided into two stages: channel attention (CA) and spatial attention (SA): The channel attention calculation formula is: ; The spatial attention calculation formula is: ; The final output features are dual-path enhanced features: ; In some embodiments of the present invention, in step S500, the non-uniform quantization measurement includes: The quantization step size is adaptively adjusted according to the semantic weight map: ; in: is the basic quantization step size; is the position in the semantic weight graph The intensity of the response; λ is the adjustment coefficient, which is used to enhance the resolution of high semantic regions; The quantization operation is defined as: ; in: Indicates location , The eigenvalue at Z i，j is the eigenvalue to be quantified, which is the output from dynamic SSM modeling; is the rounding function; It is an adaptive quantization step, which is dynamically adjusted by the semantic weight.

[0059] In this embodiment, the semantic-guided non-uniform quantization strategy and joint probability modeling process includes: receiving the dynamic feature Z= output from S400 Perform quantization and modeling to achieve a balance between compression rate and fidelity.

[0060] The adaptive quantization step size at each position is calculated using the following formula: : ; The quantization operation is defined as: ; Quantized features It is a compressed representation that can be entropy-encoded. This strategy ensures that semantically important areas (such as faces and text) are encoded more finely, while background areas can be processed more coarsely to save bit rate.

[0061] In some embodiments of the present invention, in S500, entropy coding includes: Based on the hyper-prior network of state space modeling, a joint probability model is constructed for latent variables: ; in: N represents Gaussian distribution; Represents the feature dimension , The predicted mean and standard deviation of the dimension, generated by the hyperprior network: ; ; in: It is a lightweight neural network module based on state space modeling, used to estimate the parameters of each feature position ; It is a lightweight neural network module based on state space modeling, used to estimate the parameters of each feature position ; Represents channel splicing operation; Decoded features.

[0062] In this embodiment, in order to improve the compression efficiency, the joint probability distribution of the features is modeled , the construction form is as follows: ; The super prior network uses a multi-scale feature fusion method to combine the decoded features And the semantic feature map at the corresponding scale After splicing, it is input into the state space modeling module (Mamba) for parameter prediction, specifically: ; ⊕ represents the channel concatenation operation, and the exponential function is used to ensure that the predicted standard deviation is positive.

[0063] In some embodiments of the present invention, in S600, the compressed code stream is input into the decoder, and the code stream is decoded based on the lightweight Mamba module, combined with the inverse wavelet transform and the semantic weight map. Reconstructing an image involves the following steps: Mamba module decoding status update: ; in: is the state vector at the current moment; A 、B State transition coefficients generated for semantic guidance; is the gating function; Inverse wavelet transform to reconstruct features: ; in: is a learnable inverse wavelet transform operator; Multi-scale semantic fusion output image: ; in: is the learnable fusion weight.

[0064] In specific implementation, Figure 8 As shown in Figure 2, the semantically guided image reconstruction process based on the Mamba decoder includes: Receive the code stream output from the encoding stage , combined with semantic weight graph and wavelet subbands ( ), the original image is reconstructed through the Mamba decoder and inverse wavelet transform module.

[0065] Mamba decoder based on state space modeling, the Mamba decoder uses a lightweight state space mechanism and performs state updates based on semantically guided selective activation channels. The calculation process is as follows: The state update equation is: ; in: ∈ R C is the current channel status, State transition coefficients generated for semantic guidance; is a gating function that suppresses invalid feature channels.

[0066] The inverse wavelet reconstruction module receives the output features of the Mamba decoder , with the low-frequency sub-band and high frequency sub-band , perform inverse wavelet transform: ; in It is a learnable inverse wavelet transform operator that reconstructs the current scale image feature map.

[0067] Multi-scale fusion and semantic-guided reconstruction: The decoding results of the three scales are weightedly fused with the semantic map to generate the final reconstructed image. : ; in: ∈R is the fusion weight; is the semantic weight map at the corresponding scale.

[0068] The fusion strategy gives greater influence to regions with high semantic weight, ensuring that key structures (such as facial contours and text edges) are more completely preserved in image reconstruction.

[0069] In the specific implementation, the added loss function is as follows: In order to effectively improve the restoration quality, structural fidelity and visual consistency of semantic regions during image compression, the reconstructed image is trained during the model training phase. With the original image Introduce multiple loss functions to construct a composite loss objective function , the loss function includes semantic edge loss, structure preservation loss, and traditional reconstruction loss. The details are as follows: Pixel-level reconstruction loss (MSE): This loss is used to measure the error between the reconstructed image and the original image at the pixel level and is defined as: ; Where N is the total number of pixels in the image, and The pixels in the reconstructed image and the original image are The value of .

[0070] Structure Preserving Loss (SSIM Loss): The structural similarity index (SSIM) is used to measure the consistency of image structure, which is defined as: ; This loss term mainly constrains the brightness, contrast and structural information of the image to ensure that the reconstructed image remains perceptually similar to the original image.

[0071] Considering that the edges of semantic regions usually carry important structural information, the semantic edge guided loss is introduced: ; in: M is the normalization factor; k is the pyramid level index; is the pixel space coordinate; is the gradient value of the reconstructed image; is the gradient value of the original image; Indicates the high-response semantic area at the kth level; ∈[0, 1] is the guidance strength of the position in the semantic weight map; This loss term emphasizes the fidelity of semantic region edge reconstruction.

[0072] During the training process, the compression rate target is taken into account and the bit rate control item is added: ; in: For quantitative features expected value; For quantitative features The probability value of is the information quantity function.

[0073] The loss term measures the average number of bits of the compressed code after encoding, and is derived from the joint probability model established in step S500.

[0074] It is understandable that the total loss function combination is as follows: ; in: λ 1~ λ4 is the loss term balance coefficient, which is tuned through cross-validation during training. λ3 is usually set to be greater than λ1 to highlight the reconstruction accuracy of semantic edge areas. For real-time compression applications, λ4 can be appropriately increased to control the overall bit rate.

[0075] Through the above-mentioned multi-loss joint optimization strategy, the image compression system can effectively improve its structure preservation ability and reconstruction quality in semantically salient areas (such as faces, text, and object contours), while taking into account the overall image compression efficiency. It is suitable for a variety of scenarios that require "high compression ratio and high-fidelity reconstruction".

[0076] In some embodiments of the present invention, step S500 further includes median deviation mapping quantization encoding, including the following steps: The potential feature value Map to the median reference coordinate system and calculate the interval to which it belongs median , and calculate the deviation value: ; in: is the original eigenvalue; Symmetrically discretize the deviations: ; Where: γ is the quantization step size; Generate ternary Perform compressed representation; in is the spatial position index, To quantify the deviation.

[0077] The distribution model is used to optimize bitrate allocation in the encoder and to provide prior guidance for reconstruction in the decoder.

[0078] In some embodiments of the present invention, a discrete coding mechanism based on deviation representation is further introduced on the basis of the non-uniform quantization module, such as Figure 7 As shown. First, the potential feature value or pixel value range [0, 255] is divided into N interval segments, each interval is defined as: ;

[0079] Set a middle value for each interval As the reconstruction reference value. For each encoded feature point , by finding the interval to which it belongs And calculate its relative median deviation: ;

[0080] Symmetrically discretize the deviations: ; Eventually The triples are encoded, where is the spatial position index, The triplet data can be further compressed using entropy coding or variable length coding to achieve more efficient code stream expression.

[0081] The present invention introduces a dynamic selective state space modeling mechanism, combines bidirectional scanning with semantically sensitive attention gating, and effectively reduces computational complexity while enhancing the ability to capture details and understand global context in key image areas such as faces and text, alleviating the problem of blurred image details caused by the loss of high-frequency information in traditional compression methods. Furthermore, based on the semantically guided non-uniform quantization strategy and the median deviation mapping quantization coding mechanism, the quantization step size can be dynamically adjusted according to the semantic weight map, and the spatial position and deviation information can be used for three-dimensional coordinate encoding, and a finer quantization level can be assigned to areas with high semantic value, thereby optimizing the balance between compression efficiency and reconstruction quality. By integrating a reversible neural network and an adaptive wavelet transform to construct a multi-scale-wavelet joint coding architecture, lossless compression of low-frequency subbands is effectively achieved, avoiding low-frequency distortion; at the same time, high-frequency subbands retain texture details with the help of lightweight dynamic convolution coding, meeting the real-time processing requirements of edge devices. At the decoding end, a selective state space activation mechanism based on the Mamba structure is introduced, retaining only key channels to participate in image reconstruction, significantly reducing the amount of decoding calculations, and reconstructing the output image through inverse wavelet transform and multi-scale semantic fusion. These technological breakthroughs make this application have important practical value and broad application prospects in application scenarios such as security monitoring, mobile communications, and medical imaging that require a balance between compression rate and visual fidelity.

[0082] refer to Figure 9 As shown, the present invention also discloses a multi-scale semantically guided image compression system, which includes a memory and a processor. When the processor executes a computer program stored in the memory, the multi-scale semantically guided image compression method is implemented.

[0083] Furthermore, to achieve end-to-end deployment of the image compression system, the image compression system proposed in the present invention further includes the following modules at the system architecture level: Image preprocessing module: obtains the original image, performs color space conversion, normalization and abnormal image removal, and outputs a standardized image.

[0084] Semantic guidance module: Use the DeepLabv3+ semantic segmentation model to extract semantic features, and obtain a multi-scale semantic weight map through bilinear interpolation to guide subsequent encoding.

[0085] Multi-scale wavelet coding module: Constructs a three-level coding pyramid. Each level consists of depthwise separable convolution, reversible neural network, and adaptive wavelet transform, outputting low-frequency subbands and high-frequency subbands.

[0086] Dynamic SSM modeling module: integrates semantic graph guidance and bidirectional scanning mechanism to achieve state space modeling and output dynamic convolutional coding features.

[0087] Entropy coding module: uses a hyper-prior network to extract contextual features, and estimates the probability distribution in conjunction with a semantic weight map, and executes non-uniform quantization and median deviation mapping quantization coding strategies to generate a bitstream.

[0088] Decoding and reconstruction module: Combining Mamba state update with semantic gating mechanism, it generates a reconstructed image through inverse wavelet transform and semantic weighted fusion.

[0089] The present invention also discloses a storage medium, which stores a determination program of a multi-scale semantically guided image compression method. When the determination program is executed by a processor, the multi-scale semantically guided image compression method is implemented.

[0090] Since the multi-scale semantically guided image compression system and storage medium adopt all the technical solutions of the multi-scale semantically guided image compression method of the above embodiment, they at least have all the beneficial effects brought by the technical solutions of the above embodiment, which will not be repeated here.

[0091] The embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in the relevant technical field without departing from the scope of the present invention.

Claims

1. A multi-scale semantically guided image compression method, characterized in that: The following steps are involved: S100: Obtain input image data, perform preprocessing on the input image, and obtain standardized image data. ; S200, the standardized image data Input the pre-trained semantic segmentation network to generate multi-scale semantic feature maps and its corresponding semantic weight graph ; S300, constructing a three-level pyramid encoder to decode the standardized image data Execute step by step: S301, depth-separable convolution downsampling, generating multi-scale features ; S302, reversible neural network for the multi-scale features Perform nonlinear transformations; S303, adaptive discrete wavelet transform the multi-scale features after nonlinear transformation Decompose into low-frequency subbands With high frequency sub-band ; S400, based on the semantic weight map For the high frequency sub-band Perform dynamic selective state-space modeling, including: Bidirectional scanning mechanism updates status; Dynamic convolution kernel generation; Channel-spatial dual-path attention gating enhancement features; S500, performing non-uniform quantization and entropy coding on the features after modeling at each layer in step S400 to generate a compressed bit stream; S600: Input the compressed code stream into the decoder, decode it based on the lightweight Mamba module, and combine the inverse wavelet transform with the semantic weight map. Reconstruct the image.

2. The multi-scale semantically guided image compression method according to claim 1, characterized in that: In the step S200, the standardized image data Input the pre-trained semantic segmentation network to generate multi-scale semantic feature maps and its corresponding semantic weight graph include: The Xception65 backbone of the DeepLabv3+ network is used to extract multi-scale features, which are then fused through the dilated spatial pyramid pooling module to generate a semantic feature map. And align the encoding levels through bilinear interpolation, and take the maximum value along the channel dimension to generate the semantic weight map .

3. The multi-scale semantic guided image compression method according to claim 1, characterized in that: In S300, the depth-separable convolution downsampling generates multi-scale features include: Original image scale characteristics , half-scale features , quarter-scale features ; Where: R is the real number space; H is the feature map height; W is the feature map width.

4. The multi-scale semantic guided image compression method according to claim 1, characterized in that: In S300, the reversible neural network processes the multi-scale features Performing nonlinear transformations includes: The reversible neural network performs the forward transformation: ； ； ； The reversible neural network performs the inverse transformation: ； ； ； in: 、 Multi-scale features Two parts divided along the channel dimension; is the output feature; F and G are three-layer convolution residual blocks; T represents the forward mapping function; Represents the inverse mapping function.

5. The multi-scale semantic guided image compression method according to claim 1, characterized in that: In S400, the bidirectional scanning mechanism includes: Forward state update equation: ； Backward state update equation: ； in: 、 Represent the forward and backward states respectively; ∈R C×1 is the state memory weight; ∈R C×C is the input gating weight matrix; Depthwise separable convolution operation for dynamically generated convolution kernels; is the element-wise product.

6. The multi-scale semantic guided image compression method according to claim 1, characterized in that: In S400, the dynamic convolution kernel generation includes: Based on the semantic weight map With the high frequency sub-band Generate query Q, key K, value V, and calculate the convolution kernel parameter matrix through multi-head attention : ； Where: d is the dimension of the attention head; Softmax is the normalized exponential function; T is the matrix transpose.

7. The multi-scale semantic guided image compression method according to claim 1, characterized in that: In S400, the channel-space dual-path attention gating enhancement feature includes: Channel Attention: ； Spatial Attention: ； Final output features: ； in: X is the input feature; GAP is global average pooling; MLP is a multi-layer perceptron; for The activation function of ⊙ is the element-wise product.

8. The multi-scale semantic guided image compression method according to claim 1, characterized in that: In S500, the non-uniform quantization measurement includes: The quantization step size is adaptively adjusted according to the semantic weight map: ； in: is the basic quantization step size; is the position in the semantic weight graph The intensity of the response; λ is the adjustment coefficient; The quantization operation is defined as: ; in: Indicates location , The eigenvalue at Z i，j is the eigenvalue to be quantified; is the rounding function; is the adaptive quantization step size.

9. The multi-scale semantic guided image compression method according to claim 8, characterized in that: In S500, the entropy coding includes: Based on the hyper-prior network of state space modeling, a joint probability model is constructed for the latent variables: ； in: N represents Gaussian distribution; Represents the feature dimension , The predicted mean and standard deviation of the dimension, generated by the hyperprior network: ； ； in: It is a lightweight neural network module based on state space modeling, used to estimate the parameters of each feature position ; It is a lightweight neural network module based on state space modeling, used to estimate the parameters of each feature position ; Represents channel splicing operation; Decoded features.

10. The multi-scale semantic guided image compression method according to claim 1, characterized in that: In the step S600, the compressed code stream is input into the decoder, and the code stream is decoded based on the lightweight Mamba module, combined with the inverse wavelet transform and the semantic weight map. Reconstructing an image involves the following steps: Mamba module decoding status update: ； in: is the state vector at the current moment; A 、B State transition coefficients generated for semantic guidance; is the gating function; Inverse wavelet transform to reconstruct features: ； in: For the k Layer reconstruction feature map; is a learnable inverse wavelet transform operator; Multi-scale semantic fusion output image: ； in: is the learnable fusion weight.

11. The multi-scale semantic guided image compression method according to claim 1, characterized in that: The S500 further includes median deviation mapping quantization encoding, including the following steps: The potential feature value Map to the median reference coordinate system and calculate the interval to which it belongs median , and calculate the deviation value: ； Symmetrically discretize the deviations: ； in: is the rounding function; γ is the quantization step size; Generate ternary Perform compressed representation; in: is the spatial position index, To quantify the deviation.

12. A multi-scale semantically guided image compression system, characterized in that: The method comprises a memory and a processor, wherein when the processor executes the computer program stored in the memory, the multi-scale semantically guided image compression method according to any one of claims 1 to 11 is implemented.

13. A storage medium, characterized in that: The storage medium stores a determination program of a multi-scale semantically guided image compression method, and when the determination program is executed by a processor, the multi-scale semantically guided image compression method according to any one of claims 1 to 11 is implemented.

Images