Adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution image

Abstract

The invention relates to the technical field of image processing and deep learning, in particular to a wide-area high-resolution image adaptive compression and high-frequency detail accurate reconstruction method. Comprising the following steps: collecting and segmenting a wide-area high-resolution image to obtain a plurality of segmented images, and mapping each segmented image into a low-dimensional continuous potential feature vector by adopting an encoder; constructing an adaptive mask module, obtaining texture complexity based on the segmented image, establishing a mask matrix, masking the low-dimensional continuous potential feature vector through the mask matrix to obtain a reserved feature vector, and generating a mask position map; generating a Token sequence after vector quantization operation; establishing a composite feature map, predicting and recovering the composite feature map based on context information by using a demasker in combination with global attention and local residual perception, and generating a potential feature map; and a decoder is adopted to map the potential feature map back to a continuous pixel space, edges and fine textures are restored, a reconstructed image corresponding to the segmented image is obtained, and a wide-area high-resolution image is restored.

Description

Technical Field

[0001] This invention relates to the fields of image processing and deep learning technology, specifically to a method for adaptive compression and accurate reconstruction of high-frequency details in wide-area high-resolution images. Background Technology

[0002] With the maturation of ultra-high resolution imaging chip technology, it will have wide applications in aerospace scenarios in the future. However, acquiring ultra-wide field-of-view, ultra-high resolution images, i.e., wide-area high-resolution images, involves a surge in image data, making real-time transmission difficult. Due to limitations in video memory and bandwidth, images need to be compressed and transmitted in blocks and reconstructed at the ground. Current image compression and reconstruction technologies are mainly divided into two categories: First, traditional compression methods based on transform coding, such as JPEG2000 (Joint Photographic Experts Group 2000), HEVC / H.265 (High Efficiency Video Coding), and BPG (Better Portable Graphics). Although traditional methods have mature algorithms and relatively low computational complexity, the reconstructed image quality at extremely low bitrates is mostly blurry and accompanied by severe block artifacts, making it difficult to meet the high-precision reconstruction requirements of ultra-wide field-of-view, ultra-high resolution images. Secondly, there are deep learning-based image compression methods, which utilize convolutional neural networks (CNNs) or Transformer architectures for end-to-end feature learning and generation, such as joint source-channel coding (JSCC) and image compression models based on generative adversarial networks (GANs). For example, Chinese patent application number 202511281423.X discloses an extremely low bit rate image compression method. This method extracts textual description information of the image as semantic guidance and combines it with latent variables reflecting structure and spatial distribution for joint encoding, thereby reducing the bit rate while preserving image details as much as possible and improving reconstruction quality.

[0003] However, for the problem of wide-area high-resolution image compression and reconstruction, emerging methods are often limited by GPU memory bottlenecks and require independent processing of blocks, which severs long-distance semantic dependencies across patches, resulting in a lack of global spatial consistency in the reconstructed image. In addition, existing end-to-end models usually lack adaptive transmission mechanisms for texture complexity and are prone to generating false textures that do not conform to the real scene at extremely low bitrates, making it difficult to achieve both high compression ratios and pixel-level accurate reconstruction of key details under limited bandwidth. Summary of the Invention

[0004] To address the technical problems of existing deep learning models lacking global spatial consistency and adaptive transmission mechanisms, which affect the realism of image reconstruction, this invention aims to provide an adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images. The specific technical solution adopted is as follows:

[0005] A wide-area high-resolution image is acquired and segmented to obtain several segmented images. An encoder is used to map each segmented image into a low-dimensional continuous latent feature vector.

[0006] An adaptive masking module is constructed. Based on the texture complexity obtained from the segmented image, a mask matrix is ​​established. The preserved feature vector is obtained by masking the low-dimensional continuous latent feature vector through the mask matrix, and a mask position map is generated.

[0007] Perform vector quantization on the retained feature vectors to generate a token sequence;

[0008] The token sequence is mapped back to the quantized feature vector, and a composite feature map is built by combining the mask location map. The demask is used in combination with global attention and local residual perception. Based on the context information, the composite feature map is predicted and recovered to generate a latent feature map.

[0009] A decoder is used to map the latent feature map back to the continuous pixel space to restore the edges and fine textures, thereby obtaining the reconstructed image corresponding to the segmented image. All the reconstructed images corresponding to the segmented images are integrated and stitched together to restore the wide-area high-resolution image.

[0010] Preferably, an encoder is used to map each segmented image into a low-dimensional continuous latent feature vector, including:

[0011] A convolution operation is performed on any segmented image to map it to an initial feature space. The encoder extracts features from the initial feature space through multiple levels to determine the initial feature map.

[0012] A bottleneck layer is constructed, and the initial feature map is aggregated and refined using global contextual information to obtain deep features.

[0013] The deep features are mapped to the latent space, and the output is a low-dimensional continuous latent feature vector.

[0014] Preferably, the encoder extracts features from the initial feature space through multiple levels to determine the initial feature map, specifically as follows:

[0015] Multiple residual blocks are stacked. The initial feature space is processed by the residual blocks to obtain the output features of the corresponding level. A preset resolution constraint is set. When the output features meet the resolution constraint, bias self-attention blocks are used to perform self-attention processing on the output features, and the initial feature map is obtained through downsampling.

[0016] Preferably, the bottleneck layer has a hierarchical structure, including a residual block, an autofocus block, and a residual block arranged sequentially.

[0017] Preferably, an adaptive masking module is constructed, which obtains the texture complexity based on the segmented image, establishes a mask matrix, obtains the preserved feature vector by masking the low-dimensional continuous latent feature vector through the mask matrix, and generates a mask location map, including:

[0018] The segmented image is converted into a grayscale image by brightness weighting, and the gradient magnitude map corresponding to the grayscale image is generated by edge operators.

[0019] The gradient magnitude map is mapped to the same resolution as the low-dimensional continuous latent feature vector by average pooling, thus generating a complexity map.

[0020] Set the feature retention rate according to the transmission bandwidth requirements, determine the total number of features to be retained, and design a binary mask matrix by combining the complexity graph.

[0021] The retained feature vectors are obtained by masking the low-dimensional continuous latent feature vectors using a binary mask matrix, and a binary mask location map is generated.

[0022] Preferably, the retained feature vectors are subjected to vector quantization to generate a token sequence, specifically as follows:

[0023] The pre-trained learnable discrete codebook uses a vector quantizer to analyze and retain feature vectors and discrete codebooks. The quantized feature vectors are obtained through the quantization mapping process, and the corresponding indices are obtained. The indices are then integrated to build a token sequence.

[0024] Preferably, the token sequence is mapped back to the quantized feature vector, and a composite feature map is constructed by combining the mask location map. A demasking device, combined with global attention and local residual awareness, is used to predict and recover the composite feature map based on contextual information, generating a latent feature map, including:

[0025] The token sequence is mapped back to the quantized feature vector, and combined with the mask location map, learnable placeholder vectors are filled in the missing positions to generate the initial composite feature map.

[0026] A demasker incorporating a hybrid processor is used, and an initial composite feature map is input to determine global correlations;

[0027] Zero-initialized residual connections are introduced into the demask to perform identity mapping, and global correlation is analyzed through self-attention processing. The initial composite feature maps are combined to complete the progressive repair process, resulting in the repaired latent feature map.

[0028] Preferably, a decoder is used to map the latent feature map back to a continuous pixel space to recover edges and fine textures, obtaining the reconstructed image corresponding to the segmented image. All reconstructed images corresponding to the segmented images are then integrated and stitched together to restore the wide-area high-resolution image, including:

[0029] The decoder uses convolution to project the latent feature map onto the input dimension of the decoder. The bottleneck layer of the decoder captures global contextual information and outputs the reconstructed features. Figure 1 ;

[0030] Reconstruction features Figure 1 Residual processing and upsampling transformation are performed sequentially to restore edges and fine textures, and the reconstructed features are output. Figure 2 ;

[0031] Reconstruction features Figure 2 Normalization, non-linear activation, and convolution are performed sequentially, and the image is mapped to the RGB pixel space to output the reconstructed image.

[0032] Integrate and stitch together the reconstructed images corresponding to all segmented images to restore the wide-area high-resolution image.

[0033] To address the aforementioned technical problems, the present invention further provides: an adaptive compression and high-frequency detail accurate reconstruction device for wide-area high-resolution images, the device comprising:

[0034] The image coding unit is used to: acquire and segment a wide-area high-resolution image to obtain several segmented images, and use an encoder to map each segmented image into a low-dimensional continuous latent feature vector.

[0035] The adaptive masking unit is used to: construct an adaptive masking module, obtain texture complexity based on the segmented image, establish a mask matrix, obtain preserved feature vectors by masking low-dimensional continuous latent feature vectors through the mask matrix, and generate a mask location map;

[0036] The vector quantization unit is used to: perform vector quantization on the retained feature vector to generate a token sequence;

[0037] The mask removal unit is used to: map the token sequence back to the quantized feature vector, build a composite feature map by combining the mask location map, and use the mask removal unit to combine global attention and local residual perception to predict and recover the composite feature map based on context information to generate a latent feature map.

[0038] The image reconstruction unit is used to: map the latent feature map back to the continuous pixel space using a decoder, restore the edges and fine textures, obtain the reconstructed image corresponding to the segmented image, integrate and stitch together the reconstructed images corresponding to all segmented images, and restore the wide-area high-resolution image.

[0039] To address the aforementioned technical problems, this invention also provides: an adaptive compression and high-frequency detail accurate reconstruction system for wide-area high-resolution images, the system comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus, and the processor calls logical instructions in the memory to execute the adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images described in any of the preceding claims.

[0040] The present invention has the following beneficial effects:

[0041] 1. A novel adaptive compression and high-frequency detail reconstruction method for wide-area high-resolution images is proposed. This method not only dynamically adjusts the compression ratio according to the transmission bandwidth, but also outperforms traditional image compression methods in both the perceptual metrics LPIPS (Learned Perceptual Image Patch Similarity) and FID (Fréchet Inception Distance) for different compression ratios. This advantage indicates that the reconstructed image obtained by this method is closer to the human visual system, more realistic and natural, and visually comfortable. The feature distribution is closer to the distribution of real image sets, effectively overcoming the smoothing and block effects common in traditional methods. By reconstructing realistic texture details, the subjective visual quality is improved.

[0042] 2. The adaptive compression and high-frequency detail accurate reconstruction device and system for wide-area high-resolution images provided by the present invention have the same beneficial effects as the adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images provided by the present invention, and will not be described in detail here. Attached Figure Description

[0043] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 The flowchart illustrates the steps of an adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images provided in one embodiment of the present invention.

[0045] Figure 2 This is an implementation architecture diagram of an adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images provided in one embodiment of the present invention;

[0046] Figure 3 This is a diagram illustrating the effect of an adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images provided in an embodiment of the present invention. Detailed Implementation

[0047] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a method for adaptive compression and high-frequency detail accurate reconstruction of wide-area high-resolution images proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0049] The following description, in conjunction with the accompanying drawings, details the specific scheme of the adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images provided by this invention.

[0050] For better illustration, this embodiment mainly focuses on wide-field high-resolution images, namely ultra-large field-of-view ultra-high-resolution images. Wide-field high-resolution images refer to images with extremely high spatial detail resolution capabilities that are captured within an extremely wide field of view, that is, hundreds or even thousands of pixels in horizontal and vertical dimensions. They not only cover a wider scene, but also have smaller physical sizes for each pixel, enabling the resolution of finer structures.

[0051] First, wide-area high-resolution images involve massive amounts of pixel data, placing enormous pressure on storage and transmission. For example, an 8K resolution (7680×4320) image contains approximately 33 million pixels. If it is true color (24 bits / pixel), a single image can reach approximately 100MB in size. Ultra-large field-of-view images may be composed of dozens or even hundreds of 8K images, with the total data volume reaching gigabytes. Traditional uncompressed storage not only occupies huge storage space but also faces bandwidth bottlenecks and latency issues during transmission. Second, directly processing the original images corresponding to wide-area high-resolution images consumes significant computational resources and time, making it difficult to meet the needs of real-time or near-real-time applications. Therefore, this paper proposes an adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images, aiming to solve the bottlenecks in the storage, transmission, and processing of wide-area high-resolution images.

[0052] Please combine Figure 1 and Figure 2 It illustrates a flowchart and implementation architecture diagram of an adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images provided in the first embodiment of the present invention. The method includes:

[0053] Step S1: Acquire and segment the wide-area high-resolution image to obtain several segmented images, and use an encoder to map each segmented image into a low-dimensional continuous latent feature vector;

[0054] Step S2: Construct an adaptive masking module, obtain the texture complexity based on the segmented image, establish a masking matrix, obtain the preserved feature vector by masking the low-dimensional continuous latent feature vector through the masking matrix, and generate a masking location map;

[0055] Step S3: Perform vector quantization on the retained feature vectors to generate a token sequence;

[0056] Step S4: Map the token sequence back to the quantized feature vector, combine it with the mask location map to build a composite feature map, use the demasker combined with global attention and local residual perception, predict and recover the composite feature map based on context information, and generate a latent feature map.

[0057] Step S5: Use a decoder to map the latent feature map back to the continuous pixel space, restore the edges and fine textures, obtain the reconstructed image corresponding to the segmented image, integrate and stitch together the reconstructed images corresponding to all segmented images, and restore the wide-area high-resolution image.

[0058] As an optional implementation, the wide-area high-resolution image is segmented to obtain several smaller images, which are the segmented images. In this embodiment, the resolution of the segmented images is 256×256, which can be adjusted according to the actual situation.

[0059] Further, in step S1, an encoder is used to map each segmented image into a low-dimensional continuous latent feature vector, including:

[0060] Step S11: Perform a convolution operation on any segmented image to map it to the initial feature space. The encoder extracts features from the initial feature space through multiple levels to determine the initial feature map.

[0061] The explanation clarifies that the segmented image being analyzed is... , , These represent the height and width of a wide-area high-resolution image, respectively. Represents the channel dimension of a wide-area high-resolution image; segments the image. Mapped to a channel dimension via convolution operation The initial feature space, and the corresponding calculation formula is:

[0062]

[0063] in, Represents the initial feature space; This indicates normalization processing; Indicates the convolution operation; This represents a segmented image.

[0064] Further, in step S11, the encoder extracts features from the initial feature space through multiple levels to determine the initial feature map, specifically as follows:

[0065] Multiple residual blocks are stacked. The initial feature space is processed by the residual blocks to obtain the output features of the corresponding level. A preset resolution constraint is set. When the output features meet the resolution constraint, bias self-attention blocks are used to perform self-attention processing on the output features, and the initial feature map is obtained through downsampling.

[0066] Specifically, the encoder (Encoder) through Feature extraction is performed at each level, for the first level layer, The initial feature space is sequentially processed through multiple stacked ResNet Blocks to obtain the output features of the corresponding residual blocks in that layer, i.e. , , These represent the output features of the corresponding residual blocks, and It can be the initial feature space. Indicates a hierarchical index; Represents the residual block index at any level. Indicates the number of residual blocks; This indicates residual processing. Next, preset resolution constraints are applied, only when the output features... The current resolution satisfies When this happens, self-attention is enabled using biased self-attention blocks, i.e. , This represents self-attention processing. Finally, a downsampling operation reduces the spatial resolution of the self-attention-processed feature map to half its original value while doubling the channel dimension. , This indicates a downsampling operation, which is implemented through a convolution with a stride of 2.

[0067] Step S12: Construct a bottleneck layer. Through the bottleneck layer, aggregate and refine the global context information of the initial feature map to obtain deep features.

[0068] Furthermore, the bottleneck layer has a hierarchical structure, including residual blocks, self-attention blocks, and residual blocks arranged in sequence; that is, for the initial feature map generated after encoding, at the lowest resolution level, the bottleneck layer constructed by “residual block-self-attention block-residual block” (ResNet-Attention-ResNet) achieves the aggregation and refinement of global contextual information.

[0069] Specifically, the lowest resolution level is determined based on the initial feature map, i.e. The deep features are obtained through the bottleneck layer, and the corresponding calculation formula is:

[0070]

[0071] in, It represents deep features.

[0072] Step S13: Map the deep features to the latent space and output a low-dimensional continuous latent feature vector.

[0073] Specifically, for the VQ-GAN (Vector Quantization-Generative Adversarial Network) architecture adopted, the encoder directly outputs a deterministic feature map, denoted as... That is, a low-dimensional continuous latent feature vector, the corresponding calculation formula is:

[0074]

[0075] in, Represents image segmentation The corresponding low-dimensional continuous latent feature vector; Indicates the activation function; , These represent the height and width of the segmented image, respectively. Represents image segmentation The channel dimension.

[0076] It can be noted that the encoder outputs a low-dimensional continuous latent feature vector directly from the deep features, unlike the traditional variational autoencoder (VAE), which requires decoupling the deep features into mean and variance distributions.

[0077] Understandably, the adaptive masking module is constructed by using the high-resolution spatial information corresponding to the original input image, i.e. the segmented image, as a guide to accurately identify high-frequency features in the latent space.

[0078] Further, step S2 includes:

[0079] Step S21: Convert the segmented image into a grayscale image by using brightness weighting, and generate the gradient magnitude map corresponding to the grayscale image using edge operators.

[0080] Specifically, still using image segmentation This explanation explains how to segment the image using brightness weighting. Convert to a single-channel grayscale map to output a grayscale image, thereby decoupling chromaticity information and focusing on structural texture. The corresponding calculation formula is:

[0081]

[0082] in, Represents image segmentation The corresponding grayscale image; , , These represent the weight coefficients corresponding to the RGB pixels.

[0083] Preferably, in this embodiment, , , These are brightness weighting coefficients that conform to human visual perception, typically with values ​​of 0.299, 0.587, and 0.114, which can be adjusted according to the actual situation. In addition, the edge operator uses the Sobel operator, which identifies edge regions in the image by calculating the approximate gradient values ​​of each pixel in the horizontal and vertical directions.

[0084] Using the Sobel operator Calculate grayscale image gradient magnitude plot The corresponding calculation formula is:

[0085]

[0086] in, Represents the gradient magnitude plot; This indicates a convolution operation.

[0087] Step S22: Map the gradient magnitude map to the same resolution as the low-dimensional continuous latent feature vectors by average pooling to generate a complexity map.

[0088] Specifically, the corresponding calculation formula is:

[0089]

[0090] in, Representing the complexity graph; This indicates an average pooling operation; This represents the downsampling factor of a low-dimensional continuous latent feature vector.

[0091] Step S23: Set the feature retention rate according to the transmission bandwidth requirements, determine the total number of features to be retained, and design a binary mask matrix by combining the complexity graph.

[0092] Specifically, the feature retention rate is set according to the transmission bandwidth requirements. That is, the maximum tolerable transmission bandwidth is determined based on the task requirements, and the feature retention rate is determined based on this bandwidth. Therefore, the total number of features retained is , Represents the floor operation; design a binary mask matrix. It follows the Top-K selection principle, that is, only retaining the one with the highest complexity score. The logical expression for each feature position is:

[0093]

[0094] in, Represents a binary mask matrix. , Representing feature maps respectively Spatial coordinate indexes in the height and width directions, , .

[0095] It can be explained that, It can directly reflect the low-dimensional continuous latent feature vectors The texture richness of the represented region, and therefore based on the complexity graph Perform salient feature filtering to retain or discard feature locations on the segmented image being analyzed.

[0096] Step S24: Obtain the retained feature vector by masking the low-dimensional continuous latent feature vectors through the binary mask matrix, and generate the binary mask position map.

[0097] Specifically, based on a binary mask matrix The segmented image is filtered. When the value is 1, the region is likely to correspond to high-frequency details such as city buildings and vehicles, which are retained and marked as payloads for quantization and transmission. When the value is 0, the region is likely to correspond to low-frequency flat textures such as sky and sea surface, which are discarded directly and only their mask position index is transmitted. Based on this filtering, the positions of all retained features are determined, and the retained feature vector is established, which is the adaptively filtered high-dimensional latent features. A binary mask position map is then generated.

[0098] Furthermore, in step S3, specifically:

[0099] The pre-trained learnable discrete codebook uses a vector quantizer to analyze and retain feature vectors and discrete codebooks. The quantized feature vectors are obtained through the quantization mapping process, and the corresponding indices are obtained. The indices are then integrated to build a token sequence.

[0100] Specifically, the learnable discrete codebook is , Indicates the capacity of the discrete codebook; Represents the first in the discrete codebook Each code element vector Indicates the symbol vector index; The feature dimension of the symbol vector is represented; where the symbol vector represents a specific low-level texture or semantic feature in the image, and its value is updated synchronously with the network parameters during training; in the implementation of the aforementioned steps, the retained feature vector is denoted as... The vector quantizer calculates its Euclidean distance to all symbol vectors in the discrete codebook and finds the closest, or most similar, codebook vector as a replacement. The corresponding quantization mapping process is as follows:

[0101]

[0102]

[0103] in, Represents the preserved feature vector Quantized feature vector after quantization mapping; Represents the quantized feature vector The corresponding index; Represents the square operation of the norm.

[0104] Next, after completing the quantization mapping, the entire image processing system does not directly store or transmit the high-dimensional feature vector, i.e., the quantized feature vector. Instead, it extracts its corresponding index. Due to the index Only Integers within the range, their data width ( The number of bits is much smaller than the original floating-point feature vector, i.e., the feature vector is preserved. The data bit width is used to establish a token sequence through indexing, and then combined with the corresponding adaptive binary mask position map. The subsequent channel transmits this sequence and position information to achieve semantic compression at extremely low bit rates.

[0105] Further, step S4 includes:

[0106] Step S41: Map the token sequence back to the quantized feature vector, combine it with the mask location map, fill the missing positions with learnable placeholder vectors, and generate the initial composite feature map.

[0107] Specifically, after obtaining the token sequence and the binary mask location map, the receiving end first consults the discrete codebook. The token sequence is mapped back to quantized feature vectors. Then, based on the information corresponding to the binary mask location map, the vectors obtained from the receiving end are restored to their spatial positions in the two-dimensional feature map. For the missing positions of vectors discarded due to bandwidth limitations, a special, learnable placeholder vector is used to fill each missing position, denoted as . These are stitched together to form a complete initial composite feature map with a mask, denoted as . The result is then fed into the mask remover for repair.

[0108] Step S42: Use a demasker containing a hybrid processor and input the initial composite feature map to determine global correlation.

[0109] To explain, in order to simultaneously capture local high-frequency details and global long-range dependencies of an image, a context-aware demasker network is used. That is, the demasker does not use a single Transformer architecture, but instead constructs a hybrid processor. The hybrid processor consists of multiple stacked hybrid processing blocks, and each processing block internally connects the residual block and the bias self-attention block from step S1 above.

[0110] Specifically, the initial composite feature map is input into the demask. First, the data passes through a residual block, utilizing the local receptive field provided by convolution to extract high-frequency detail information such as edges and textures. Then, a biased self-attention block calculates global correlations in the feature space, integrating long-distance contextual information to overcome the limitations of pure Transformer in processing local textures and the shortcomings of pure CNN in capturing global structure. The corresponding calculation formula is as follows:

[0111]

[0112] in, Indicates global correlation; Indicates biased self-attention block operation; Indicates residual block operations; This represents the initial composite feature map.

[0113] Step S43: Introduce zero-initialized residual connections in the demasker for identity mapping, analyze global correlation through self-attention processing, and perform progressive repair processing by combining the initial composite feature maps to obtain the repaired latent feature map.

[0114] To ensure the training stability of the entire image processing network architecture and to achieve progressive feature restoration, a zero-initialized residual connection is introduced in the demasker. Then, global correlation is analyzed through the aforementioned biased self-attention block. Instead of directly predicting the final feature map, a correction term is predicted. This correction term is output through a convolutional layer with zero initial weights and biases to achieve progressive restoration processing in conjunction with the initial composite feature map.

[0115] Specifically, the corresponding calculation formula is:

[0116]

[0117] in, Represents the latent feature map; This indicates a zero-initialization residual join operation.

[0118] It can be explained that in the early stages of training, since the ZeroConv output is zero, the Demasker is equivalent to an identity mapping. That is, in the early stages of training, the output is 0, allowing the entire network architecture to perform an identity mapping, ensuring stable convergence and gradual repair during training. As training progresses, the image processing network gradually learns non-zero correction values ​​to fill in the missing semantic features, thereby gradually obtaining the repaired latent feature map. This significantly accelerated network convergence and improved reconstruction quality.

[0119] Further, step S5 includes:

[0120] Step S51: The latent feature map is projected onto the input dimension of the decoder through convolution. The global context information is captured by the bottleneck layer of the decoder, and the reconstructed features are output. Figure 1 .

[0121] Specifically, latent feature maps First, the decoder processes the latent feature maps through convolutional layers. The image is then projected onto the input dimension of the decoder. Next, to ensure the coherence of the reconstructed image in its global structure, it passes through a bottleneck layer consisting of "residual block - bias self-attention block - residual block." This layer utilizes a self-attention mechanism to capture the global context in the low-resolution space, repairing any structural discontinuities that may arise during feature quantization and patching, thus obtaining the reconstructed features. Figure 1 The corresponding calculation formula is:

[0122]

[0123] in, Represents reconstructed features Figure 1 ; Indicates residual block operations; Indicates biased self-attention block operation; This indicates convolution processing.

[0124] Step S52: Reconstruct features Figure 1 Residual processing and upsampling transformation are performed sequentially to restore edges and fine textures, and the reconstructed features are output. Figure 2 .

[0125] Specifically, for reconstructed features Figure 1 Each upsampling level Let the input features of this layer be denoted as . ,in, That is, reconstructing features Figure 1 By performing a series of nonlinear transformations and detail enrichments on the stacked residual blocks, the feature map of this level after residual processing is obtained, i.e. Subsequently, an upsampling transformation is performed using transpose convolution or interpolation operations to improve the feature map of this level. The spatial size is doubled, effectively reversing the encoder's downsampling path to obtain the output of that level, i.e. , This represents an upsampling transformation to recover high-frequency edges in the feature map; according to the... The same process is applied to all layers until the output of the last upsampled layer is obtained, which is then used as the reconstructed feature. Figure 2 , recorded as .

[0126] Step S53: Reconstruct features Figure 2 Normalization, non-linear activation, and convolution are performed sequentially, mapping the image to the RGB pixel space to output the reconstructed image.

[0127] Specifically, for reconstructed features Figure 2 Normalization and non-linear activation function processing are performed, and the image is mapped to the RGB pixel space through the terminal convolutional layer to obtain the final high-fidelity reconstructed image. The corresponding calculation formula is as follows:

[0128]

[0129] in, Represents image segmentation The corresponding reconstructed image; This indicates normalization processing; Represents a nonlinear activation function; This indicates convolution processing.

[0130] Preferably, in this embodiment, the nonlinear activation function is the Swish function, which has smoother gradient characteristics; and the reconstructed image While preserving the macro background and structure, it accurately reproduces the subtle textures and high-frequency edge information of the semantics.

[0131] Step S54: Integrate and stitch together the reconstructed images corresponding to all segmented images to restore the wide-area high-resolution image; that is, based on the segmented images... Obtain the reconstructed image The same operation is used to obtain the reconstructed images of all segmented images in the wide-area high-resolution image, and then stitch all the reconstructed images together to restore the wide-area high-resolution image.

[0132] Please see Figure 3 To better illustrate and verify the reliability of the proposed adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images, wide-area high-resolution images were obtained and processed according to the steps of this method. The compression ratio varied depending on the mask rate, resulting in different reconstruction effects. The first row of images represents the original transmitted image, i.e., the wide-area high-resolution image; the second row of images represents the reconstruction effect with a mask rate of 0% and a compression ratio of 110; and the third row of images represents the reconstruction effect with a mask rate of 25% and a compression ratio of 134.

[0133] Understandably, the proposed adaptive compression and high-frequency detail reconstruction method for wide-area high-resolution images not only dynamically adjusts the compression ratio according to the transmission bandwidth, but also outperforms traditional image compression methods in both perceptual metrics LPIPS and FID for different compression ratios. This advantage indicates that the reconstructed image obtained by this method is closer to the human visual system, more realistic and natural, and visually comfortable. The feature distribution is closer to the distribution of real image sets, effectively overcoming the smoothing and block effects common in traditional methods. By reconstructing realistic texture details, it achieves an improvement in subjective visual quality.

[0134] The second embodiment of the present invention provides an adaptive compression and high-frequency detail accurate reconstruction device for wide-area high-resolution images, the device comprising:

[0135] The image coding unit is used to: acquire and segment a wide-area high-resolution image to obtain several segmented images, and use an encoder to map each segmented image into a low-dimensional continuous latent feature vector.

[0136] The adaptive masking unit is used to: construct an adaptive masking module, obtain texture complexity based on the segmented image, establish a mask matrix, obtain preserved feature vectors by masking low-dimensional continuous latent feature vectors through the mask matrix, and generate a mask location map;

[0137] The vector quantization unit is used to: perform vector quantization on the retained feature vector to generate a token sequence;

[0138] The mask removal unit is used to: map the token sequence back to the quantized feature vector, build a composite feature map by combining the mask location map, and use the mask removal unit to combine global attention and local residual perception to predict and recover the composite feature map based on context information to generate a latent feature map.

[0139] The image reconstruction unit is used to: map the latent feature map back to the continuous pixel space using a decoder, restore the edges and fine textures, obtain the reconstructed image corresponding to the segmented image, integrate and stitch together the reconstructed images corresponding to all segmented images, and restore the wide-area high-resolution image.

[0140] It can be explained that the device comprises five core units: an image encoding unit, an adaptive masking unit, a vector quantization unit, a mask removal unit, and an image reconstruction unit. Specifically, the wide-area high-resolution image is first segmented into smaller images to obtain multiple segmented images. An encoder maps these segmented images into low-dimensional continuous latent feature vectors containing deep semantic information. Then, an adaptive masking module establishes a mask matrix, which masks the aforementioned low-dimensional continuous latent feature vectors to output preserved feature vectors, accurately preserving high-frequency detail regions and masking low-frequency regions to a certain extent. Finally, vector quantization is performed on the preserved feature vectors, i.e., de-masking... The data is processed by dispersion to generate a token sequence for efficient data compression. Simultaneously, the mask location map generated based on the masking operation is transmitted to the receiving end along with the token sequence. Next, the data is removed from the mask. A demasking stage is then performed, utilizing a demasker combined with global attention and local perception capabilities to predict and recover missing features based on contextual information of the unmasked regions, generating a latent feature map. Finally, the image reconstruction stage is initiated. A decoder maps the recovered latent feature map back to continuous pixel space, restoring image edges and fine textures, outputting a high-fidelity reconstructed image, and stitching together the reconstructed images corresponding to all segmented images to form a complete wide-area high-resolution image.

[0141] Understandably, when the device is in operation, it needs to utilize an adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images. Therefore, whether the device and program data are integrated or different hardware is configured to produce functions similar to those achieved by the present invention, they all fall within the protection scope of the present invention.

[0142] The third embodiment of the present invention provides an adaptive compression and high-frequency detail accurate reconstruction system for wide-area high-resolution images. The system includes a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other through the communication bus. The processor calls logical instructions in the memory to execute the adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images described in any of the foregoing embodiments.

[0143] Furthermore, this device or system has the same beneficial effects as the aforementioned method for adaptive compression and high-frequency detail accurate reconstruction of wide-area high-resolution images, which will not be elaborated upon here.

[0144] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0145] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for adaptive compression and accurate reconstruction of high-frequency details in wide-area high-resolution images, characterized in that, The method includes: A wide-area high-resolution image is acquired and segmented to obtain several segmented images. An encoder is used to map each segmented image into a low-dimensional continuous latent feature vector. An adaptive masking module is constructed. Based on the texture complexity obtained from the segmented image, a mask matrix is ​​established. The preserved feature vector is obtained by masking the low-dimensional continuous latent feature vector through the mask matrix, and a mask position map is generated. Perform vector quantization on the retained feature vectors to generate a token sequence; The token sequence is mapped back to the quantized feature vector, and a composite feature map is built by combining the mask location map. The demask is used in combination with global attention and local residual perception. Based on the context information, the composite feature map is predicted and recovered to generate a latent feature map. A decoder is used to map the latent feature map back to the continuous pixel space to restore the edges and fine textures, thereby obtaining the reconstructed image corresponding to the segmented image. All the reconstructed images corresponding to the segmented images are integrated and stitched together to restore the wide-area high-resolution image.

2. The method for adaptive compression and accurate high-frequency detail reconstruction of wide-area high-resolution images according to claim 1, characterized in that, An encoder is used to map each segmented image into a low-dimensional continuous latent feature vector, including: A convolution operation is performed on any segmented image to map it to an initial feature space. The encoder extracts features from the initial feature space through multiple levels to determine the initial feature map. A bottleneck layer is constructed, and the initial feature map is aggregated and refined using global contextual information to obtain deep features. The deep features are mapped to the latent space, and the output is a low-dimensional continuous latent feature vector.

3. The adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images according to claim 2, characterized in that, The encoder extracts features from the initial feature space through multiple levels to determine the initial feature map, specifically: Multiple residual blocks are stacked. The initial feature space is processed by the residual blocks to obtain the output features of the corresponding level. A preset resolution constraint is set. When the output features meet the resolution constraint, bias self-attention blocks are used to perform self-attention processing on the output features, and the initial feature map is obtained through downsampling.

4. The adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images according to claim 2, characterized in that, The bottleneck layer has a hierarchical structure, including a residual block, an autofocus block, and another residual block arranged sequentially.

5. The adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images according to claim 1, characterized in that, An adaptive masking module is constructed. Based on the texture complexity obtained from the segmented image, a mask matrix is ​​established. The preserved feature vectors are obtained by masking the low-dimensional continuous latent feature vectors through the mask matrix, and a mask location map is generated, including: The segmented image is converted into a grayscale image by brightness weighting, and the gradient magnitude map corresponding to the grayscale image is generated by edge operators. The gradient magnitude map is mapped to the same resolution as the low-dimensional continuous latent feature vector by average pooling, thus generating a complexity map. Set the feature retention rate according to the transmission bandwidth requirements, determine the total number of features to be retained, and design a binary mask matrix by combining the complexity graph. The retained feature vectors are obtained by masking the low-dimensional continuous latent feature vectors using a binary mask matrix, and a binary mask location map is generated.

6. The adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images according to claim 1, characterized in that, The retained feature vectors are subjected to vector quantization to generate a token sequence, specifically: The pre-trained learnable discrete codebook uses a vector quantizer to analyze and retain feature vectors and discrete codebooks. The quantized feature vectors are obtained through the quantization mapping process, and the corresponding indices are obtained. The indices are then integrated to build a token sequence.

7. The adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images according to claim 1, characterized in that, The token sequence is mapped back to the quantized feature vector, and a composite feature map is constructed by combining it with the mask location map. A demasking process, combined with global attention and local residual awareness, is used to predict and recover the composite feature map based on contextual information, generating a latent feature map, including: The token sequence is mapped back to the quantized feature vector, and combined with the mask location map, learnable placeholder vectors are filled in the missing positions to generate the initial composite feature map. A demasker incorporating a hybrid processor is used, and an initial composite feature map is input to determine global correlations; Zero-initialized residual connections are introduced into the demask to perform identity mapping, and global correlation is analyzed through self-attention processing. The initial composite feature maps are combined to complete the progressive repair process, resulting in the repaired latent feature map.

8. The adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images according to claim 1, characterized in that, A decoder is used to map the latent feature map back to continuous pixel space, recovering edges and fine textures to obtain the reconstructed image corresponding to the segmented image. All reconstructed images corresponding to the segmented images are then integrated and stitched together to restore a wide-area high-resolution image, including: The latent feature map is projected onto the input dimension of the decoder through convolution, and the global context information is captured by the bottleneck layer of the decoder to output the reconstructed feature map 1. Residual processing and upsampling transformation are performed on the reconstructed feature map 1 in sequence to restore the edges and fine textures, and the reconstructed feature map 2 is output. The reconstructed feature map 2 is then normalized, non-linearly activated, and convolved sequentially, mapped to the RGB pixel space, and the reconstructed image is output. Integrate and stitch together the reconstructed images corresponding to all segmented images to restore the wide-area high-resolution image.

9. A device for adaptive compression and high-frequency detail accurate reconstruction of wide-area high-resolution images, characterized in that, The device includes: The image coding unit is used to: acquire and segment a wide-area high-resolution image to obtain several segmented images, and use an encoder to map each segmented image into a low-dimensional continuous latent feature vector. The adaptive masking unit is used to: construct an adaptive masking module, obtain texture complexity based on the segmented image, establish a mask matrix, obtain preserved feature vectors by masking low-dimensional continuous latent feature vectors through the mask matrix, and generate a mask location map; The vector quantization unit is used to: perform vector quantization on the retained feature vector to generate a token sequence; The mask removal unit is used to: map the token sequence back to the quantized feature vector, build a composite feature map by combining the mask location map, and use the mask removal unit to combine global attention and local residual perception to predict and recover the composite feature map based on context information to generate a latent feature map. The image reconstruction unit is used to: map the latent feature map back to the continuous pixel space using a decoder, restore the edges and fine textures, obtain the reconstructed image corresponding to the segmented image, integrate and stitch together the reconstructed images corresponding to all segmented images, and restore the wide-area high-resolution image.

10. A system for adaptive compression and accurate high-frequency detail reconstruction of wide-area high-resolution images, characterized in that, The system includes a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other through the communication bus. The processor calls logical instructions in the memory to execute the adaptive compression and high-frequency detail accurate reconstruction method for wide-area high-resolution images as described in any one of claims 1 to 8.

Images