Principal component analysis and deep learning based frequency domain autoencoding method and system

Abstract

The application relates to a frequency domain self-encoding method based on principal component analysis and deep learning, which comprises the following steps: performing multistage discrete wavelet transform on a target image, extracting a low-frequency subband and a high-frequency subband, splicing all the subbands after spatial alignment into a packed frequency domain feature tensor in the channel dimension, inputting the packed frequency domain feature tensor into a linear anchor encoding branch and a nonlinear residual encoding branch respectively, obtaining a basic linear latent variable and a nonlinear residual feature, fusing to obtain a compact latent variable, respectively passing through a linear inverse mapping algorithm and a deep nonlinear neural network, obtaining a baseline restoration feature and a frequency domain residual compensation feature according to the compact latent variable, fusing to obtain a frequency domain restoration feature, performing inverse channel unpacking on the frequency domain restoration feature to recover multi-scale wavelet coefficients, and reconstructing an image according to the multi-scale wavelet coefficients. Through the application, the problems of high-frequency detail loss and calculation redundancy of a self-encoder under a high compression ratio are solved, and high-fidelity reconstruction under a high compression ratio is realized.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a frequency domain autoencoder method and system based on principal component analysis and deep learning. Background Technology

[0002] In practical applications of high-resolution digital images from acquisition to transmission, dimensionality reduction and compression of the high-dimensional pixel space are necessary to adapt to downstream processing (such as real-time rendering on mobile devices and cloud storage). Existing technologies typically employ variational autoencoders or pure convolutional networks to map pixel physical values ​​to latent space features. However, at the physical signal processing level, these methods face severe technical bottlenecks at high compression ratios: because high-frequency components in digital image signals (corresponding to physical abrupt changes in object edges and material textures) are easily filtered out in purely linear transformations (such as traditional PCA), and nonlinear networks struggle to stably fit this complex inverse mapping end-to-end, the decoded image exhibits visible blurring and ringing artifacts at the physical display end (such as screens and printers).

[0003] In current image generation applications based on diffusion models (such as text-to-image generation), systems typically need to call an autoencoder to decode the generated low-dimensional latent variables into a high-dimensional pixel matrix in real time. Existing technologies (such as the VAEs used in the Stable Diffusion series) directly perform a black-box mapping from latent space to pixel space. This operation ignores the inherent physical distribution patterns of natural images in the frequency domain—that is, the structural prior of concentrated low-frequency energy and sparse high-frequency details. Due to the lack of explicit constraints on the physical frequency band components, the high-frequency components of the latent variables generated by the diffusion model are often submerged in the reconstruction errors of the low-frequency contours during decoding, resulting in a "plastic" appearance or blurred edges in the generated results.

[0004] In practical engineering deployments, such as running real-time image generation or compression tasks on embedded devices with limited computing resources, like smartphones or autonomous vehicles, the inference process of existing autoencoders (such as FLUX.2-VAE and Hunyuan-VAE) requires billions of floating-point operations (often tens of thousands of GFLOPS), with parameters reaching hundreds of megabytes. This extremely high computational complexity directly leads to the exhaustion of the physical computing power of the device's graphics processing unit (GPU) or neural network processing unit (NPU) and memory bandwidth overload, causing inference latency of up to hundreds of milliseconds (failing to meet real-time requirements) and overheating and a sharp drop in battery life due to prolonged high load. Summary of the Invention

[0005] This application provides a frequency domain autoencoder method, system, electronic device, and storage medium based on principal component analysis and deep learning, to at least solve the problems of high-frequency detail loss and computational redundancy in existing autoencoders with high compression ratios in related technologies.

[0006] In a first aspect, embodiments of this application provide a frequency domain autoencoder method based on principal component analysis and deep learning, the method comprising: Acquire the target image, perform multi-level discrete wavelet transform on the target image, extract at least one low-frequency sub-band and multiple high-frequency sub-bands of different scales, perform spatial alignment operation on each high-frequency sub-band, and stitch all sub-bands together according to the channel dimension to form a packed frequency domain feature tensor with a unified spatial scale. The packed frequency domain feature tensor is input into the linear anchored coding branch and the nonlinear residual coding branch respectively to obtain the basic linear latent variables and the nonlinear residual features. Based on the basic linear latent variables and the nonlinear residual features, the fused compact latent variables are obtained. The baseline restoration features and frequency domain residual compensation features are obtained based on the compact latent variables using a linear inverse mapping algorithm and a deep nonlinear neural network, respectively. Based on the baseline restoration features and the frequency domain residual compensation features, the fused frequency domain restoration features are obtained. The frequency domain restored features are subjected to inverse channel unpacking to recover multi-scale wavelet coefficients. Multi-level inverse discrete wavelet transform is then performed based on the multi-scale wavelet coefficients to output the reconstructed image.

[0007] In some embodiments, the step of inputting the packed frequency domain feature tensor into the linear anchored coding branch and the nonlinear residual coding branch respectively to obtain the basic linear latent variable and nonlinear residual features includes: The packed frequency domain feature tensor is decentralized through the linear anchored coding branch. The basic linear latent variables are obtained by linear mapping based on the pre-trained principal component analysis projection matrix. The nonlinear residual coding branch inputs the packed frequency domain feature tensor into the deep nonlinear neural network encoder to extract the nonlinear residual features.

[0008] In some embodiments, obtaining the fused compact latent variables based on the underlying linear latent variables and the nonlinear residual characteristics includes: The nonlinear residual characteristics are input into a preset latent dynamic gating module to obtain the injected weight vector. The compact latent variable is obtained based on the injected weight vector, the nonlinear residual feature, and the basic linear latent variable.

[0009] In some embodiments, obtaining the fused frequency domain restoration features based on the baseline restoration features and the frequency domain residual compensation features includes: The baseline restoration features are obtained by performing a linear inverse mapping of the compact latent variables using the principal component analysis inverse projection matrix. The compact latent variables are input into the decoder of the deep nonlinear neural network to obtain the frequency domain residual compensation features; The frequency domain residual compensation features are input into a preset frequency domain dynamic gating module to obtain a frequency domain injection weight vector; Based on the frequency domain injection weight vector, the frequency domain residual compensation feature, and the baseline restoration feature, the fused frequency domain restoration feature is obtained.

[0010] In some embodiments, the method further includes: In the first stage of model training, all nonlinear neural network parameters are fixed and statistical calculations are performed only on the training sample set to obtain the mean vector and principal component projection matrix required for principal component analysis. In the second stage of model training, the principal component analysis parameters that have been calculated are fixed, and the deep nonlinear neural network encoder, deep nonlinear neural network decoder and corresponding gating modules are enabled. Joint optimization is performed using a composite loss function, which includes the Charbonnier loss function and the perceptual loss function.

[0011] In some embodiments, both the latent domain dynamic gating module and the frequency domain dynamic gating module are learnable neural network modules. The output layers of the latent domain dynamic gating module and the frequency domain dynamic gating module adopt the Sigmoid activation function, and the output value range is [0,1]. In the second stage of model training, the final network weights of the latent domain dynamic gating module and the frequency domain dynamic gating module are initialized to zero.

[0012] In some embodiments, the spatial alignment operation is a pixel inversion rearrangement operation; The wavelet basis of the multi-level discrete wavelet transform is selected from any one of the Haar wavelet, Daubechies wavelet family, or Bior2.2 wavelet; The channel dimension of the packed frequency domain feature tensor is determined based on the original number of channels of the target image and the level of wavelet decomposition.

[0013] Secondly, embodiments of this application provide a frequency domain autoencoder system based on principal component analysis and deep learning, the system comprising: The extraction module is used to acquire the target image, perform multi-level discrete wavelet transform on the target image, extract at least one low-frequency sub-band and multiple high-frequency sub-bands of different scales, perform spatial alignment operation on each high-frequency sub-band, and stitch all sub-bands together according to the channel dimension to form a packaged frequency domain feature tensor with a unified spatial scale. The first fusion module is used to input the packed frequency domain feature tensor into the linear anchored coding branch and the nonlinear residual coding branch respectively to obtain the basic linear latent variables and the nonlinear residual features, and to obtain the fused compact latent variables based on the basic linear latent variables and the nonlinear residual features. The second fusion module is used to obtain baseline restoration features and frequency domain residual compensation features based on the compact latent variables through a linear inverse mapping algorithm and a deep nonlinear neural network, respectively, and to obtain fused frequency domain restoration features based on the baseline restoration features and the frequency domain residual compensation features. The reconstruction module is used to perform inverse channel unpacking operation on the frequency domain restored features to recover multi-scale wavelet coefficients, perform multi-level inverse discrete wavelet transform based on the multi-scale wavelet coefficients, and output the reconstructed image.

[0014] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the frequency domain autoencoder method based on principal component analysis and deep learning as described in the first aspect above.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the frequency domain autoencoder method based on principal component analysis and deep learning as described in the first aspect above.

[0016] Compared to related technologies, the frequency domain autoencoder method based on principal component analysis and deep learning provided in this application explicitly decomposes the image into low-frequency contours and high-frequency details by introducing multi-level discrete wavelet transform. Combined with a dual-path parallel encoding architecture, it achieves high-fidelity reconstruction while maintaining an extremely lightweight model. Specifically, wavelet decomposition ensures that high-frequency texture information is completely preserved and processed separately, avoiding the problem of severe loss of high-frequency details under high compression ratios in traditional black-box autoencoders. The linear anchoring encoding branch provides a stable and controllable compression baseline, while the nonlinear residual encoding branch only learns the difficult-to-fit residual parts. The fusion of these two methods inherits the training stability and gradient explosion resistance of linear methods while endowing the network with powerful nonlinear fitting capabilities. This solves the problem of high-frequency detail loss and computational redundancy under high compression ratios in autoencoders, achieving high-fidelity reconstruction under high compression ratios and ensuring the stability and physical interpretability of the training process. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a frequency domain autoencoder method based on principal component analysis and deep learning according to an embodiment of this application; Figure 2 This is a schematic diagram of a self-encoder architecture according to an embodiment of this application; Figure 3 This is a structural block diagram of a frequency domain autoencoder system based on principal component analysis and deep learning according to an embodiment of this application; Figure 4 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0019] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0020] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0021] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0022] This embodiment provides a frequency domain autoencoder method based on principal component analysis and deep learning. Figure 1 This is a flowchart of a frequency domain autoencoder method based on principal component analysis and deep learning according to an embodiment of this application, as shown below. Figure 1 As shown, the process includes the following steps: Step S101: Acquire the target image, perform multi-level discrete wavelet transform on the target image, extract at least one low-frequency sub-band and multiple high-frequency sub-bands of different scales, perform spatial alignment operation on each high-frequency sub-band, and stitch all sub-bands together according to the channel dimension to form a packaged frequency domain feature tensor with a unified spatial scale.

[0023] Input the input image x into the wavelet transform module, perform multi-level discrete wavelet transform (DWT), and extract the low-frequency sub-band of the image (optionally, the low-frequency sub-band is set to LL) and multiple high-frequency sub-bands of different scales (optionally, the high-frequency sub-bands are set to LH, HL, HH).

[0024] In some embodiments, the spatial alignment operation is a pixel inversion rearrangement operation; The wavelet basis for the multi-level discrete wavelet transform is selected from any one of the Haar wavelet, Daubechies wavelet family, or Bior2.2 wavelet.

[0025] The channel dimension of the packed frequency domain feature tensor is determined based on the original number of channels of the target image and the level of wavelet decomposition.

[0026] Spatial alignment operations (such as pixel unshuffle) are performed on high-frequency subbands at different levels to map all subbands to a uniform spatial scale. Then, they are concatenated into a fixed-channel sequence according to preset rules, forming a unified packed frequency domain feature tensor r (raw feature), whose dimension is defined as in_channel × 4. level , where in_channel is the original number of channels of the target image, and level is the level of wavelet decomposition.

[0027] Step S102: Input the packed frequency domain feature tensor into the linear anchored coding branch and the nonlinear residual coding branch respectively to obtain the basic linear latent variables and nonlinear residual features. Based on the basic linear latent variables and nonlinear residual features, obtain the fused compact latent variables.

[0028] In some embodiments, step S102 involves inputting the packed frequency domain feature tensor into the linear anchored coding branch and the nonlinear residual coding branch, respectively, to obtain the basic linear latent variable and nonlinear residual features, including: Step S1021: The packed frequency domain feature tensor is decentralized through the linear anchored coding branch.

[0029] Step S1022: Perform linear mapping based on the pre-trained principal component analysis projection matrix to obtain the basic linear latent variables.

[0030] Step S1023: Input the packed frequency domain feature tensor into the deep nonlinear neural network encoder through the nonlinear residual coding branch to extract the nonlinear residual features.

[0031] The processor distributes the packed frequency domain feature tensor r to two parallel coding branches: a linear anchored coding (PCA) branch and a nonlinear residual coding (deep learning) branch.

[0032] Linear anchored coding (PCA) branch: Decentralizes feature r (r c = r - Then, a linear mapping is performed using the principal component analysis (PCA) projection matrix V to obtain the basic linear latent variable z. base =r c *V This branch provides a stable and controllable compression baseline.

[0033] Nonlinear residual coding (deep learning branch): Input the feature r into a deep nonlinear neural network encoder to extract high-dimensional nonlinear residual features z. nl =Enc nl (r).

[0034] The packed frequency domain features are decentralized, and then linear mapping is performed using the pre-trained principal component analysis projection matrix to obtain stable and controllable basic linear latent variables. This design ensures that the encoder can output physically reasonable compression results even in the worst case, effectively preventing the gradient explosion or mode collapse problems common in purely nonlinear networks. Simultaneously, a deep nonlinear neural network encoder is introduced to independently extract nonlinear residual features, simplifying the network's learning task from a complete end-to-end mapping to fitting the residual parts that are difficult to express by linear encoding alone, significantly reducing optimization difficulty and training cost.

[0035] In some embodiments, step S102, which involves obtaining the fused compact latent variables based on the characteristics of the basic linear latent variables and the nonlinear residuals, specifically includes: Step S1024: Input the nonlinear residual characteristics into the preset latent dynamic gating module to obtain the injected weight vector.

[0036] Step S1025: Based on the injected weight vector, nonlinear residual features, and basic linear latent variables, compact latent variables are obtained.

[0037] Latent variable domain dynamic gating fusion: Introducing a latent domain dynamic gating module G α The injection weights of residual features are calculated by channel, and then expressed using the formula. The mixture is then fused to output the final compact latent variable z.

[0038] It should be noted that the latent dynamic gating module is a learnable, lightweight neural network submodule located between the end of the encoder's nonlinear residual branch and the fusion node. Its core function is to adaptively calculate the injection ratio of nonlinear residual features in the channel dimension, thereby determining how much nonlinear correction information is ultimately added to the linear coding baseline.

[0039] This embodiment takes nonlinear residual features as input and outputs an injection weight vector with the same dimension as the channel. This weight vector is then multiplied element-wise with the nonlinear residual features and superimposed onto the basic linear latent variables to obtain the final compact latent variables. This gating mechanism allows the network to autonomously determine how much nonlinear information to inject for different channels. For channels that can be recovered using a linear base (such as low-frequency contours), the gating weights approach 0; for channels requiring detailed refinement (such as high-frequency textures), the gating weights approach 1. Furthermore, a zero-initialization training strategy is employed. The latent dynamic gating module completely shields the influence of nonlinear branches in the early stages of training, making the model behave like a pure PCA encoder. Nonlinear corrections are then smoothly introduced as training progresses, thus completely avoiding the divergence risk in the early stages of end-to-end training.

[0040] For example, suppose the latent dimension (C) is set to 4.

[0041] Assume the linear baseline feature obtained after PCA projection is z base = [1.0, 2.0, -1.0, 0.5], which is a stable basic characteristic.

[0042] The nonlinear residual feature obtained by the deep nonlinear network extraction is z. nl = [0.8, -0.4, 1.2, 0.1], which represents the high-frequency or nonlinear details that the network believes need to be supplemented.

[0043] Submarine Dynamic Gating Module G α (For example: a small multilayer perceptron (MLP) with a sigmoid activation function at the end) receives z nl As input, it outputs a 4-dimensional weight vector, known as the injected weights. Each element of this weight vector has a value between 0 and 1, representing the proportion of the non-linear features of the corresponding channel that are injected into the final result.

[0044] enter:

[0045] Processing (conceptual example): The network determines that channels 1 and 3 require more correction, while channels 2 and 4 only need minor adjustments.

[0046] Output injection weight α (i.e., G) α (z nl Assume the output weight vector is α = [0.9, 0.1, 0.8, 0.01].

[0047] It should be noted that in the early stages of training, due to the zero initialization strategy adopted in this embodiment, the value of this α vector is usually very close to all 0 (e.g., [0.001, 0.001, 0.001, 0.001]). The larger differential weights mentioned above are gradually learned as training progresses.

[0048] The fusion process is based on channel-wise element-wise multiplication (Hadamard Product, symbol...). ) and add them together.

[0049] Calculate the weighted residuals ( The injected weights are element-wise multiplied with the nonlinear features:

[0050] This step achieves gating of the nonlinear residuals, retaining most of the information in channels 1 and 3, while suppressing the information in channels 2 and 4.

[0051] Add the result from the previous step to the linear baseline feature:

[0052] The final compact latent variable is z = [1.72, 1.96, -0.04, 0.501]. This final variable contains both the stable linear component of PCA and adaptively incorporates the necessary nonlinear detail correction components.

[0053] Step S103: Baseline restoration features and frequency domain residual compensation features are obtained based on compact latent variables using linear inverse mapping algorithm and deep nonlinear neural network, respectively. Based on the baseline restoration features and frequency domain residual compensation features, the fused frequency domain restoration features are obtained.

[0054] In some embodiments, step S103 specifically includes: Step S1031: Linear inverse mapping of compact latent variables is performed using principal component analysis inverse projection matrix to obtain baseline restoration features.

[0055] Step S1032: Input the compact latent variables into the decoder of the deep nonlinear neural network to obtain the frequency domain residual compensation features.

[0056] Step S1033: Input the frequency domain residual compensation features into the preset frequency domain dynamic gating module to obtain the frequency domain injection weight vector.

[0057] Step S1034: Based on the frequency domain injection weight vector, frequency domain residual compensation features, and baseline restoration features, the fused frequency domain restoration features are obtained.

[0058] At the decoding end, the processor receives the compact latent variable z (in the compressed transmission task, this variable comes from the network reception; in the generation task, this variable comes from the generation output of the diffusion model): The latent variable z is recovered using the PCA inverse projection matrix, as shown in the formula: r base_rec = z·V T +μ yields the baseline restoration features, μ is the mean of PCA, and V is the principal component matrix.

[0059] latent variables Input a deep nonlinear neural network decoder to calculate the frequency domain residual compensation feature r. nl_rec =Dec nl (z). Among them, Dec nl (·) represents the mathematical representation of a deep nonlinear neural network decoder.

[0060] Introducing a frequency domain dynamic gating module G β Through formula The frequency domain reconstruction features r after fusion are calculated. rec .

[0061] By directly restoring the baseline restoration features through the inverse projection matrix of principal component analysis, the stable restoration of the low-frequency contours and overall structure of the image is ensured, avoiding the structural collapse or artifact problems common in pure nonlinear decoders. Furthermore, a deep nonlinear neural network decoder is introduced to independently extract frequency domain residual compensation features, which are specifically responsible for learning high-frequency textures and edge details that are difficult to restore by linear inverse mapping. This decomposes the decoding task into two orthogonal sub-problems: subject restoration and detail compensation, which significantly reduces the learning difficulty of the network.

[0062] A frequency-domain dynamic gating module is introduced to achieve adaptive weighted fusion of frequency-domain residual compensation features along the channel dimension. The module takes the frequency-domain residual compensation features as input and outputs a frequency-domain injection weight vector consistent with the channel dimension. This vector is then superimposed on the baseline restoration features after element-wise multiplication with the residual features. This gating mechanism enables the decoder to make autonomous decisions for different frequency subbands: for well-restored low-frequency channels, the gating weights approach zero to suppress unnecessary nonlinear interference; for high-frequency channels lacking detail, the gating weights approach one to maximize the injection of nonlinear refinement information. The frequency-domain dynamic gating module employs a zero-initialization training strategy. This mechanism makes the decoder behavior equivalent to a pure PCA inverse mapping in the early stages of training, and smoothly introduces nonlinear compensation as training progresses, completely avoiding the divergence risk of end-to-end training. This is one of the key technologies for achieving extremely low-cost training and high-fidelity reconstruction.

[0063] In the decoding and reconstruction stage, the model needs to restore the latent variables to frequency domain features (i.e., wavelet coefficients), and then reconstruct the image through inverse wavelet transform. (Formula) Adaptive fusion of linear baseline features and nonlinear detail features in the frequency domain (especially across different frequency bands) has been achieved. To clearly illustrate this process, an example based on specific tensor dimensions and physical meaning is provided below: Suppose that at a certain spatial location (x,y), we need to reconstruct a set of frequency domain features containing 4 channels (e.g., the LL low frequency, LH horizontal high frequency, HL vertical high frequency, and HH diagonal high frequency subbands corresponding to wavelet transform).

[0064] Fundamental frequency domain restoration feature r base_rec Generated by a linear decoder (such as inverse PCA), it primarily recovers the overall energy and low-frequency contours of the image. Assume the baseline restoration feature at this location is r. base_rec = [10.0, 0.5, -0.5, 0.1] (representing a larger value for the first low-frequency channel).

[0065] Nonlinear frequency domain residual characteristics r nl_recGenerated by a deep nonlinear decoding network, it primarily aims to supplement lost high-frequency textures and edge details. Assume the output nonlinear frequency domain residual features are r. nl_rec = [0.2, 1.5, -2.0, 0.8].

[0066] Frequency Domain Dynamic Gating Module G β Receive nonlinear feature r nl_rec (Or combined with other contextual features) as input, passed through network layers (same as G above) α The normalized weight vector β (range 0~1) is calculated and output, which is consistent with the channel dimension.

[0067] The module determines that the low frequency (channel 1) has been recovered well enough from the basic features and does not need much nonlinear interference; while the high frequency (channels 2 and 3) has serious loss of details and requires a large amount of nonlinear residual injection.

[0068] Output weight β (i.e. G) β (r nl_rec Assuming the calculated injection weights are β = [0.05, 0.09, 0.85, 0.40].

[0069] Multiply the weights by the nonlinear residuals along the channels (elements) to calculate the weighted frequency domain residuals. ):

[0070] The weighted frequency domain residual effectively suppresses unwanted low-frequency residual noise and amplifies the high-frequency details that need to be supplemented.

[0071] The weighted residuals are superimposed back onto the basic frequency domain features to obtain the frequency domain restored features r. rec :

[0072] Step S104: Perform inverse channel unpacking operation on the frequency domain restored features to recover multi-scale wavelet coefficients, perform multi-level inverse discrete wavelet transform based on the multi-scale wavelet coefficients, and output the reconstructed image.

[0073] The fused frequency domain restored features r rec The channel is unpacked (reverse splitting operation), followed by a multi-level inverse discrete wavelet transform (IDWT), and finally the reconstructed image x is output. rec .

[0074] Through the steps described above, a multi-level discrete wavelet transform is introduced to explicitly decompose the image into low-frequency contours and high-frequency details. Combined with a dual-path parallel coding architecture, high-fidelity reconstruction is achieved while maintaining the model's extreme lightweight nature. Specifically, wavelet decomposition ensures that high-frequency texture information is fully preserved and processed separately, avoiding the problem of severe loss of high-frequency details in traditional black-box autoencoders under high compression ratios. The linear anchoring coding branch provides a stable and controllable compression baseline, while the nonlinear residual coding branch only learns the difficult-to-fit residual parts. The fusion of these two methods inherits the training stability and gradient explosion resistance of linear methods while endowing the network with powerful nonlinear fitting capabilities. This solves the problem of high-frequency detail loss and computational redundancy under high compression ratios in autoencoders, achieving high-fidelity reconstruction under high compression ratios while ensuring the stability and physical interpretability of the training process.

[0075] In some embodiments, the method further includes: In the first stage of model training, all nonlinear neural network parameters are fixed and statistical calculations are performed only on the training sample set to obtain the mean vector and principal component projection matrix required for principal component analysis.

[0076] In the second stage of model training, the principal component analysis parameters that have been calculated are fixed, and the deep nonlinear neural network encoder, deep nonlinear neural network decoder and corresponding gating modules are enabled. Joint optimization is performed using composite loss functions, including the Charbonnier loss function and the perceptual loss function.

[0077] To ensure training convergence, the training system in this embodiment performs the following steps: Phase 1 (Linear Fitting): Freeze all nonlinear networks and only perform statistical fitting on the sample set to calculate the PCA mean μ and principal component matrix V.

[0078] Phase Two (Residual Learning): Freeze PCA parameters and enable the nonlinear network. Specifically, the dynamic gating module G... α and G β The weights of the final layer are initialized to zero. This makes the output behavior of the network equivalent to the pure PCA baseline in the early stages of training, and then the nonlinear residuals are gradually learned during backpropagation.

[0079] Joint loss optimization is performed using the composite loss function L = λ1·L charbonnier (x rec ,x)+λ2·L lpips (x rec ,x) Update the weights of the non-linear branches.

[0080] This approach completely solves the technical challenges of unstable training and slow convergence in traditional end-to-end autoencoders by employing a divide-and-conquer strategy. The first stage fixes all nonlinear network parameters and only calculates the mean vector and projection matrix of principal component analysis based on the training sample set. Essentially, this provides the model with a physically optimal linear compression baseline. This stage requires no gradient backpropagation or iterative optimization, resulting in extremely low computational cost and deterministic results, ensuring usable reconstruction capabilities even in the worst-case scenario. The second stage keeps the trained principal component analysis parameters unchanged and only uses the nonlinear network and gating module for joint optimization. This design simplifies the learning task of the nonlinear network from a complete end-to-end mapping to simply fitting the residual portion that the principal component analysis baseline cannot express, significantly reducing the optimization difficulty and the risk of getting trapped in local optima.

[0081] Furthermore, in the second stage, the composite loss function employs a strategy combining Charbonnier loss and perceptual loss. Charbonnier loss is a robust variant of L2 loss, insensitive to outliers, and can stably constrain pixel-level reconstruction accuracy; perceptual loss is calculated based on feature maps of a pre-trained deep network, enabling better supervision of high-fidelity restoration of high-frequency textures and semantic structures. The joint optimization of these two loss mechanisms achieves the optimal balance between pixel-level accuracy and perceptual quality. Combined with the aforementioned zero-initialization gating strategy, the model output in the early stages of training is equivalent to the pure principal component analysis baseline, avoiding the gradient explosion risk caused by random initialization; as training progresses, the gating module gradually opens, allowing the network to smoothly learn nonlinear residual compensation capabilities.

[0082] In some embodiments, both the latent domain dynamic gating module and the frequency domain dynamic gating module are learnable neural network modules. The output layers of the latent domain dynamic gating module and the frequency domain dynamic gating module adopt the Sigmoid activation function, and the output value range is [0,1]. In the second phase of model training, the weights of the last layer networks in the latent domain dynamic gating module and the frequency domain dynamic gating module are initialized to zero.

[0083] The Sigmoid function smoothly maps the gated output to the 0-1 interval, allowing the model to learn how much nonlinear information to inject in a gradient-friendly manner channel by channel. When the gated output is close to 0, the nonlinear residual of the corresponding channel is effectively suppressed, retaining only the stable output of the linear baseline; when the gated output is close to 1, the nonlinear residual of the corresponding channel is fully injected, maximizing compensation for high-frequency details. This soft-switching mechanism avoids the non-differentiability problem caused by discrete decision-making, enabling the network to optimize the gated weights end-to-end through backpropagation, while maintaining the continuity and differentiability of the fusion process.

[0084] Furthermore, in the second stage of model training, the final layer network weights of the latent domain dynamic gating module and the frequency domain dynamic gating module are initialized to zero. This strategy has key technical significance. The direct consequence of zero final layer weights is that regardless of the input value, the initial output of the gating module is always a zero vector. At this point, the behavior of the entire encoder and decoder is completely equivalent to a pure principal component analysis linear encoder and decoder, while the nonlinear residual branches, although present in the network, are completely masked. This ensures that the training starting point is at a physically optimal stable point verified in the first stage, completely avoiding the risk of gradient explosion or mode collapse caused by random initialization of the nonlinear network. As training iterates, the final layer weights are gradually updated from zero to non-zero values. The gating module smoothly and progressively opens the nonlinear injection paths of each channel, enabling the network to automatically identify which channels require nonlinear compensation and which channels only require a linear basis.

[0085] In some embodiments, the method further includes frequency domain co-generation and reconstruction in a downstream diffusion model. Specifically, this includes: Latent space diffusion and generation: Downstream diffusion models (such as DiT, UViT, etc.) based on the latent variables generated by the aforementioned Training and denoising sampling are performed in the low-dimensional space formed by the diffusion generation space. To unify the variance of the diffusion generation space, the following steps are taken: A scaling factor 1 / std based on standard deviation can be introduced before inputting the diffusion model.

[0086] Wavelet frequency domain feature reconstruction (key difference): Diffusion model denoising sampling outputs the final latent variable z. gen Afterwards, the system does not directly decode it into RGB pixels, but instead converts the z-axis to RGB pixels. gen The input is fed into the aforementioned dual-path parallel decoder, where multi-scale wavelet high and low frequency coefficients (i.e., packed features r) are explicitly calculated and reconstructed through the fusion of linear inverse mapping and nonlinear residual gating. rec ).

[0087] Physical inverse transform output: the reconstructed wavelet coefficients r rec After unpacking according to spatial rules, a multi-level inverse discrete wavelet transform (IDWT) is performed to restore the final generated image. (During this process, the autoencoder can reside directly in GPU memory as a unified interface component (Wrapper), enabling seamless online transformation of latent variables and avoiding disk write-to-disk conversion.)

[0088] Figure 2 This is a schematic diagram of a self-encoder architecture according to an embodiment of this application, as shown below. Figure 2 As shown, the dashed line represents the process that can be applied to the downstream diffusion model generation process.

[0089] This application completely eliminates the drawback of traditional VAEs heavily relying on massive convolutional stacks. Table 1 is a comparison table of inference efficiency and computational overhead between the method according to the embodiments of this application and mainstream VAE schemes. The table takes 1024 resolution inference as an example. As shown in the experimental data in Table 1, under the same spatial downsampling rate, the resource overhead of this application is reduced by an order of magnitude: With an 8x spatial downsampling configuration, compared to mainstream FLUX.2-VAE and SD-VAE, the parameter size of this application is drastically reduced to 1.77 MB (approximately 1 / 47 of the comparable schemes), and the computational cost is only 57.6 GFLOPS (approximately 1 / 247 of the existing schemes). In terms of throughput, this scheme achieves 280.6 img / s (an improvement of approximately 84 times), and the single-image latency is drastically reduced from approximately 367ms to a staggering 4.97ms.

[0090] With a 32x downsampling configuration, compared to state-of-the-art Hunyuan-VAE and DC-AE, this application achieves only 14.54 MB of parameters (less than 1 / 20th of the comparison schemes) and 33.79 GFLOPS of computation (less than 1 / 400th of the comparison schemes). The throughput reaches an unprecedented 502.0 img / s (an improvement of over 114 times), with a latency of only 7.09 ms. In summary, these metrics demonstrate that this method is fully capable of real-time encoding / decoding and generative diffusion inference on resource-constrained edge devices.

[0091] Table 1

[0092] Traditional deep nonlinear VAEs require extremely long periods and massive computing power to complete distribution fitting (typically requiring monthly cluster training). This application, however, significantly reduces optimization difficulty by employing a two-stage architecture: PCA anchoring to provide a linear foundation and a gated network for zero-initialization learning of residuals. Experiments show that on the ImageNet dataset, this application requires only a single A800 GPU and can complete the entire model training and convergence in less than 4 hours (consuming only 50,000 steps). This not only greatly reduces the computational cost of developing the underlying layer of frontier diffusion models but also makes it possible to quickly fine-tune / retrain dedicated frequency domain autoencoders for specific vertical scenarios (such as medical imaging and satellite remote sensing).

[0093] This application can flexibly adjust the latent dimension according to actual needs, and maintain excellent perceptual quality even under extremely high compression rates, thus solving the bottleneck of high-frequency detail loss under high compression.

[0094] Table 2 shows the performance data of 8x spatial downsampling with different latent variable dimensions (ImageNet256) according to the embodiments of this application. As shown in Table 2, when the latent dimension is 8, a stable reconstruction baseline can still be maintained at a compression rate of up to 95.83% (PSNR=25.07, LPIPS=0.2262); as the dimension is increased to 64 (66.67% compression rate), the model exhibits extremely high fidelity with almost no loss (PSNR=35.22, LPIPS is only 0.0168, rFID is extremely low to 0.0635).

[0095] Table 2

[0096] Table 3 shows the performance data of a 32x spatial downsampling method with different latent variable dimensions (ImageNet256) according to an embodiment of this application. As shown in Table 3, even with a 32x spatial compression, when the latent dimension reaches 1024 (66.67% compression rate), the reconstructed image metrics still achieve top-tier levels of PSNR=34.45 and LPIPS=0.0291. Summary: This result demonstrates that the dual-domain gated residual scheme of this invention can adaptively capture signals in different frequency bands and possesses excellent scalability.

[0097] Table 3

[0098] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0099] This embodiment also provides a frequency domain autoencoder system based on principal component analysis and deep learning. This system is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0100] Figure 3 This is a structural block diagram of a frequency domain autoencoder system based on principal component analysis and deep learning according to an embodiment of this application, as shown below. Figure 3 As shown, the system includes: Extraction module 31 is used to acquire the target image, perform multi-level discrete wavelet transform on the target image, extract at least one low-frequency sub-band and multiple high-frequency sub-bands of different scales, perform spatial alignment operation on each high-frequency sub-band, and stitch all sub-bands together according to the channel dimension to form a packaged frequency domain feature tensor with a unified spatial scale. The first fusion module 32 is used to input the packed frequency domain feature tensor into the linear anchored coding branch and the nonlinear residual coding branch respectively to obtain the basic linear latent variables and nonlinear residual features, and to obtain the fused compact latent variables based on the basic linear latent variables and nonlinear residual features. The second fusion module 33 is used to obtain baseline restoration features and frequency domain residual compensation features based on compact latent variables through linear inverse mapping algorithm and deep nonlinear neural network, respectively, and to obtain fused frequency domain restoration features based on baseline restoration features and frequency domain residual compensation features; The reconstruction module 34 is used to perform inverse channel unpacking operation on the frequency domain restored features, recover the multi-scale wavelet coefficients, perform multi-level inverse discrete wavelet transform based on the multi-scale wavelet coefficients, and output the reconstructed image.

[0101] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0102] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0103] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0104] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program: S1. Acquire the target image, perform multi-level discrete wavelet transform on the target image, extract at least one low-frequency sub-band and multiple high-frequency sub-bands of different scales, perform spatial alignment operation on each high-frequency sub-band, and stitch all sub-bands together according to the channel dimension to form a packaged frequency domain feature tensor with a unified spatial scale.

[0105] S2, input the packed frequency domain feature tensor into the linear anchored coding branch and the nonlinear residual coding branch respectively to obtain the basic linear latent variables and nonlinear residual features. Based on the basic linear latent variables and nonlinear residual features, the fused compact latent variables are obtained.

[0106] S3 obtains baseline restoration features and frequency domain residual compensation features based on compact latent variables through linear inverse mapping algorithm and deep nonlinear neural network, respectively. Based on the baseline restoration features and frequency domain residual compensation features, the fused frequency domain restoration features are obtained.

[0107] S4 performs inverse channel unpacking on the frequency domain restored features to recover multi-scale wavelet coefficients. Based on the multi-scale wavelet coefficients, it performs multi-level inverse discrete wavelet transform and outputs the reconstructed image.

[0108] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0109] In one embodiment, Figure 4 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 4 As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 4 As shown, this electronic device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a frequency domain autoencoding method based on principal component analysis and deep learning.

[0110] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0111] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0112] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0113] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A frequency domain autoencoder method based on principal component analysis and deep learning, characterized in that, The method includes: Acquire the target image, perform multi-level discrete wavelet transform on the target image, extract at least one low-frequency sub-band and multiple high-frequency sub-bands of different scales, perform spatial alignment operation on each high-frequency sub-band, and stitch all sub-bands together according to the channel dimension to form a packed frequency domain feature tensor with a unified spatial scale. The packed frequency domain feature tensor is input into the linear anchored coding branch and the nonlinear residual coding branch respectively to obtain the basic linear latent variables and the nonlinear residual features. Based on the basic linear latent variables and the nonlinear residual features, the fused compact latent variables are obtained. The baseline restoration features and frequency domain residual compensation features are obtained based on the compact latent variables using a linear inverse mapping algorithm and a deep nonlinear neural network, respectively. Based on the baseline restoration features and the frequency domain residual compensation features, the fused frequency domain restoration features are obtained. The frequency domain restored features are subjected to inverse channel unpacking to recover multi-scale wavelet coefficients. Multi-level inverse discrete wavelet transform is then performed based on the multi-scale wavelet coefficients to output the reconstructed image.

2. The method according to claim 1, characterized in that, The step of inputting the packed frequency domain feature tensor into the linear anchored coding branch and the nonlinear residual coding branch respectively to obtain the basic linear latent variable and nonlinear residual features includes: The packed frequency domain feature tensor is decentralized through the linear anchored coding branch. The basic linear latent variables are obtained by linear mapping based on the pre-trained principal component analysis projection matrix. The nonlinear residual coding branch inputs the packed frequency domain feature tensor into the deep nonlinear neural network encoder to extract the nonlinear residual features.

3. The method according to claim 2, characterized in that, The fused compact latent variables obtained based on the basic linear latent variables and the nonlinear residual characteristics include: The nonlinear residual characteristics are input into a preset latent dynamic gating module to obtain the injected weight vector. The compact latent variable is obtained based on the injected weight vector, the nonlinear residual feature, and the basic linear latent variable.

4. The method according to claim 3, characterized in that, The fused frequency domain restoration features obtained based on the baseline restoration features and the frequency domain residual compensation features include: The baseline restoration features are obtained by performing a linear inverse mapping of the compact latent variables using the principal component analysis inverse projection matrix. The compact latent variables are input into the decoder of the deep nonlinear neural network to obtain the frequency domain residual compensation features; The frequency domain residual compensation features are input into a preset frequency domain dynamic gating module to obtain a frequency domain injection weight vector; Based on the frequency domain injection weight vector, the frequency domain residual compensation feature, and the baseline restoration feature, the fused frequency domain restoration feature is obtained.

5. The method according to claim 4, characterized in that, The method further includes: In the first stage of model training, all nonlinear neural network parameters are fixed and statistical calculations are performed only on the training sample set to obtain the mean vector and principal component projection matrix required for principal component analysis. In the second stage of model training, the principal component analysis parameters that have been calculated are fixed, and the deep nonlinear neural network encoder, deep nonlinear neural network decoder and corresponding gating modules are enabled. Joint optimization is performed using a composite loss function, which includes the Charbonnier loss function and the perceptual loss function.

6. The method according to claim 5, characterized in that, Both the latent domain dynamic gating module and the frequency domain dynamic gating module are learnable neural network modules. The output layers of the latent domain dynamic gating module and the frequency domain dynamic gating module adopt the Sigmoid activation function, and the output value range is [0,1]. In the second stage of model training, the final network weights of the latent domain dynamic gating module and the frequency domain dynamic gating module are initialized to zero.

7. The method according to claim 1, characterized in that, The spatial alignment operation is a pixel inverse rearrangement operation; The wavelet basis of the multi-level discrete wavelet transform is selected from any one of the Haar wavelet, Daubechies wavelet family, or Bior2.2 wavelet; The channel dimension of the packed frequency domain feature tensor is determined based on the original number of channels of the target image and the level of wavelet decomposition.

8. A frequency domain autoencoder system based on principal component analysis and deep learning, characterized in that, The system includes: The extraction module is used to acquire the target image, perform multi-level discrete wavelet transform on the target image, extract at least one low-frequency sub-band and multiple high-frequency sub-bands of different scales, perform spatial alignment operation on each high-frequency sub-band, and stitch all sub-bands together according to the channel dimension to form a packaged frequency domain feature tensor with a unified spatial scale. The first fusion module is used to input the packed frequency domain feature tensor into the linear anchored coding branch and the nonlinear residual coding branch respectively to obtain the basic linear latent variables and the nonlinear residual features, and to obtain the fused compact latent variables based on the basic linear latent variables and the nonlinear residual features. The second fusion module is used to obtain baseline restoration features and frequency domain residual compensation features based on the compact latent variables through a linear inverse mapping algorithm and a deep nonlinear neural network, respectively, and to obtain fused frequency domain restoration features based on the baseline restoration features and the frequency domain residual compensation features. The reconstruction module is used to perform inverse channel unpacking operation on the frequency domain restored features to recover multi-scale wavelet coefficients, perform multi-level inverse discrete wavelet transform based on the multi-scale wavelet coefficients, and output the reconstructed image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the frequency domain autoencoder method based on principal component analysis and deep learning as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the frequency domain autoencoder method based on principal component analysis and deep learning as described in any one of claims 1 to 7.

Images