Device-adaptive image encoding and decoding method

Abstract

The present invention discloses a device-adaptive image encoding and decoding method, comprising: obtaining hardware parameters of a target device and mapping them into a device feature vector; performing feature extraction on the image to be encoded and decoded to obtain an image content feature vector; inputting the parameters into a hypernetwork generator to dynamically generate a codec parameter set adapted to the target device based on a gated residual structure; configuring a codec to encode and decode the image to be encoded and decoded to obtain a codec image; calculating the distribution difference between the codec image and the output of a baseline codec, and adjusting the encoding and decoding process through an online distillation mechanism to obtain an optimized codec image. This method solves the problem of traditional codecs requiring individual optimization for each device, reducing fluctuations in peak signal-to-noise ratio between different devices.

Description

Technical Field

[0001] The present invention belongs to the field of image coding and decoding, and in particular to a device adaptive image coding and decoding method. Background Art

[0002] Image coding and decoding technology, as a core component of multimedia communications and edge computing, faces severe challenges brought about by device heterogeneity. The International Telecommunication Union points out that 68% of global image transmission delays are due to the mismatch between codecs and hardware. In scenarios such as AR / VR surgical navigation, the inability of traditional fixed-parameter models to adapt to device differences leads to average PSNR fluctuations of up to 4.2dB. This phenomenon is particularly prominent in the field of medical image transmission. US FDA statistics show that 23% of remote diagnosis disputes are directly related to cross-device image quality degradation. The industry urgently needs an intelligent compression architecture that can achieve single-training and full-end adaptation to cope with the rapid iteration of new hardware such as foldable phones and holographic projection devices.

[0003] Existing solutions suffer from systemic flaws in dynamic adaptation, creating a bottleneck in technical optimization. Universal codec standards improve compatibility by adding coding unit division patterns, but their algorithmic complexity increases exponentially, increasing power consumption when deployed on mobile devices. Dedicated optimization solutions require maintaining independent model libraries for different devices, increasing model storage overhead when supporting 20 types of IoT devices. NAS-based lightweight methods are limited by the preset search space, and Google's FALSR solution suffers from reduced compression efficiency when encountering unregistered device types.

[0004] The limitations of device feature extraction and parameter adjustment mechanisms are key factors that restrict performance. Traditional methods rely on manually set hardware indicators and cannot capture the nonlinear relationship between NPU memory bandwidth and GPU tensor cores, resulting in a waste of computing resources. Although device fingerprint libraries have been established in existing technologies, their rule-based feature matching has difficulty handling dynamic changes in the screen color gamut, resulting in color difference anomalies. In terms of parameter adjustment, the linear scaling strategy adopted by existing technologies has large fluctuations in encoding and decoding delays when processing 4K / 8K resolution switching. In addition, existing methods usually freeze the teacher model parameters. When the target device exceeds the training set range, the peak signal-to-noise ratio (PSNR) of the student model decreases. Summary of the Invention

[0005] The purpose of the invention is to provide a device adaptive image encoding and decoding method, in order to solve at least one technical problem existing in the prior art.

[0006] The technical solution is a device-adaptive image encoding and decoding method, including:

[0007] Obtain the hardware parameters of the target device and map them into device feature vectors through the device feature encoding network;

[0008] Perform feature extraction on the image to be encoded and decoded to obtain the image content feature vector;

[0009] The device and image content feature vectors are input into the hypernetwork generator, which dynamically generates a codec parameter set adapted to the target device based on the gated residual structure.

[0010] Configure the codec using the codec parameter set, perform encoding and decoding on the image to be coded and decoded, and obtain a coded and decoded image;

[0011] The distribution difference between the codec image and the benchmark codec output is calculated, and the encoding and decoding process is adjusted based on the distribution difference through an online distillation mechanism to obtain the optimized codec image.

[0012] Beneficial effects: The present invention solves the problem that traditional codecs need to be optimized separately for each device, reduces the fluctuation of peak signal-to-noise ratio between different devices, and at the same time reduces model storage requirements and simplifies computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A flowchart of the steps of a device adaptive image encoding and decoding method provided in an embodiment of the present application.

[0014] Figure 2 A flowchart of the steps for mapping hardware parameters into device feature vectors through a device feature encoding network provided in an embodiment of the present application.

[0015] Figure 3 This is a flowchart of the steps for dynamically generating a codec parameter set adapted to a target device based on a gated residual structure provided in an embodiment of the present application.

[0016] Figure 4 A flowchart of the steps for implementing soft parameter selection using a learnable threshold function in a first gating unit provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or are inherent to these processes, methods, products or apparatus.

[0019] Research has revealed that the core technical challenge facing current image encoding and decoding systems when deployed across devices lies in the inherent conflict between static model architectures and dynamic hardware environments. Traditional approaches, using fixed-parameter models, are unable to adapt to differences in computing power (2-8 cores on mobile CPUs vs. 16-80 cores on cloud GPUs), screen characteristics (120% NTSC color gamut on mobile OLEDs vs. 72% NTSC on surveillance screens), and bandwidth fluctuations (dynamic variations of 20-100Mbps on 5G networks), resulting in PSNR fluctuations exceeding 32%. In edge computing scenarios, when the same model needs to be deployed simultaneously on drone vision modules (4TOPS of computing power) and cloud inference cards (200TOPS of computing power), existing technologies either sacrifice compression rate to ensure compatibility or require separate model training for each device type, increasing storage overhead by over 300%. This results in an additional decoding delay of 17-23ms for ultra-high-definition video transmission, severely hindering the user experience for real-time applications such as AR / VR. A breakthrough solution is urgently needed to address the compatibility issues between distillation optimization mechanisms and new devices. Traditional distillation loss functions suffer from vanishing gradients when the feature dimensions of a device suddenly change. When the computing power difference of devices exceeds 15% of the maximum value of the training set, the compression efficiency of the traditional solution decreases.

[0020] like Figure 1 As shown, a device adaptive image encoding and decoding method is provided, comprising the following steps:

[0021] Obtain the hardware parameters of the target device and map them into device feature vectors through the device feature encoding network;

[0022] Perform feature extraction on the image to be encoded and decoded to obtain the image content feature vector;

[0023] The device and image content feature vectors are input into the hypernetwork generator, which dynamically generates a codec parameter set adapted to the target device based on the gated residual structure.

[0024] Configure the codec using the codec parameter set, perform encoding and decoding on the image to be coded and decoded, and obtain a coded and decoded image;

[0025] The distribution difference between the codec image and the benchmark codec output is calculated, and the encoding and decoding process is adjusted based on the distribution difference through an online distillation mechanism to obtain the optimized codec image.

[0026] Specifically, the hardware parameters of the target device are obtained and mapped into a device feature vector using a device feature encoding network. These hardware parameters include, but are not limited to, 12 dimensions, such as the number of CPU cores, GPU floating-point computing power, screen resolution, screen color gamut coverage, and memory bandwidth. For example, the hardware parameters of a mobile device equipped with an A15 chip include a 6-core CPU, a 5-core GPU, 18TOPS computing power, a 2796×1290 resolution, and a 120Hz refresh rate. The device feature encoding network applies a nonlinear transformation to these raw parameters, ultimately mapping them into an 8-dimensional feature vector that uniquely identifies the device's hardware characteristics, like a device fingerprint. Feature extraction is performed on the image to be encoded or decoded, generating an image content feature vector. Specifically, a pre-trained ResNet-18 network can be used to extract features from the input image, outputting a 64-dimensional image content feature vector that captures key information such as texture, edges, and color distribution. The device feature vector and image content feature vector are then fed into a hypernetwork generator, which dynamically generates a codec parameter set tailored to the target device based on a gated residual structure. The hypernetwork acts as a parameter factory, generating optimal codec parameters in real time based on different devices and image content. The generated parameter set includes codec parameters such as quantization tables, transform coefficients, and prediction modes, and the entire generation process takes only 8-10 milliseconds. The codec is configured using the codec parameter set to encode and decode the image to be encoded, resulting in a coded image. Loaded with device-specific parameters, the codec can fully utilize the device's hardware performance while maintaining image quality. The codec calculates the distribution difference between the coded image and the baseline codec output. Based on this difference, an online distillation mechanism adjusts the encoding and decoding process to produce an optimized coded image. This process acts as a quality control for the encoding and decoding results. By comparing the output with industry-standard codecs (such as VTM11.0), it ensures that the adaptive encoding does not deviate too far while maintaining device characteristics.

[0027] In practice, when a user watches a 4K video on a Huawei P60 phone and then casts the video to a Sony TV, the system automatically detects the device switch. The phone's codec parameters are optimized for a 6.1-inch OLED screen, with a compression ratio of 3:1. Upon detecting the Sony TV's 65-inch 4K screen, the system regenerates the parameters for the larger screen within 200 milliseconds, adjusting the compression ratio to 1.8:1 while enhancing color depth preservation to ensure optimal viewing on the larger screen. These steps achieve the technical effect of one-time training and full-end adaptation, resolving the issue of traditional codecs requiring separate optimization for each device. This reduces PSNR fluctuations between devices and reduces model storage requirements.

[0028] In one embodiment of the present application, when a smartphone equipped with a Snapdragon 8 Gen2 processor is connected to the system, the system first collects the 12-dimensional hardware parameters of the device, including 8 CPU cores, 12TFLOPS GPU floating-point computing power, 480 screen PPI, 120% NTSC color gamut coverage, 51.2GB / s memory bandwidth and other parameters. These original parameters are normalized. For example, the normalized calculation of GPU computing power is (12-0.5) / (15.3-0.5)=0.777, where 0.5 and 15.3 are the preset minimum and maximum values, respectively. After normalization, a 12-dimensional normalized parameter vector is obtained. The system uses a pre-trained ResNet-18 network to extract features from the input 4K resolution medical image. After processing through the convolution layer and pooling layer, a 64-dimensional feature vector is extracted before the last fully connected layer. The vector contains key information such as the texture, edge, and color distribution of the image. In actual operation, the 8-dimensional device feature vector and the 64-dimensional image content feature vector are concatenated to form a 72-dimensional joint feature vector. This vector is combined with the first weight matrix W h 1The 256-dimensional intermediate feature vector is obtained by multiplying the vectors (72 × 256 dimensions). Each element x in the intermediate feature vector is processed using the gating function G(x) = x·σ(5x-1). When x = 0.15, σ(5 × 0.15-1) = σ(-0.25) ≈ 0.438, resulting in G(0.15) = 0.15 × 0.438 ≈ 0.066, partially suppressing the feature. When x = 0.3, σ(5 × 0.3-1) = σ(0.5) ≈ 0.622, resulting in G(0.3) = 0.3 × 0.622 ≈ 0.187, moderately preserving the feature. After ReLU activation and a second gating unit, a 256-dimensional codec parameter set is generated. This parameter set is directly loaded into the codec to adjust key modules such as the quantization table, transform coefficients, and entropy coding parameters. For example, for the 12 TFLOPS computing power of the Snapdragon 8 Gen2, the system automatically sets the quantization step size of the DCT transform to 16, while for lower-end devices it may be set to 32 to reduce computational complexity. The system uses layers 3, 8, and 15 of the VGG19 network to extract feature maps from the encoded and decoded images and the baseline VTM11.0 output, respectively, and calculates the L1 norm difference. For example, the L1 difference of the feature map for layer 3 is 0.082, 0.156 for layer 8, and 0.234 for layer 15. Weighted by the weights [0.3, 0.5, 0.2], the resulting overall feature difference is 0.145. Both images are simultaneously converted to probability distributions. During the initial training phase, the temperature coefficient γ = 1.0, resulting in a calculated KL divergence of 0.187. As training progresses, γ decays linearly to 0.5, ensuring more accurate distribution matching in later stages. The final distribution difference is 0.7×0.145+0.3×0.187=0.158. The system reversely adjusts the codec parameters based on this difference value. After about 50 iterations, the distribution difference drops below 0.032, completing the optimization process.

[0029] Through the above steps, this embodiment automatically generates optimal codec parameters based on the hardware characteristics of different devices. Actual measurements show that when the same medical image needs to be displayed on a high-end workstation (NVIDIA RTX 4090, 82.6 TFLOPS) and a mobile device (Snapdragon 8 Gen2, 12 TFLOPS), the PSNR difference between the two devices can reach 4.2dB using a traditional fixed-parameter codec. However, using this method, the difference is reduced to within 0.3dB, effectively resolving the issue of inconsistent image quality when deploying across devices.

[0030] like Figure 2 As shown, according to one aspect of the present application, mapping hardware parameters into device feature vectors through a device feature encoding network includes:

[0031] Obtaining multi-dimensional hardware parameters of the target device, normalizing the multi-dimensional hardware parameters to obtain a normalized parameter vector;

[0032] Apply different nonlinear transformation functions to different types of parameters in the normalized parameter vector to obtain the transformed parameter vector;

[0033] The transformed parameter vector is multiplied by the learnable embedding matrix and the bias term is added. After processing through the ReLU activation function, the device feature vector is obtained. The dimension of the device feature vector is lower than the dimension of the multi-dimensional hardware parameters.

[0034] According to one aspect of the present application, different nonlinear transformation functions are applied to different types of parameters in the normalized parameter vector, specifically including:

[0035] Identify the parameter types in the normalized parameter vector and classify the parameters into computing power parameters, performance parameters, and binary parameters;

[0036] Apply a logarithmic transformation to the hashrate parameter: log2(x+1), where x is the normalized hashrate value. The addition of 1 avoids numerical issues with logarithmic operations.

[0037] Apply a hyperbolic tangent transformation tanh(x' / α) to the performance parameter, where x' is the normalized performance value and α is a scaling factor used to adjust the sensitivity of the transformation;

[0038] Apply the sigmoid transformation σ(βx″) to the binary parameters, where σ( ) is the sigmoid function, x″ is the normalized parameter value, and β is the steepness factor that controls the transition interval of the transformation.

[0039] The transformation results of various parameters are combined in the original order to form a transformed parameter vector.

[0040] Specifically, the multi-dimensional hardware parameters of the target device are obtained, and the multi-dimensional hardware parameters are normalized to obtain a normalized parameter vector. For example, the normalization formula for the GPU computing power parameter is (d2-0.5) / (15.3-0.5), where d2 is the original computing power value, and the normalization range covers the computing power range from embedded chips to server GPUs. Different nonlinear transformation functions are applied to different types of parameters in the normalized parameter vector. Among them, the logarithmic transformation log2(x+1) is applied to the computing power parameter, the hyperbolic tangent transformation tanh(x / α) is applied to the performance parameter, and the sigmoid transformation σ(βx) is applied to the binary parameter to obtain the transformed parameter vector. The transformed parameter vector is compared with the learnable embedding matrix W. e ∈R 8×12 Multiply and add the bias term b e , after processing with the ReLU activation function, the device feature vector e is obtained d ∈R 8. Different types of hardware parameters have different numerical distribution characteristics, so the device feature encoding adopts differentiated nonlinear transformation. The computing power parameter grows exponentially and is suitable for logarithmic transformation; the performance parameter has a limited range of variation, and the hyperbolic tangent transformation can enhance the discrimination; the binary parameters require a smooth 0-1 mapping. This embodiment compresses the 12-dimensional sparse hardware parameters into 8-dimensional dense features through differentiated processing, which improves the effectiveness of feature expression and enables the super network to perceive device differences more accurately.

[0041] In actual use, when a Huawei P60 Pro phone is connected to the system, the system obtains its 12-dimensional hardware parameters, including: 8 CPU cores, 3.2 TFLOPS GPU floating-point computing power, 460 PPI screen, 110% DCI-P3 color gamut coverage, 44.8 GB / s memory bandwidth, 2.4 TOPS NPU computing power, 2700 × 1228 screen resolution, 1200 nits maximum brightness, HDR10+ support (binary parameter 1), H.265 hardware decoding support (binary parameter 1), 120Hz refresh rate, and 12GB RAM capacity. These parameters are normalized. Taking GPU computing power as an example, the normalized calculation is (3.2 - 0.5) / (15.3 - 0.5) = 0.182, where 0.5 TFLOPS corresponds to the minimum computing power of embedded devices and 15.3 TFLOPS corresponds to the upper limit of high-end GPU computing power. The memory bandwidth is normalized to (44.8-10) / (100-10)=0.387, and the screen PPI is normalized to (460-200) / (600-200)=0.65. After normalization, the resulting 12-dimensional vector is [0.375, 0.182, 0.65, 0.85, 0.387, 0.16, 0.675, 0.8, 1.0, 1.0, 0.75, 0.6]. Nonlinear transformations are applied to different parameter types. For parameters related to computing power, such as the normalized value of 0.375 for the number of CPU cores, applying a logarithmic transformation yields log²(0.375+1)=log²(1.375)≈0.459; the normalized value of 0.182 for GPU computing power becomes log²(1.182)≈0.241 after transformation; and the normalized value of 0.16 for NPU computing power becomes log²(1.16)≈0.214 after transformation. Logarithmic transformation can better reflect the nonlinear growth characteristics of computing power parameters and prevent the computing power advantages of high-end devices from being overly exaggerated. For performance parameters, such as the normalized memory bandwidth value of 0.387, a hyperbolic tangent transformation is applied: tanh(0.387 / 0.8)=tanh(0.484≈0.449, where α=0.8 is the preset scaling factor. For the normalized refresh rate value of 0.75, the transformation is tanh(0.75 / 0.8)=tanh(0.938≈0.735. The S-shaped curve characteristic of the hyperbolic tangent function moderately amplifies the differences in moderate performance parameters while compressing extreme values. For binary parameters, such as the HDR10+ support flag 1.0, a sigmoid transformation is applied: σ(3×1.0)=σ(3)≈0.953, where β=3 is the steepness factor. For the H.265 support flag 1.0, the transformation is also applied: 0.953. This ensures that binary parameters remain close to 0 or 1 after transformation.Combining all the transformed values ​​in their original order gives us a 12-dimensional transformed parameter vector [0.459, 0.241, 0.742, 0.812, 0.449, 0.214, 0.725, 0.785, 0.953, 0.953, 0.735, 0.587]. This vector is combined with the learnable embedding matrix W. e (8×12 dimensions) multiplication, assuming W e The first row of is [0.82, -0.31, 0.45, 0.27, -0.18, 0.63, -0.22, 0.51, 0.38, -0.15, 0.44, -0.29], then the first output dimension is calculated as 0.82×0.459+(-0.31)×0.241+0.45×0.742+...≈1.287. Add the bias term b e After [0]=-0.15, 1.137 is obtained, and it remains at 1.137 after ReLU activation. Similarly, the remaining 7 dimensions are calculated, and the 8-dimensional device feature vector [1.137, 0.892, 0.0, 0.756, 1.234, 0.423, 0.0, 0.981] is finally obtained, where the 3rd and 7th dimensions are set to zero due to ReLU activation. The multi-level feature encoding method effectively captures the differences between devices. For example, comparing the device feature vectors of Huawei P60 Pro (mid-range) and iPhone 14Pro Max (high-end), the cosine similarity between the two is 0.76, reflecting a hardware difference of about 25%, which will directly affect the encoding and decoding parameters generated by the subsequent hypernetwork.

[0042] like Figure 3 As shown, according to one aspect of the present application, a codec parameter set adapted to a target device is dynamically generated based on a gated residual structure, including:

[0043] Splicing the device and image content feature vectors to obtain a spliced ​​feature vector;

[0044] Multiplying the concatenated feature vector by the first weight matrix to obtain an intermediate feature vector, and processing the intermediate feature vector through a first gating unit, wherein the first gating unit uses a learnable threshold function to implement soft parameter selection;

[0045] The output of the first gating unit is activated by ReLU, processed by the second gating unit, and multiplied by the second weight matrix to generate a codec parameter set that meets the L2 norm constraint.

[0046] like Figure 4 As shown, according to one aspect of the present application, the process of implementing parameter soft selection by the first gating unit using a learnable threshold function includes:

[0047] For each eigenvalue x of the intermediate eigenvector, calculate the gating coefficient σ(5x-1), where σ is the sigmoid function, the coefficient 5 controls the steepness of the gating transition, and the constant 1 determines the threshold position to be 0.2;

[0048] Multiply each eigenvalue x by its corresponding gating coefficient. When the eigenvalue is less than 0.2, the gating coefficient is close to 0 so that the feature is suppressed; when the eigenvalue is greater than 0.2, the gating coefficient is close to 1 so that the feature is retained, and a smooth transition is achieved around 0.2.

[0049] Through gating processing, feature channels that contribute greatly to device adaptation are adaptively selected, and feature channels with small contributions are suppressed, forming a gated feature vector with sparse activation characteristics.

[0050] Specifically, the device feature vector and the image content feature vector are spliced ​​to obtain a spliced ​​feature vector. The dimension of the device feature vector is 8, and the dimension of the image content feature vector is 64. After splicing, a 72-dimensional feature vector is obtained. The spliced ​​feature vector is combined with the first weight matrix W h 1 ∈R 72×256 The intermediate eigenvector is obtained by multiplication and processed by the first gating unit. The first gating unit uses a learnable threshold function G(x)=x·σ(5x-1) to achieve soft parameter selection, where x is the input eigenvalue and σ is the sigmoid function. The output of the first gating unit is activated by ReLU and then processed by the second gating unit and combined with the second weight matrix W h 2 ∈R 256×256 Multiply them together to generate the codec parameter set θ codec This parameter set satisfies the L2 norm constraint ||θ codec ||2≤1.5. This embodiment achieves adaptive feature selection through a learnable threshold function. Feature values ​​less than 0.2 are suppressed, while those greater than 0.2 are retained, forming a sparse activation pattern. This enables the network to automatically focus on feature dimensions that contribute most to device adaptation. The parameter generation speed reaches 153 FPS, meeting the requirements of 8K@60fps real-time encoding while maintaining parameter stability and effectiveness.

[0051] In one embodiment of the present application, following the above-mentioned Huawei P60 Pro case, the 8-dimensional device feature vector [1.137, 0.892, 0.0, 0.756, 1.234, 0.423, 0.0, 0.981] is spliced ​​with the 64-dimensional image content feature vector. Taking the processing of a chest CT image as an example, the first 8 dimensions of the image feature vector extracted by ResNet-18 are [0.234, -0.187, 0.445, 0.0, -0.089, 0.623, 0.312, -0.156], reflecting the low contrast and high detail characteristics of medical images. After splicing, a 72-dimensional vector is formed. The 72-dimensional vector and the first weight matrix W h 1 Multiplying together, taking the 47th output dimension as an example, the dot product of its corresponding weight vector and the input vector is calculated to be 0.178. After processing by the first gating unit G1, G1(0.178)=0.178×σ(5×0.178-1)=0.178×σ(-0.11)≈0.178×0.473≈0.084. This indicates that this feature channel is partially suppressed because its activation value is lower than the threshold of 0.2. In contrast, the value of the 23rd dimension is 0.412, G1(0.412)=0.412×σ(5×0.412-1)=0.412×σ(1.06)≈0.412×0.743≈0.306, and this feature is well preserved. Of the 256-dimensional vectors processed by the first gating unit, statistics show that approximately 38% of the dimensions are suppressed (output value < 0.1), 47% are moderately retained (0.1-0.3), and only 15% are fully activated (> 0.3). This enables the hypernetwork to adaptively select the feature channels that are most relevant to the current device-image combination. After ReLU activation, the vector passes through the second gating unit G2, which is processed in the same way as G1 but with independent parameter learning. Finally, it is combined with the second weight matrix W h 2 Multiplication generates a 256-dimensional codec parameter set. The generated parameters must satisfy the L2 norm constraint. The calculated result is ||θ||2=1.423<1.5, which satisfies the constraint. This parameter set includes configurations for multiple submodules, such as the quantization table scaling factors [0.82, 0.91, 1.05, 1.18] (corresponding to the Y, Cb, Cr channels and high-frequency coefficients), 128 entropy coding probability model parameters, and 16 loop filter strength parameters. These parameters are directly loaded into the codec, implementing a device-adaptive compression strategy.

[0052] According to one aspect of the present application, calculating a distribution difference between a codec image and a reference codec output includes:

[0053] Based on the codec image and the baseline codec output, the L1 norm difference of the feature map is calculated at multiple preset feature levels, and different weight coefficients are assigned to different feature levels to obtain the weighted multi-scale L1 norm difference;

[0054] Convert the codec image and the baseline codec output into probability distributions p and q, respectively, and calculate the KL divergence KL(p||q)=Σp(i)log[p(i) / q(i)]. Soften the KL divergence by the temperature coefficient γ to obtain the softened KL divergence; where i is the distribution index;

[0055] The distribution difference is obtained by linearly combining the weighted multi-scale L1 norm difference and the softened KL divergence, where the temperature coefficient γ decays linearly from the initial value to the target value during training.

[0056] According to one aspect of the present application, obtaining the softened KL divergence includes:

[0057] According to the current training step number t and the total training step number T, calculate the temperature coefficient γ(t)=γ init -(γ init -γ final )×(t / T), where γ init is the initial temperature coefficient, γ final For the target temperature coefficient, the init to γ final Linear attenuation of

[0058] Use the temperature coefficient γ(t) to soften the probability distribution and convert the original probability value p i and q i Convert to p i 1 / γ(t) and q i 1 / γ(t) , and renormalize to obtain the softened probability distribution;

[0059] The KL divergence is calculated based on the softened probability distribution. The initial first γ value makes the distribution smoother and facilitates rapid convergence, while the later second γ value makes the distribution sharper and facilitates fine optimization. The first γ value is greater than the second γ value to achieve adaptive adjustment of the distillation process and obtain the softened KL divergence.

[0060] Specifically, multi-scale feature extraction is performed on the codec image and the baseline codec output. Specifically, feature maps are extracted using layers 3, 8, and 15 of the VGG19 network, and the L1 norm difference is calculated at each feature level. The codec image and the baseline codec output are converted to probability distributions p and q, respectively, and the KL divergence (KL(p||q)) = Σp(i)log[p(i) / q(i)] is calculated, where i is the distribution index. The KL divergence is softened using a temperature coefficient γ. The weighted multi-scale L1 norm difference is linearly combined with the softened KL divergence to obtain the distribution difference. The temperature coefficient γ decays linearly from an initial value of 1.0 to a target value of 0.5 during training. This embodiment utilizes a dual mechanism of multi-scale feature matching and KL divergence constraint. The multi-scale features capture image information at different levels, while the KL divergence ensures the consistency of the probability distribution. This allows for an initial focus on global distribution matching and a later focus on detail optimization. When switching between different platforms, the SSIM difference of the output image is minimized, ensuring visual consistency across devices.

[0061] In one embodiment of the present application, when processing the above-mentioned chest CT images, the system runs the codec configured with the generation parameters and the baseline VTM11.0 codec simultaneously. Multi-scale feature extraction and distribution difference calculation are performed on the outputs of the two codecs. When using the VGG19 network to extract features, the 3rd layer (64×256×256) mainly captures low-level edge features, the 8th layer (128×128×128) captures mid-level texture features, and the 15th layer (512×32×32) captures high-level semantic features. For the lung nodule area (32×32 pixels) in the medical image, the L1 difference of the 3rd layer feature is calculated as Σ|f 3ref -f 3gen | / N=0.067, the 8th layer is 0.124, and the 15th layer is 0.189. Weighted according to the weights [0.3, 0.5, 0.2], we get 0.3×0.067+0.5×0.124+0.2×0.189=0.120. For the KL divergence calculation, the pixel values ​​of the two images are converted into probability distributions. At the 1000th step of training (total 10,000 steps), the temperature coefficient γ(1000)=1.0-(1.0-0.5)×(1000 / 10000)=0.95. The original probability distribution is softened, for example, the probability p of a pixel value in the benchmark output is i =0.15, after softening it is 0.15 1 / 0.95 ≈0.141; the probability of generating output corresponding to q i =0.18, after softening it is 0.18 1 / 0.95≈0.169. The KL divergence contribution of this dimension is calculated to be 0.141×log(0.141 / 0.169)≈-0.025. The total KL divergence is 0.156 when summed over all 256 gray levels. The total loss is calculated as 0.7×0.156+0.3×(mean square error 0.0034)=0.110. Based on this loss value, the system adjusts the codec parameters by gradient descent. After about 35 iterations, the loss dropped to 0.028, of which the KL divergence dropped to 0.038, indicating that the output distribution of the generative model has been well approximated to the baseline model. In the later stage of training (step 8000), the temperature coefficient dropped to γ=0.6, making the probability distribution sharper, for example, p i =0.15 softened to 0.15 1 / 0.6 ≈0.079, which places more emphasis on matching the distribution peaks to ensure accurate reconstruction of key gray levels.

[0062] According to one aspect of the present application, it also includes dynamically constraining the codec parameter set:

[0063] Perform spectral normalization on the codec parameter set and calculate the maximum singular value σ of the parameter matrix max ;

[0064] When σ max When the preset Lipschitz constant threshold is exceeded, the parameter matrix is ​​divided by σ max Scaling is performed to obtain a dynamically constrained codec parameter set.

[0065] Specifically, the Lipschitz constant threshold is 1.8. During the deployment phase, the numerical stability of the codec parameter set is monitored in real time. When a non-numeric value is detected in the parameter, it is automatically rolled back to the most recent valid parameter set. The device characteristics that triggered the rollback are recorded for subsequent analysis. The valid parameter set is used to configure the codec for image encoding and decoding. This embodiment ensures that the output of the neural network is insensitive to input perturbations by limiting the Lipschitz constant of the parameter matrix. Scaling is performed when the maximum singular value exceeds the threshold. The robustness of the system is improved, and numerical stability can be maintained under extreme device configurations, avoiding parameter anomalies.

[0066] In one embodiment of the present application, after generating the codec parameter set, the system performs spectral normalization on the parameter matrix. Taking the quantization matrix parameters (8×8 dimensions) as an example, the maximum singular value σ is calculated by singular value decomposition (SVD). max =2.13. Since it exceeds the preset Lipschitz constant threshold of 1.8, the system automatically divides the entire matrix by 2.13 / 1.8≈1.183 for scaling. After scaling, σ is recalculated. max=1.8, ensuring that the constraints are met. Limit the impact of parameter changes on the output. For example, when a DCT coefficient of the input image changes by 1 unit, the output will not change by more than 1.8 units after the constrained quantization matrix processing, thus avoiding instability caused by overly sensitive parameters. In actual deployment, the system performs a numerical stability test every 100 frames. During one test, a NaN value was found in the entropy coding probability parameter. The reason was traced back to the fact that a completely black image was input, resulting in a division by 0 error in the probability calculation. The system immediately triggered the rollback mechanism and restored the valid parameter set [θ valid ], while recording device fingerprints and abnormal image features: {Device ID: P60Pro 001 , GPU computing power: 3.2TFLOPS, Exception type: Division by zero error, Image entropy: 0.0}. Used to improve the robustness of hypernetwork training.

[0067] According to one aspect of the present application, a hot switching mechanism is also included when the device is switched:

[0068] Monitor the device feature vectors at the current moment and the previous moment in real time, calculate the cosine similarity between the two, and determine that a device switch has occurred when the value of 1 minus the cosine similarity exceeds the preset threshold;

[0069] In response to a determination of device switching, re-execute the device feature encoding, super network parameter generation, and codec configuration processes within a predetermined processing period to generate a codec image adapted to the new device;

[0070] The predetermined processing cycle does not exceed 3 encoding and decoding processing cycles to ensure service continuity during device switching.

[0071] According to one aspect of the present application, calculating the cosine similarity between the two includes:

[0072] Get the device feature vectors at the current moment and the previous moment, calculate the dot product of the two vectors and their respective L2 norms;

[0073] Divide the dot product by the product of the two L2 norms to obtain the cosine similarity cos(θ), where θ is the angle between the two device feature vectors. The range of cosine similarity is [-1, 1].

[0074] Specifically, real-time monitoring of the device feature vector e at the current moment d t and the device feature vector e at the previous moment d t-1. Calculate the cosine similarity cos(θ) of the two vectors, where θ is the angle between the two vectors. Calculate the device change degree as 1-cos(θ). When the device change degree exceeds 0.15, trigger device switching. Re-execute the device feature encoding, super network parameter generation and codec configuration process within 3 encoding and decoding processing cycles. The cosine similarity measures the angle between the two device feature vectors, reflecting the overall difference in the device hardware characteristics. The threshold of 0.15 corresponds to an angle of approximately 31 degrees, achieving a balance between sensitivity and stability. In this embodiment, the device switching response time is controlled within 3 processing cycles (approximately 300ms), achieving seamless switching, and the user can hardly perceive the change in image quality.

[0075] In one embodiment of the present application, during a video conference, a user switches from a Huawei P60 Pro phone to a Huawei MateBook X Pro laptop. The system monitors the changes in the device feature vectors in real time. The phone's 8-dimensional feature vector is [1.137, 0.892, 0.0, 0.756, 1.234, 0.423, 0.0, 0.981]. The laptop, equipped with a more powerful Intel Iris Xe graphics card (5.8 TFLOPS) and a 2K screen (PPI 227), has a feature vector of [1.423, 1.156, 0.268, 0.923, 1.567, 0.645, 0.189, 1.234]. The dot product of the two vectors is calculated as 1.137 × 1.423 + 0.892 × 1.156 + ... ≈ 7.823, and their respective L2 norms are ||e phone ||=2.435, ||e laptop || = 3.187. The calculated cosine similarity is 7.823 / (2.435×3.187) ≈ 0.784. Therefore, the device variation is 1-0.784=0.216>0.15, triggering a device switch. Upon detecting the switch, the system immediately initiates the parameter update process. The first processing cycle (0-100ms): completes the laptop hardware parameter acquisition and feature encoding. The second cycle (100-200ms): the hypernetwork generates new codec parameters, including adjusting the quantization step size from 16 to 12 to match the higher display resolution. The third cycle (200-300ms): the new parameters are loaded and take effect. Throughout the switch, the video stream remains continuous, and users may only notice slight fluctuations in image quality during the second cycle. After the switch is complete, the encoding bitrate automatically adjusts from 2.5Mbps on the phone to 3.8Mbps, and the PSNR improves from 32.1dB to 34.7dB, fully leveraging the laptop's stronger hardware performance and higher screen resolution.

[0076] According to one aspect of the present application, in a medical image transmission scenario, the method includes:

[0077] Identify the type of medical device and extract its hardware parameters, generate codec parameters adapted to medical display requirements through the device feature encoding network, and set the compression ratio and DCT coefficient retention strategy;

[0078] Perform DCT transformation on medical images to obtain a coefficient matrix, identify high-frequency coefficient areas based on a preset frequency threshold, and retain high-frequency DCT coefficients of no less than a preset ratio to preserve pathological texture details;

[0079] Extract the grayscale histogram of the encoded image and calculate the KL divergence with the DICOM standard histogram. When the KL divergence exceeds a preset threshold, dynamically adjust the DCT quantization table parameters and re-encode until the KL divergence meets the requirements.

[0080] When a display device switch is detected, the codec parameter set is regenerated according to the resolution and color depth parameters of the new device.

[0081] Specifically, the processing flow for endoscopes is as follows: The hardware parameters of the endoscope are identified, including the 2TOPS computing power of the Renesas RZ / V2M chip and the 720p display. The device's feature encoding network generates codec parameters tailored to medical display requirements, setting a compression ratio of 3.2:1. A DCT transform is performed on the medical image to obtain a coefficient matrix. High-frequency coefficient regions are identified based on a frequency threshold, retaining 92% of the high-frequency DCT coefficients. The grayscale histogram of the encoded image is extracted, and the KL divergence with the DICOM standard histogram is calculated. If the KL divergence exceeds 0.03, the DCT quantization table parameters are dynamically adjusted, and the image is re-encoded until the KL divergence meets the required value. Upon detecting a switch to a 4K teaching display, the system regenerates the codec parameters within 214ms and adjusts the compression ratio to 1.8:1. Pathological features are primarily reflected in texture details, and the DICOM standard histogram is monitored to ensure that the compressed image still meets diagnostic requirements.

[0082] In the specific implementation of the smart medical image transmission scenario, the system operates according to the following timing: (1) When the endoscope device is turned on, the feature encoding network detects its equipped Renesas RZ / V2M chip (2TOPS computing power) and 720p display; (2) The super network generates lightweight model parameters with a compression ratio of 3.2:1 within 82ms, focusing on retaining the high-frequency components of the mucosal texture; (3) If the device is switched to a 4K teaching display (Sony LMD-X3200) during surgery, the system recognizes the device change through HDCP handshake information and regenerates the adaptation parameters within 214ms, adjusting the compression ratio to 1.8:1 to match the surgeon's detailed observation needs; (4) The online distillation module continuously compares the histogram distribution of the output image with the standard DICOM format to ensure the effectiveness of the diagnosis. Actual measurements show that in the transmission of cholecystectomy surgery video, the difference in lesion visibility scores between different display terminals is reduced, and the reliability of remote diagnosis is improved.

[0083] In one embodiment of the present application, during a laparoscopic cholecystectomy, the surgeon uses an endoscope system equipped with a Renesas RZ / V2M chip. After the system recognizes the 2TOPS computing power and 720p (1280×720) display of the device, it generates adaptive encoding and decoding parameters. In view of the special texture characteristics of gallbladder tissue, the system sets a compression ratio of 3.2:1 and specially optimizes the DCT coefficient retention strategy. After performing an 8×8 block DCT transform on the surgical video frame, 64 frequency coefficients are obtained. The system generates the frequency coefficients according to the frequency threshold f th =12 identifies the high frequency area, that is, the frequency coordinates (u, v) satisfy u 2 +v 2>144 coefficients. Statistics show that although high-frequency coefficients only account for 28% of the total coefficients, they contain 92% of tissue edge information. Therefore, the system retains the full accuracy of these 28%×64≈18 high-frequency coefficients, and uses more aggressive quantization for the remaining low-frequency coefficients. When the operation reaches a critical step, the assistant switches the screen to the 4K display (Sony LMD-X3200) in the demonstration room. The system recognizes the new device within 87ms through the HDCP 2.3 handshake protocol and then completes parameter reconfiguration within 214ms: the compression ratio is adjusted to 1.8:1, the DCT high-frequency coefficient retention ratio is increased to 96%, and the quantization step size is reduced from 32 to 16. In terms of quality monitoring, the system continuously compares the grayscale histogram of the encoded image with the DICOM standard. A histogram of one frame shows that in the critical mid-grayscale region (100-150), the encoded distribution is [0.082, 0.091, 0.103, ...], while the DICOM standard is [0.079, 0.095, 0.101, ...]. The calculated KL divergence is 0.082 × log(0.082 / 0.079) + ... ≈ 0.024 < 0.03, meeting medical standards. When the KL divergence for a frame reached 0.041, the system automatically fine-tuned the quantization table, reducing the quantization step size for the mid-grayscale by 15%. After three frames, the KL divergence dropped to 0.027, ensuring consistent visibility of critical surgical information across different devices.

[0084] According to one aspect of the present application, in a cross-platform content distribution scenario of an XR device, the method includes:

[0085] Extracting the display parameters and rendering capability parameters of the XR device, including resolution, refresh rate, and posture prediction computing power, and generating a device feature vector through the device feature encoding network;

[0086] Separate the spatiotemporal features of 3D scene texture data and extract temporal feature vectors and spatial feature vectors. Temporal features include motion vectors and inter-frame difference information, while spatial features include texture details and edge information.

[0087] Determine the feature weight based on the ratio of device refresh rate to resolution. For high refresh rate devices, increase the timing feature weight w. t , for high-resolution devices, increase the spatial feature weight w s , calculate the weighted eigenvector w t Multiply by the time series eigenvector plus w s Multiply by the spatial eigenvector;

[0088] Differentiated encoding parameters are generated based on the weighted feature vectors to achieve adaptive compression for different XR devices.

[0089] In one embodiment of the present application, the display parameters and rendering capability parameters of the XR device are extracted, including resolution, refresh rate, and posture prediction computing power. For example, the Quest Pro device has a 4K resolution and a 120Hz refresh rate. The spatiotemporal feature separation is performed on the 3D scene texture data, and the timing feature vector and the spatial feature vector are extracted. The timing feature contains motion vectors and inter-frame difference information, and the spatial feature contains texture details and edge information. The feature weight is determined based on the ratio of the device refresh rate to the resolution. For high refresh rate devices, the timing feature weight w is set. t =0.7, spatial feature weight w s =0.3; for high-resolution devices, set w t =0.3, w s =0.7. Calculate the weighted eigenvector w t ×Time series feature vector + w s × spatial feature vectors, generating differentiated encoding parameters based on weighted feature vectors. High refresh rate devices require smooth motion transitions, emphasizing temporal continuity; high-resolution devices require clear image details, emphasizing spatial fidelity. In Unity engine integration testing, cross-device rendering quality variance was reduced, effectively solving the problem of XR content adaptability across different hardware platforms.

[0090] According to another aspect of the present application, a device-adaptive image encoding and decoding method is constructed to construct a three-level adaptive architecture, including: extracting hardware fingerprints through a device feature encoding network, whose mathematical model is F=Φ(D)=[log2(FLOPs), ColorGamut / 100, MemBandwidth]∈R 3 , where Φ(D) is the mapping function of the device feature encoding network, which converts the device feature D into the feature vector F; FLOPs is the number of floating-point operations of the computing device, obtained through the device benchmark test; ColorGamut is the color gamut coverage of the device, and the coverage area is calculated using CIE1931 xy chromaticity coordinates; MemBandwidth is the memory bandwidth of the device, quantified through the memory copy test. The hypernetwork H generates the adaptation parameter θ=H(F;W) based on the feature vector F. Its network structure contains 3 residual blocks and 1 gated attention layer. The parameter scale is controlled within 1.2M to meet the real-time generation requirements, where W is the weight parameter of the hypernetwork. Construct an online distillation loss function: L=αL MSE +(1-α)L KL , where the mean square error loss function L MSE Constrained output image quality, Kullback-Leibler divergence L KL=Σp(x)log[p(x) / q(x)], ensuring statistical consistency between the generated model and the baseline video coding reference model (VTM11.0); p(x) is the probability distribution of the generated model; q(x) is the probability distribution of the baseline video coding reference model. The trade-off coefficient α = 0.7 was determined via grid search. A two-stage training strategy was employed: a meta-learner was pre-trained on a heterogeneous cluster of 200 devices (batch size = 256, learning rate 3e-4), followed by deployment-time parameter calibration using an online fine-tuning module (convergence required <50 iterations).

[0091] In terms of compression performance, this embodiment reduces the PSNR fluctuation between Huawei P40 and NVIDIA V100 from 31.7dB±2.1dB in the traditional method to 32.5dB±0.3dB, while maintaining a compression ratio of 2.8:1. In terms of resource consumption, dynamic parameter generation only adds 11ms of latency (accounting for 3.2% of the total processing time) but reduces model storage requirements by 83%. In terms of engineering implementation, it supports a hot switching mechanism, reducing the adaptation time during device changes from minutes in traditional solutions to less than 200ms. Specifically, the device feature encoding network quantifies hardware differences into three-dimensional interpretable features, enabling the super-network to accurately perceive screen color gamut differences (ΔE < 1.5) and computing power fluctuations (prediction error < 8%). An online distillation mechanism ensures that the SSIM difference in output images on the Apple A15 chip (18TOPS computing power) and the Qualcomm Snapdragon 8Gen2 (12TOPS computing power) is within 0.02. The super-network architecture design enables parameter generation at a speed of 153 FPS, meeting the requirements of 8K@60fps real-time encoding. Test data shows that in a multi-camera autonomous driving system, this embodiment reduces the standard deviation of decoding time between cameras from 47ms to 6ms.

[0092] The complete workflow of this embodiment includes two stages: offline training and online deployment. Offline stage: Build a device feature database, collect 12 original indicators such as the number of GPU cores and memory bandwidth of 200+ devices through automated testing tools, and obtain a three-dimensional feature vector after PCA dimensionality reduction. The meta-learner is trained using the MAML framework. The inner loop is quickly adapted to a single device (5 gradient updates), and the outer loop optimizes the initial parameters of the hypernetwork. Online deployment stage: Form a closed-loop system: feature extraction is triggered when the device is connected → the hypernetwork generator outputs the adaptation parameters within 120ms → the codec loads dynamic parameters and runs → the quality monitoring module feeds back PSNR data to the online fine-tuner, forming a continuous optimization loop. When a device change is detected (such as casting the mobile phone screen to the TV), the system completes the parameter switching within 3 processing cycles (about 300ms).

[0093] In a specific embodiment, a device adaptive image encoding and decoding method includes the following steps:

[0094] Step 1: Construction of device feature encoding network and hardware fingerprint extraction.

[0095] The quantitative modeling of device hardware characteristics is achieved through the feature encoding network, whose input is the 12-dimensional original hardware parameter vector D=[d1, d2, ..., d 12 ], including key indicators such as the number of CPU cores (d1), GPU floating point computing power (d2, unit TFLOPS), screen PPI (d3), color gamut coverage (d4, %NTSC). The feature conversion process uses the embedding matrix W e ∈R 8×12 Perform dimensionality reduction: e d =ReLU(W e ·[log2 (d1+1), tanh(d2 / 10),..., sigmoid(d 12 )] T +b e ), where b e ∈R 8 is the bias term, and the activation function is selected to take into account both nonlinear expression ability and gradient stability; d ∈R 8 is the compressed device feature vector. All input parameters need to be normalized to the range [0, 1] through min-max. For example, the normalization formula for GPU computing power item d2 is (d2-0.5) / (15.3-0.5), covering the computing power range from embedded chips to server GPUs. Output device feature vector e d ∈R 8 It will be used as the input condition of the hypernetwork.

[0096] Step 2: Hypernetwork dynamic parameter generation mechanism.

[0097] Based on the device feature vector e d and image content features f c ∈R 64 (Extracted by ResNet-18), the super network generates the encoder-decoder parameters through the gated residual structure: θ codec =G2 (ReLU(G1 ([e d Θf c ]·W h 1 )))·W h 2 , where Θ represents the vector concatenation operation, W h 1 ∈R 72×256 and W h 2 ∈R 256×256is a learnable weight matrix, G1 and G2 are gate units: G(x) = x·σ(5x-1), σ is a sigmoid function; this enables the network to adaptively select important parameter paths. In the actual measurement on the Snapdragon 8 Gen2 platform, a 256-dimensional parameter vector θ is generated. codec It takes only 8.3ms. The parameter generation process must satisfy the L2 norm constraint |θ codec |2≤1.5, stable training is achieved through the gradient clipping algorithm.

[0098] Step 3: Online distillation loss calculation and back propagation.

[0099] In order to maintain the distribution consistency between the generated model and the baseline VTM11.0 codec, a multi-scale feature distillation loss is designed: L distill =∑ l=1 3 λ l |Φ l (y ref )-Φ l (y gen )|1+γ·KL(p|q), where Φ l is the VGG19 layer l feature extractor (l=3, 8, 15), λ l =[0.3, 0.5, 0.2] is the hierarchical weight coefficient, KL(p|q) calculates the KL divergence between the output distribution p of the baseline model and the generated model q, the temperature coefficient γ=0.5 softens the probability distribution, y ref is the reference output of the benchmark model, y gen is the output of the generative model. The total loss function is defined as: L total =0.7L distill +0.3|y ref -y gen |2 2 , Adam optimizer is used for back propagation, and the learning rate is set to 3×10 -4 , it only takes 2.1 seconds to complete 50 iterations on the Huawei Ascend 910 chip.

[0100] Step 4: Dynamic parameter constraints and stability assurance.

[0101] The system ensures the physical feasibility of the generated parameters through three mechanisms: 1). Device characteristic vector e d The L2 regularization term is 0.1|e d |2 2 Prevent overfitting; 2). Codec parameters θ codecSpectral normalization is performed to constrain the Lipchitz constant to ≤ 1.8. 3) A temperature annealing strategy is used for online distillation loss, with an initial γ = 1.0 that decays linearly to 0.5 over the number of training steps. During the deployment phase, the real-time monitoring module checks the numerical stability of output parameters. If a NaN value appears, it automatically rolls back to the most recent valid parameter set and records the device fingerprint in the log for subsequent analysis.

[0102] Step 5: Implementation of closed-loop workflow.

[0103] The system operation consists of two stages: offline training and online deployment: Offline training: Pre-training is performed using the MAML meta-learning framework on a heterogeneous cluster containing 200 types of devices. The inner loop (single device adaptation) has a learning rate of 0.1, the outer loop (meta-parameter update) has a learning rate of 3e-4, and the batch size is 256. Online deployment: Feature extraction is triggered when the device is connected (completed in 120ms); the hypernetwork generates adaptation parameters (NVIDIA T4 GPU takes 9.8ms); the codec loads the parameters and runs, and the quality monitoring module calculates the PSNR; when the PSNR fluctuation exceeds the threshold Δ=2dB, online fine-tuning is triggered (iterations <50). Hot switching mechanism: The change in device features is calculated by cosine similarity. When 1-cos(e d t , e d t-1 )>0.15, the parameter refresh is completed within 3 processing cycles.

[0104] Step 6: Verification of medical image transmission scenarios.

[0105] In the endoscope system driven by the Renesas RZ / V2M chip, the system implementation process is as follows: when the device starts, it automatically collects hardware parameters (2TOPS computing power / 720p screen) and generates a feature vector e d =[0.21, 0.67, ..., 0.38] T The hypernetwork generated encoding parameters with a compression ratio of 3.2:1, focusing on retaining the high-frequency components of the mucosal texture (DCT coefficient retention rate of 92%). When switching to a 4K teaching display, the system acquired new device features through the HDCP 2.3 protocol and generated 1.8:1 compression parameters within 214ms. The online distillation module continuously monitored the histogram difference between the output image and the DICOM standard (KL divergence <0.03) and dynamically adjusted the quantization table. Surgical video playback tests showed that the difference in VIS scores between different terminals dropped from 2.4 to 0.7, and the standard deviation of key frame decoding time was ≤6ms.

[0106] This embodiment differs from the pre-stored model matching strategy used by Huawei's EMUI system. It constructs a differentiable device feature encoding network to map heterogeneous features such as floating-point computing power and screen color depth into a 128-dimensional latent space vector. Based on the dynamic parameter generation mechanism of the hypernetwork, actual measurements on the MediaTek Dimensity 9200 platform show that it maintains a 94.3% compression efficiency consistency on an unprecedented new AI accelerator. By constraining the KL divergence between the generated model and the baseline model through an online distillation loss function, the image quality fluctuation is reduced from 32% of the traditional method to 4.7% when switching between the Xiaomi 13 Ultra and NVIDIA A100 platforms.

[0107] This invention can be applied to mobile multimedia. The device-adaptive image codec system can improve the consistency of the mobile multimedia experience. In scenarios such as short video platforms and real-time video calls, different mobile phone models vary in computing power (e.g., an octa-core GPU in a flagship phone versus a quad-core CPU in a mid-range phone), screen resolution (ranging from 1080p to 4K), and decoding capabilities. Traditional fixed-parameter codecs can cause lag or image quality degradation on low-end devices, while preventing high-end devices from fully realizing their performance advantages. This system's meta-learning-driven architecture generates codec parameters in real time that adapt to the specific phone's characteristics. For example, it dynamically enhances color compression fidelity for OLED screens or automatically reduces computational complexity for low-power chips. Field tests have shown that in applications such as TikTok and WeChat videos, the system reduces decoding latency differences between different device models while maintaining the SSIM image quality metric. In cloud-edge-end collaborative intelligent monitoring systems, this invention can effectively address codec adaptation issues in video analysis pipelines. When surveillance video requires preliminary analysis at edge nodes (such as the HiSilicon Hi3559 chip) before being transmitted to the cloud (such as an NVIDIA T4 server) for further processing, traditional methods require deploying two separate codecs, resulting in bandwidth waste and feature distortion. This application's hypernetwork architecture dynamically generates a layered compression strategy based on meta-features extracted by the device's feature encoding network (2TOPS computing power at the edge / 30TOPS computing power at the cloud): high compression ratios are used to preserve motion features at the edge, while detailed textures are restored on the cloud. Tests in a smart city project have shown that this system improves video analysis accuracy while reducing cross-device transmission bandwidth. Addressing the fragmentation of AR / VR devices, this application's online distillation mechanism provides key technical support for metaverse content distribution. When the same 3D scene needs to be rendered on different XR devices, such as the Hololens 2 (2K resolution / 60Hz refresh rate) and the QuestPro (4K / 120Hz), traditional methods require pre-storing multiple versions of the source material for each hardware type. After identifying display parameters and gesture prediction computing power through the system's device feature encoding network, the meta-learner generates an adaptive texture compression scheme: prioritizing temporal coherence for high-refresh-rate devices and preserving spatial detail for high-resolution devices. Unity engine integration testing shows that while maintaining a 90 FPS frame rate, this system reduces cross-device rendering quality variance, reducing developers' cost of maintaining multiple versions.

[0108] The preferred embodiments of the present invention are described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the scope of protection of the present invention.

Claims

1. A device-adaptive image encoding and decoding method, characterized in that: include: Obtain the hardware parameters of the target device and map them into device feature vectors through the device feature encoding network; Perform feature extraction on the image to be encoded and decoded to obtain the image content feature vector; The device and image content feature vectors are input into the hypernetwork generator, which dynamically generates a codec parameter set adapted to the target device based on the gated residual structure. Configure the codec using the codec parameter set, perform encoding and decoding on the image to be coded and decoded, and obtain a coded and decoded image; Calculate the distribution difference between the codec image and the baseline codec output, and adjust the encoding and decoding process based on the distribution difference through an online distillation mechanism to obtain the optimized codec image; Dynamically generates a codec parameter set adapted to the target device based on a gated residual structure, including: Splicing the device and image content feature vectors to obtain a spliced ​​feature vector; Multiplying the concatenated feature vector by the first weight matrix to obtain an intermediate feature vector, and processing the intermediate feature vector through a first gating unit, wherein the first gating unit uses a learnable threshold function to implement soft parameter selection; The output of the first gating unit is activated by ReLU, processed by the second gating unit, and multiplied by the second weight matrix to generate a codec parameter set that meets the L2 norm constraint.

2. The method according to claim 1, characterized in that The hardware parameters are mapped into device feature vectors through the device feature encoding network, including: Obtain the multi-dimensional hardware parameters of the target device, normalize them, and obtain a normalized parameter vector; Apply different nonlinear transformation functions to different types of parameters in the normalized parameter vector to obtain the transformed parameter vector; The transformed parameter vector is multiplied by the learnable embedding matrix and the bias term is added. After processing through the ReLU activation function, the device feature vector is obtained. The dimension of the device feature vector is lower than the dimension of the multi-dimensional hardware parameters.

3. The method according to claim 2, characterized in that Different nonlinear transformation functions are applied to different types of parameters in the normalized parameter vector, including: Identify the parameter types in the normalized parameter vector and classify the parameters into computing power, performance, and binary parameters; Apply a logarithmic transformation log2(x+1) to the hashrate parameter, where x is the normalized hashrate value. Apply the hyperbolic tangent transformation tanh(x' / α) to the performance parameter, where x' is the normalized performance value and α is the scaling factor; Apply the sigmoid transformation σ(βx″) to the binary parameter, where x″ is the normalized parameter value, β is the steepness factor, and σ( ) is the sigmoid function.

4. The method according to claim 1, wherein The process of implementing soft parameter selection using a learnable threshold function in the first gating unit includes: For each eigenvalue x of the intermediate eigenvector, calculate the gating coefficient σ(5x-1), where σ is the sigmoid function, the coefficient 5 controls the steepness of the gating transition, and the constant 1 determines the threshold position to be 0.2; Multiply each eigenvalue x by its corresponding gating coefficient. When the eigenvalue is less than 0.2, the gating coefficient is close to 0 so that the feature is suppressed; when the eigenvalue is greater than 0.2, the gating coefficient is close to 1 so that the feature is retained, and a smooth transition is achieved around 0.

2.

5. The method according to claim 1, wherein Computes the distribution difference between the codec image and the baseline codec output, including: Based on the codec image and the baseline codec output, the L1 norm difference of the feature map is calculated at multiple preset feature levels, and different weight coefficients are assigned to different feature levels to obtain the weighted multi-scale L1 norm difference; Convert the codec image and the baseline codec output into probability distributions p and q, respectively, and calculate the KL divergence KL(p||q)=Σp(i)log[p(i) / q(i)]. Soften the KL divergence by the temperature coefficient γ to obtain the softened KL divergence; where i is the distribution index; The distribution difference is obtained by linearly combining the weighted multi-scale L1 norm difference and the softened KL divergence, where the temperature coefficient γ decays linearly from the initial value to the target value during training.

6. The method according to claim 5, characterized in that Get the softened KL divergence, including: According to the current training step number t and the total training step number T, calculate the temperature coefficient γ(t)=γ init -(γ init -γ final )×(t / T), where γ init is the initial temperature coefficient, γ final For the target temperature coefficient, the init to γ final Linear attenuation of Use the temperature coefficient γ(t) to soften the probability distribution and convert the original probability value p i and q i Convert to p i 1 / γ(t) and q i 1 / γ(t) , and renormalize to obtain the softened probability distribution; The KL divergence is calculated based on the softened probability distribution to obtain the softened KL divergence.

7. The method according to claim 1, characterized in that This also includes dynamic constraints on codec parameter sets: Perform spectral normalization on the codec parameter set and calculate the maximum singular value σ of the parameter matrix max ; When σ max When the preset Lipschitz constant threshold is exceeded, the parameter matrix is ​​divided by σ max Scaling is performed to obtain a dynamically constrained codec parameter set.

8. The method according to claim 1 or 2, characterized in that It also includes a hot switching mechanism when switching devices: Monitor the device feature vectors at the current moment and the previous moment in real time, calculate the cosine similarity between the two, and determine that a device switch has occurred when the value of 1 minus the cosine similarity exceeds the preset threshold; In response to a determination of device switching, re-execute the device feature encoding, super network parameter generation, and codec configuration processes within a predetermined processing period to generate a codec image adapted to the new device; The predetermined processing cycle does not exceed 3 encoding and decoding processing cycles.

9. The method according to claim 8, characterized in that Calculating the cosine similarity between the two includes: Get the device feature vectors at the current moment and the previous moment, calculate the dot product of the two vectors and their respective L2 norms; Divide the dot product by the product of the two L2 norms to obtain the cosine similarity cos(θ), where θ is the angle between the two device feature vectors. The range of cosine similarity is [-1, 1].

Images