Online Adaptive Methods and Related Equipment for Normalized Structure Reconstruction and Feature Sharpness Perception

Abstract

This application provides an online adaptation method and related equipment for normalized structure reconstruction and feature sharpness perception, belonging to the field of transfer learning technology. The method includes: a normalized layer reconstruction stage: obtaining a batch normalized model; generating pseudo-data matching the source domain data distribution using data-free distillation techniques; replacing the batch normalized layer in the batch normalized model with a grouped normalized layer or a layer normalized layer to form an initial student model; performing knowledge distillation on the initial student model using pseudo-data to transfer the knowledge of the batch normalized model to the reconstructed student model, obtaining a normalized layer reconstruction model; an online adaptation stage: obtaining test samples; filtering the prediction confidence of the test samples based on the normalized layer reconstruction model to obtain a reliable sample subset; calculating the feature sharpness loss based on the reliable sample subset; updating the parameters of the normalized layer reconstruction model with the goal of minimizing the feature sharpness loss to complete the online adaptation.

Description

Technical Field

[0001] This application relates to the fields of deep learning and transfer learning technology, and in particular to an online adaptation method and related equipment for normalized structure reconstruction and feature sharpness perception. Background Technology

[0002] Deep neural networks (DNNs) have achieved great success in many fields. Batch normalization (BN), a key component of DNNs, significantly accelerates the training process and improves the model's generalization performance by reducing internal covariate bias. During training, BN normalizes data using the statistics (mean and variance) of the current batch and records the running statistics; during testing (inference), it uses the global running statistics obtained during training. This method is based on a strong assumption: the training data and the test data are independently and identically distributed (i.i.d.).

[0003] However, in real-world open scenarios (such as autonomous driving and intelligent monitoring), the testing environment is complex and variable, and test data often exhibits a domain shift compared to training data. In such cases, fixed Batch Normalization (BN) statistics cannot accurately characterize the distribution of test data, becoming a key bottleneck that severely degrades model performance. To address this issue, existing technologies have proposed Test-Time Adaptation (TTA) or Online Adaptation methods, aiming to dynamically adjust model parameters to adapt to the target domain using unlabeled test data.

[0004] Existing TTA methods mainly improve model robustness by dynamically calibrating BN statistics, such as dynamically fusing current batch statistics with global statistics, or using instance-level and group-level statistics for normalization. However, these methods have inherent limitations: (1) They heavily rely on the quality of current batch statistics. When there is class imbalance or multiple distribution shifts (such as simultaneous changes in lighting, weather, and scene) in the test batch, the current batch statistics are not representative enough, leading to inaccurate normalization and performance degradation. (2) They cannot effectively handle inference scenarios with extremely small batches (such as batch size = 1). In this scenario, batch statistics lose statistical meaning and may even introduce noise, leading to unstable model output. (3) They are still fundamentally dependent on the BN structure, only using statistics calibration to deal with distribution shifts, failing to fundamentally improve robustness from the model structure level, and their adaptability has a clear ceiling.

[0005] Therefore, there is an urgent need for an online adaptation method that can fundamentally overcome the limitations of BN layers to ensure the stability and reliability of the model in complex dynamic environments. Summary of the Invention

[0006] The main objective of this application is to propose an online adaptation method, electronic device, storage medium, and program product for normalized structure reconstruction and feature sharpness perception without data distillation, aiming to overcome the bottlenecks of insufficient adaptability and stability caused by the dependence of existing technologies on batch normalized structures and local statistics.

[0007] To achieve the above objectives, one aspect of this application proposes an online adaptation method for normalized structure reconstruction and feature sharpness perception, the method comprising:

[0008] Normalization layer reconstruction stage: Obtain a pre-trained batch normalization model; use data-free distillation to generate pseudo-data that matches the distribution of source domain data; replace the batch normalization layer in the pre-trained batch normalization model with a grouped normalization (GN) layer or a layer normalization (LN) layer to form an initial student model; use the pseudo-data to perform knowledge distillation on the initial student model to transfer the knowledge of the batch normalization model to the reconstructed student model, thus obtaining the normalization layer reconstruction model;

[0009] Online adaptation phase: Unlabeled test samples are acquired; the prediction confidence of the test samples is filtered based on the normalized layer reconstruction model to obtain a reliable sample subset; based on the reliable sample subset, feature sharpness loss is calculated, which is used to characterize the sharpness of the decision boundary of the normalized layer reconstruction model in the feature space; the parameters of the normalized layer reconstruction model are updated with the goal of minimizing the feature sharpness loss, so that the model learns robust feature representations far from the decision boundary in the feature space, thereby improving the robustness of the model in the online adaptation process.

[0010] In some embodiments, generating pseudo-data that matches the distribution of source domain data using data-free distillation includes:

[0011] Initialize noise data;

[0012] The noise data is input into the batch normalization model to calculate the batch statistics generated at each batch normalization layer during the forward propagation process.

[0013] Calculate the statistical matching loss between the batch statistics and the pre-stored runtime statistics in the batch normalization layer;

[0014] By optimizing the noisy data to minimize the statistical matching loss, optimized pseudo-data is obtained, and the batch statistics of the pseudo-data are matched with the running statistics.

[0015] In some embodiments, the statistical matching loss Calculated using the following formula:

[0016]

[0017] Where l is the layer index, Let z be the mean and variance of the noise data z in the current batch at layer l. These are the mean and variance of the l-th layer obtained by training the model in the source domain. ∈ is the square of the L2 norm; ∈ is a small constant.

[0018] In some embodiments, the batch normalization layer is replaced with a group normalization layer or a layer normalization layer, and the learnable parameters γ and β of the batch normalization layer are inherited.

[0019] In some embodiments, the knowledge distillation of the initial student model using the pseudo-data includes:

[0020] Calculate the outputs of the batch normalization model and the student model for the same pseudo-data;

[0021] The parameters of the student model are updated with the goal of minimizing the difference between the output of the batch normalization model and the student model.

[0022] The difference between the outputs is calculated using KL divergence to determine distillation loss, and / or using mean square error to determine feature layer alignment loss.

[0023] In some embodiments, the screening of prediction confidence scores for the test samples includes:

[0024] Calculate the prediction entropy for each test sample;

[0025] Samples whose predicted entropy is lower than a preset entropy threshold are selected as reliable samples, forming the reliable sample subset.

[0026] In some embodiments, calculating the feature sharpness loss includes:

[0027] Calculate the average entropy loss on the reliable sample subset.

[0028] Calculate the average entropy loss The gradient of the model feature representation φ;

[0029] A normalized perturbation direction v is constructed based on the gradient;

[0030] The feature φ is perturbed along the perturbation direction v to obtain the perturbed feature φ′;

[0031] Calculate the average entropy of the prediction obtained based on the perturbated feature φ′, and use it as the feature sharpness loss.

[0032] In some embodiments, the disturbance direction v is calculated using the following formula

[0033]

[0034] in, Let φ be the gradient of the feature, and ||||2 be the L2 norm.

[0035] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0036] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0037] To achieve the above objectives, another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0038] The embodiments of this application include at least the following beneficial effects: This application provides an online adaptation method, electronic device, storage medium, and program product for normalized structure reconstruction and feature sharpness perception. By generating representative pseudo-data using data-free distillation technology before online model adaptation, it guides the model to complete the structural transformation from batch normalization to group normalization or layer normalization, thus constructing an inherently robust adaptation foundation. This application introduces a stability enhancement mechanism during the online adaptation process to improve the model's stability in complex and ever-changing online adaptation scenarios. Attached Figure Description

[0039] Figure 1 This is a flowchart of the online adaptation method for normalized structure reconstruction and feature sharpness perception provided in the embodiments of this application;

[0040] Figure 2 This is a flowchart of the stable online adaptation method for normalized structure reconstruction and feature sharpness perception without data distillation provided in the embodiments of this application;

[0041] Figure 3 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0044] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0045] 1) Batch Normalization (BN) is a technique used in deep neural network training to improve the stability and convergence speed of model training. It alleviates the gradient vanishing / exploding problem by normalizing the distribution of inputs to each layer.

[0046] To address distribution shifts during the testing phase, existing techniques primarily improve model robustness by adjusting the use of statistics in the batch normalization layer. This can be summarized into two main approaches: One approach integrates current batch statistics with global statistics obtained during pre-training to construct a dynamically weighted hybrid statistic for normalization. The weights adaptively adjust based on the degree of change in the input data distribution, relying more on the current statistics when the distribution changes significantly and retaining more of the global statistics when the distribution is stable. The other approach uses instance-level or group-level statistics for modeling, independently calculating normalization parameters for individual samples or similar sample groups and calibrating them using global statistics, thus improving the stability of the batch normalization model to some extent.

[0047] Existing technologies have improved the model's adaptability to distribution shifts to some extent by dynamically adjusting the use of statistics in the normalization layer. However, these methods still have the following limitations when facing complex and dynamic scenarios in real open environments: 1) Existing methods rely on the estimation of statistics for the current input data. When the test data has class imbalance within a batch or multiple biases mixed within a batch, the local statistics are not representative enough, leading to inaccurate normalization. For example, in autonomous driving scenarios, a vehicle may suddenly enter a rainy tunnel environment from a sunny city road, while multiple factors such as lighting, weather, and road structure undergo complex changes (i.e., multiple biases mixed within a batch). At this time, the mean and variance calculated based solely on the current batch cannot accurately reflect the overall distribution characteristics, leading to inaccurate batch normalization (BN) layer statistics and a sharp decline in model performance. In addition, in intelligent monitoring systems, the images captured by cameras may contain only background or a few categories for a long time (such as occasional pedestrians in an empty corridor), causing severe class imbalance within a batch. At this time, the BN statistics are severely biased by the dominant class and cannot correctly normalize minority class features, resulting in a degradation in the model's ability to identify rare categories. 2) Existing methods fail to operate stably in scenarios with extremely small batches or single-sample inputs. When the batch size is very small, batch-based statistics lose their statistical significance. For example, in real-time face recognition systems on edge devices, inference is often performed using single images or extremely small batches (e.g., batch size = 1). In this case, the Batch Normalization (BN) layer cannot effectively calculate the batch mean and variance, leading to instability in the normalization process and even introducing noise, severely affecting the reliability of the model output. 3) Existing methods do not escape their dependence on the batch normalization structure itself; their essence remains to address distribution shifts by calibrating statistics, rather than improving robustness at the model structure level. When facing unknown and variable testing environments, their adaptability is limited, and their stability is insufficient.

[0048] To address the aforementioned issues, this application aims to resolve the limitations of existing online adaptation techniques in complex dynamic scenarios such as batch-based class imbalance, mixed batch offsets, or extremely small batch inputs. These limitations include: reliance on local statistics leading to normalization inaccuracies, instability under small batch or single-sample inputs, and dependence on the batch normalization (BN) structure itself limiting the model's robustness. To overcome these technical bottlenecks, this application proposes a stable online adaptation method and related equipment based on data-free distillation and normalization structure reconstruction. By addressing the issue at the model structure level, it fundamentally solves the problem of batch normalization statistic inaccuracies in BN models during adaptation, and improves the stability of online adaptation by minimizing feature sharpness, effectively handling challenging scenarios such as batch-based class imbalance, mixed offsets, and extremely small batch inputs. The method includes two working modes: 1) a normalization layer reconstruction mode without data distillation, which reconstructs the normalization layer from the model structure level to improve the inherent robustness of the model; 2) an online adaptation mode based on feature sharpness minimization, which improves the robustness of the model in the adaptation process by learning robust feature representations far from the decision boundary in the feature space.

[0049] Specifically, the data-free distillation normalization layer reconstruction mode utilizes data-free distillation technology to generate pseudo-data from a pre-trained batch normalization (BN) model and guide the model to complete the structural transformation from BN layers to group normalization (GN) layers or layer normalization (LN) layers. This method does not require access to real source domain data, relying only on the internal statistics of the pre-trained BN model. By optimizing the noisy data to match its batch statistics with the running statistics of the pre-trained BN model, pseudo-sample proxies similar to the source domain data are generated. After obtaining the pseudo-data, the BN layer is replaced with a GN or LN layer, inheriting the learnable parameters of the BN layer. Subsequently, knowledge distillation is performed using the pseudo-data. By minimizing the loss between the output of the teacher model (pre-trained BN model) and the student model (model after normalization layer reconstruction), the knowledge of the teacher model is transferred to the student model while preserving its decision boundary. This mode improves the intrinsic robustness of the model at the model structure level, laying the foundation for subsequent online adaptation.

[0050] The online adaptation model based on minimizing feature sharpness aims to select reliable samples and enable the model to learn more robust feature representations in the feature space that are far from the decision boundary, thereby improving its robustness to test data perturbations and the stability of online adaptation. This method first calculates the entropy value of unlabeled test samples and filters out a subset of "reliable" samples by setting an entropy threshold. On the selected reliable samples, the gradient of the entropy loss with respect to the feature representation is calculated, and this gradient is used to construct the perturbation direction. By finding the maximum step size in the perturbation direction, the sharpness of the decision boundary is quantified. Finally, the average entropy loss after perturbation is used as the feature sharpness loss, and this is used as the optimization objective to update the model parameters. By minimizing this loss, the model learns robust feature representations in the feature space that are far from the decision boundary, thereby improving the model's robustness during the adaptation process.

[0051] This application innovatively proposes a normalization structure reconstruction without data distillation, which improves the model's robustness to shifts in unknown domains from the model structure level, and further enhances the stability of online adaptation by minimizing feature sharpness. This approach not only solves the problem of limited adaptability of existing technologies in complex dynamic scenarios and with extremely small batch inputs, but also overcomes the bottlenecks brought about by traditional batch normalization structures, providing a new solution for the stable deployment of models in real open environments.

[0052] This application provides an online adaptation method for normalized structure reconstruction and feature sharpness perception, relating to the field of transfer learning. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited thereto. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the online adaptation method for normalized structure reconstruction and feature sharpness perception, but is not limited to the above forms.

[0053] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0054] Figure 1 This is an optional flowchart of an online adaptation method for normalized structure reconstruction and feature sharpness perception provided in an embodiment of this application. Figure 1 The method may include, but is not limited to, steps S1 to S2.

[0055] Step S1, Normalization Layer Reconstruction Phase: This phase is performed before model deployment and aims to build an inherently robust base model.

[0056] Specifically, step S1 includes the following steps S11-S14:

[0057] S11. Obtain the pre-trained batch normalized (BN) model as the teacher model.

[0058] S12. Using data-free distillation techniques, generate pseudo-data that matches the distribution of the source domain data. Specifically, this includes: initializing noisy data; inputting the noisy data into the teacher model and calculating the batch statistics generated in each BN layer during its forward propagation; calculating the statistical matching loss between the batch statistics and the pre-stored running statistics in the BN layer; and optimizing the noisy data to minimize this statistical matching loss to obtain optimized pseudo-data.

[0059] S13. Replace the BN layer in the teacher model with a group normalization (GN) layer or layer normalization (LN) layer that is insensitive to batch size to form the initial student model, and inherit the learnable parameters (scale and shift parameters γ and β) of the original BN layer.

[0060] S14. The initial student model is subjected to knowledge distillation using the pseudo-data to transfer the knowledge from the teacher model to the reconstructed student model, resulting in a normalized layer reconstructed model. The knowledge distillation loss includes the KL divergence loss of the output logits and the mean squared error loss of the intermediate layer features.

[0061] Step S2, Online Adaptation Phase: This phase is executed in real time after the model is deployed and in response to the test data stream.

[0062] Specifically, step S2 includes the following steps S21-S24:

[0063] S21. Obtain unlabeled test samples.

[0064] S22. Based on the normalized layer reconstruction model, the prediction confidence (such as entropy) of the test samples is screened, and samples with entropy values ​​lower than a preset threshold are screened as reliable samples to form a reliable sample subset.

[0065] S23. Based on the reliable sample subset, calculate the feature sharpness loss. Specifically, this includes: calculating the average entropy loss on the reliable sample subset; calculating the gradient of the average entropy loss with respect to the model feature representation, and constructing a normalized perturbation direction accordingly; perturbing the features along the perturbation direction; calculating the predicted average entropy of the perturbed features, and using it as the feature sharpness loss.

[0066] S24. With the goal of minimizing the feature sharpness loss, update the parameters of the normalization layer reconstruction model to achieve online adaptation and improve the robustness of the model to the current test environment.

[0067] The solutions of the embodiments of the present invention will now be described in detail and explained with reference to the accompanying drawings and specific application examples:

[0068] like Figure 2 As shown in the embodiments of this application, a stable online adaptation method based on normalization structure reconstruction and feature sharpness perception without data distillation is proposed, aiming to overcome the bottlenecks of insufficient adaptability and stability caused by the dependence of existing technologies on batch normalization structures and local statistics. This method includes two working modes: 1) a normalization layer reconstruction mode without data distillation, which reconstructs the normalization layer at the model structure level to improve the model's inherent robustness; 2) an online adaptation mode based on feature sharpness minimization, which improves the model's robustness during the adaptation process by learning robust feature representations far from the decision boundary in the feature space.

[0069] (1) Normalized layer reconstruction mode without data distillation

[0070] This method proposes a technique to generate pseudo-data from pre-trained batch normalized (BN) models using data-free distillation and then distill grouped normalized (GN) / layer normalized (LN) models, aiming to improve the robustness of online adaptation at the model structure level. This method does not require access to real source domain data and operates solely on the internal statistics of the pre-trained BN model. The algorithm includes the following steps:

[0071] Step 1: Initialize noise and teacher model. Use the pre-trained Batch Normalization (BN) model as the teacher model f. teacher It contains multiple batch normalization (BN) layers, each with runtime statistics obtained from training the model on the source domain data. and (l represents the layer index); the initial noise z0~N(0,I) has a shape of (B,C,H,W), where B is the size of the synthesis batch. This step prepares the noise as the basis for pseudo-data generation and loads the BN statistic of the teacher model as a constraint prior.

[0072] Step 2: Optimize the pseudo-data to match the BN statistic. Specifically, pseudo-data optimization includes the following steps:

[0073] Step 2.1: For each BN layer l, calculate the batch mean of the noise z using the teacher model forward propagation. and batch variance Where B is the batch size, z l Activate the input of layer l.

[0074] Step 2.2: Define the statistical matching loss L stats This is used to minimize the difference between batch statistics and runtime statistics.

[0075]

[0076] in, ∈ is the square of the L2 norm, and ∈ is a small constant (e.g., 1e-5) to avoid numerical instability.

[0077] Step 2.3: Define the optimization objective as follows:

[0078]

[0079] The SGD optimizer is used to iterate over the noise z (considered a learnable parameter) to be optimized. This optimization ensures that the pseudo-data simulates the low-order moment distribution of the source domain in the feature space, i.e., μ. batch (z * )≈E source [x], This generates pseudo-sample proxies that are similar to the source domain.

[0080] Step 3: Replace the normalization layer and initialize the student model. Specifically, the normalization layer replacement includes the following steps:

[0081] Step 3.1: For each BN layer, replace the student model with GN or LN, inheriting the learnable parameters γ and β of BN. The original normalization formula for BN is:

[0082]

[0083] Step 3.2, If replaced with GN: Divide the channels into G groups, and normalize independently within each group:

[0084]

[0085] in, C g =C / G, initial γ and β are the same as BN.

[0086] Step 3.3, If replaced with LN: Normalize across the entire layer (channel and spatial dimensions):

[0087]

[0088] in, Initially, γ and β are the same as BN.

[0089] Step 4: Perform knowledge distillation using pseudo-data. Specifically, knowledge distillation includes the following process:

[0090] Step 4.1: For each pseudo-data batch z * Calculate the logit output for teachers and students: logit t =f teacher (z * ),logit s =f student (z * ).

[0091] Step 4.2, Define the distillation loss function as follows:

[0092]

[0093] in, τ is the KL divergence, which measures the difference between two probability distributions; τ is a temperature parameter used to soften the softmax distribution.

[0094] Step 4.3: Add feature-level alignment loss Where φ l The activation features of the l-th layer, This represents the mean square error.

[0095] Step 4.4, Total Loss:

[0096]

[0097] Using the SGD optimizer based on The student model weights are updated while preserving the decision boundary of the teacher model. At the same time, the GN / LN weights of the student model are adjusted to adapt to pseudo-data, thereby achieving knowledge transfer without data.

[0098] Step 5: Repeat steps 1-4 to generate more pseudo-data for knowledge distillation until the student model performance is stable, at which point the method stops.

[0099] The grouped normalization (GN) or layered normalization (LN) models obtained after the above reconstruction no longer rely on batch statistics for normalization operations, but instead perform independent normalization based on channel or intra-group statistics within a single sample. Therefore, based on the reconstructed GN / LN models, independent normalization of each sample can be achieved, effectively solving the performance degradation problem caused by inaccurate batch statistics in traditional BN models in scenarios such as real-time inference of edge devices (e.g., batch size=1), highly imbalanced categories in surveillance videos (e.g., containing only background and rare pedestrians), and mixed scenarios with multiple offsets in batches due to synchronous changes in lighting, weather, and scene in autonomous driving.

[0100] (2) Online adaptation mode based on feature sharpness minimization

[0101] This paper proposes an online adaptation method based on feature sharpness minimization. Its goal is to select reliable samples and enable the model to learn more robust feature representations in the feature space that are far from the decision boundary, thereby improving its robustness to test data perturbations and the stability of online adaptation. This method effectively avoids potential model crashes during online adaptation in complex dynamic environments (such as the three scenarios mentioned above) by precisely controlling the samples used for adaptation and the optimization objective. The detailed steps of this method are as follows:

[0102] Step 1: Given a batch of unlabeled test samples This method first obtains the feature representation φ of each sample through the forward propagation of the model. i Then, the feature φ i The classification head input to the model In the process, the original output prediction is obtained. in Denotes the classifier, θ h Its parameters. Based on the prediction result, the entropy value E of each sample is calculated. i :

[0103]

[0104] Where C represents the total number of categories. Let be the predicted probability that sample i belongs to category c.

[0105] Step 2: To avoid the negative impact of noise and uncertain samples on online adaptation, this method introduces a reliable sample selection mechanism. A predefined entropy threshold E0 is set, and only samples with entropy values ​​below this threshold are retained. These samples are typically those that the model can predict with high confidence and are considered "reliable." The selected sample subset is denoted as... in All subsequent adaptation and optimization steps are performed only on the subset. The above will be carried out.

[0106] Step 3: In the selected reliable sample subset Calculate its average original entropy loss.

[0107]

[0108] Step 4: Calculate the feature gradient and construct the perturbation direction for the selected reliable sample subset. Calculate its entropy loss Gradient of feature representation φ This gradient vector indicates in the feature space, along which a small shift will most rapidly increase the model's predictive uncertainty. A normalized perturbation direction v is constructed using this gradient:

[0109]

[0110] Step 5: To quantify the sharpness of the decision boundary, this method finds a maximum step size s in the perturbation direction v, such that after moving the feature φ in the feature space to φ′=φ+s·v, the predicted category of the model remains unchanged, and the corresponding step size s directly reflects the stability of the region.

[0111] Step 6: Apply the maximum step size s and perturbation direction v determined in Step 5 to the original feature φ of the reliable sample to obtain the feature φ′ = φ + s·v located at the edge of the decision boundary. Then, input φ′ into the classification head to obtain the perturbed prediction. Finally, we calculate the perturbed average entropy loss as the feature sharpness loss.

[0112]

[0113] Step 7: Calculate the sharpness loss obtained in Step 6. As the optimization objective, the parameters θ of the model are updated using the backpropagation mechanism. Formalized as:

[0114]

[0115] Here, θ represents the learnable parameters of the normalized layer of the model. By minimizing this loss, the model learns robust feature representations far from the decision boundary in the feature space, thereby improving the model's robustness during the adaptation process. Repeat the above steps until all test data has been processed, at which point the model stops online adaptation.

[0116] In summary, this application aims to address the performance degradation and instability issues of existing online adaptation techniques in complex dynamic scenarios such as batch class imbalance, mixed offsets in batches, or extremely small batch inputs, due to their reliance on batch normalization structures and local statistics. To address this, this application proposes a stable online adaptation method, electronic device, storage medium, and program product based on normalization structure reconstruction and feature sharpness awareness without data distillation. This method includes two operating modes: 1) Normalization layer reconstruction mode without data distillation: This mode utilizes data distillation technology to reconstruct the pre-trained batch normalization model into a grouped normalization or layer normalization model that is insensitive to batch size at the model structure level. This fundamentally solves the limitation of existing methods in failing to operate stably in scenarios with batch class imbalance, mixed offsets in batches, or extremely small batch inputs. 2) Online adaptation mode based on feature sharpness minimization: This mode effectively improves the robustness of the model in the adaptation process by screening high-confidence samples and allowing the model to learn robust feature representations far away from the decision boundary in the feature space, thereby improving the model's adaptability and stability in complex and ever-changing environments.

[0117] In summary, this invention innovatively applies data-free distillation technology to normalization layer reconstruction, improving the model's inherent robustness at the model structure level; and enhances the model's stability during online adaptation by minimizing feature sharpness. This method overcomes the dependence of existing test-time adaptation techniques based on batch normalization statistics calibration on batch normalization statistics, solving its adaptation bottleneck in small-batch and complex dynamic scenarios, and enabling the model to have stronger generalization ability and practicality in open worlds. Compared with existing technologies, it has at least the following advantages and beneficial effects:

[0118] 1) Fundamental structural innovation: By reconstructing the normalization layer without data distillation, the model is transformed from a BN structure to a GN or LN structure, completely eliminating the dependence on test batch statistics and fundamentally solving the problem of inaccurate batch normalization statistics in scenarios such as very small batches, intra-batch class imbalance, and mixed offsets.

[0119] 2) Strong adaptability and high stability: An online adaptation mechanism based on feature sharpness minimization is introduced. By selecting reliable samples and allowing the model to learn robust feature representations far from the decision boundary in the feature space, the robustness of the model to distribution shifts and input perturbations is significantly enhanced, effectively avoiding the potential collapse problem of the model when adapting online in complex dynamic environments (such as the three scenarios mentioned above).

[0120] 3) High practicality and security: The entire process does not require access to the original training data, but only uses the pre-trained model itself and unlabeled test data, which reduces the risk of data privacy leakage and is more in line with the deployment needs of actual industrial applications.

[0121] 4) Significantly improved performance: This method provides an effective technical solution for the stable and reliable deployment of models in complex and dynamic open-world environments, with significant performance advantages and broad applicability.

[0122] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0123] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0124] Please see Figure 3 , Figure 3 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0125] The processor 301 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0126] The memory 302 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 302 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 302 and is called and executed by the processor 301 using the methods described in the embodiments of this application.

[0127] Input / output interface 303 is used to implement information input and output;

[0128] The communication interface 304 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0129] Bus 305 transmits information between various components of the device (e.g., processor 301, memory 302, input / output interface 303, and communication interface 304);

[0130] The processor 301, memory 302, input / output interface 303, and communication interface 304 are connected to each other within the device via bus 305.

[0131] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0132] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0133] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0134] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0135] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented in the embodiments of this program product are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments. The executable computer program code or "code" used to perform the various embodiments can be written in high-level programming languages ​​such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0136] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0137] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0138] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0139] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0140] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0141] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0142] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0143] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0144] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0145] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0146] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. An online adaptive method for normalized structure reconstruction and feature sharpness perception, characterized in that, The method includes the following steps: Normalization layer reconstruction stage: Obtain a pre-trained batch normalization model; use data-free distillation technology to generate pseudo-data that matches the distribution of source domain data; replace the batch normalization layer in the pre-trained batch normalization model with a group normalization layer or a layer normalization layer to form an initial student model; use the pseudo-data to perform knowledge distillation on the initial student model to transfer the knowledge of the batch normalization model to the reconstructed student model, thus obtaining the normalization layer reconstruction model; Online adaptation phase: Obtain unlabeled test samples; filter the prediction confidence of the test samples based on the normalized layer reconstruction model to obtain a reliable sample subset; calculate the feature sharpness loss based on the reliable sample subset, the feature sharpness loss is used to characterize the decision boundary sharpness of the normalized layer reconstruction model in the feature space; update the parameters of the normalized layer reconstruction model with the goal of minimizing the feature sharpness loss; The calculation of feature sharpness loss includes: Calculate the average entropy loss on the reliable sample subset. ; Calculate the average entropy loss Regarding model feature representation The gradient; A normalized perturbation direction is constructed based on the gradient. ; Along the direction of the disturbance Features Perturbation is performed to obtain the perturbed features. ; Calculate the features based on the perturbation The obtained average entropy of the prediction is used as the feature sharpness loss. .

2. The method according to claim 1, characterized in that, The method of generating pseudo-data that matches the distribution of source domain data using data-free distillation includes: Initialize noise data; The noise data is input into the batch normalization model to calculate the batch statistics generated at each batch normalization layer during the forward propagation process. Calculate the statistical matching loss between the batch statistics and the pre-stored runtime statistics in the batch normalization layer; By optimizing the noisy data to minimize the statistical matching loss, optimized pseudo-data is obtained, and the batch statistics of the pseudo-data are matched with the running statistics.

3. The method according to claim 2, characterized in that, The statistical matching loss Calculated using the following formula: in, For layer index, , Noise data for the current batch In the The mean and variance of the layer The model obtained by training on the source domain is the first... The mean and variance of the layer; The square of the L2 norm; It is a small constant.

4. The method according to claim 1, characterized in that, The batch normalization layer is replaced with a group normalization layer or a layer normalization layer, and the learnable parameters of the batch normalization layer are inherited. γ and β .

5. The method according to claim 1, characterized in that, The step of using the pseudo-data to perform knowledge distillation on the initial student model includes: Calculate the outputs of the batch normalization model and the student model for the same pseudo-data; The parameters of the student model are updated with the goal of minimizing the difference between the output of the batch normalization model and the student model. The difference between the outputs is calculated using KL divergence to determine distillation loss, and / or using mean square error to determine feature layer alignment loss.

6. The method according to claim 1, characterized in that, The process of filtering the prediction confidence of the test samples includes: Calculate the prediction entropy for each test sample; Samples whose predicted entropy is lower than a preset entropy threshold are selected as reliable samples, forming the reliable sample subset.

7. The method according to claim 1, characterized in that, The direction of the disturbance Calculated using the following formula in, For feature representation gradient, It is an L2 norm.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images