Federated learning based ar scene dynamic deep learning model training and inference method

Abstract

The application relates to the field of federated learning and augmented reality technology, and discloses an AR scene dynamic deep learning model training and reasoning method based on federated learning. The method comprises the following steps: a cloud end initializes a global model and distributes the global model to terminal equipment; after local AR scene data collected by the terminal equipment is preprocessed, forward calculation and loss evaluation of the model are performed, local gradient updates are generated, and distribution characteristics of the local gradient updates are analyzed to identify bias patterns and device-specific patterns, and the local gradient updates are uploaded after being encrypted; the cloud end combines historical parameter snapshots to construct a dynamic adjustment module, evaluates update consistency, determines an optimization direction, fuses multiple source gradient updates to a global model, and finally performs parameter pruning and precision quantization on the model to generate a lightweight reasoning model. Through gradient distribution characteristic analysis and historical consistency evaluation, the application improves adaptability, training stability and communication security of the model in a dynamic AR environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of federated learning and augmented reality technology, in particular to an AR scene dynamic deep learning model training and inference method based on federated learning. BACKGROUND

[0002] The performance of AR applications depends on the real-time perception and interaction ability of deep learning models for complex scenes. The optimization of the model requires continuous learning of massive and diverse terminal data. However, AR data contains visual information and behavior data of the user's environment, which is highly privacy-sensitive. Direct uploading to the cloud for centralized training poses a significant risk of privacy leakage. Federated learning technology enables training on terminals locally and only uploads model updates to the cloud for aggregation, providing a basic framework for privacy-protected model training in AR scenarios.

[0003] Existing federated learning solutions have limitations when dealing with dynamic and heterogeneous AR data. The standard process treats gradient updates from different devices as homogeneous inputs for simple aggregation, ignoring the heterogeneity of gradient distribution caused by device hardware differences, user habits, and environmental dynamics. This processing cannot distinguish between beneficial device characteristic patterns and harmful biases caused by environmental interference, resulting in poor global model adaptability and unstable convergence.

[0004] The existing cloud aggregation mechanism is static and has no memory, only relying on the current round of updates for decision-making, lacking reference to the historical training state of the model. AR data has strong temporal correlation, and model updates need to evaluate their consistency with long-term optimization trajectories. The current technology cannot identify the impact of short-term data fluctuations on optimization direction, easily leading to model performance oscillation. Communication security policies are extensive and cannot differentiate protection based on gradient content risk, making it difficult to balance security and efficiency. A new method is needed that can deeply analyze gradient features, have a historical perspective, and implement intelligent security policies. SUMMARY

[0005] The purpose of the present application is to provide an AR scene dynamic deep learning model training and inference method based on federated learning to solve the problems raised in the background.

[0006] To achieve the above purpose, the present application provides an AR scene dynamic deep learning model training and inference method based on federated learning, which comprises:

[0007] Initializing a cloud federated learning environment, configuring global model parameters and distributing them to multiple terminal devices;

[0008] The terminal device collects local AR scene data, including image frames and sensor readings, timestamps the collected data and standardizes the resolution, generating a preprocessed local dataset;

[0009] performing a model forward calculation using the pre-processed local dataset and the global model parameters to obtain a prediction output, and calculating a loss value based on the prediction output and the real label;

[0010] deriving a local model gradient update from the loss value, and performing normalization processing on the local model gradient update to generate a standardized gradient update;

[0011] analyzing the distribution characteristics in the standardized gradient update to identify bias patterns and device-specific patterns in the gradient update;

[0012] applying an encrypted transmission protocol to the identified bias patterns and device-specific patterns to perform data encryption and integrity protection to generate secure gradient data;

[0013] receiving the secure gradient data at the cloud, combining past parameter snapshots in a historical model parameter library to construct a dynamic adjustment module for evaluating model update consistency;

[0014] determining a model optimization direction based on the output of the dynamic adjustment module and fusing multi-source gradient data;

[0015] updating the global model parameters according to the fusion result, and performing structural simplification processing on the updated model, including parameter pruning and precision quantization, to generate a lightweight inference model.

[0016] Preferably, the application of the encrypted transmission protocol to the identified bias patterns and device-specific patterns to perform data encryption and integrity protection to generate secure gradient data comprises:

[0017] extracting key feature vectors from the bias patterns and device-specific patterns, and performing dimensionality reduction processing using a feature selection algorithm to obtain a reduced feature set;

[0018] applying a symmetric encryption algorithm to the reduced feature set using a dynamically generated key to generate initial encrypted data;

[0019] verifying the integrity and authenticity of the initial encrypted data by calculating a data digest using a hash function and attaching a digital signature;

[0020] based on the verification result, performing block transmission optimization on the initial encrypted data, adjusting the data packet size to adapt to network conditions, and finally generating secure gradient data.

[0021] Preferably, the verification of the integrity and authenticity of the initial encrypted data by calculating a data digest using a hash function and attaching a digital signature comprises:

[0022] calculating the SHA-256 hash value of the initial encrypted data to obtain the data digest;

[0023] The terminal device uses its private key to sign the data digest, generating a digital signature;

[0024] The digital signature is bound to the initial encrypted data to form a signed data packet;

[0025] The cloud uses the terminal device's public key to verify the digital signature, confirming data integrity and source, and discarding the data packet if verification fails.

[0026] Preferably, the cloud receives security gradient data, combines past parameter snapshots in the historical model parameter library, and constructs a dynamic adjustment module for evaluating model update consistency, including:

[0027] Load the model parameters of the last several iterations from the historical model parameter library as a reference parameter set;

[0028] After decrypting the security gradient data, compare it with the reference parameter set to calculate the parameter change;

[0029] Based on the parameter change, apply a sliding window algorithm to analyze the trend and identify long-term consistency and short-term fluctuations;

[0030] According to the analysis results, construct a dynamic weight distribution mechanism to adjust the contribution of different terminal device gradients in aggregation.

[0031] Preferably, based on the parameter change, applying a sliding window algorithm to analyze the trend and identify long-term consistency and short-term fluctuations, includes:

[0032] Define the size of the time window and extract the parameter change sequence within the window;

[0033] Calculate the moving average and standard deviation of the sequence to evaluate the stability of the change;

[0034] Identify rising or falling trends and mark abnormal fluctuation points through trend detection algorithms;

[0035] Combine the long-term consistency score and short-term fluctuation index to generate a change trend report.

[0036] Preferably, the analysis of the distribution characteristics in the standardized gradient update to identify bias patterns and device-specific patterns in the gradient update includes:

[0037] Calculate the statistical moments of the standardized gradient update, including mean and variance, to generate a distribution summary;

[0038] Apply the Isolation Forest algorithm to the distribution summary for anomaly scoring to identify outliers;

[0039] Based on device hardware configuration information such as processor type and memory size, cluster the gradient updates to identify device group-specific patterns;

[0040] Fusing the anomaly score and the device group information, outputting the bias pattern and the device-specific pattern.

[0041] Preferably, the device hardware configuration information, such as processor type and memory size, is used to cluster gradient updates to identify device group-specific patterns, including:

[0042] Collecting hardware configuration data of terminal devices, including CPU core number and GPU performance indicators;

[0043] Using the K-means algorithm to cluster the hardware configuration data to generate device groups;

[0044] For each device group, analyze the common characteristics of its gradient updates and extract typical patterns of the group;

[0045] Comparing the typical patterns of different groups to identify device dependency differences.

[0046] Preferably, the preprocessed local data set is used to perform forward calculation of the model with the global model parameters to obtain a predicted output, and a loss value is calculated based on the predicted output and the true label, including:

[0047] Loading the global model parameters into the local memory of the terminal device to perform forward propagation of the deep learning model;

[0048] Inputting the preprocessed local data set to obtain a predicted output tensor;

[0049] Comparing the predicted output tensor with the true label and using a cross-entropy loss function to calculate a loss value;

[0050] Smoothing the loss value to reduce noise effects and generating an optimized loss value.

[0051] Preferably, the loss value is smoothed to reduce noise effects and generate an optimized loss value, including:

[0052] Applying an exponential moving average algorithm to smooth the loss value sequence to obtain a smoothed loss;

[0053] Calculating the first derivative of the smoothed loss to evaluate the loss change rate;

[0054] Adjusting the learning rate parameter based on the change rate to adaptively control the training process;

[0055] Outputting the optimized loss value for gradient calculation.

[0056] Preferably, the global model parameters are updated according to the fusion result, and the updated model is subjected to structural simplification processing, including parameter pruning and precision quantization, to generate a lightweight inference model, including:

[0057] Update the global model parameters using the gradient fusion result to generate a new version of the model;

[0058] Apply a model pruning algorithm to remove redundant connections and neurons, reducing the number of parameters;

[0059] Quantize the pruned model to convert floating-point parameters to low-precision fixed-point representations;

[0060] Verify the inference accuracy of the simplified model in the AR scene, and output the final lightweight inference model.

[0061] Compared with the prior art, the beneficial effects of the present application are:

[0062] By analyzing the distribution characteristics of the standardized gradient updates, common deviation patterns caused by AR environment mutations and specific patterns derived from device hardware and usage habits can be accurately identified. This process converts gradient data from a simple parameter adjustment vector into a signal carrying environmental and device information. Based on the identified specific patterns, an encrypted transmission protocol is applied to achieve fine-grained deployment of security policies. More targeted data encryption and integrity protection is implemented for identified key patterns or abnormal patterns, rather than uniform and possibly redundant or insufficient security overhead for all gradient data. This content-aware differentiated security mechanism optimizes the utilization of communication and computing resources while ensuring the security of critical data.

[0063] The cloud aggregation link introduces past parameter snapshots from the historical model parameter library to build a dynamic adjustment module to evaluate the consistency of the current model update with the historical evolution trajectory. This module enables the aggregation process to have time sequence awareness, allowing it to determine whether the current gradient update direction deviates from the long-term learning trend of the model. Based on the consistency evaluation results, the model optimization direction is dynamically determined, and multiple sources of gradient data are intelligently fused. When the current update is highly consistent with the historical trend, it is fully adopted; when there are short-term drastic fluctuations or potential interference, it is smoothed or corrected. This mechanism effectively suppresses model oscillation caused by temporary disturbances or abnormal data from some devices in the AR scene, improving the stability of the training process and the robustness of the final model, allowing the global model to evolve in a more reliable direction. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 The working principle diagram of the AR scene dynamic deep learning model training and inference method based on federated learning described in the present application;

[0065] Figure 2 Flowchart for generating secure gradient data;

[0066] Figure 3 Flowchart for building a dynamic adjustment module and consistency evaluation;

[0067] Figure 4 Fig. 2 is a diagram of the change of the optimized loss value of the AR model local training for different terminal devices with the iteration number;

[0068] Figure 5 Fig. 3 is a performance comparison diagram of different quantization schemes in the AR model. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0070] Please refer to Figure 1 The present application provides an AR scene dynamic deep learning model training and inference method based on federated learning, which comprises the following steps: initializing a cloud-side federated learning environment, configuring global model parameters and distributing them to multiple terminal devices; the terminal device collects local AR scene data, including image frames and sensor readings, performs timestamp alignment and resolution standardization processing on the collected data, and generates a preprocessed local data set; using the preprocessed local data set and the global model parameters to perform model forward calculation to obtain a prediction output, and calculating a loss value based on the prediction output and the real label; deriving a local model gradient update from the loss value, and performing normalization processing on the local model gradient update to generate a standardized gradient update; analyzing the distribution characteristics in the standardized gradient update, identifying the bias mode and device-specific mode in the gradient update; applying an encryption transmission protocol to the identified bias mode and device-specific mode to perform data encryption and integrity protection, and generating secure gradient data; the cloud side receives the secure gradient data, combines the past parameter snapshots in the historical model parameter library to construct a dynamic adjustment module for evaluating model update consistency; based on the output of the dynamic adjustment module, determining the model optimization direction and fusing multi-source gradient data; updating the global model parameters according to the fusion result, and performing structure simplification processing on the updated model, including parameter pruning and precision quantization, to generate a lightweight inference model.

[0071] Embodiment 1: Please refer to Figure 2In specific implementation, the extraction of the key feature vectors in the bias pattern and the device-specific pattern is the first step of dimensionality reduction using a feature selection algorithm. The key feature vectors are derived from the distribution characteristic analysis of the normalized gradient updates. The feature selection algorithm uses principal component analysis method to project the high-dimensional feature vectors into a low-dimensional subspace. By calculating the covariance matrix and eigenvalue decomposition of the features, the principal components with the largest variance contribution rate are retained to form a reduced feature set. For example, assuming that the distribution characteristic analysis of the normalized gradient updates obtains a 100-dimensional feature vector (such as [0.2, 0.5, 0.3, …, 0.1]), the eigenvalue sequence is obtained after calculating the covariance matrix, and the cumulative variance contribution rate of the first three principal components reaches 95%. Therefore, the reduced feature set is formed by retaining the three principal components. In specific implementation, the application of the symmetric encryption algorithm to the reduced feature set is the core step of generating the initial encrypted data. The symmetric encryption algorithm selects the AES-256 standard. A dynamically generated key is used in the encryption process. The dynamic key is generated by a cryptographically secure pseudo-random number generator. Each transmission session of each terminal device uses an independent key. The encryption operation divides the reduced feature set data into fixed-size data blocks. Each data block is encrypted using the AES encryption algorithm to generate the initial encrypted data. For example, the reduced feature set data block is a hexadecimal string '0x1234567890ABCDEF'. The dynamic key is generated by a cryptographically secure pseudo-random number generator (such as '0xA1B2C3D4E5F60718G9H0I1J2K3L4M5N6O7P8Q9R0'). After encryption, the initial encrypted data block '0x9876543210FEDCBA...' is obtained. In specific implementation, the verification of the integrity and authenticity of the initial encrypted data is achieved by calculating the data digest using a hash function and attaching a digital signature. The SHA-256 hash value of the initial encrypted data is calculated to obtain the data digest. The SHA-256 algorithm generates a 256-bit hash value for the input data as a digital fingerprint of the data. For example, the initial encrypted data is'sample_gradient_data_2025'. The calculated SHA-256 hash value is '0x5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8'.

[0072] The terminal device uses its private key to sign the data digest to generate a digital signature, and the signature algorithm uses the RSA public key cryptography system. The private key of the terminal device is stored in the hardware security module or the trusted execution environment. For example, the data digest is '0x5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8', and the digital signature '0x1234567890ABCDEF...' is obtained after signing with the RSA private key (such as modulus 3233 and private key exponent 2753). The digital signature is bound with the initial encrypted data to form a signed data packet. The structure of the signed data packet includes a data header, an initial encrypted data payload, a digital signature field, and a timestamp field. The cloud uses the public key of the terminal device to verify the digital signature to confirm the data integrity and source. The verification process includes parsing the signed data packet, extracting the digital signature and the initial encrypted data, using the pre-stored public key of the terminal device to decrypt the digital signature to obtain the original digest value, and recalculating the SHA-256 hash value of the received initial encrypted data. The consistency of the two digest values is compared. If the verification fails, the data packet is discarded and a security event is recorded. In specific implementation, based on the verification result, the initial encrypted data is subjected to block transmission optimization to adapt to the network conditions. The block transmission optimization dynamically adjusts the data packet size according to the real-time network bandwidth and delay indicators. In a high-bandwidth and low-delay network environment, a larger data packet is used to reduce transmission overhead, and in a low-bandwidth and high-delay network environment, a smaller data packet is used to improve the transmission success rate. The final generated security gradient data contains complete data units that have been encrypted, signed, and transmission-optimized. For example, when the network bandwidth is monitored to be 100 Mbps and the delay is 10 ms, the data packet size is set to 1024 KB; when the bandwidth is 10 Mbps and the delay is 100 ms, the data packet size is adjusted to 128 KB.

[0073] In some embodiments, the implementation of the feature selection algorithm includes data standardization, covariance matrix calculation and eigenvalue sorting steps. The data standardization scales the values of each dimension of the key feature vector to the same scale, for example, the original feature vector is [10, 200, 3], and after standardization, it becomes [0.1, 0.8, 0.03] (assuming scaling to the 0-1 range); the covariance matrix reflects the linear relationship between features, for example, two feature vectors X = [1, 2, 3] and Y = [4, 5, 6], the covariance calculation is ((1-2)(4-5)+(2-2)(5-5)+(3-2)(6-5)) / 2 = 1; the eigenvalue sorting determines the number of principal components selected, and the dimension of the reduced feature set after dimensionality reduction is usually reduced to 10% to 30% of the original features. In some embodiments, the implementation details of the symmetric encryption algorithm include key generation, encryption mode and padding scheme. The key generation uses a random number generator that meets the FIPS140-3 standard, for example, generating a 256-bit random key '0xA1B2C3D4E5F60718G9H0I1J2K3L4M5N6O7P8Q9R0'; the encryption mode selects the CBC mode to enhance security, and the padding scheme uses the PKCS7 padding to ensure that the data block length meets the encryption requirements.

[0074] Optionally, the digital signature verification process can add a retransmission mechanism. When the cloud fails to verify the digital signature, a retransmission request is sent to the terminal device, and the terminal device re-executes the encryption and signature process and then sends the data again. Optionally, the block transmission optimization can be combined with the forward error correction coding technology to add redundant check information in the data packet to improve the fault tolerance of data transmission. It can be understood that the implementation of the encrypted transmission protocol requires the terminal device and the cloud to have corresponding cryptographic computing capabilities. The terminal device needs to integrate a security chip or a software cryptographic library to support high-strength encryption operations. It can be understood that the entire generation process of the security gradient data is a multi-step pipeline operation, and the output of each step is used as the input of the next step to form a complete data security processing chain.

[0075] In specific implementation, the execution of the principal component analysis method needs to set a variance retention threshold, usually set to 95% of the cumulative variance contribution rate, and the number of principal components to be retained is determined by calculating the cumulative percentage of eigenvalues, for example, the eigenvalue sequence is [5, 3, 2, 1, 0.5], the cumulative variance contribution rate is calculated as 5 / (5+3+2+1+0.5)=40%, 5+3=8 / 11.5≈69.57%, 5+3+2=10 / 11.5≈86.96%, 5+3+2+1=11 / 11.5≈95.65%, then the first 4 principal components are retained. The reduced feature set after dimensionality reduction not only reduces the data volume, but also eliminates the correlation between features, which is beneficial to improve the efficiency of subsequent encryption operation. In specific implementation, the implementation of AES-256 encryption algorithm involves initial round key addition, multiple round byte substitution, row shifting, column mixing and round key addition operations. The encryption process uses a 256-bit key and needs to perform 14 rounds of transformation. Each round of transformation performs nonlinear transformation and linear transformation on the data block to ensure that the encryption strength reaches the military-level security standard. For example, the initial data block '0x00112233445566778899AABBCCDDEEFF' becomes '0x637c777bf26b6fc530059c608517f32bfe2ce282f90400b20709379259047913' after 1 round of byte substitution. In specific implementation, SHA-256 hash calculation is an iterative compression process. After the input data is padded to a multiple of 512 bits, it is divided into multiple message blocks. Each message block is applied to 64 rounds of logical function and constant addition operation, and finally outputs a 256-bit hash value. The avalanche effect of the hash value ensures that a small change in the input data will result in a large difference in the output hash value. For example, the input data'sample' is padded to get the message block '0x73616d706c6580000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000028', and after 64 rounds of operation, the hash value is output as '0x5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8'.

[0076] In specific implementation, the RSA digital signature scheme uses PKCS#1 v1.5 or PSS padding mode, and the data digest is encoded and padded before signature, and then the private key is used for modular exponentiation to generate the signature. When verifying, the public key is used to decrypt the signature to obtain the encoded digest, which is compared with the calculated digest. For example, the data digest '0x5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8' becomes '0x0001ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff003031300d0609608648016503040201050004205e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8' after padding with PKCS#1 v1.5. In specific implementation, the implementation of block transmission optimization needs to monitor network state parameters in real time, including bandwidth, packet loss rate and round trip time, and dynamically calculate the optimal data block size according to these parameters. In the transmission process, a sliding window mechanism is used for flow control to ensure the rational use of network resources. For example, when the bandwidth is 50 Mbps, the packet loss rate is 0.5%, and the round trip time is 20 ms, the optimal data block size is calculated as 512 KB.

[0077] In specific implementation, the dimension reduction processing of feature selection algorithm can combine variance filtering and correlation analysis, remove features with low variance first, remove highly correlated features, and finally apply principal component analysis for dimension compression. For example, feature A variance 0.01 (remove for being too low), feature B variance 0.8 (retain), feature C correlation coefficient with feature B 0.95 (remove feature C). In specific implementation, the key management of symmetric encryption algorithm uses key derivation function to derive session key from master key, and different salt values are used for each session to ensure the uniqueness of the key. In specific implementation, the digital signature binding process includes serializing the initial encrypted data and digital signature, adding version identification and length field, and forming a signature data packet conforming to the ASN.1 encoding specification. In specific implementation, the network adaptation algorithm for block transmission optimization can use the method of reinforcement learning to dynamically adjust the block strategy according to the historical transmission success rate, and optimize the overall transmission efficiency. The eigenvalue calculation of principal component analysis uses power iteration method or QR algorithm, and the eigenvector is obtained by solving the characteristic equation, and the projection transformation multiplies the original eigenvector by the eigenvector matrix to complete the dimension reduction. In specific implementation, the byte substitution operation of AES encryption uses S-box for nonlinear substitution, and S-box is a precomputed 256-byte lookup table that realizes byte confusion transformation, for example, byte '0x00' is replaced by '0x63', '0x01' is replaced by '0x7c'. In specific implementation, the compression function of SHA-256 is composed of 64 logical functions, each logical function contains bit operation and modulo addition operation, and the message block is compressed into an intermediate hash value through multiple rounds of iteration, for example, the message block '0x1234567890ABCDEF' is changed to '0xabcd1234efgh5678' after one round of logical function. In specific implementation, the key generation of RSA signature needs to select two large prime numbers, calculate the modulus and Euler function, select the public key exponent to calculate the private key exponent, and the key length is usually 2048 bits or longer. For example, select prime p=61, q=53, modulus n=61*53=3233, Euler function φ(n)=60*52=3120, public key exponent e=17, private key exponent d=2753 (17*2753 mod3120=1). In specific implementation, the packet size adjustment algorithm for block transmission optimization is based on the principle of congestion control, and uses a slow start and congestion avoidance mechanism similar to TCP to dynamically adjust the transmission rate according to network feedback.

[0078] Example 2: see Figure 3In specific implementation, loading the model parameters of the last several iterations from the historical model parameter library as the reference parameter set is the first step of the dynamic adjustment module initialization, the historical model parameter library is stored in a cloud distributed file system, the model parameters are saved in binary format and are attached with version marks and time stamps, the loading process retrieves the parameter snapshots of the last five iterations through a database query interface, the reference parameter set contains complete copies of the model weight matrix and the bias vector, and data integrity is checked when loading to prevent damage. In specific implementation, the secure gradient data is decrypted and compared with the reference parameter set to calculate the parameter change amount, the decryption operation uses the symmetric decryption algorithm matching the AES-256 standard and the decryption key used during encryption, the difference comparison is performed separately for each model parameter dimension, the absolute difference or relative difference between the current gradient value and the reference parameter value is calculated, and the parameter change amount is organized in matrix form to reflect the overall model update amplitude. For example, the reference parameter value is 0.5, the current gradient value is 0.6, and the absolute difference is 0.1; the reference parameter matrix is [[0.2, 0.3], [0.4, 0.5]], the current gradient matrix is [[0.25, 0.35], [0.45, 0.55]], and the parameter change amount matrix is [[0.05, 0.05], [0.05, 0.05]]. In specific implementation, based on the parameter change amount, a sliding window algorithm is applied to analyze the change trend to identify long-term consistency and short-term fluctuations, the sliding window algorithm defines the time window size as ten iteration periods, the parameter change amount sequence within the window is extracted and arranged in chronological order, and the moving average and standard deviation of the parameter change amount sequence are calculated using an iterative update method, the moving average smooths random fluctuations and the standard deviation quantifies the dispersion degree. The moving average calculation formula of the parameter change amount is:

[0079]

[0080] wherein: MA m represents the moving average value of the mth iteration, w represents the sliding window size and takes the value of 10, Δθ n represents the parameter change amount of the nth iteration. For example, assuming that the sliding window size w = 10, the parameter change amount Δθ 11 of the 11th iteration is 0.5, the moving average MA 10 of the previous 10 times is 0.3, then MA 11 = (0.3 * 9 + 0.5) / 10 = 0.32.

[0081] In specific implementations, the trend detection algorithm identifies an upward or downward trend by fitting the slope of the parameter change sequence with linear regression, and the outlier fluctuation point detection uses the Z-score method to calculate the deviation multiple of each data point from the moving average, and marks the data point as an outlier fluctuation point when the Z-score exceeds the threshold value 3.0. For example, the moving average MA = 0.3, the standard deviation σ = 0.1, the data point Δθ = 0.8, the Z-score = (0.8-0.3) / 0.1 = 5.0 > 3.0, and it is marked as an outlier. In specific implementations, the long-term consistency score is calculated based on the coefficient of variation of the parameter change within the sliding window, such as the mean of the parameter change being 0.2, the standard deviation being 0.04, and the coefficient of variation = 0.04 / 0.2 = 0.2; the short-term fluctuation index is quantified by the density of abnormal fluctuation points, and the change trend report is output in JSON format including the trend direction, stability index and abnormal list. In specific implementations, a dynamic weight allocation mechanism is constructed to adjust the contribution of different terminal devices in the aggregation, and the weight allocation function uses the consistency score as input, and the high-score device is given an exponentially growing weight value, and the weight value is normalized to ensure the stability of the gradient norm after aggregation.

[0082] In some embodiments, the loading process of the historical model parameter library uses a caching mechanism to accelerate data access, and the most recently used parameter snapshots are kept in the memory cache to reduce disk I / O delay. In some embodiments, the difference comparison introduces normalization processing when calculating the parameter change to eliminate parameter scale differences, and the normalization method uses Min-Max scaling to map the change to the [0, 1] interval, for example, the original change range is [0.1, 0.9], and a change Δθ = 0.5, the normalized value = (0.5-0.1) / (0.9-0.1) = 0.5. Optionally, the sliding window algorithm can adaptively adjust the window size, and dynamically expand or shrink the window range according to the variance of the parameter change to balance the trend capture sensitivity. Optionally, the trend detection algorithm can integrate a machine learning model such as a support vector machine to classify the trend type and improve the accuracy of complex pattern recognition. It can be understood that the implementation of the dynamic adjustment module depends on the continuous accumulation of historical data, and the time series analysis of the parameter change requires complete and non-missing iteration cycles. The design of the weight allocation mechanism needs to avoid the gradient aggregation bias, and the weight normalization step prevents individual devices from dominating the update process.

[0083] In specific implementation, the loading process of reference parameter set includes version consistency check, comparing hash value of parameter snapshot ensures data is not tampered, triggering retry mechanism to recover data from backup storage when loading fails. In specific implementation, the decryption operation of secure gradient data is executed in secure isolation environment, decryption key is dynamically obtained through key management service, and decrypted data is immediately used for difference comparison to reduce memory residence time. In specific implementation, the calculation of parameter change amount uses Euclidean distance to measure overall change amplitude, for example, reference parameter vector θ_ref = [0.2, 0.3], current gradient vector θ_curr = [0.4, 0.5], Euclidean distance = √((0.4-0.2) 2 +(0.5-0.3) 2 )=√(0.04+0.04)=√0.08≈0.2828; the distance value is divided by the number of parameters to obtain the average change amount, avoiding the influence of model size on comparison result. In specific implementation, the moving average calculation of sliding window uses circular buffer data structure to efficiently manage window data, and the newest data replaces the oldest data to maintain fixed window size. In specific implementation, the linear regression algorithm of trend detection uses least square method to fit the relationship between parameter change amount sequence and time index, the slope sign indicates the trend direction and the slope absolute value reflects the trend strength. In specific implementation, the Z-score calculation of abnormal fluctuation point detection relies on real-time update of moving average and standard deviation, and the standard deviation uses Bessel correction formula to handle small sample bias, for example, sample [1, 2, 3], mean = 2, standard deviation = √[((1-2) 2 +(2-2) 2 +(3-2) 2 ) / (3-1)]=√(2 / 2)=1. In specific implementation, the long-term consistency score combines the autocorrelation coefficient of parameter change amount within sliding window for evaluation, and the autocorrelation lag one period calculates the correlation between the sequence before and after, for example, sequence [1, 2, 3, 4], autocorrelation coefficient lag one period ≈ 0.98. In specific implementation, the short-term fluctuation index is obtained by dividing the number of abnormal fluctuation points within the window by the window size, for example, there are 2 abnormal points within the window, and the window size is 10, ratio = 2 / 10 = 0.2. In specific implementation, the weight function of dynamic weight allocation mechanism uses Sigmoid function to map consistency score to weight value, for example, consistency score = 0.8, Sigmoid(0.8) = 1 / (1+e^-0.8) ≈ 0.689. In specific implementation, the weight normalization process sums the weights of all devices and divides the sum to obtain normalized weights, for example, the weights of three devices are 0.4, 0.6, and 1.0, the sum is 2.0, and after normalization, the weights are 0.2, 0.3, and 0.5.

[0084] In specific implementation, the query interface of the historical model parameter library supports filtering parameter snapshots by time range, and the time range is set to data within about one hour corresponding to the last five rounds of iterations depending on the training frequency. In specific implementation, the difference comparison operation parallelizes processing each element of the parameter matrix to accelerate calculation using a multi-core CPU, and the comparison result is aggregated as a scalar change or retained in vector form. In specific implementation, the window size of the sliding window algorithm is determined based on experimental data, and a too small window size leads to noise sensitivity and a too large window size delays trend detection. In specific implementation, the confidence evaluation of the trend detection algorithm is realized by calculating the R-square value of the regression model, and an R-square value higher than 0.7 is considered to be a significant trend, for example, the R-square of the actual value sequence [1, 2, 3] and the predicted value sequence [1.1, 2.0, 2.9] is approximately 0.99. In specific implementation, the abnormal fluctuation point is marked to trigger an alarm mechanism to record the device ID and iteration time for subsequent analysis. In specific implementation, the structure of the change trend report includes an abstract part and a detailed data part, the abstract part is for quick decision-making, and the detailed part is for in-depth diagnosis. In specific implementation, the dynamic weight allocation mechanism is implemented and integrated into the gradient aggregation server, and the weight value is calculated and applied in real time without storing the intermediate state.

[0085] In specific implementation, the statistical moments of the standardized gradient update include mean and variance to generate a distribution summary, which is the first step. The standardized gradient update is derived from the normalized processing output of the local model gradient update. The statistical moment calculation is performed separately for each dimension of the gradient vector. The mean value is calculated by summing all dimension values and dividing by the number of dimensions to obtain the arithmetic mean. For example, the mean of the gradient vector [0.1, 0.2, 0.3, 0.4] is (0.1+0.2+0.3+0.4) / 4=0.25. The variance calculation measures the dispersion of gradient values around the mean. The variance of the gradient vector [0.1, 0.2, 0.3, 0.4] is ((0.1-0.25) 2 +(0.2-0.25) 2 +(0.3-0.25) 2 +(0.4-0.25) 2 ) / 4≈0.0125. The distribution summary is organized in a matrix form containing the mean and variance values of each dimension. In specific implementation, the Isolation Forest algorithm is applied to the distribution summary for anomaly scoring to identify outliers. The Isolation Forest algorithm constructs multiple isolation trees, each tree randomly selects features and split values. In the training phase, the distribution summary data is recursively split until the data points are isolated. In the scoring phase, the path length of each data point is calculated. The shorter the path, the higher the likelihood of abnormality. The anomaly scoring formula is represented as:

[0086]

[0087] where: is the anomaly score of data point o, E(h(o)) represents the path length expectation value of data point o in all isolation trees, c(N) represents the normalization factor for given sample size N, abnormal score close to 1 indicates high abnormality, normal score close to 0 indicates normal. For example, path length expectation value E(h(o)) = 5 for data point o, sample size N = 100, normalization factor c(N) = log2(100) ≈ 6.64, abnormal score = 2^(-5 / 6.64) ≈ 0.65.

[0088] In implementation, device hardware configuration information such as processor type and memory size is used to cluster gradient updates to identify device group-specific patterns, device hardware configuration data is collected through system calls including CPU core number, GPU floating point performance indicator and memory capacity, and categorical variables such as processor type are encoded as numerical features in data preprocessing. In implementation, K-means algorithm is used to cluster hardware configuration data to generate device groups, K-means algorithm randomly initializes k cluster centers, iteratively assigns data points to the nearest center and updates the center position, and the number of clusters k is determined by the elbow rule, where the elbow point corresponds to the optimal number of clusters. For example, when the number of clusters k = 2, the sum of squared errors (SSE) = 100, when k = 3, SSE = 50, and when k = 4, SSE = 45, the elbow point is at k = 3. In implementation, common features of gradient updates are analyzed for each device group to extract group typical patterns, common feature analysis calculates the mean vector and covariance matrix of gradient updates within the group, and typical patterns are identified by pattern matching algorithm to repeatedly appear gradient distribution shape. Comparing typical patterns of different groups identifies device dependency differences, and Mahalanobis distance is used to measure the difference considering the covariance structure, for example, group A mean vector μ_A = [0.2, 0.3], group B mean vector μ_B = [0.5, 0.6], covariance matrix Σ = [[0.1, 0], [0, 0.1]], Mahalanobis distance = √((0.3) 2 / 0.1 + (0.3) 2 / 0.1) = √(1.8) ≈ 1.3416; a large distance value indicates significant difference in gradient behavior between groups.

[0089] In some embodiments, the statistical moment calculation handles unbalanced data with a weighted average method, with weights giving higher importance to recent data based on timestamps of gradient updates, e.g. three data points with timestamps weights of 0.1, 0.3, 0.6, corresponding to gradient values of 0.2, 0.4, 0.6, the weighted average = (0.2*0.1 + 0.4*0.3 + 0.6*0.6) / (0.1+0.3+0.6) = 0.5. In some embodiments, the isolation forest algorithm limits the depth of isolation trees to a logarithmic level to prevent overfitting, such as a depth limit of log2(256) = 8 levels when the sample size is 256; the number of trees is set to 100 to balance computational efficiency and accuracy. Optionally, the hardware configuration data clustering can integrate a hierarchical clustering method as a supplement to K-means, generating a tree diagram to visualize device relationships. The typical pattern extraction can use principal component analysis for dimensionality reduction to identify the dominant pattern direction and simplify the comparison process. It can be understood that the distribution feature analysis relies on high-quality standardized processing of gradient updates, and the effectiveness of the anomaly score requires a sufficient number of data points to support statistical significance. The device clustering result can be dynamically updated as the hardware configuration changes, and periodic re-clustering is suitable for adapting to new devices joining.

[0090] In specific implementation, mean calculation uses incremental update algorithm to adapt to streaming data scenario, new gradient update arrives to update mean without recalculation of all data, for example, current mean μ = 0.2 (based on 10 data), new data x = 0.3, new mean μ_new = (μ * 10 + x) / (10 + 1) = 2.3 / 11 ≈ 0.209; variance calculation uses Welford algorithm to calculate online to avoid numerical instability, for example, existing mean μ = 2, variance M = 5, n = 3, new data x = 4, new μ = (2 * 3 + 4) / 4 = 2.5, new M = 5 + (4 - 2.5) * (4 - 2.5) * 3 / 4 = 6.6875, new variance = 6.6875 / (4 - 1) ≈ 2.229. The encoding processing of hardware configuration data maps the processor type to an integer index, such as Intel → 0, AMD → 1; the GPU performance index is normalized to the range of 0-1 to eliminate the dimensional influence, for example, the original range of GPU performance is 1000-5000, and a certain GPU is 3000 → normalized value = (3000 - 1000) / (5000 - 1000) = 0.5. In specific implementation, the distance metric of K-means algorithm uses Euclidean distance, such as the distance from data point (1, 2) to center (3, 4) = √((2)^2 + (2)^2) = √8 ≈ 2.828; the cluster center is updated to the arithmetic mean of the data points in the cluster, and the iteration stopping condition is that the center change is less than a threshold. In specific implementation, the time complexity of common feature analysis is reduced by sampling, and a part of gradient update calculation is randomly selected to represent the mode. In specific implementation, Mahalanobis distance calculation needs to calculate the inverse covariance matrix, and pseudo-inverse is used to handle singular cases to ensure numerical stability, for example, covariance matrix [[1, 0], [0, 0]] (singular), its pseudo-inverse is [[1, 0], [0, 0]].

[0091] In specific implementation, the input data for statistical moment calculation is the standardized gradient update matrix, with matrix rows corresponding to different training iterations and matrix columns corresponding to model parameter dimensions, for example, the matrix is [[0.1, 0.2], [0.3, 0.4], [0.5, 0.6]] (3 rows of iterations, 2 columns of parameters); the mean vector is generated by summing and averaging in the column direction, the first column mean = (0.1 + 0.3 + 0.5) / 3 = 0.3, the second column mean = (0.2 + 0.4 + 0.6) / 3 = 0.4 → mean vector [0.3, 0.4]; the variance vector is generated by calculating the squared deviation sum in the column direction, the first column variance ≈ ((0.1 - 0.3) 2 + (0.3 - 0.3) 2 + (0.5 - 0.3) 2) / 3 ≈ 0.0267. In specific implementation, the sampling ratio of the training data of the Isolation Forest algorithm is set to 256 samples each time, and the path length expectation value is obtained by averaging the path lengths of all trees in the forest. Hardware configuration data collection is realized through a device agent program, which calls system API to read hardware specifications and encrypts transmission to the analysis module. In specific implementation, the K-means algorithm uses the K-means++ method for initialization to improve convergence speed, such as first randomly selecting the first center, then selecting the point farthest from it as the second center; cluster evaluation uses the silhouette coefficient to measure the intra-cluster tightness and inter-cluster separation, such as a data point with a silhouette coefficient of 0.8 indicating high tightness and separation. The mean vector calculation for typical pattern extraction uses weighted average, with weights based on the confidence of gradient update, and the covariance matrix calculation uses an unbiased estimator with N-1 in the denominator. The significance test of device dependency difference uses t-test to compare the gradient means of different groups, for example, group A mean 0.5, group B mean 0.7, t value = 2.5, p value = 0.01 < 0.05 → the difference is statistically significant.

[0092] In a specific implementation, loading global model parameters into local memory of terminal device and performing forward propagation of deep learning model are initialization steps, global model parameters are distributed from cloud in binary format containing weight and bias parameters, loading process efficiently reads parameter file through memory mapping technique to reduce I / O delay, forward propagation performs computational graph operations of convolutional neural network or recurrent neural network, input pre-processed local dataset including image frames and sensor readings computes activation values layer by layer. In a specific implementation, input pre-processed local dataset obtains prediction output tensor, local dataset is divided into batches with batch size dynamically adjusted according to device memory, usually set to 32 or 64, prediction output tensor is a multi-dimensional array representing inference results of model on input data such as class probability or regression value, tensor data is retained in device memory for loss calculation. In a specific implementation, prediction output tensor is compared with true label and loss value is calculated using cross-entropy loss function, true label comes from annotation data of AR scene corresponding to input data, cross-entropy loss function calculates difference between predicted probability distribution and true distribution, loss value calculation formula is represented as:

[0093]

[0094] wherein: represents batch average loss value, represents batch size, represents number of categories, represents true label value of i-th sample j-th category, represents predicted probability value of i-th sample j-th category. In a specific implementation, loss value is smoothed to reduce noise effect to generate optimized loss value, smoothing applies exponential moving average algorithm to weighted average loss value sequence, exponential moving average algorithm uses smoothing factor to control weight proportion of historical value and current value. Referring to Table 1, key parameters of loss value smoothing.

[0095] Table 1: Key parameter table of loss value smoothing

[0096]

[0097] In specific implementation, the loss value sequence is smoothed by applying exponential moving average algorithm to obtain smoothed loss, the exponential moving average algorithm iteratively updates state variable each time combined with new loss value and historical state, and the smoothing factor is set to 0.9 to balance noise suppression and response speed. In specific implementation, the first derivative of the smoothed loss is calculated to evaluate the loss change rate, the first derivative is calculated by central difference method using the difference between the current smoothed loss and the previous smoothed loss divided by the time step, and the change rate value indicates whether the training process is improved or deteriorated. In specific implementation, the learning rate parameter is adjusted based on the change rate to adaptively control the training process, and the learning rate adjustment adopts rules that increase the learning rate to accelerate convergence when the change rate is negative and the absolute value is large, and reduce the learning rate to prevent oscillation when the change rate is close to zero. In specific implementation, the optimized loss value is output for gradient calculation, and the optimized loss value is used as the input of back propagation to drive model parameter update.

[0098] In specific implementation, the global model parameter is loaded to verify the checksum and ensure data integrity, and the checksum mismatch triggers the re-download process. In specific implementation, the device resource usage is monitored during forward propagation execution, and the batch size is reduced or the swap space is enabled when the memory is insufficient. In specific implementation, the Softmax function is applied to the predicted output tensor for post-processing to normalize the probability value and ensure that the output meets the probability distribution requirements. In specific implementation, a small epsilon value is added to the cross-entropy loss calculation to prevent numerical errors caused by the input of the logarithmic function being zero. In specific implementation, the state initialization of the exponential moving average algorithm uses the arithmetic mean of the first few loss values to reduce the initial deviation. In specific implementation, the first derivative calculation uses bidirectional difference to improve accuracy while considering the loss values at the previous and next time points. In specific implementation, the learning rate adjustment sets upper and lower limits to prevent the learning rate from being too large or too small to affect convergence, and the upper limit is usually 10 times the initial learning rate and the lower limit is 0.0001. In specific implementation, the optimized loss value is recorded to the training log for subsequent analysis and visualization.

[0099] The local dataset loading uses pipeline technology to overlap data loading and model calculation, and the next batch of data is prefetched while the current batch is processed. In specific implementation, the predicted output tensor is compressed to reduce memory usage, and half-precision floating-point numbers or quantization representation are used. In specific implementation, the loss value calculation is parallelized to process batch samples, and multi-threading is used to accelerate the summation operation. In specific implementation, the smoothing factor of the exponential moving average algorithm is dynamically adjusted based on the loss variance, and the smoothing strength is increased when the variance is high. In specific implementation, the change rate calculation filters high-frequency fluctuations through a low-pass filter, and the filter cutoff frequency is adjusted according to the training stage. In specific implementation, the learning rate adjustment rule integrates a warm-up phase, and the learning rate is linearly increased in the initial iterations to avoid instability. In specific implementation, the optimized loss value is verified by comparing the original loss to check the smoothing effect, and a warning is triggered when the difference is too large.

[0100] ReferenceFigure 4 The loss values of the five types of terminal devices, high-end mobile phones, mid-end mobile phones, tablet devices, AR glasses, and low-end mobile phones, after optimization, are presented in relation to the number of training iterations. Specifically, the loss values of each device overall decrease with the increase in the number of iterations, reflecting the convergence process of model training: In the initial stage (iteration number 0-20), the loss value fluctuates greatly and is at a relatively high level, reflecting the unstable state of preliminary adjustment of model parameters; as the iteration progresses (20-100), the loss value gradually decreases and tends to be stable, indicating that the fitting effect of the model on local data is continuously optimized. The loss value changes differ among different devices: The loss value of high-end devices (such as high-end mobile phones and AR glasses) decreases at a relatively faster rate and fluctuates less in the later stage, while the loss value of low-end mobile phones fluctuates relatively significantly, which is related to the influence of device hardware configuration (such as processor performance and memory capacity) on model training efficiency. At the parameter level, the training process corresponds to the loss value calculation and smoothing processing (such as the exponential moving average algorithm) in the forward calculation of the local model. The smoothed fluctuations of the loss value in the figure can reflect the actual effect of the loss value optimization strategy, providing a performance basis for subsequent gradient update and cloud model aggregation.

[0101] In a specific implementation, using the gradient fusion result to update the global model parameters to generate a new version of the model is the initial step, and the gradient fusion result comes from the output of the cloud dynamic adjustment module, which contains the weighted average gradient vector. The update process applies the stochastic gradient descent algorithm or its variants such as the Adam optimizer, and the learning rate is dynamically adjusted according to the training progress. The new version of the model parameter is stored as a floating-point matrix to prepare for subsequent simplification processing. In a specific implementation, the model pruning algorithm is applied to remove redundant connections and neurons to reduce the number of parameters. The model pruning algorithm is based on the weight amplitude importance score, which is determined by calculating the absolute value of the weight. The threshold is dynamically adjusted by the change in the validation set accuracy. The pruned model is retrained for a small number of iterations to restore performance and prevent accuracy from decreasing. Quantization processing is performed on the pruned model to convert floating-point parameters to low-precision fixed-point representation. The quantization processing uses a uniform quantization scheme to map floating-point values to the integer domain. The quantization formula is:

[0102]

[0103] where: represents the quantized parameter value, represents the original floating-point parameter value, represents the quantization step. The quantization step is calculated according to the parameter range, and the dynamic range is determined by the minimum and maximum values of the calibration data set.

[0104] In some embodiments, the model pruning algorithm employs a structured pruning method to remove entire convolutional channels or fully connected layer nodes, reducing model complexity while maintaining hardware compatibility. In some embodiments, the quantization process integrates training-aware quantization techniques, fine-tuning parameters during quantization to compensate for precision loss and improve model robustness. Optionally, pruning threshold selection can use automated search algorithms such as grid search or Bayesian optimization to find the optimal pruning rate that balances model size and performance. The quantization scheme can choose a non-uniform quantization method that better matches the parameter distribution to improve quantization efficiency. Model simplification processing requires a trade-off between computational resource savings and inference accuracy maintenance, and excessive pruning or quantization can cause the model to fail. It can be understood that the deployment of a lightweight inference model needs to consider the hardware differences of terminal devices and dynamically adjust the inference parameters to adapt to different performance constraints.

[0105] In specific implementations, the gradient fusion result updates the global model parameters using momentum to accelerate convergence, with a momentum coefficient set to 0.9 to reduce update oscillation, and the parameter update formula combines first and second moment estimates of the gradient. The redundancy connection identification of the model pruning algorithm is based on correlation analysis, calculating the covariance matrix of neuron activation values to identify redundant dimensions, and the pruning operation removes connections with low contribution. The calibration dataset for quantization processing is randomly sampled from the training set to represent the parameter distribution, and the calibration process determines the quantization step size by calculating the parameter histogram, with the quantization step size calculation formula using the maximum-minimum value difference divided by the number of quantization levels. The test dataset for verifying inference accuracy includes various AR scenarios such as image recognition, pose estimation, and scene understanding tasks, and the test environment simulates terminal device running conditions including changes in lighting and motion blur.

[0106] The new version of the model is immediately subjected to integrity checks, calculating the model output compared to the benchmark to ensure the correctness of the update. The pruning algorithm iteratively performs multiple steps of removing parameters, with performance changes evaluated after each pruning. The iteration stopping condition is a decrease in accuracy exceeding a pre-set threshold. The quantized model applies weight sharing techniques to further compress the model size, quantizing similar weights to the same value to reduce storage overhead. The verification process uses cross-validation techniques to split the test data and evaluate multiple times to ensure the stability of the results.

[0107] Referring to Figure 5, the original floating point (32bit) and four types of quantization scheme under the model relative index change characteristics. Specifically, in the original floating point scheme, the relative model size, accuracy, and inference delay are maintained at 100% of the benchmark level; uniform quantization (16bit) reduces the model size to about 50%, the inference delay to about 65%, and the relative accuracy remains 100%; the relative accuracy of the three types of 8bit quantization schemes (uniform quantization, pair quantization, and training-aware quantization) is close to 100%, but the model size and inference delay show differential optimization effects: the relative model size of uniform quantization (8bit) is compressed to about 25%, and the inference delay is reduced to about 38%; the relative model size of pair quantization (8bit) and training-aware quantization (8bit) is about 22% and 24% respectively, and the inference delay is optimized to about 42% and 40% respectively. The index changes reflect the performance trade-off logic of the quantization scheme: low-bit-width quantization (such as 8bit) can significantly compress the model size and reduce the inference delay, but needs to compensate for the potential loss of accuracy through training-aware techniques; uniform quantization (16bit) achieves a relatively balanced compromise between model compression and performance maintenance. In the parameter dimension, the configuration differences of dynamic range, quantization step, etc. of different quantization schemes directly affect the optimization amplitude of model size and inference efficiency, and training-aware quantization achieves a better balance of model size, accuracy, and inference delay at 8bit precision through parameter fine-tuning during the quantization process.

[0108] It should be noted that the relational terms herein such as first and second and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any such actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0109] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and changes can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for training and inference of dynamic deep learning models for AR scenes based on federated learning, characterized in that, The method includes: Initialize the cloud-based federated learning environment, configure global model parameters, and distribute them to multiple terminal devices; The terminal device collects local AR scene data, including image frames and sensor readings, performs timestamp alignment and resolution standardization on the collected data, and generates a pre-processed local dataset. The model forward computation is performed using the preprocessed local dataset and global model parameters to obtain the predicted output, and the loss value is calculated based on the predicted output and the true label. The local model gradient update is derived from the loss value, and the local model gradient update is normalized to generate a standardized gradient update. Analyze the distribution characteristics in standardized gradient updates, and identify bias patterns and device-specific patterns in gradient updates, including: Calculate the statistical moments of the standardized gradient update, including the mean and variance, and generate a distribution summary; The isolated forest algorithm is used to score anomalies in the distribution profile and identify outliers. Based on device hardware configuration information, including processor type and memory size, clustering gradient updates are performed to identify typical patterns in device groups. This includes: collecting hardware configuration data of terminal devices, including the number of CPU cores and GPU performance metrics; using the K-means algorithm to cluster the hardware configuration data to generate device groups; and for each device group, analyzing the common features of its gradient updates and extracting typical patterns of the group. Compare typical patterns of different groups to identify differences in device dependence; fuse anomaly scores and device group information to output deviation patterns and device-specific patterns; Encryption transmission protocols are applied to the identified deviation patterns and device-specific patterns to encrypt and protect the data integrity, generating secure gradient data. The cloud receives security gradient data and combines it with historical parameter snapshots from the historical model parameter library to build a dynamic adjustment module for evaluating the consistency of model updates. Based on the output of the dynamic adjustment module, the direction of model optimization is determined, and multi-source gradient data are fused. The global model parameters are updated based on the fusion results, and the updated model is structurally simplified, including parameter pruning and precision quantization, to generate a lightweight inference model.

2. The method for training and inference of dynamic deep learning models for AR scenes based on federated learning according to claim 1, characterized in that, The process involves applying an encrypted transmission protocol to the identified deviation patterns and device-specific patterns to encrypt and protect the data's integrity, generating secure gradient data, including: Key feature vectors are extracted from deviation patterns and device-specific patterns, and dimensionality reduction is performed using a feature selection algorithm to obtain a simplified feature set. A symmetric encryption algorithm is applied to the simplified feature set, and the initial encrypted data is generated using a dynamically generated key. Verify the integrity and authenticity of the initial encrypted data, calculate the data digest using a hash function, and attach a digital signature; Based on the verification results, the initial encrypted data is optimized by segmenting and transmitting it in blocks, adjusting the data packet size to adapt to network conditions, and finally generating security gradient data.

3. The method for training and inference of dynamic deep learning models for AR scenes based on federated learning according to claim 2, characterized in that, The verification of the integrity and authenticity of the initial encrypted data, including calculating a data digest using a hash function and attaching a digital signature, includes: Calculate the SHA-256 hash value of the initial encrypted data to obtain the data digest; A digital signature is generated by signing the data digest using the private key of the terminal device. The digital signature is bound to the initial encrypted data to form a signed data packet; The cloud uses the public key of the terminal device to verify the digital signature, confirm the integrity and origin of the data, and discards the data packet if the verification fails.

4. The method for training and inference of dynamic deep learning models for AR scenes based on federated learning according to claim 1, characterized in that, The cloud receives security gradient data and, combined with historical parameter snapshots from the historical model parameter library, constructs a dynamic adjustment module to evaluate the consistency of model updates, including: Load the model parameters from the most recent iterations from the historical model parameter library as a reference parameter set; After decrypting the security gradient data, compare the differences with the reference parameter set and calculate the parameter changes; Based on the parameter changes, the sliding window algorithm is used to analyze the changing trends and identify long-term consistency and short-term fluctuations. Based on the analysis results, a dynamic weight allocation mechanism is constructed to adjust the contribution of different terminal device gradients in the aggregation.

5. The method for training and inference of dynamic deep learning models for AR scenes based on federated learning according to claim 4, characterized in that, The method of analyzing trends based on parameter changes using a sliding window algorithm to identify long-term consistency and short-term fluctuations includes: Define the time window size and extract the parameter change sequence within the window; Calculate the moving average and standard deviation of the series to assess the stability of changes; The algorithm identifies upward or downward trends and marks abnormal fluctuation points. A trend report is generated by combining long-term consistency scores and short-term volatility indices.

6. The method for training and inference of dynamic deep learning models for AR scenes based on federated learning according to claim 1, characterized in that, The step of using the preprocessed local dataset and global model parameters to perform forward computation of the model to obtain the predicted output, and calculating the loss value based on the predicted output and the true labels, includes: Load global model parameters into the local memory of the terminal device and perform forward propagation of the deep learning model; Input the preprocessed local dataset to obtain the predicted output tensor; Compare the predicted output tensor with the true label, and calculate the loss value using the cross-entropy loss function; The loss value is smoothed to reduce the impact of noise and generate an optimized loss value.

7. The method for training and inference of dynamic deep learning models for AR scenes based on federated learning according to claim 6, characterized in that, The process of smoothing the loss value to reduce the impact of noise and generate an optimized loss value includes: The loss value sequence is smoothed by applying the exponential moving average algorithm to obtain the smoothed loss; Calculate the first derivative of the smoothed loss and evaluate the rate of change of the loss. The training process is adaptively controlled by adjusting the learning rate parameter based on the rate of change. The optimized loss value is used for gradient calculation.

8. The method for training and inference of dynamic deep learning models for AR scenes based on federated learning according to claim 1, characterized in that, The process of updating the global model parameters based on the fusion results and simplifying the updated model structure, including parameter pruning and precision quantization, to generate a lightweight inference model includes: Update the global model parameters using the gradient fusion results to generate a new version of the model; The model pruning algorithm is applied to remove redundant connections and neurons, thereby reducing the number of parameters; The pruned model is quantized to convert floating-point parameters into low-precision fixed-point representations; Verify the inference accuracy of the simplified model in AR scenarios and output the final lightweight inference model.

Images