Deep learning optimization method based on dynamic domain adaptive adjustment

Through dynamic domain-aware initialization and cyclic domain adjustment, combined with gradient perturbation and dual feedback mechanisms, the generalization performance degradation of deep learning models in cross-domain applications is solved, achieving higher adaptability and efficiency.

CN118917377AInactive Publication Date: 2024-11-08ABD SMART EYE ELECTRONICS CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410949484.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Deep learning models have deteriorated generalization performance when applied across domains, making it difficult to maintain good predictive capabilities on new domain data that have not been seen, especially when the data source has different distribution characteristics.

Method used

Dynamic domain-aware initialization, gradient perturbation-based parameter exploration, dual feedback mechanism and domain adaptability regularization are adopted. Through circular domain adjustment and performance evaluation, model parameters are adjusted in real time to adapt to new domain data and reduce dependence on additional target domain data.

Benefits of technology

It significantly improves the generalization ability and adaptability of the model, improves performance in a diversified environment, reduces computing costs and resource consumption, and achieves faster model optimization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118917377A_ABST
    Figure CN118917377A_ABST
Patent Text Reader

Abstract

The invention relates to a deep learning optimization method based on dynamic domain adaptive adjustment. The deep learning optimization method comprises the following steps of performing dynamic domain perception initialization; performing parameter exploration based on gradient disturbance; a dual feedback mechanism; carrying out domain adaptive regularization; adjusting the cyclic domain and evaluating the performance of the model, and if the performance meets the requirement, ending; and if not, continuing to adjust the cyclic domain. According to the method, through dynamic domain perception initialization and parameter exploration based on gradient disturbance, the model can better understand and adapt to different data distributions at the initial stage of training, so that good prediction performance can be kept without data; by introducing a domain adaptive regularization and cyclic domain adjustment strategy, the expression difference between different domains of the model can be remarkably reduced, and the model can be ensured to achieve the optimal performance in various domains; and the dual feedback mechanism allows the model to be self-adjusted according to the real-time effect of the parameters in the training process, so that the training process is more intelligent and efficient, and the resource consumption and the training time are reduced while the model performance is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and machine learning technology, and in particular to a deep learning optimization method based on dynamic domain adaptive regulation. Background Art

[0002] In current deep learning applications, the generalization ability and domain adaptability of models have become key research focuses. With the widespread application of artificial intelligence technology, from medical diagnosis to autonomous driving, from natural language processing to image recognition, models need to show excellent adaptability and stability in a changing and complex data environment. Although deep learning technology has demonstrated strong performance in many fields, it often encounters the problem of degraded generalization performance when applied across domains, that is, the model performs well on the training set, but performs poorly on unseen new domain data.

[0003] The core of the generalization problem is how to make the model maintain good prediction ability when facing new and untrained data. The domain adaptability problem further challenges the model's ability to adapt between different data sources, especially when these data sources have different distribution characteristics. Traditional solutions include but are not limited to the following:

[0004] Data augmentation: Expanding the training set through technical means, such as rotating, flipping, scaling, and other image operations, or expanding the corpus through synonymous substitution of text. Although these methods can improve the robustness of the model to a certain extent, they are still limited by the original distribution of the data and it is difficult to fully cover potential application scenarios.

[0005] Transfer learning: fine-tuning a model trained in one domain to adapt to a new domain. This method relies on the similarity between the source domain and the target domain, but when the two domains are very different, the adaptability of the model will be greatly reduced.

[0006] Adversarial training: By training the model to identify the difference between real samples and generated samples, the model's robustness to input data perturbations is enhanced. This method effectively improves the generalization ability of the model, but the computational cost is high and it may still fail under specific adversarial attacks.

[0007] Domain adaptation technology: attempts to narrow the distribution difference between training data and target data, such as adjusting the model through domain adaptation layers or statistical feature matching to make it better adapt to the new domain. Usually, by adding specific adaptation layers to the model or using special loss functions, the model can establish effective knowledge transfer between the source domain and the target domain. However, these methods often require explicit definition of the source domain and the target domain and rely on sufficient target domain data for training. Summary of the invention

[0008] In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide a deep learning optimization method based on dynamic domain adaptive adjustment to solve one or more problems in the prior art.

[0009] To achieve the above object, the technical solution of the present invention is as follows:

[0010] A deep learning optimization method based on dynamic domain adaptive adjustment includes the following steps:

[0011] Dynamic domain-aware initialization;

[0012] Parameter exploration based on gradient perturbations;

[0013] Double feedback mechanism;

[0014] Domain-adaptive regularization;

[0015] The loop domain is adjusted and the model performance is evaluated. If the performance meets the requirements, the process ends; if not, the loop domain adjustment continues.

[0016] Furthermore, the dynamic domain awareness initialization includes the following steps:

[0017] Adjust weights to suit data characteristics;

[0018] Adjust the model data bias term.

[0019] Furthermore, the calculation formula for adjusting the weight to adapt to the data characteristics is as follows:

[0020] W=alpha*((mu / sigma)+beta*(skew / kurt))

[0021] Where: W is the parameter, mu is the mean of data X, sigma is the variance of data X, skew is the skewness of the data, kurt is the kurtosis of the data, alpha and beta are adjustment factors.

[0022] Furthermore, the calculation formula for adjusting the model data bias term is as follows:

[0023] b=-gamma*log(sigma / mu)

[0024] Where: b is the bias, sigma is the variance of data X, mu is the mean of data X, and gamma is the adjustment factor.

[0025] Furthermore, the parameter exploration calculation formula based on gradient perturbation is shown as follows:

[0026] Delta W=eta*dL / dW+epsilon*sqrt(var(dL / dW))*sign(dL / dW)

[0027] Where: Delta W is the update amount of parameter W, dL / dW is the gradient of the loss function L with respect to parameter W, eta is the learning rate, epsilon is the perturbation intensity adjustment coefficient, var(dL / dW) is the variance of the gradient, sqrt(var(dL / dW)) is the square root of the gradient variance, and sign(dL / dW) is the sign function of the gradient.

[0028] Furthermore, the calculation formula of the dual feedback mechanism is as follows:

[0029] Lc=L(W+Delta W)-L(W)+rho*norm(Delta W)

[0030] Where: Lc is the contrast loss, L(W) is the loss value of the model under the current parameter W, Delta W is the change in parameter W caused by this update, L(W+Delta W) is the loss value of the model after updating the parameters, rho is the adjustment factor, and norm(Delta W) is the norm function.

[0031] Furthermore, the domain adaptive regularization calculation formula is as follows:

[0032] R = lambda * sum ((Li - Lavg) 2 / var(L))

[0033] Where: Li represents the loss of the model on the i-th data domain, Lavg is the average of the losses on all considered domains, var(L) is the variance of the loss, and lambda is the regularization coefficient, which is a hyperparameter.

[0034] Furthermore, the cycle domain adjustment calculation formula is as follows:

[0035] Wi+1=Wi-eta*dL / dWi-kappa*Adjust(Domain Performance)

[0036] Where Wi is the parameter of the model in the current iteration, Wi+1 is the parameter of the model in the next iteration, eta is the learning rate, dL / dWi is the gradient of the loss function L with respect to the current parameter Wi, kappa is the adjustment coefficient, which is a hyperparameter, and Adjust(Domain Performance) is a function that is dynamically adjusted according to the performance evaluation results of each domain.

[0037] Compared with the prior art, the beneficial technical effects of the present invention are as follows:

[0038] (i) The present invention provides a more dynamic and adaptive solution through dynamic domain-aware initialization and cyclic domain adjustment strategies. Compared with traditional technologies that often rely on pre-defined domain knowledge or require a large amount of target domain data for fine-tuning, the present invention does not require explicit definition of the source domain and target domain, nor does it rely on additional target domain data. By considering the statistical characteristics of the data at each training stage and adjusting the training strategy in real time, the present invention enables the model to automatically adapt to the new domain without additional data support, providing higher flexibility and adaptability.

[0039] (ii) The present invention significantly improves the generalization ability of the model by combining parameter exploration based on gradient perturbation with domain adaptive regularization. Compared with the data enhancement or adversarial training methods commonly used in the prior art, although data enhancement can expand the training set, it is still limited by the distribution of the original data; although adversarial training can improve robustness, it has a high computational cost and may not be able to solve the adaptation problem of all new scenarios. The present invention ensures that the model can maintain good performance in diverse environments, especially when facing unknown data, by intelligently adjusting the parameter update strategy in the learning process.

[0040] (III) Compared with the prior art, the dual feedback mechanism and loop domain adjustment strategy introduced in the present invention increase the intelligence level of model training. These innovations make model training no longer a static, one-time process, but a process of dynamic adjustment and continuous optimization. Model training in the prior art usually needs to be evaluated after training is completed, and may require multiple iterative adjustments to achieve the ideal state; the present invention greatly reduces the need for repeated adjustments through real-time evaluation and adjustment during the training process, improves training efficiency, reduces resource consumption, and enables the model to achieve better performance in a shorter time. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A structural schematic diagram of a deep learning optimization method based on dynamic domain adaptive adjustment according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention clearer, the following is a further detailed description of a deep learning optimization method based on dynamic domain adaptive adjustment proposed by the present invention in combination with the accompanying drawings and specific embodiments. According to the following description, the advantages and features of the present invention will be clearer. It should be noted that the drawings are in a very simplified form and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, please refer to the accompanying drawings. It should be noted that the structure, proportion, size, etc. illustrated in the drawings of this specification are only used to match the content disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the limiting conditions for the implementation of the present invention, so it has no technical substantive significance. Any modification of the structure, change in the proportional relationship or adjustment of the size, without affecting the effect that the present invention can produce and the purpose that can be achieved, should still fall within the scope of the technical content disclosed in the present invention.

[0043] A deep learning optimization method based on dynamic domain adaptive adjustment includes the following steps:

[0044] Step 1: Dynamic domain-aware initialization:

[0045] The statistical characteristics of the input data, such as mean, variance, skewness, and kurtosis, are used to optimize the initial parameter configuration of the model so that the model can better adapt to the characteristics of the specific data domain.

[0046] Step 1.1: Adjust weights to suit data characteristics:

[0047] The calculation formula for adjusting the weight to adapt to the data characteristics is shown in Formula 1:

[0048] W = alpha * ((mu / sigma) + beta * (skew / kurt)) (1)

[0049] Where:

[0050] W is a parameter.

[0051] mu is the mean of data X, which represents the average level of data points and is used to capture the central trend of the data.

[0052] Sigma is the variance of data X, which indicates the dispersion of data points around the mean.

[0053] Skew is the skewness of the data, which describes the asymmetry of the data distribution. Positive or negative skewness will affect the model's ability to process data.

[0054] Kurt is the kurtosis of the data, which measures the sharpness of the data distribution. A high kurtosis indicates that there are many extreme values ​​in the data.

[0055] Alpha and beta are adjustment factors, which are used to adjust the influence of the mean-to-variance ratio and the skewness-to-kurtosis ratio, so that the model can flexibly adapt to different data characteristics.

[0056] Formula 1 above adjusts the weights by combining the mean-to-variance ratio and the skewness-to-kurtosis ratio of the data. This combination ensures that the weights are adapted to the center position, dispersion, deviation from symmetry, and kurtosis of the data.

[0057] Step 1.2: Adjust the model data bias term:

[0058] Furthermore, the calculation formula for adjusting the model data bias term is shown in Formula 2 below:

[0059] b = -gamma * log(sigma / mu) (2)

[0060] Where:

[0061] b is the bias.

[0062] Sigma is the variance of data X, which indicates the dispersion of data points around the mean.

[0063] mu is the mean of data X, which indicates the average level of data points and is used to capture the central trend of the data.

[0064] Gamma is a regulation factor that controls the sensitivity of bias initialization.

[0065] Equation 2 above uses a logarithmic function to adjust the bias b, which is based on the ratio of the variance to the mean, and helps adjust the bias term of the model to better adapt to the scale and position offset of the data.

[0066] Through dynamic domain-aware initialization, the model parameters are initialized using the statistical characteristics of each language dataset, such as word frequency distribution, average sentence length, etc. This initialization method enables the model to adapt to the basic characteristics of different languages ​​at the beginning of training, providing a good starting point for subsequent deep learning.

[0067] This initialization method takes into account the distribution of data, helping the model to more accurately reflect and adapt to the real characteristics of the data in the early stages of training, showing better convergence in the early stages of training, and showing better performance in a changing real-world environment. It provides the model with a starting parameter configuration optimized for a specific data domain, effectively improving the model's response speed and accuracy to data features in the early stages of training, and providing the model with a more reasonable starting point, thus laying a solid foundation for the subsequent learning process. It is particularly suitable for deep learning tasks in application scenarios where data sources are diverse and highly dynamic.

[0068] Step 2: Parameter exploration based on gradient perturbation:

[0069] It comes into play after the dynamic domain perception is initialized. By introducing perturbations during the model training process, the model's exploration ability in the parameter space is enhanced, and a wider parameter space is explored, thereby enhancing the model's generalization ability for new and unseen data, avoiding falling into local optimal solutions, and improving the model's adaptability and robustness.

[0070] Furthermore, the parameter exploration calculation formula based on gradient perturbation is shown in the following formula 3:

[0071] Delta W = eta * dL / dW + epsilon * sqrt(var(dL / dW)) * sign(dL / dW)(3)

[0072] Where:

[0073] Delta W is the update amount of parameter W.

[0074] dL / dW is the gradient of the loss function L with respect to the parameter W, indicating the direction of reducing the loss function in the parameter space.

[0075] eta is the learning rate, which controls the magnitude of each step of gradient descent update.

[0076] Epsilon is the perturbation intensity adjustment coefficient, which determines the magnitude of the perturbation, and its value depends on the variability of the gradient.

[0077] var(dL / dW) is the variance of the gradient, which measures the instability of gradient changes in different data batches or different model states.

[0078] sqrt(var(dL / dW)) is the square root of the gradient variance and is used to adjust the scale of the perturbation so that the addition of perturbation is proportional to the instability of the gradient.

[0079] sign(dL / dW) is the sign function of the gradient, which is used to ensure that the direction of the perturbation is consistent with the direction of the gradient to ensure that the perturbation does not deviate from the optimal descent direction indicated by the original gradient.

[0080] In each iteration, in addition to the regular gradient descent update, this step also introduces a perturbation based on the current gradient variability. This approach enhances the model's ability to explore unknown data domains by adding a perturbation term proportional to the gradient variability to the parameter update. Such exploration not only helps the model avoid falling into local optimality, but also increases the model's ability to adapt to new data environments, especially when faced with diverse and dynamically changing data.

[0081] In this way, each update not only considers the current best descent direction, but also dynamically adjusts the size of the perturbation based on the instability of the gradient. This dynamic adjustment makes the model more robust when facing complex data or dealing with changes in data distribution.

[0082] Step 3: Double feedback mechanism:

[0083] The dual feedback mechanism calculates the loss change of the model before and after the parameter update, and evaluates the effect of each parameter update based on the size of the parameter change. The introduced contrast loss helps adjust the training process to ensure that the parameter update is not just based on a single loss reduction, but comprehensively reflects the performance of the model in practical applications. By comparing the changes in loss before and after and the magnitude of parameter changes, the training strategy is adjusted in real time, the model update process is optimized, and the risk of overfitting is effectively reduced, making the model more robust, especially when dealing with complex or noisy data.

[0084] Furthermore, the calculation formula of the dual feedback mechanism is shown in Formula 4 below:

[0085] Lc = L(W + Delta W) - L(W) + rho * norm(Delta W) (4)

[0086] Where:

[0087] Lc is the contrast loss.

[0088] L(W) is the loss value of the model under the current parameter W.

[0089] Delta W is the change in parameter W caused by this update.

[0090] L(W+Delta W) is the loss value of the model after updating the parameters.

[0091] Rho is a regulating factor used to adjust the impact weight of the additional loss caused by the change in parameter size.

[0092] norm(Delta W) is a norm function used to measure the update amplitude of the parameter W. The norm can be L1 norm, L2 norm, etc. The specific choice depends on the requirements and characteristics of the model.

[0093] Formula 4 above evaluates the performance change of the model after the parameter update and the actual size of the parameter change by comparing the loss Lc. This mechanism not only considers the change of the loss function, but also evaluates the magnitude of the parameter update, thereby providing real-time, multi-dimensional feedback information to help adjust subsequent training strategies.

[0094] By applying a dual feedback mechanism, combined with the dual perspectives of loss change and parameter update amplitude, the training process can be adjusted more finely. After each update, not only the degree of reduction of the loss function is examined, but also whether the size of the parameter update is appropriate is evaluated. This method not only enhances the adaptability of the model when facing diverse and complex data, but also helps avoid common pitfalls in the training process, thereby improving the performance and reliability of the model in practical applications.

[0095] Step 4: Domain Adaptive Regularization:

[0096] Based on the dual feedback mechanism, domain adaptive regularization further enhances the generalization ability of the model by reducing the performance differences between different data domains, especially focusing on the consistency of performance in multiple different data domains.

[0097] Furthermore, the domain adaptive regularization calculation formula is shown in Formula 5 below:

[0098] R = lambda * sum ((Li - Lavg) 2 / var(L)) (5)

[0099] Where:

[0100] Li represents the loss of the model on the i-th data domain, which can be any metric that quantifies the error of the model, such as cross entropy loss or mean squared error.

[0101] Lavg is the average of the losses over all considered domains, which provides a benchmark for measuring the deviation of the losses of individual domains from the average level.

[0102] var(L) is the variance of the loss, which indicates the degree of variation in the loss between different domains. This metric helps the model understand in which domains its performance deviates significantly from the average level.

[0103] Lambda is the regularization coefficient, which is a hyperparameter that controls the strength of the regularization term. A higher lambda value emphasizes the importance of maintaining consistent performance across all domains.

[0104] This regularization strategy and dual feedback mechanism complement each other. Regularization focuses on the long-term, global consistency of model performance by adding a regularization term to the loss function to balance the performance of the model in different data domains, while dual feedback focuses on local, immediate update effects. Together, they ensure that the model achieves good generalization and consistency in multiple domains.

[0105] In real-world applications, data often comes from different distributions, which may differ significantly from each other. Conventional training methods may cause the model to perform well on some data domains but poorly on other domains. Domain adaptation regularization forces the model to maintain consistent performance as much as possible across all domains, thereby reducing this performance difference. Through cross-validation and parameter adjustment experiments, the optimal lambda value can be found to ensure that the model can maintain consistency in performance across domains while not overfitting to the specific features of any single domain.

[0106] Step 5: Adjust the loop domain and evaluate the model performance. If the performance meets the requirements, the process ends; if not, continue to adjust the loop domain:

[0107] Loop domain adjustment adjusts the training strategy based on the performance feedback of the model on different data domains, forming a continuous optimization cycle to cope with the changing data environment.

[0108] Furthermore, the cycle domain adjustment calculation formula is shown in the following formula 6:

[0109] Wi+1=Wi - eta * dL / dWi - kappa * Adjust(Domain Performance) (6)

[0110] Where:

[0111] Wi is the parameter of the model in the current iteration.

[0112] Wi+1 is the parameter of the model in the next iteration.

[0113] eta is the learning rate, which controls the amplitude of parameter updates based on gradient descent.

[0114] dL / dWi is the gradient of the loss function L with respect to the current parameter Wi, indicating the direction of reducing the loss in the parameter space.

[0115] Kappa is the adjustment coefficient and is a hyperparameter used to control the intensity of the adjustment based on domain performance feedback.

[0116] Adjust (Domain Performance) is a function that is dynamically adjusted according to the performance evaluation results of each domain. The specific form of this function can be a mathematical model based on performance indicators, such as weighted average, proportional scale adjustment, etc.

[0117] Cyclic domain adjustment is an advanced training strategy that not only optimizes performance on a single domain, but also provides a dynamic and adaptive method to continuously optimize and refine the model to better adapt to the ever-changing and diverse data environment. This method periodically evaluates the performance of the model on different data domains and adjusts the parameter update strategy during training accordingly. The core idea is to guide model learning through detailed performance feedback, so that the model can show good stability and accuracy in a variety of application environments.

[0118] Example 1: Multilingual Text Classification

[0119] In multilingual text classification tasks, the model needs to accurately classify texts in different languages, such as distinguishing between news, blogs, comments, etc. Due to the significant differences in the structure and word usage of texts in different languages, traditional text classification models often perform poorly on cross-language tasks, especially in languages ​​with insufficient training data.

[0120] Step 1: Dynamic domain-aware initialization:

[0121] The model parameters are initialized using the statistical characteristics of each language data set, such as word frequency distribution, average sentence length, etc. This initialization method enables the model to adapt to the basic characteristics of different languages ​​at the beginning of training, providing a good starting point for subsequent deep learning.

[0122] Step 2: Parameter exploration based on gradient perturbation:

[0123] During the training process, gradient perturbations are introduced to dynamically adjust the size and direction of the perturbations according to the language characteristics of the current batch of data. Such measures are intended to explore a wider parameter space and help the model find better parameter settings in multiple language environments.

[0124] Step 3: Double feedback mechanism:

[0125] The model performance is evaluated after each parameter update, and feedback is provided by calculating the performance differences of the model on multilingual datasets. This mechanism ensures that each update is made in the direction of reducing cross-language performance differences.

[0126] Step 4: Domain Adaptive Regularization:

[0127] A regularization term is added to the loss function. This regularization term is calculated based on the performance differences of the model on different language datasets. The purpose is to minimize these differences and promote balanced performance of the model on all languages.

[0128] Step 5: Loop Domain Adjustment:

[0129] The training strategy is adjusted cyclically according to the classification performance of the model on different languages ​​to directly meet the model performance requirements, such as increasing the sample proportion of poorly performing languages, or adjusting the model training parameters for specific languages.

[0130] Through the technical solution of the present invention, the model shows significant performance improvement in cross-language text classification tasks, reduces the performance difference between the model in the training language and unseen languages, and improves the classification accuracy of the model in the case of few language samples, thereby effectively solving the limitations of traditional models in cross-language applications.

[0131] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A deep learning optimization method based on dynamic domain adaptive adjustment, characterized in that The following steps are involved: Dynamic domain-aware initialization; Parameter exploration based on gradient perturbations; Double feedback mechanism; Domain-adaptive regularization; The loop domain is adjusted and the model performance is evaluated. If the performance meets the requirements, the process ends; if not, the loop domain adjustment continues.

2. The deep learning optimization method based on dynamic domain adaptive adjustment according to claim 1, characterized in that: The dynamic domain-aware initialization comprises the following steps: Adjust weights to suit data characteristics; Adjust the model data bias term.

3. A deep learning optimization method based on dynamic domain adaptive adjustment as claimed in claim 2, characterized in that: The calculation formula for adjusting the weight to adapt to data characteristics is as follows: W=alpha*((mu / sigma)+beta*(skew / kurt)) Where: W is the parameter, mu is the mean of data X, sigma is the variance of data X, skew is the skewness of the data, kurt is the kurtosis of the data, alpha and beta are adjustment factors.

4. The deep learning optimization method based on dynamic domain adaptive adjustment according to claim 3, characterized in that: The calculation formula of the adjustment model data bias term is as follows: b=-gamma*log(sigma / mu) Where: b is the bias, sigma is the variance of data X, mu is the mean of data X, and gamma is the adjustment factor.

5. The deep learning optimization method based on dynamic domain adaptive adjustment according to claim 1, characterized in that: The parameter exploration calculation formula based on gradient perturbation is shown as follows: Delta W=eta*dL / dW+epsilon*sqrt(var(dL / dW))*sign(dL / dW) Where: Delta W is the update amount of parameter W, dL / dW is the gradient of the loss function L with respect to parameter W, eta is the learning rate, epsilon is the perturbation intensity adjustment coefficient, var(dL / dW) is the variance of the gradient, sqrt(var(dL / dW)) is the square root of the gradient variance, and sign(dL / dW) is the sign function of the gradient.

6. The deep learning optimization method based on dynamic domain adaptive adjustment according to claim 1, characterized in that: The calculation formula of the dual feedback mechanism is as follows: Lc=L(W+Delta W)-L(W)+rho*norm(Delta W) Where: Lc is the contrast loss, L(W) is the loss value of the model under the current parameter W, Delta W is the change in parameter W caused by this update, L(W+Delta W) is the loss value of the model after updating the parameters, rho is the adjustment factor, and norm(Delta W) is the norm function.

7. The deep learning optimization method based on dynamic domain adaptive adjustment according to claim 1, characterized in that: The domain adaptation regularization calculation formula is shown as follows: <h2 style=";text-align:left;direction:ltr">R = lambda * sum ((Li-Lavg))<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> / var(L)) Where: Li represents the loss of the model on the i-th data domain, Lavg is the average of the losses on all considered domains, var(L) is the variance of the loss, and lambda is the regularization coefficient, which is a hyperparameter.

8. The deep learning optimization method based on dynamic domain adaptive adjustment according to claim 1, characterized in that: The cycle domain adjustment calculation formula is as follows: Wi+1=Wi-eta*dL / dWi-kappa*Adjust(Domain Performance) Where Wi is the parameter of the model in the current iteration, Wi+1 is the parameter of the model in the next iteration, eta is the learning rate, dL / dWi is the gradient of the loss function L with respect to the current parameter Wi, kappa is the adjustment coefficient, which is a hyperparameter, and Adjust(DomainPerformance) is a function that is dynamically adjusted according to the performance evaluation results of each domain.

Citation Information

Patent Citations

  • Motion state quantity evaluation method based on artificial intelligence

    CN117009876A

  • Air compression station adaptive transfer learning tuning method based on adversarial training

    CN118259589A