Deep neural network training optimization method and system based on adaptive gradient cutting

Abstract

The invention relates to the technical field of neural computing, in particular to a deep neural network training optimization method and system based on adaptive gradient clipping, and the method comprises the steps: obtaining a parameter set of a deep neural network model used for training a robot with a body, and carrying out the distinguishing of the parameter set to obtain a parameter subset; constructing a time sequence batch sample set based on the time sequence interaction data; performing forward propagation and loss calculation based on the time sequence batch sample set to obtain a loss value, and performing calculation based on the loss value to obtain a gradient set; determining an independent proportion cutting threshold of each parameter subset based on a dynamic threshold generation mechanism; calculating a scaling parameter ratio based on the gradient set, and executing a gradient set clipping operation based on the scaling parameter ratio and a proportion clipping threshold to obtain a clipped gradient set; driving the parameter set to update based on the clipped gradient set; and deploying the updated parameter set to the robot with the body, and repeating the process until training convergence.

Description

Technical Field

[0001] This invention relates to the field of neural computing, specifically to a method and system for training and optimizing deep neural networks based on adaptive gradient pruning. Background Technology

[0002] Deep neural networks (DNNs) have been widely applied in recent years in various fields such as image recognition, natural language processing, and speech recognition. To achieve more powerful model performance, the architecture of neural networks tends to become more complex and deeper, often containing hundreds or even thousands of layers. Ensuring numerical stability and convergence efficiency during the training of these complex deep neural networks is a core challenge.

[0003] Gradient explosion is a common problem in backpropagation of deep neural networks, leading to uncontrolled parameter updates and model training failure. To address this, conventional gradient pruning methods are used, which limit the norm of the gradient vector by setting a fixed pruning threshold. However, conventional gradient pruning suffers from several drawbacks: Difficulty in hyperparameter tuning and lack of adaptability: Conventional methods rely solely on a fixed threshold, failing to consider the ratio of the gradient norm to the parameter norm. Therefore, this fixed threshold is highly sensitive to hyperparameters such as network depth, learning rate, and batch size, requiring complex fine-grained tuning, significantly increasing the difficulty and time cost of model optimization. Insufficient pruning control precision: Due to the lack of consideration for the magnitude of the parameters themselves, conventional pruning may lead to over-pruning (slowing down learning) in some training stages or under-pruning (failing to effectively prevent instability) in others. Performance bottleneck in large network training: When training larger-scale neural networks, the gain effect of conventional gradient pruning diminishes, making it difficult to continuously and effectively improve model performance.

[0004] In summary, conventional gradient pruning methods in existing technologies lack adaptability and struggle to precisely control parameter updates, failing to meet the widespread demands of current deep neural networks for high efficiency, high stability, and low cost. Therefore, there is an urgent need in this field for an optimization method and system that can overcome the hyperparameter sensitivity of traditional fixed pruning and achieve automatic and precise control over parameter update stability. To address this, a method and system for optimizing deep neural network training based on adaptive gradient pruning are proposed. Summary of the Invention

[0005] The purpose of this invention is to provide a deep neural network training optimization method and system based on adaptive gradient clipping. By dynamically generating independent clipping thresholds for different network layers and calculating the ratio of gradient norm to parameter norm, the gradient is adaptively clipped by combining the ratio with the dynamic threshold. Finally, the clipped gradient is used to update the model parameters, thereby achieving efficient and stable training.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a deep neural network training optimization method based on adaptive gradient pruning, comprising: S1: Obtain the parameter set of the deep neural network model used to train the embodied robot, and distinguish the parameter set into parameter subsets based on different network layers; construct a time-series batch sample set based on the time-series interaction data generated by the interaction between the embodied robot and the environment; perform forward propagation and loss calculation based on the time-series batch sample set to obtain the loss value, and calculate the gradient set corresponding to the parameter subset through backpropagation based on the loss value. S2: A dynamic threshold generation mechanism is used to determine the independent proportional clipping threshold for each parameter subset; the ratio of the gradient norm of the gradient tensor to the parameter norm of the parameter tensor in the gradient set is calculated to obtain the scaling parameter ratio, and the gradient set clipping operation is performed based on the scaling parameter ratio and the proportional clipping threshold to obtain the clipped gradient set; S3: Based on the pruned gradient set, drive the parameter set update of the deep neural network model; deploy the updated parameter set on the embodied robot, and repeat S1 to S3 until the preset termination condition is reached.

[0007] Preferably, the specific process of obtaining the parameter set of the deep neural network model used to train the android and distinguishing parameter subsets based on different network layers includes: Traverse all network layers of the deep neural network model, lock the network layer containing trainable parameters, and extract weight tensors and bias tensors from the network layer; combine the weight tensors and bias tensors to form a parameter subset, and establish a direct correspondence between the parameter subset and the network layer.

[0008] Preferably, the specific process of constructing a time-series batch sample set based on the time-series interaction data generated by the interaction between the embodied robot and the environment includes: The time-series interaction data generated by the embodied robot in its interaction cycle with the environment is stored in the data storage space inside the embodied robot. The time-series interaction data includes state information, executed actions, and environmental feedback information. The time-series interaction data is randomly extracted from the data storage space, and all the extracted time-series interaction data are aggregated to generate the time-series batch sample set.

[0009] Preferably, the specific process of obtaining the loss value through forward propagation and loss calculation based on the time-series batch sample set, and obtaining the gradient set corresponding to the parameter subset through backpropagation based on the loss value includes: The time-series batch sample set is input into the deep neural network model. The time-series batch samples are processed layer by layer based on the network structure, and the prediction results are output. The target value is calculated based on the environmental feedback information and state information in the time-series batch sample set. A preset loss function is called to quantize and compare the prediction results with the target value to obtain the loss value. The backpropagation process is started based on the loss value. The backpropagation process calculates the partial derivatives of the loss value with respect to all parameters in the parameter subset from the output layer to the input layer. The partial derivative arrays are merged to form the gradient set corresponding to the parameter subset.

[0010] Preferably, the specific process of determining the independent proportional pruning threshold for each parameter subset using the dynamic threshold generation mechanism includes: Establish a preset adjustment function that maps network layer information to threshold adjustment coefficients; obtain network layer information corresponding to parameter subsets, input the network layer information into the preset adjustment function, calculate the threshold adjustment coefficient of the network layer; multiply the threshold adjustment coefficient by the current baseline pruning threshold to obtain the independent pruning threshold of each parameter subset.

[0011] Preferably, the specific process of performing gradient set clipping operation based on the scaling parameter ratio and the proportional clipping threshold to obtain the clipped gradient set includes: For different parameter subsets, calculate the L2 norm of the gradient tensor to obtain the gradient norm value, and calculate the L2 norm of the corresponding parameter tensor to obtain the parameter norm value. Divide the gradient norm value by the parameter norm value to obtain the scaling parameter ratio. When the scaling parameter ratio is greater than the scaling parameter threshold, perform a pruning operation on the gradient tensor to make the scaling parameter ratio equal to the scaling parameter threshold. When the scaling parameter ratio is not greater than the scaling parameter threshold, keep the gradient tensor value unchanged. Combine the pruned gradient tensors of all parameter subsets to obtain the pruned gradient set.

[0012] Preferably, the specific process of updating the parameter set of the deep neural network model based on the pruned gradient set includes: Based on the clipped gradient set, the corresponding parameter update amount is calculated for the parameters in the parameter set using the gradient descent optimization algorithm; the parameter values ​​in the parameter set are adjusted based on the parameter update amount to obtain the updated parameter set.

[0013] Preferably, the deep neural network training optimization system based on adaptive gradient pruning specifically includes: Gradient generation module: Obtains the parameter set of the deep neural network model used to train the embodied robot, and distinguishes the parameter set into parameter subsets based on different network layers; constructs a time-series batch sample set based on the time-series interaction data generated by the interaction between the embodied robot and the environment; performs forward propagation and loss calculation based on the time-series batch sample set to obtain the loss value, and calculates the gradient set corresponding to the parameter subset through backpropagation based on the loss value. The adaptive pruning module uses a dynamic threshold generation mechanism to determine an independent proportional pruning threshold for each parameter subset; it calculates the ratio of the gradient norm of the gradient tensor to the parameter norm of the parameter tensor in the gradient set to obtain the scaling parameter ratio, and performs a gradient set pruning operation based on the scaling parameter ratio and the proportional pruning threshold to obtain the pruned gradient set. Iterative update module: Based on the pruned gradient set, it drives the parameter set update of the deep neural network model; the updated parameter set is deployed on the embodied robot, and all functions of the gradient generation module, adaptive pruning module and iterative update module are repeated until the preset termination condition is reached.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention solves the problem of fixed thresholds being sensitive to hyperparameters by constructing dynamic thresholds for different network layers. When training embodied robots, it can adaptively handle the training requirements of different modules such as vision and force feedback without tedious manual tuning, significantly simplifying the process of model iteration and deployment of embodied robots in real or simulated environments.

[0015] 2. This invention overcomes the shortcomings of traditional methods that lack control precision due to neglecting parameter scaling by introducing the ratio of gradient to parameter norm for pruning. It precisely suppresses destructive updates that lead to uncontrolled behavior in the android, ensuring high numerical stability of the android's complex control strategy during training.

[0016] 3. This invention overcomes the problem of insufficient control accuracy caused by the single pruning force in conventional pruning methods in large-scale heterogeneous networks through hierarchical adaptive gradient control. By assigning differentiated pruning strategies to different levels, the numerical stability of complex embodied robot models processing high-dimensional, multimodal data is significantly improved during training, enabling efficient and stable convergence, thereby effectively increasing the final performance ceiling of the embodied robot. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the deep neural network training optimization method based on adaptive gradient pruning proposed in this invention. Figure 2 This is a schematic diagram of the structure of the deep neural network training and optimization system based on adaptive gradient pruning proposed in this invention; Figure 3 This is a flowchart illustrating the adaptive pruning method for a single parameter subset proposed in this invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figure 2 The diagram below shows the structure of the deep neural network training and optimization system based on adaptive gradient pruning proposed in this invention. The technical solution is as follows: A deep neural network training optimization system based on adaptive gradient pruning includes: Gradient generation module: Obtains the parameter set of the deep neural network model used to train the embodied robot, and distinguishes the parameter set into parameter subsets based on different network layers; constructs a time-series batch sample set based on the time-series interaction data generated by the interaction between the embodied robot and the environment; performs forward propagation and loss calculation based on the time-series batch sample set to obtain the loss value, and calculates the gradient set corresponding to the parameter subset through backpropagation based on the loss value. The adaptive pruning module uses a dynamic threshold generation mechanism to determine an independent proportional pruning threshold for each parameter subset; it calculates the ratio of the gradient norm of the gradient tensor to the parameter norm of the parameter tensor in the gradient set to obtain the scaling parameter ratio, and performs a gradient set pruning operation based on the scaling parameter ratio and the proportional pruning threshold to obtain the pruned gradient set. Iterative update module: Based on the pruned gradient set, it drives the parameter set update of the deep neural network model; the updated parameter set is deployed on the embodied robot, and all functions of the gradient generation module, adaptive pruning module and iterative update module are repeated until the preset termination condition is reached.

[0020] Example 1 Reference Figure 1 This is a flowchart illustrating the deep neural network training optimization method based on adaptive gradient pruning proposed in this invention. This embodiment provides a specific application of the deep neural network training optimization method based on adaptive gradient pruning. Its typical application scenario is to train a six-axis robotic arm to perform high-precision desktop object grasping and placement tasks.

[0021] The dynamic threshold generation mechanism is implemented as a parameterized mapping mechanism. Its construction process specifically includes: establishing a preset adjustment function that maps network layer information to threshold adjustment coefficients. The design of the preset adjustment function should incorporate prior knowledge of the network structure. For example, considering that deep networks are generally more prone to gradient instability than shallow networks, the function is designed as a monotonically decaying form. This design principle ensures that deep networks or key functional layers responsible for core decisions receive lower threshold adjustment coefficients than shallow networks, thereby imposing stricter pruning constraints to maintain global numerical stability. The preset adjustment function uses the network layer information to calculate the threshold adjustment coefficients. Its specific mapping mechanism can be differentiated according to the type of prior information used: when only layer depth information is used, the function can be implemented as a linear decay mechanism. First, the ratio of the current network layer depth to the total number of network layers is calculated. Then, this ratio is multiplied by a preset decay coefficient (e.g., selected between 0.3 and 0.7, 0.5 in this embodiment), and the result is subtracted from the value to obtain the threshold adjustment coefficient. When utilizing layer type or functional module information: the function can be implemented as a mapping mechanism based on preset rules. For the layer type (such as convolutional layer, fully connected layer, recurrent layer) or the functional module (such as visual encoder, ontology perception processor, or decision output layer) contained in the prior information of the network structure, the system pre-sets a corresponding threshold adjustment coefficient value. For example, for the decision output layer, which is more sensitive to gradient updates and located in the core decision path, a lower adjustment coefficient value is assigned; for the primary visual encoding layer located at the front end of the network and with relatively stable gradients, a higher adjustment coefficient value is assigned. This mapping mechanism directly maps the layer type or functional module to its preset threshold adjustment coefficient. For any parameter subset, the system first obtains its corresponding network layer information. The network layer information is input into the preset adjustment function, and the threshold adjustment coefficient of the network layer is calculated based on the preset mapping rules in the function. The threshold adjustment coefficient is multiplied by the current baseline pruning threshold to obtain the independent pruning threshold for each parameter subset. The parameter subset, as the core data object, will have its data structure encapsulated and stored parameter norm values ​​corresponding to the current parameter tensor after initialization by the gradient generation module or update by the iterative update module. The adaptive pruning module's function of "determining an independent proportional pruning threshold for each parameter subset" is calculated in real-time by accessing the preset network layer information in the parameter subset data structure and calling the dynamic threshold generation mechanism, rather than storing a fixed value. The function of "calculating the ratio of the gradient norm of the gradient tensor to the parameter norm of the parameter tensor in the gradient set" calculates the gradient norm in real-time and performs the ratio operation by accessing the preset parameter norm values ​​in the parameter subset data structure to reduce redundant calculations.

[0022] Furthermore, the specific process of obtaining the parameter set of the deep neural network model used to train the embodied robot, and distinguishing the parameter set into parameter subsets based on different network layers, includes: The deep neural network model used in training the six-axis robotic arm is a 120-layer deep residual network (ResNet) for processing 480x480 pixel RGB-D images input from the end-effector camera. At the start of the training process, a complete and ordered traversal of the 120 layers of the deep residual network is performed first. When a network layer containing trainable parameters is reached, such as the 25th convolutional layer, that layer is locked, and a parameter extraction instruction is triggered. This instruction reads all predefined weight tensors (3x3x128x256 dimensions) and bias tensors (256 dimensions) from the data space of that layer. Subsequently, these two extracted tensors are combined in memory and encapsulated into a data structure-independent subset of parameters; the specific process of constructing the parameter subset includes encapsulating the parameter subset into an independent data structure. The data structure further encapsulates and stores metadata associated with the parameter subset; the metadata includes the parameter norm value of the parameter subset and a field for storing the current independent pruning threshold. By pre-setting and encapsulating metadata such as the parameter norm value and independent pruning threshold in the parameter subset data structure, the above process transforms the high-frequency repetitive computation in training iterations into efficient memory access, significantly reducing the computation time of the adaptive pruning algorithm itself, thereby improving the practicality and overall training efficiency of this method when applied to large-scale deep models. To ensure the precise correspondence between subsequent gradient calculations and parameter updates, the parameter subset is assigned a unique "Layer-25" identifier bound to the source layer. This process of locking, extracting, combining, and identifying will be repeatedly executed for all parameterized layers in the model until the complete parameter set of the deep neural network model is completely divided into 120 parameter subsets that directly correspond to each network layer.

[0023] The above process systematically decomposes the overall parameter set of the deep neural network model into multiple parameter subsets that strictly correspond to the network layers, providing an indispensable technical prerequisite for subsequent hierarchical adaptive gradient control. Furthermore, by establishing a precise layer-parameter mapping relationship, subsequent gradient pruning can be differentiated and finely adjusted according to the specific characteristics of each layer, thus avoiding the training instability caused by applying a single, general update strategy to the entire heterogeneous network.

[0024] Furthermore, the specific process of constructing a time-series batch sample set based on the time-series interaction data generated by the interaction between the embodied robot and the environment includes: A data storage space with a capacity of 1,000,000 data records is configured on the computing unit inside the six-axis robotic arm. This data storage space is technically implemented as an experience playback pool. During the exploration phase of training, the six-axis robotic arm continuously interacts with a desktop object in a simulated environment. In each interaction cycle, the system generates a time-series interaction data record. This time-series interaction data specifically includes: a 480x480 pixel RGB-D image captured by the end effector and the current angle readings of the six joints as state information; three-dimensional spatial displacement and grasping commands issued to the end effector as actions; a scalar reward value calculated based on the action result as environmental feedback information; and the next state information after the action execution. This time-series interaction data is completely stored in the data storage space. When performing a model parameter update, the system randomly selects 512 time-series interaction data records from the data storage space based on a uniform distribution. Subsequently, the 512 time-series interaction data records were aggregated to form a time-series batch sample set, which was used as input data for the subsequent forward propagation steps.

[0025] The above process effectively breaks the strong correlation of time-series data generated by the embodied robot during continuous interaction by establishing a data storage space and randomly sampling from it to construct training batches; and by storing and randomly replaying historical experience, it ensures that the data samples used for each training are diverse and independent, significantly improving the stability of the training process and the efficiency of data utilization, and ultimately enabling the trained embodied robot control strategy to have stronger robustness and generalization ability to different scenarios.

[0026] Furthermore, the specific process of obtaining the loss value through forward propagation and loss calculation based on the time-series batch sample set, and then obtaining the gradient set corresponding to the parameter subset through backpropagation based on the loss value, includes: The time-series batch sample set containing 512 time-series interaction data records is fed into the input of the 120-layer deep residual network (ResNet). The data is processed layer by layer according to the network topology of the model, ultimately generating 512 corresponding predicted action commands at the model's output. Subsequently, the system calls the mean squared error loss function (MSELoss) to quantitatively compare the 512 predicted action commands with the target value contained in the sample set. The target value is calculated online using the temporal-difference algorithm based on the next state information and reward value recorded in the time-series batch sample set, thus calculating a single scalar loss value. Starting from this scalar loss value, the system automatically initiates the backpropagation process. The backpropagation process begins at the model's output layer and proceeds layer by layer along the network structure back to the input layer, calculating the partial derivatives of the loss value with respect to all parameters within each parameter subset. Finally, all the calculated partial derivative values ​​are combined to form 120 gradient sets that correspond one-to-one with the parameter subsets from “Layer-1” to “Layer-120”. These gradient sets will serve as the direct input for the subsequent adaptive pruning step.

[0027] The above process establishes a complete and precise mathematical pathway from macroscopic task performance to microscopic parameter adjustment. For the complex task of controlling a six-axis robotic arm, this process successfully transforms an abstract performance error (i.e., the loss value) into guiding signals (i.e., the gradient set) on how each parameter in the neural network should be adjusted. This ensures that each step of model optimization has a clear direction and mathematical basis, solving the technical challenge of blindly searching or inefficiently adjusting within a massive parameter space.

[0028] Furthermore, the specific process of determining the independent proportional pruning threshold for each parameter subset using the dynamic threshold generation mechanism includes: A globally uniform baseline pruning threshold is set, with a value of 0.02. This baseline pruning threshold is a preset constant used to set the overall pruning intensity benchmark, preferably ranging from 0.005 to 0.05. The initial setting of this value should be estimated based on the specific task type of the embodied robot and the scale of the training model. Generally, more complex models or tasks with high stability requirements (such as balance control) require a relatively low baseline pruning threshold to strengthen the overall pruning effect. Simultaneously, a preset adjustment function is established that maps network layer depth to threshold adjustment coefficients. The dynamic threshold generation mechanism incorporates prior knowledge of the network structure. Its preset adjustment function is designed as a monotonically decaying function to ensure that the threshold adjustment coefficient of deep networks or functionally critical layers is lower than that of shallow networks, thus imposing stricter pruning constraints. Specifically, the preset adjustment function can be implemented using the following mapping rule: first, calculate the ratio of the current network layer depth to the total number of network layers (120); then, subtract half of this ratio from the value 1.0, and use the difference as the threshold adjustment coefficient. The mechanism for mapping network layer information to the threshold adjustment function preferably employs a differentiated pruning strategy based on prior network structure information (e.g., the layer type or functional role of the parameter subset). This process dynamically generates differentiated pruning thresholds based on the mapping of layer depth or functional role, achieving strict gradient constraints on deep networks and significantly enhancing the numerical stability and convergence efficiency of training. For any parameter subset, such as "Layer-25", the system first obtains its corresponding network layer information as 25 and inputs it into the adjustment function. The threshold adjustment coefficient for the current network layer is calculated to be approximately 0.8958. This threshold adjustment coefficient is then multiplied by a baseline pruning threshold of 0.02 to obtain the independent pruning threshold for the "Layer-25" parameter subset, which is approximately 0.017916. In contrast, for a deeper parameter subset, such as "Layer-110" (layer depth 110), the threshold adjustment coefficient obtained using the same calculation method is approximately 0.5417, and its independent pruning threshold gamma_110 is approximately 0.010834. This process is applied to all 120 parameter subsets, thereby dynamically generating an independent pruning threshold for each layer.

[0029] The above process, by automatically assigning stricter pruning thresholds to deep networks, can effectively suppress runaway parameter updates during training, while avoiding unnecessary over-pruning of shallow networks. This significantly enhances the overall stability and convergence efficiency of training, ultimately resulting in a superior robot control strategy.

[0030] Furthermore, the specific process of performing gradient set clipping based on the scaling parameter ratio and the scaling clipping threshold to obtain the clipped gradient set includes: The core principle behind this step, which uses a scaling ratio (i.e., the ratio of gradient norm to parameter norm) for pruning, is to ensure that the magnitude of parameter updates matches its current scale, thereby maintaining training stability. In deep neural network training, the parameter norms (i.e., measures of parameter size) of different layers can differ by several orders of magnitude. Traditional absolute gradient pruning (i.e., limiting the gradient norm itself to no more than a fixed value) cannot perceive this scale difference, which may lead to applying relatively excessive updates to parameter layers with smaller norms (such as shallow layers or bias terms) (whose gradient norm, although less than the fixed value, is much larger than their parameter norm), causing their parameter values ​​to be washed away; at the same time, it may under-prune parameter layers with larger norms (such as deep layers). This invention calculates the scaling parameter ratio (i.e., the gradient norm divided by the parameter norm) and restricts it within an independent scaling threshold. Essentially, it limits the magnitude of the parameter update (approximately equal to the learning rate multiplied by the gradient norm) to the product of the learning rate, the scaling threshold, and the parameter norm. In other words, the relative magnitude of the parameter update to its corresponding parameter size is controlled within a stable ratio determined by the learning rate and the scaling threshold. This parameter-scale-based relative scaling method can impose scale-adaptive constraints on different layers, thereby more accurately suppressing destructive updates and ensuring high numerical stability during training. For any one of the 120 gradient sets generated in the preceding steps, for example, the gradient tensor g_110 corresponding to the parameter subset "Layer-110", the L2 norm of the gradient tensor is first calculated to obtain the gradient norm value, which is 0.8. Simultaneously, based on the correspondence between the gradient tensor and the parameter subset, the corresponding parameter tensor p_110 is indexed and retrieved from the current deep neural network model, and its L2 norm is calculated to obtain the parameter norm value, which is 20. Then, the gradient norm value is divided by the parameter norm value to obtain the scaling parameter ratio r_110 corresponding to the gradient tensor, which is the ratio of the gradient norm value 0.8 to the parameter norm value 20, i.e., 0.04. This scaling parameter ratio 0.04 is compared with the independent proportional clipping threshold γ_110 determined for this layer in the preceding steps. Since the scaling factor ratio is greater than the proportional clipping threshold, the condition is met. Therefore, a clipping operation is performed on the gradient tensor g_110. Specifically, the gradient tensor is multiplied by the scaling factor (γ_110 / r_110), i.e., (0.010834 / 0.04), so that the scaling factor ratio of the clipped gradient tensor is exactly equal to 0.010834. For other gradient tensors whose scaling factor ratio is not greater than their corresponding threshold, their values ​​are kept unchanged. Finally, the gradient tensors of all 120 parameter subsets processed by this process are combined to form the clipped gradient set.

[0031] The above process precisely distinguishes and suppresses destructive gradients that could cause critical parameters to "overshoot" and lead to divergent robot movements, while retaining reasonable gradient updates that are beneficial to the convergence of the overall model. This solves the problem of over- or under-pruning caused by neglecting parameter scale, ensuring that each step of parameter updates is smoother and more stable, thereby significantly improving the success rate of the robot learning complex skills.

[0032] Furthermore, the specific process of updating the parameter set of the deep neural network model based on the pruned gradient set includes: The Adam optimizer is invoked as the gradient descent optimization algorithm, with a learning rate of 0.0001. Upon initial startup of the training process, the first and second moment estimates internally used by the Adam optimizer to describe gradient history are initialized to zero. The clipped gradient set, composed of the gradient tensors, is used as input to the Adam optimizer. The Adam optimizer first updates its internally maintained first and second moment estimates using the input gradient set, and then calculates the corresponding parameter update amount for each parameter in the 120 parameter subsets based on this internal state. Finally, the system subtracts the current value of each parameter in the parameter set from the calculated corresponding parameter update amount, and uses the result as the new value of the parameter, thus obtaining a complete updated parameter set containing all 120 parameter subsets. One epoch is set to 2000 parameter updates. Every 2000 parameter updates, the system triggers a verification process. This process first pauses training, loads the updated parameter set into the six-axis robotic arm and sets it to pure evaluation mode. It then drives the robot to execute a predefined verification set containing 100 standardized grasping and placement tasks. The system records the number of successful tasks and calculates the average task success rate as a performance metric. If the performance metric no longer improves or reaches a pre-set performance threshold (99%) for three consecutive epochs, training is considered converged and the process is terminated. Furthermore, after each epoch verification, the system reviews the average gradient norm g_avg before pruning within the 2000 iterations of the current epoch. The average gradient norm g_avg is the arithmetic mean of the L2 norm values ​​of the gradient tensors of all parameter subsets within the current epoch. It then calculates the dynamic adjustment threshold T_adjust, which is equal to the product of the arithmetic mean p_avg of the current model parameter norms and the baseline pruning threshold gamma_base. A macro-periodic feedback control logic is constructed: if g_avg is greater than T_adjust, then gamma_base is multiplied by 0.9 for reduction, and the pruning intensity is globally strengthened in the next Epoch; if g_avg is less than the value obtained by multiplying T_adjust by 0.5, then gamma_base is multiplied by 1.1 for amplification, and the pruning constraint is globally relaxed in the next Epoch. A dual-loop feedback control mechanism is constructed, specifically including: a micro-instantaneous feedback loop that performs instantaneous pruning operation on the gradient based on the ratio of the gradient norm to the parameter norm and the independent proportional pruning threshold of each parameter subset in each parameter update iteration; and a macro-periodic feedback loop that, after completing one or more preset number of parameter updates, dynamically adjusts the baseline proportional pruning threshold based on the macro-statistical data of the training process collected in that cycle, and applies the adjusted baseline proportional pruning threshold to subsequent training cycles.The above process achieves dynamic regulation of the training process across time scales by constructing a dual-loop control mechanism that combines microscopic real-time feedback with macroscopic periodic feedback. It can both suppress instantaneous gradient anomalies in real-time through the inner loop to ensure single-step stability and adjust the global pruning strategy according to the macroscopic training trend through the outer loop, significantly enhancing the optimization method's adaptability and overall robustness to different training stages.

[0033] The above process provides an efficient and stable application path for gradient sets. By employing advanced gradient descent optimization algorithms such as Adam, it ensures that the fine-grained control applied in the preceding steps to prevent training divergence can be transformed into effective and smooth optimization of model parameters.

[0034] The above embodiment 1, for training a six-axis robotic arm, provides an efficient adaptive gradient pruning optimization method. By constructing a dynamic threshold associated with the network layers and innovatively using the gradient-parameter norm ratio for pruning, it fundamentally solves the problems of poor adaptability and insufficient control precision of fixed thresholds for complex robot models. This optimization method can accurately suppress destructive updates that cause the robotic arm's movements to diverge, while avoiding excessive suppression of beneficial gradients, significantly enhancing the stability and convergence efficiency of training. Ultimately, this enables the six-axis robotic arm to learn more refined grasping strategies and achieve a higher task success rate.

[0035] Example 2 This embodiment provides another specific application of a deep neural network training optimization method based on adaptive gradient pruning. A typical application scenario is the construction process of a temporal batch sample set for training a quadrupedal inspection robot to learn stable gait in unstructured rugged terrain. The deep neural network model used to train the quadrupedal inspection robot is a hybrid neural network architecture with 88 layers. This model jointly processes 128x128 pixel depth image information acquired by the robot's forward-facing depth camera and the proprioception information of its 12 joints. When the training process starts, the system program first performs a complete and ordered traversal of the 88-layer hybrid neural network architecture. When traversing to a network layer containing trainable parameters, such as the 75th fully connected layer which is the core of the proprioception processing module, this layer is locked, and a parameter extraction command is triggered. The parameter extraction command reads all predefined weight tensors (dimension 512x256) and bias tensors (dimension 256) from the data space of this layer. Subsequently, the two extracted tensors are combined in memory and encapsulated into a parameter subset with an independent data structure. To ensure the precise correspondence between subsequent gradient calculations and parameter updates, this parameter subset is assigned a unique "layer 75" identifier bound to the source layer. The above process of locking, extracting, combining, and identifying is repeated for all parameterized layers in the model until the complete parameter set of the deep neural network model is thoroughly divided into 88 parameter subsets that directly correspond to each network layer. A data storage space with a capacity of 2,000,000 data records is configured on the computing unit inside the quadruped inspection robot. This data storage space is technically implemented as an experience replay pool. During the exploration phase of training, the quadruped inspection robot explores gait learning in a simulated rugged terrain environment. In each interaction loop, the system generates a temporal interaction data. The temporal interaction data specifically includes: a 128x128 pixel depth image captured by the forward-looking depth camera as current state information; current angle and angular velocity readings of the 12 joints; target position commands issued to the servo motors of the 12 joints as action execution information; and a scalar reward value calculated by integrating the robot's forward speed, trunk stability, and energy consumption as environmental feedback information. The temporal interaction data is completely stored in the data storage space. When performing a model parameter update, the system randomly extracts 1024 temporal interaction data records from the data storage space based on a uniform distribution. Subsequently, these 1024 temporal interaction data records are aggregated to form a temporal batch sample set; the aggregation operation specifically involves stacking data elements of the same type in the 1024 temporal interaction data records along a new batch dimension to generate multiple independent batch tensors.For example, 1024 depth images (each with a dimension of 128x128) are combined into a single image batch tensor with a dimension of 1024x128x128; 1024 sets of joint angle readings (each set containing 12 values) are combined into a single joint angle batch tensor with a dimension of 1024x12; this stacking method is repeated to combine joint angular velocities, executed actions, and scalar reward values ​​into their respective batch tensors. These collections of batch tensors, each storing different types of data, collectively constitute the temporal batch sample set and serve as input data for subsequent forward propagation steps.

[0036] This embodiment addresses the technical challenge of training a quadrupedal inspection robot to maintain a stable gait in complex terrain by providing a highly structured and computationally efficient data and model preprocessing scheme. By precisely dividing the parameters of the hybrid neural network into layers, a crucial structural foundation is laid for applying differentiated adaptive gradient control to different functional modules such as visual perception and joint control.

[0037] Example 3 This embodiment provides another specific application of the deep neural network training optimization method based on adaptive gradient pruning. A typical application scenario is training a humanoid robot to perform balance control on a dynamically unstable platform. This embodiment will describe in detail the complete technical process of generating a gradient set based on a time-series batch sample set. First, a time-series batch sample set containing 256 time-series interaction data records is fed into the input of a 56-layer deep neural network model used for the humanoid robot's balance control. The model uses a Long Short-Term Memory (LSTM) network as its core, specifically designed to handle the high temporal dependence of the robot's proprioceptive information. During forward propagation, the time-series data stream, consisting of inertial measurement unit (IMU) readings, angles and angular velocities of 20 joints, etc., contained in the time-series batch sample set, is first fed into the LSTM network layer. The LSTM network layer processes the input sequence through its internal gating mechanism to capture key temporal dynamic features such as momentum and oscillation patterns in the robot's motion posture. Subsequently, the feature vectors containing temporal information output by the Long Short-Term Memory (LSTM) network layer are passed to subsequent fully connected layers for deep nonlinear transformation, ultimately generating 256 corresponding predicted correction moment instructions with 20 dimensions at the model's output. The system calls a smoothed L1 loss function to perform element-wise quantization comparisons of these 256 predicted correction moment instructions with target values ​​contained in the sample set. These target values, within an actor-critic framework, are calculated by an independent critic network estimating the value of the current state; the target value represents the optimal correction moment direction in that state. Through this quantization comparison, the system calculates and obtains a single scalar loss value of 0.853. Finally, using this scalar loss value of 0.853 as the starting point, the system automatically initiates the backpropagation process. The backpropagation process starts from the model's output layer and calculates the partial derivatives of the loss value with respect to all parameters within each parameter subset layer by layer along the network structure back to the input layer. When gradients flow through fully connected layers, they are calculated following the standard chain rule. When gradients flow to long short-term memory (LSTM) layers, they are calculated using a backpropagation algorithm to ensure that error gradients propagate correctly along the time series. Finally, all calculated partial derivative values ​​are combined to form 56 gradient sets, each corresponding to a subset of parameters from layer 1 to layer 56. These gradient sets serve as direct inputs for subsequent adaptive pruning steps.

[0038] The above process establishes a complete and accurate mathematical pathway from macroscopic performance errors to microscopic parameter adjustments for complex tasks such as balance control of humanoid robots, which are highly time-dependent. By employing a long short-term memory network and a backpropagation algorithm, the model is ensured to effectively learn and utilize temporal information. Simultaneously, by selecting a smooth L1 loss function, a more stable and robust gradient signal is provided for subsequent parameter updates, laying a solid data foundation for ultimately training a high-level dynamic balance strategy.

[0039] Example 4 This embodiment provides another specific application of the deep neural network training optimization method based on adaptive gradient pruning. A typical application scenario is training an autonomous mobile delivery robot to navigate and avoid obstacles in a dynamically changing indoor environment. (Reference) Figure 3This is a flowchart illustrating the adaptive pruning method for a single parameter subset proposed in this invention. This embodiment will describe in detail the specific technical process of dynamic threshold generation and adaptive pruning operation. In this embodiment, the deep neural network model used for navigation of the autonomous mobile delivery robot is an architecture containing 37 layers, combining convolutional neural networks and fully connected layers. First, a globally unified baseline pruning threshold is set, with a value of 0.015. Simultaneously, a preset adjustment function is established to map the network layer depth to a threshold adjustment coefficient. To reflect stronger constraints on deeper decision layers, this function is set to an exponential decay form. Specifically, the function is: taking the natural constant e as the base, using the negative value of the quotient obtained by dividing the network layer depth by the total number of network layers as the exponent, performing a power operation, and using the result as the threshold adjustment coefficient. For any parameter subset, such as the "fifth layer" parameter subset corresponding to the fifth convolutional layer processing visual input, the system first obtains its corresponding network layer information as 5 and inputs it into the adjustment function. The threshold adjustment coefficient for a given layer is obtained by calculating the negative value of the quotient (divided by the layer depth and the total number of layers in the network) and then exponentiating it with the natural constant e as the base. The calculated result is approximately 0.8737. This adjustment coefficient is then multiplied by the baseline proportional pruning threshold of 0.015 to obtain the independent proportional pruning threshold for the "Layer 5" parameter subset, which is approximately 0.0131. In contrast, for the "Layer 36" parameter subset, a deeper fully connected layer responsible for outputting final steering and speed commands, with a layer depth of 36, the threshold adjustment coefficient obtained using the same calculation method is approximately 0.3779, and its independent proportional pruning threshold is approximately 0.00567. This process is applied to all 37 parameter subsets, thereby dynamically generating an independent pruning threshold for each layer that matches its position and function within the network. Further, the specific process of performing gradient set pruning based on the scaling parameter ratio and the proportional pruning threshold to obtain the pruned gradient set includes: For any one of the 37 gradient sets generated in the aforementioned steps, for example, the gradient tensor corresponding to the parameter subset of the "thirty-sixth layer", the system first calculates the L2 norm of the gradient tensor to obtain the gradient norm value, which is 0.5; simultaneously, based on the correspondence between the gradient tensor and the parameter subset, the corresponding parameter tensor is indexed and obtained from the current deep neural network model, and the L2 norm of the parameter tensor is calculated to obtain the parameter norm value, which is 40. Subsequently, the gradient norm value is divided by the parameter norm value to obtain the scaling parameter ratio corresponding to the gradient tensor, which is 0.0125. This scaling parameter ratio of 0.0125 is compared with the independent proportional pruning threshold of approximately 0.00567 determined for this layer in the aforementioned steps. Since 0.0125 is greater than 0.00567, the judgment condition is met, and therefore, the pruning operation is performed on the gradient tensor.The pruning operation specifically involves multiplying the gradient tensor by a scaling factor. This scaling factor is obtained by dividing the independent scaling threshold for that level by the scaling parameter ratio of the gradient tensor; for example, dividing 0.00567 by 0.0125 yields approximately 0.4536. This multiplication ensures that the scaling parameter ratio of the pruned gradient tensor is exactly 0.00567. For another subset of parameters, such as the "fifth layer," if its calculated scaling parameter ratio is 0.01, since it is less than the corresponding independent scaling threshold of 0.0131, its gradient tensor value remains unchanged. Finally, the gradient tensors of all 37 parameter subsets processed through this procedure are combined to form the pruned gradient set.

[0040] This embodiment addresses the technical challenge of navigation for autonomous mobile delivery robots in dynamic environments by providing a refined gradient control scheme. By employing an exponential decay function to generate independent pruning thresholds strongly correlated with network depth, this embodiment imposes extremely strict stability constraints on the deep network responsible for final path decisions, while providing a more reasonable learning space for the shallow network responsible for initial visual perception. Through pruning operations based on the gradient-parameter norm ratio, destructive updates that could lead to drastic deviations from the robot's trajectory are precisely suppressed. This differentiated and precise control strategy ensures the efficiency and stability of the training process, ultimately enabling the delivery robot to learn more reliable and smoother obstacle avoidance and navigation capabilities.

[0041] Example 5 This embodiment provides another specific application of the deep neural network training optimization method based on adaptive gradient pruning. A typical application scenario is training a multi-UAV swarm for cooperative formation and obstacle avoidance. This embodiment will describe in detail the specific technical process of driving parameter updates and completing the training loop based on the pruned gradient set. The deep neural network model used for the cooperative control of the UAV swarm is a graph neural network (GNN) with 45 layers. First, the system calls the Adam optimizer as the gradient descent optimization algorithm and sets its learning rate to 0.0005. At the initial start of the training process, the first and second moment estimates used to describe the gradient history information within the Adam optimizer are initialized to zero vectors. The pruned gradient set, consisting of 45 processed gradient tensors, is used as the input to the Adam optimizer. Since the input gradient set has undergone the adaptive pruning process of this invention, its values ​​are stable and its scale is reasonable. This allows the Adam optimizer to more reliably utilize this gradient information to update its internally maintained first and second moment estimates, effectively avoiding internal state contamination caused by extreme values ​​that may exist in the original gradients. Subsequently, the Adam optimizer, based on its updated internal state, calculates a unique, adaptively scaled parameter update for each individual parameter in the 45 parameter subsets. Finally, the system subtracts the calculated parameter update from the current value of each parameter in the parameter set, using the result as the new value for that parameter, thus obtaining a complete updated parameter set containing all 45 parameter subsets. An epoch is set to 5000 parameter updates. Every 5000 parameter updates, the system triggers a verification process. This process first pauses training, loads the currently updated parameter set into the UAV swarm's control system and sets it to pure evaluation mode, then drives it to execute a predefined verification set containing 50 standardized cooperative formation and obstacle avoidance tasks. The system records the average formation error after all tasks are completed as a performance indicator. If the average formation error is below 0.1 meters for five consecutive epochs, training is considered converged and the process is terminated. Furthermore, after each epoch of validation, the system reviews the average gradient norm g_avg before pruning over the 5000 iterations contained in that epoch and calculates a dynamic adjustment threshold T_adjust. This dynamic adjustment threshold is equal to the product of the arithmetic mean p_avg of the current model parameter norms and the baseline pruning threshold γ_base. If g_avg is greater than T_adjust, γ_base is multiplied by 0.95 for reduction, and the pruning intensity is globally increased in the next epoch.

[0042] This embodiment addresses the technical challenges of collaborative control of multi-UAV swarms by providing an automated training closed-loop scheme. By inputting adaptively pruned stable gradients into the Adam optimizer, the complex parameter update process in the graph neural network is ensured to converge smoothly. Simultaneously, the constructed mechanism, incorporating periodic verification and dynamic threshold adjustment, forms a macroscopic feedback control loop. This loop dynamically and adaptively adjusts the strength of the pruning strategy based on the overall training of the swarm, ultimately ensuring that the UAV swarm can efficiently learn accurate collaborative formation and obstacle avoidance capabilities.

[0043] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A deep neural network training optimization method based on adaptive gradient clipping, characterized in that, The method comprises the following steps: S1: obtaining a parameter set of a deep neural network model used for training a body-equipped robot, and dividing the parameter set into parameter subsets based on different network levels; constructing a time-series batch sample set based on time-series interaction data generated by the body-equipped robot interacting with the environment; performing forward propagation and loss calculation based on the time-series batch sample set to obtain a loss value, and calculating a gradient set corresponding to the parameter subsets based on the loss value through back propagation; S2: determining a proportional clipping threshold for each parameter subset independently using a dynamic threshold generation mechanism; calculating the ratio of the gradient norm of the gradient tensor to the parameter norm of the parameter tensor in the gradient set to obtain a scaling parameter ratio, and performing a gradient set clipping operation based on the scaling parameter ratio and the proportional clipping threshold to obtain a clipped gradient set; S3: updating the parameter set of the deep neural network model based on the clipped gradient set; deploying the updated parameter set to the body-equipped robot, and repeating S1 to S3 until a predetermined termination condition is reached. 2.The deep neural network training optimization method based on adaptive gradient clipping according to claim 1, wherein, The specific process of obtaining the parameter set of the deep neural network model used for training the body-equipped robot, and dividing the parameter set into parameter subsets based on different network levels comprises: traversing all network levels of the deep neural network model, locking the network level containing trainable parameters, and extracting the weight tensor and bias tensor from the network level; combining the weight tensor and the bias tensor to form a parameter subset, and establishing a direct correspondence between the parameter subset and the network level. 3.The deep neural network training optimization method based on adaptive gradient clipping according to claim 1, wherein, The specific process of constructing a time-series batch sample set based on time-series interaction data generated by the body-equipped robot interacting with the environment comprises: storing the time-series interaction data generated by the body-equipped robot in the environment interaction cycle in the data storage space inside the body-equipped robot, wherein the time-series interaction data includes state information, executed actions, and environmental feedback information; randomly extracting time-series interaction data from the data storage space, aggregating all extracted time-series interaction data, and generating the time-series batch sample set. 4.The deep neural network training optimization method based on adaptive gradient clipping according to claim 1, wherein, The specific process of performing forward propagation and loss calculation based on the time-series batch sample set to obtain a loss value, and calculating a gradient set corresponding to the parameter subsets based on the loss value through back propagation comprises: inputting the time-series batch sample set into the deep neural network model, processing the time-series batch sample layer by layer based on the network structure and outputting a prediction result; calculating a target value based on the environmental feedback information and the state information in the time-series batch sample set; calling a preset loss function to quantitatively compare the prediction result with the target value to obtain the loss value; starting the back propagation process based on the loss value, wherein the back propagation process calculates the partial derivative of the loss value with respect to all parameters in the parameter subset from the output layer to the input layer, and combines the partial derivatives to form a gradient set corresponding to the parameter subset. 5.The deep neural network training optimization method based on adaptive gradient clipping according to claim 1, wherein, The specific process of determining the independent scaling parameter ratio threshold of each parameter subset by the dynamic threshold generation mechanism comprises: establishing a preset adjustment function for mapping network level information to a threshold adjustment coefficient; obtaining network level information corresponding to the parameter subset, inputting the network level information into the preset adjustment function, and calculating the threshold adjustment coefficient of the network level; and multiplying the threshold adjustment coefficient and the current reference scaling parameter ratio threshold to obtain the independent scaling parameter ratio threshold of each parameter subset. 6.The deep neural network training optimization method based on adaptive gradient clipping according to claim 1, wherein, The specific process of performing gradient set clipping operation based on the scaling parameter ratio and the scaling parameter ratio threshold to obtain the clipped gradient set comprises: calculating the L2 norm of the gradient tensor to obtain a gradient norm value, and calculating the L2 norm of the parameter tensor corresponding to the gradient tensor to obtain a parameter norm value, and dividing the gradient norm value by the parameter norm value to obtain the scaling parameter ratio; when the scaling parameter ratio is greater than the scaling parameter ratio threshold, performing clipping operation on the gradient tensor so that the scaling parameter ratio is equal to the scaling parameter ratio threshold; when the scaling parameter ratio is not greater than the scaling parameter ratio threshold, the gradient tensor value remains unchanged; and combining all parameter subset clipped gradient tensors to obtain the clipped gradient set.

7. The method of claim 1, wherein the method further comprises: The specific process of driving the parameter set update of the deep neural network model based on the clipped gradient set comprises: calculating the corresponding parameter update amount of the parameters in the parameter set based on the clipped gradient set by using the gradient descent optimization algorithm; and adjusting the parameter values in the parameter set based on the parameter update amount to obtain the updated parameter set.

8. A deep neural network training optimization system based on adaptive gradient clipping, characterized in that, Comprise: A gradient generation module: obtaining a parameter set of a deep neural network model used for training a robot, and dividing the parameter set into parameter subsets based on different network levels; Based on the time series interaction data generated by the interaction between the embodied robot and the environment, a time series batch sample set is constructed; Based on the time series batch sample set, forward propagation and loss calculation are performed to obtain a loss value, and based on the loss value, a gradient set corresponding to the parameter subset is calculated by back propagation; An adaptive clipping module: a dynamic threshold generation mechanism is used to determine the independent scaling parameter ratio threshold of each parameter subset; The ratio of the gradient norm of the gradient tensor in the gradient set and the parameter norm of the parameter tensor is calculated to obtain a scaling parameter ratio, and gradient set clipping operation is performed based on the scaling parameter ratio and the scaling parameter ratio threshold to obtain a clipped gradient set; An iterative update module: based on the clipped gradient set, the parameter set update of the deep neural network model is driven; the updated parameter set is deployed on the embodied robot, and the functions of the gradient generation module, the adaptive clipping module and the iterative update module are repeated until a preset termination condition is reached.

Images