Mechanical arm path planning control method based on deep learning

Abstract

The invention discloses a mechanical arm path planning control method based on deep learning, and the method comprises the steps: collecting and preprocessing the state of a mechanical arm and obstacle information, and constructing a training and testing data set; training an improved Shenchang differential equation model, and updating model parameters; taking the trained model as a dynamics core, and constructing a simulation environment; setting a trust region threshold and a security weight rule, generating a moving trajectory and constructing a sampling data set; calculating a true and model advantage value, and generating a comprehensive advantage value; obtaining a target strategy according to the security weight and the comprehensive advantages; state information is input in actual control, and a control action is generated and corrected to drive the mechanical arm to execute. According to the method, the improved Shenchang differential equation model and the trust region strategy optimization algorithm are combined, so that the high precision, smoothness and safety of path planning and control of the mechanical arm are improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology, and in particular to a path planning and control method for a robotic arm based on deep learning. Background Technology

[0002] Existing robotic arm path planning and control technologies primarily rely on traditional planning models based on geometry, kinematics, or optimization methods. Path planning typically employs discretized spatial search or analytical equations of motion, such as Dijkstra's algorithm, fast random trees, and trajectory generation methods based on model predictive control. While these methods have achieved good results in terms of path feasibility in static environments, their reliance on simplified dynamic models and static environment assumptions often results in paths that are only geometrically feasible but lack physical executability in complex dynamic environments. Although the path may spatially avoid obstacles, in actual control, factors such as the robotic arm's dynamic constraints, joint limits, and actuator saturation may not be considered, leading to unexecutable or unevenly executed path control commands. Traditional control methods also rely heavily on empirical parameter tuning, making it difficult to adaptively handle nonlinear disturbances and external perturbations, exhibiting poor stability and insufficient robustness in complex tasks or unknown environments.

[0003] In recent years, deep learning technology has been introduced into robotic arm control and path planning. By learning nonlinear dynamic relationships and complex control strategies through neural networks, it provides a new solution for the integration of path planning and control. However, existing deep learning-based path planning and control methods generally have two limitations: First, neural network structures are mostly static mappings, which cannot effectively capture the continuous-time dynamic characteristics of the system, making the planning results sensitive to changes in time scale and difficult to maintain consistency under different control frequencies. Second, at the policy optimization level, although existing reinforcement learning methods can learn control strategies through reward signals, the optimization process lacks explicit constraints on safety and stability, which can easily lead to unstable policy updates or overexploration, resulting in collision risks or over-limit control of the path.

[0004] Therefore, how to provide a deep learning-based robotic arm path planning and control method is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a deep learning-based method for robotic arm path planning and control. This invention integrates an improved neural network constant differential equation (NDI) model with a trust-region policy optimization (TGR) algorithm, constructing an integrated framework for path planning and control decision-making. The improved NDI model enables continuous-time modeling of robotic arm dynamics and expression of safety constraints. Furthermore, the TGR algorithm introduces safety weights and trust-region constraints into policy updates, achieving stable iteration and adaptive correction of the control policy. This method can generate smooth, executable, and safe robotic arm control trajectories in complex dynamic environments, significantly improving the feasibility of path planning and the stability and robustness of the control process.

[0006] A deep learning-based robotic arm path planning and control method according to an embodiment of the present invention includes:

[0007] Collect robotic arm status information and obstacle information from the external environment, preprocess the status information, encode the obstacle information using a distance field, and construct training and testing datasets.

[0008] The improved divine ordinary differential equation model is trained using the training dataset, continuous-time integral calculation is performed to generate a safe cumulative quantity, and the error is calculated with the test dataset. The model parameters are updated according to the error to obtain the trained improved divine ordinary differential equation model.

[0009] The improved neural ordinary differential equation model after training is used as the core of the dynamics to construct the simulation environment;

[0010] In the simulation environment, the trust region threshold and safety weight calculation rules are set, the control strategy of the trust region policy optimization algorithm is initialized, and multiple running trajectories are generated in the simulation environment. A sampling dataset is constructed based on the running trajectories.

[0011] The true advantage value is calculated based on the sampled dataset. Short-term prediction is performed using the trained improved neural network differential equation model to generate the model advantage value. The true advantage value and the model advantage value are then weighted and fused to obtain the comprehensive advantage value.

[0012] The safety weights are calculated based on the accumulated safety amount. The relative entropy of the control strategy under each state is summed in weights, and a trust region constraint objective function is constructed. Under the condition that the weighted relative entropy meets the threshold, the control strategy is updated using the comprehensive advantage function as the optimization objective and the conjugate gradient method and the line search method to obtain the target control strategy.

[0013] The target control strategy is applied to the control of the robotic arm. The current state information is input to obtain the control action. The improved neural constant differential equation model after training is used for short-time prediction. The control action is verified and corrected for safety, and the final control action is obtained to drive the robotic arm to execute.

[0014] Optionally, the preprocessing of the state information includes denoising, alignment, normalization, interpolation, and filtering.

[0015] Optionally, constructing the training and test datasets includes:

[0016] The robot arm state information under preset working conditions is collected. The robot arm state information includes the joint angles, joint angular velocities, joint angular accelerations, and the pose data of the end effector in a unified coordinate system. The collected robot arm state information is preprocessed to obtain the preprocessed robot arm state sequence.

[0017] Information on external environmental obstacles within the robotic arm's workspace is collected. This information includes obstacle point cloud data or grid occupancy information. Interpolation and resampling are performed on the 3D grid. Each grid cell is assigned a distance value corresponding to the distance between the grid cell and the nearest obstacle surface, generating an obstacle distance field code. The preprocessed robotic arm state sequence is then paired with the obstacle distance field code, and training and testing datasets are constructed according to sample partitioning rules.

[0018] Optionally, obtaining the trained improved neural frequent differential equation model includes:

[0019] Read the state information and safety-related information of the robotic arm from the training dataset, set the safety accumulation variable according to the state information and safety-related information, combine the state information and safety accumulation variable to form an extended state vector sequence, and divide the extended state vector sequence according to the preset time length and time step to obtain the extended state training sequence for model training.

[0020] The extended state training sequence is input into the improved neural ordinary differential equation model. A physical consistency submodule is set up to calculate the physical consistency output of the basic dynamic changes of the robotic arm through the dynamic relationship of the robotic arm. A residual compensation submodule is introduced to generate the residual compensation output, and a safety potential field submodule is introduced to generate the safety potential field output. The physical consistency output, residual compensation output and safety potential field output are combined.

[0021] Set up event functions to describe joint angle limits and robotic arm end contact with obstacles, monitor the process of extended state training sequence changing over time, record the event occurrence time when the event function meets the preset trigger conditions, divide the continuous time axis into multiple sub-time intervals according to the occurrence time of each event, and perform state reset processing on the extended state training sequence at the occurrence time of the event.

[0022] The initial extended state vector, control input sequence, and improved neural network differential equation model for each sub-time interval are input into the numerical integrator. Continuous time integration is performed according to the set time step to obtain the extended state prediction sequence within the corresponding sub-time interval. The extended state prediction sequence includes the robotic arm state prediction sequence and the safety accumulation prediction sequence.

[0023] The extended state prediction sequence and the test dataset are time-aligned, and a loss function containing the robotic arm state deviation term and the safety cumulative deviation term is constructed. A corresponding backpropagation calculation process is established for each sub-time interval. Gradient calculation is performed on the improved God ordinary differential equation model in chronological order within each sub-time interval. At the time of the event, the gradient within the sub-time interval is corrected according to the state reset process. The gradient results obtained from all sub-time intervals are accumulated, and the parameters of the physical consistency submodule, residual compensation submodule and safety potential field submodule are updated simultaneously based on the accumulated gradient to complete the training of the improved God ordinary differential equation model.

[0024] Optionally, the construction of the simulation environment includes:

[0025] The trained improved neural ordinary differential equation model is loaded as the dynamic core, and the state vector, control input, simulation time step, simulation start state and simulation termination condition of the simulation environment are set.

[0026] During the simulation, the state vector and control input of the current simulation time step are fed into the improved neural network constant differential equation model. The state vector and safety accumulation of the next simulation time step are calculated through continuous time integration. The simulation time is iteratively updated until the simulation termination condition is met, thus obtaining the running simulation environment.

[0027] Optionally, constructing the sampling dataset based on the running trajectory includes:

[0028] In the simulation environment, the trust region threshold and safety weight calculation function are set, and the control strategy of the trust region policy optimization algorithm is initialized. The safety weight calculation function takes the state information of the robotic arm in the simulation environment and the safety accumulation output by the improved neural network constant differential equation model after training as input to obtain the corresponding safety weight coefficient.

[0029] In the simulation environment, multiple simulation experiments are performed based on the initialization control strategy. In each simulation experiment, starting from the initial state of the simulation, the current state information is input into the control strategy to obtain the control action. The control action is input into the simulation environment for continuous time integration calculation to obtain the predicted state information and the updated safety accumulation amount at the next moment. The state information, control action, immediate reward, predicted state information and safety accumulation amount of each simulation time step are recorded in time sequence to form the corresponding running trajectory.

[0030] A dual-advantage path structure of real advantage and model advantage is set up. The state information and control actions in the running trajectory are used as the data required for real advantage calculation, and the predicted state information and safety accumulation in the running trajectory are used as the data required for model advantage calculation. All recorded results obtained from multiple simulation experiments are sorted out to construct a sampled dataset containing real advantage calculation data and model advantage calculation data.

[0031] Optionally, the step of weightedly fusing the true advantage value and the model advantage value to obtain the comprehensive advantage value includes:

[0032] The instant rewards are discounted and accumulated to obtain the corresponding discount cumulative return sequence. The state value is calculated for the state at each time step. The state value is subtracted from the discount cumulative return at each time step to obtain the real advantage value sequence corresponding to the control action at each time step.

[0033] The improved neural network differential equation model after training is invoked, and continuous time integration is performed within a short-time prediction window with the state and control action as initial conditions to obtain the predicted state sequence and the predicted safety cumulative quantity sequence. The model evaluation quantity is obtained based on the predicted state sequence and the predicted safety cumulative quantity sequence. The model evaluation quantity is subtracted from the reference evaluation quantity for the state at each time step to obtain the model advantage value sequence corresponding to the control action at each time step.

[0034] The true advantage value sequence and the model advantage value sequence at the same time step are linearly combined according to weighting coefficients. The true advantage value is multiplied by the first weighting coefficient, the model advantage value is multiplied by the second weighting coefficient, and the two are summed to obtain the comprehensive advantage value at the time step.

[0035] Optionally, obtaining the target control strategy includes:

[0036] Call the safety weight calculation function, taking the robotic arm state information and safety accumulation as input, to calculate the safety weight coefficient corresponding to each time step;

[0037] Read the comprehensive advantage value of each time step, calculate the action probability distribution according to the current control strategy, construct candidate control strategies based on the current control strategy, calculate the corresponding action probability distribution, and perform KL divergence calculation on the action probability distribution of the current control strategy and candidate control strategies to obtain the relative entropy value of each time step.

[0038] The relative entropy value is multiplied by the safety weight coefficient and summed to obtain the weighted relative entropy. The weighted relative entropy is compared with the preset trust region threshold. When the weighted relative entropy meets the trust region threshold condition, the comprehensive advantage value of each time step is multiplied by the action probability distribution of the current control strategy at the time step, and the results of the multiplication are summed to form the optimization objective function. The conjugate gradient method and the line search method are used to determine the parameter update direction and update step size of the control strategy, and the parameters of the control strategy are updated to obtain the target control strategy.

[0039] Optionally, obtaining the final control action and driving the robotic arm to execute it includes:

[0040] During the online control of the robotic arm, the current state information of the robotic arm and the obstacle distance field encoding information of the external environment are obtained, the target control strategy is input, and the first control action corresponding to the current state is obtained.

[0041] The current state information of the robotic arm and the first control action are input into the improved neural ordinary differential equation model to obtain the prediction results. The constraint determination results are generated based on the relationship between the prediction results and the joint angle range, joint speed range and obstacle distance constraints.

[0042] When the constraint determination result meets the preset constraint conditions, the first control action is taken as the final control action. When the constraint determination result does not meet the preset constraint conditions, the first control action is adjusted to obtain the second control action, which is then taken as the final control action and sent to the robotic arm servo drive device for execution.

[0043] The beneficial effects of this invention are:

[0044] This invention proposes a deep learning-based robotic arm path planning and control method, which effectively solves the problems in existing technologies where the path planning and control stages are separated, and model simplification leads to feasible paths but unexecutable or non-smooth control commands. By introducing an improved neural frequent differential equation model, this invention establishes a high-precision dynamic description of the robotic arm in the continuous time domain, achieving unified modeling of robotic arm state changes and safety constraints. Through the fusion design of a three-branch structure of physical consistency, residual compensation, and safety potential field, the model exhibits higher accuracy and stability in dynamic prediction and safety constraint expression, and can adapt to dynamic disturbances and multiple constraints in complex environments, thus providing a reliable modeling foundation for the dynamic feasibility and execution smoothness of the robotic arm path.

[0045] This invention further integrates a trust-region policy optimization algorithm, introducing safety weights and trust-region constraints during the policy update process. Through weighted fusion of true advantage values ​​and model advantage values, a dynamic balance between safety and convergence in the control policy is achieved. A weighted relative entropy is used to construct the constraint objective function, and conjugate gradient and line search methods are employed for parameter updates, making the policy optimization process more stable and efficient. Using this method, robotic arms can achieve integrated path planning and control in complex environments, significantly reducing collision and limit-crossing risks and improving motion smoothness, controllability, and robustness. This invention not only enhances the interpretability and reliability of deep learning in robotic arm control but also provides a scalable, high-performance control solution for intelligent manufacturing and flexible automation, demonstrating significant engineering application value and promising prospects. Attached Figure Description

[0046] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0047] Figure 1 This is a flowchart of a deep learning-based robotic arm path planning and control method proposed in this invention.

[0048] Figure 2 This is a block diagram of the improved neural ordinary differential equation model of a deep learning-based robotic arm path planning and control method proposed in this invention.

[0049] Figure 3 This is a functional diagram of the trust region policy optimization algorithm of the deep learning-based robotic arm path planning and control method proposed in this invention. Detailed Implementation

[0050] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0051] refer to Figure 1 , Figure 2 and Figure 3 A deep learning-based path planning and control method for robotic arms includes:

[0052] Collect robotic arm status information and obstacle information from the external environment, preprocess the status information, encode the obstacle information using a distance field, and construct training and testing datasets.

[0053] The improved divine ordinary differential equation model is trained using the training dataset, continuous-time integral calculation is performed to generate a safe cumulative quantity, and the error is calculated with the test dataset. The model parameters are updated according to the error to obtain the trained improved divine ordinary differential equation model.

[0054] The improved neural ordinary differential equation model after training is used as the core of the dynamics to construct the simulation environment;

[0055] In the simulation environment, the trust region threshold and safety weight calculation rules are set, the control strategy of the trust region policy optimization algorithm is initialized, and multiple running trajectories are generated in the simulation environment. A sampling dataset is constructed based on the running trajectories.

[0056] The true advantage value is calculated based on the sampled dataset. Short-term prediction is performed using the trained improved neural network differential equation model to generate the model advantage value. The true advantage value and the model advantage value are then weighted and fused to obtain the comprehensive advantage value.

[0057] The safety weights are calculated based on the accumulated safety amount. The relative entropy of the control strategy under each state is summed in weights, and a trust region constraint objective function is constructed. Under the condition that the weighted relative entropy meets the threshold, the control strategy is updated using the comprehensive advantage function as the optimization objective and the conjugate gradient method and the line search method to obtain the target control strategy.

[0058] The target control strategy is applied to the control of the robotic arm. The current state information is input to obtain the control action. The improved neural constant differential equation model after training is used for short-time prediction. The control action is verified and corrected for safety, and the final control action is obtained to drive the robotic arm to execute.

[0059] In this embodiment, the preprocessing of the state information includes denoising, alignment, normalization, interpolation, and filtering.

[0060] In this embodiment, constructing the training dataset and the test dataset includes:

[0061] The robot arm state information under preset working conditions is collected. The robot arm state information includes the joint angles, joint angular velocities, joint angular accelerations, and the pose data of the end effector in a unified coordinate system. The collected robot arm state information is preprocessed to obtain the preprocessed robot arm state sequence.

[0062] Information on external environmental obstacles within the robotic arm's workspace is collected. This information includes obstacle point cloud data or grid occupancy information. Interpolation and resampling are performed on a 3D mesh. Each mesh cell is assigned a distance value corresponding to the distance between the mesh cell and the nearest obstacle surface, generating an obstacle distance field code. The preprocessed robotic arm state sequence is then paired with the obstacle distance field code. Training and test datasets are constructed according to sample partitioning rules, where:

[0063] The generation of obstacle distance field coding is specifically as follows: a three-dimensional regular grid is established in the workspace of the robotic arm, and obstacle point cloud or grid data is mapped to the grid after coordinate unification, noise reduction and downsampling. The distance value from each grid cell to the nearest obstacle surface is calculated by three-dimensional Euclidean distance transformation, and positive and negative signs are assigned according to the occupancy relationship. Finally, the distance value is truncated and normalized to form an obstacle distance field coding that can be used for model input.

[0064] The construction of training and test datasets according to sample partitioning rules is as follows: extracting the robotic arm state sequence, control input, and distance field encoding at corresponding times in units of time windows; generating samples in a sliding window manner; partitioning the samples hierarchically according to the task scenario and time sequence to ensure that different trajectory samples are not used interchangeably; and dividing the samples into training and test sets proportionally, with 80% of the data used for the training set and 20% used for the test set.

[0065] In this embodiment, obtaining the trained improved neural ordinary differential equation model includes:

[0066] The state information and safety-related information of the robotic arm are read from the training dataset. A safety accumulator variable is set based on the state information and safety-related information. The state information and the safety accumulator variable are combined to form an extended state vector sequence. This extended state vector sequence is then divided according to a preset time length and time step to obtain the extended state training sequence for model training.

[0067] The process of combining state information with safety cumulative variables to form an extended state vector sequence is as follows: based on the variable dimension, safety-related information is mapped to safety cumulative variables, which are then concatenated with the robotic arm state information at the same time step to form an extended state vector for a single time step. The extended state vectors of all time steps are concatenated sequentially to construct a complete extended state vector sequence in chronological order, and the dimension and unit of the sequence are standardized.

[0068] The step of dividing the extended state vector sequence according to a preset time length and time step to obtain the extended state training sequence for model training is as follows: According to the preset time window length and time step, continuous time segments are extracted from the extended state vector sequence in a sliding window manner. Each time segment contains a fixed number of continuous extended state vectors as a training sample. The starting point of the window between adjacent samples slides backward according to the time step until the entire extended state vector sequence is covered. All extracted time segments are numbered in sequence to form the extended state training sequence.

[0069] The extended state training sequence is input into the improved neural network constant differential equation model. A physical consistency submodule is set up to calculate the physical consistency output of the basic dynamic changes of the robotic arm through the dynamic relationship of the robotic arm. A residual compensation submodule is introduced to generate the residual compensation output, and a safety potential field submodule is introduced to generate the safety potential field output. The physical consistency output, residual compensation output, and safety potential field output are combined, where:

[0070] The physical consistency submodule receives the input information of the extended state training sequence, extracts the state information of the robotic arm, performs matrix processing on the state vector according to the structural parameters and dynamic constraints of the robotic arm, calculates the state derivative corresponding to each time step using the set mass matrix, inertial parameters and joint constraint variables, obtains the physical consistency output characterizing the basic dynamic changes of the robotic arm, and temporarily stores it with the time step as the index.

[0071] The residual compensation submodule is used to compensate for the complex dynamic features that the physical consistency submodule fails to fully characterize. The input of the residual compensation submodule includes the extended state training sequence and the output of the physical consistency submodule. The two are first concatenated at the same time step to form a residual input vector. The residual input vector is then processed by a multi-layer nonlinear neural network structure for feature mapping and high-dimensional feature extraction to capture the feature patterns corresponding to the unmodeled dynamic parts. A residual compensation output signal is generated at the output layer, which represents the dynamic correction terms of the model for the parts not covered by the basic dynamic equations.

[0072] The safety potential field submodule is used to characterize the safety constraint information between the robotic arm and the external environment. The safety potential field submodule first inputs safety-related variables into a spatial mapping function or embedding network. Through coordinate normalization and spatial embedding operations, it maps constraint information from different sources to a unified vector space and integrates obstacle, joint and environmental features to generate an output vector that represents the overall safety constraint. The output vector corresponds to the physical consistency output and residual compensation output in the time step dimension.

[0073] Event functions are set to describe joint angle limits and end-effector contact with obstacles. Event monitoring is performed on the extended state training sequence as it changes over time. When the event function meets a preset trigger condition, the event occurrence time is recorded. The continuous time axis is divided into multiple sub-time intervals according to the event occurrence times, and a state reset is performed on the extended state training sequence at each event occurrence time. The event functions include joint limit event functions and end-effector contact with obstacle event functions, wherein:

[0074] The node limit event function is used to detect whether the angle of each joint of the robotic arm has reached the allowable range of motion. Each joint has a corresponding minimum allowable angle value and a maximum allowable angle value, which represent the lower limit and upper limit of the joint in the physical structure, respectively. When the actual joint angle value is equal to the minimum allowable angle value or the maximum allowable angle value, it means that the joint has reached the angle limit position, the output of the node limit event function is zero, which is regarded as the occurrence of a joint limit event, and the occurrence time of the event is recorded.

[0075] The end-effector contact obstacle event function is used to determine whether the distance between the end of the robotic arm or a key part and an obstacle is zero. By encoding the obstacle distance field, the shortest distance value from any point on the robotic arm to the nearest obstacle surface can be obtained. When the distance value is zero, it means that the end of the robotic arm has made contact with the obstacle surface. The end-effector contact obstacle event function outputs zero, triggering a contact event.

[0076] During continuous time integration, the numerical changes of these two types of event functions are continuously monitored. When the output of any event function reaches zero, the current time is recorded as the event trigger time, and the time axis is divided into new sub-time intervals with the current time as the dividing point. At the event time, the extended state vector is reset.

[0077] The initial extended state vector, control input sequence, and improved neural network differential equation model for each sub-time interval are input into the numerical integrator. Continuous time integration is performed according to the set time step to obtain the extended state prediction sequence within the corresponding sub-time interval. The extended state prediction sequence includes the robotic arm state prediction sequence and the safety accumulation prediction sequence.

[0078] The extended state prediction sequence and the test dataset are time-aligned. A loss function containing a robotic arm state deviation term and a safety accumulation deviation term is constructed. A corresponding backpropagation calculation process is established for each sub-time interval. Gradient calculations are performed on the improved God ordinary differential equation model in chronological order within each sub-time interval. At the time of the event, the gradients within the sub-time interval are corrected according to the state reset process. The gradient results obtained from all sub-time intervals are accumulated, and the parameters of the physical consistency submodule, residual compensation submodule, and safety potential field submodule are updated simultaneously based on the accumulated gradients to complete the training of the improved God ordinary differential equation model. The construction of the loss function containing the robotic arm state deviation term and the safety accumulation deviation term is as follows:

[0079] Using each time step as an index, the corresponding robotic arm status information and safety cumulative value are read from the prediction sequence. At the same time, the real robotic arm status information and real safety cumulative value recorded at the same time step are extracted from the test dataset. The difference between the predicted value and the real value is calculated to form the basic deviation data.

[0080] The robotic arm state deviation term and the safety accumulation deviation term are constructed separately. The robotic arm state deviation term is used to describe the difference between the predicted state and the actual state. It is obtained by squaring and summing the differences between the predicted and actual values ​​of joint angle, joint angular velocity and end pose at each time step. The safety accumulation deviation term is used to describe the difference between the predicted safety accumulation and the actual safety accumulation. It is obtained by calculating the square of the difference between the two values ​​at each time step and performing a weighted sum.

[0081] The robotic arm state deviation term and the safety cumulative deviation term are combined into a single overall loss function according to weighting coefficients. The loss function is calculated independently in each sub-time interval. The calculated loss function value is used as the input for backpropagation calculation to guide gradient calculation and parameter update in each sub-time interval.

[0082] In this embodiment, the construction of the simulation environment includes:

[0083] The trained improved neural ordinary differential equation model is loaded as the dynamic core, and the state vector, control input, simulation time step, simulation start state and simulation termination condition of the simulation environment are set.

[0084] During the simulation, the state vector and control input of the current simulation time step are fed into the improved neural network constant differential equation model. The state vector and safety accumulation of the next simulation time step are calculated through continuous time integration. The simulation time is iteratively updated until the simulation termination condition is met, thus obtaining the running simulation environment.

[0085] In this embodiment, constructing the sampling dataset based on the running trajectory includes:

[0086] In the simulation environment, the trust region threshold and safety weight calculation function are set, and the control strategy of the trust region policy optimization algorithm is initialized. The safety weight calculation function takes the robotic arm state information in the simulation environment and the safety accumulation output by the trained improved neural network constant differential equation model as input to obtain the corresponding safety weight coefficients. The safety weight calculation function is specifically as follows:

[0087] The safety weight calculation function is a scalar mapping function used to calculate the safety weight coefficient corresponding to each time step based on the operating state of the robotic arm in the simulation environment. The input variables of the function include the robotic arm state information, the distance value of the obstacle distance field encoding, the safety accumulation output by the trained improved neural network differential equation model, and the limit margin of joint angle and angular velocity.

[0088] In the feature combination stage, the function performs linear or nonlinear combination of the input variables according to the weighting rules to form an intermediate risk quantity between zero and one. A monotonically increasing scalar mapping function is applied to the intermediate risk quantity, and the result is truncated with upper and lower bounds so that the value is strictly limited between the minimum weight coefficient and the maximum weight coefficient. Finally, the safety weight coefficient of the current time step is obtained.

[0089] In the simulation environment, multiple simulation experiments are performed based on the initialization control strategy. In each simulation experiment, starting from the initial state of the simulation, the current state information is input into the control strategy to obtain the control action. The control action is input into the simulation environment for continuous time integration calculation to obtain the predicted state information and the updated safety accumulation amount at the next moment. The state information, control action, immediate reward, predicted state information and safety accumulation amount of each simulation time step are recorded in time sequence to form the corresponding running trajectory.

[0090] A dual-advantage path structure of real advantage and model advantage is set up. State information and control actions in the trajectory are used as data for real advantage calculation, while predicted state information and safety accumulation in the trajectory are used as data for model advantage calculation. All recorded results from multiple simulation experiments are compiled to construct a sampled dataset containing both real advantage calculation data and model advantage calculation data.

[0091] The dual-dominance path structure is used to simultaneously calculate the true dominance value and the model dominance value. The core lies in parallel processing of data from different sources. First, multiple control strategy experiments are run in the simulation environment to record the state information, control actions, immediate rewards, predicted state information, and safety accumulation of the robotic arm at each time step. The data is then divided according to the source characteristics. The state information and control actions are observation data based on real interactions and are used to construct the true dominance path. The predicted state information and safety accumulation are generated by the improved ordinary differential equation model during the simulation and are used to construct the model dominance path. Through this division, two independent but structurally corresponding dominance calculation paths are formed to ensure the integrity and consistency of the sampled data.

[0092] After the data differentiation was completed, the recorded results from multiple simulation experiments were organized according to the time step and simulation round number. The organized results of all simulation rounds were merged to form a sampled dataset containing data from both types of pathways. This dataset serves as the input basis for the comprehensive advantage calculation and provides unified data support for the update of the trust region policy optimization algorithm.

[0093] In this embodiment, the step of weightedly fusing the actual advantage value and the model advantage value to obtain the comprehensive advantage value includes:

[0094] The instant rewards are discounted and accumulated to obtain a corresponding cumulative discount reward sequence. The state value is calculated for each time step, and the cumulative discount reward for each time step is subtracted from the state value to obtain a sequence of true advantage values ​​corresponding to the control actions at each time step. Specifically, the calculation of the state value for each time step involves:

[0095] The sampled running trajectories are organized in chronological order. For each trajectory, the cumulative discounted return sequence is calculated at each time step. Binning rules are established in the state dimension. Discrete intervals are divided according to the value range of each state dimension. The state at each time step is mapped to a unique bin index. For samples in all trajectories that fall into the same bin, the corresponding cumulative discounted return is collected. The average cumulative discounted return of the bin is calculated to obtain the state value estimate of the bin. For bins with few samples or no samples, neighbor bin weighted interpolation is used to complete them. The weights are distributed according to the decreasing distance between the bin centers. The obtained bin state value is then globally smoothed to form a lookup table from bin to state value.

[0096] The state value of each time step is retrieved from the lookup table based on the bin index. If a state happens to be located at the bin boundary, the weighted average of the adjacent bins is used for transition. The weights are distributed according to the distance of the state to the center of the adjacent bins. The state value is obtained in the order of time steps and aligned with the running trajectory.

[0097] The trained improved neural network differential equation model is invoked, and continuous-time integration is performed within a short-time prediction window using the state and control action as initial conditions to obtain a predicted state sequence and a predicted safety cumulative quantity sequence. Based on the predicted state sequence and the predicted safety cumulative quantity sequence, a model evaluation value is obtained. This model evaluation value is then subtracted from a pre-set reference evaluation value for each time step state to obtain a model dominance value sequence corresponding to the control action at each time step. The pre-set reference evaluation value specifically includes:

[0098] To measure the relative merits of the model's predictions, a fixed reference evaluation value needs to be set for the state at each time step. The reference evaluation value is determined by the robotic arm's performance under ideal conditions of no collisions, no constraint violations, and minimal motion energy consumption. First, based on the kinematic and dynamic constraints of the robotic arm, an ideal running trajectory is constructed in the simulation environment. Then, based on the state information and control input corresponding to the trajectory, the ideal energy consumption, joint speed smoothness, and distance error between the end effector and the target point at each time step are calculated. The three are then weighted and summed according to preset weights to obtain the ideal comprehensive index value for each time step. The ideal comprehensive index value is used as the reference evaluation value.

[0099] The true advantage value sequence and the model advantage value sequence at the same time step are linearly combined according to weighting coefficients. The true advantage value is multiplied by the first weighting coefficient, the model advantage value is multiplied by the second weighting coefficient, and the two are summed to obtain the comprehensive advantage value at the time step.

[0100] In this embodiment, obtaining the target control strategy includes:

[0101] Call the safety weight calculation function, taking the robotic arm state information and safety accumulation as input, to calculate the safety weight coefficient corresponding to each time step;

[0102] The comprehensive advantage value of each time step is read, the action probability distribution is calculated according to the current control strategy, a candidate control strategy is constructed based on the current control strategy, the corresponding action probability distribution is calculated, and the KL divergence of the action probability distribution of the current control strategy and the candidate control strategy is calculated to obtain the relative entropy value of each time step. The calculation of the action probability distribution is to obtain the score result of each action dimension through a linear transformation layer, perform exponential operation on the score result and normalize it so that the sum of the probability values ​​corresponding to each action is one, thus forming the action probability distribution.

[0103] The relative entropy value is multiplied by the safety weight coefficient and summed to obtain the weighted relative entropy. The weighted relative entropy is compared with the preset trust region threshold. When the weighted relative entropy meets the trust region threshold condition, the comprehensive advantage value of each time step is multiplied by the action probability distribution of the current control strategy at the time step, and the results of the multiplication are summed to form the optimization objective function. The conjugate gradient method and the line search method are used to determine the parameter update direction and update step size of the control strategy, and the parameters of the control strategy are updated to obtain the target control strategy.

[0104] In this embodiment, obtaining the final control action and driving the robotic arm to execute it includes:

[0105] During the online control of the robotic arm, the current state information of the robotic arm and the obstacle distance field encoding information of the external environment are obtained, the target control strategy is input, and the first control action corresponding to the current state is obtained.

[0106] The current state information of the robotic arm and the first control action are input into the improved neural ordinary differential equation model to obtain the prediction results. The constraint determination results are generated based on the relationship between the prediction results and the joint angle range, joint speed range and obstacle distance constraints.

[0107] When the constraint determination result meets the preset constraint conditions, the first control action is taken as the final control action. When the constraint determination result does not meet the preset constraint conditions, the first control action is adjusted to obtain the second control action, which is then taken as the final control action and sent to the robotic arm servo drive device for execution.

[0108] Example 1:

[0109] To verify the feasibility of this invention in practice, it was applied to an electronic assembly production line of a smart manufacturing equipment company. The production line employs a dual-robotic arm collaborative assembly architecture, with each arm having six degrees of freedom to perform high-precision insertion, positioning, and soldering tasks on densely packed component boards. The assembly area is densely packed with obstacles, including positioning fixtures, testing equipment, and conveying mechanisms, resulting in minimal spatial redundancy. The control scheme proposed in this invention, based on an improved neural network differential equation model and an improved trust region strategy optimization algorithm, was introduced into the actual production line. The aim is to obtain a path control strategy with higher smoothness and executability through deep learning.

[0110] During implementation, a high-frequency sampling module installed in the robotic arm control cabinet was used to collect real-time robotic arm status data at 5-millisecond intervals, including joint angles, angular velocities, angular accelerations, and end-effector poses. A 3D vision system was used to scan the point cloud of obstacles in the assembly work area, with a sampling density of 25 sampling points per cubic centimeter. Distance field encoding was used to interpolate and resample the environmental point cloud, generating an obstacle distance field with a resolution of 0.01 meters. Training and test datasets were constructed according to data acquisition standards, with 120,000 training samples and 30,000 test samples. When training the improved neural network constant differential equation model, an event-triggered piecewise integration mechanism and a piecewise adjoint backpropagation strategy were used for parameter optimization, with training taking approximately 16 hours. After model training, the prediction accuracy was improved by approximately 20% compared to the standard model. After constructing the simulation environment, the trust region threshold was set to 0.03, and the strategy step size was dynamically adjusted using a safety weight calculation function combined with the safety accumulation output. The trust region strategy optimization algorithm converged after 320 iterations, generating a stable target control strategy, which was then deployed in a real system for verification.

[0111] In field testing in August 2025, the production line ran continuously for five days, performing 1,000 high-precision insertion tasks per day, and the operational data was collected for comparative analysis. Experimental verification shows that this invention significantly outperforms existing solutions in terms of path planning feasibility, motion smoothness, and control robustness.

[0112] Table 1. Experimental data comparing the path planning and control performance of the robotic arm.

[0113] Test Project Method of the present invention RRT algorithm PID control Q network control Performance improvement Average path planning time (milliseconds) 61.2 95.3 - - +35.8% Mean attitude deviation (degrees) 0.37 - 0.74 0.58 -50.0% End trajectory error (mm) 0.88 1.9 1.5 1.1 -41.6% Smoothness index (acceleration standard deviation) 0.026 0.045 0.042 0.038 +42.2% Mission success rate (12 obstacles) 99.2% 91.5% 93.4% 95.1% +6.3% Mission success rate (18 obstacles) 97.6% 81.9% 86.8% 89.3% +15.7% Collision count (per 1000 missions) 3 11 8 6 -72.7% Model training time (hours) 16.0 - - 22.5 -28.8% Average energy consumption (joules / cycle) 12.5 15.1 14.7 13.6 -17.2%

[0114] As can be seen from the overall results in Table 1, the deep learning-based robotic arm path planning and control method proposed in this invention outperforms traditional methods in terms of path planning time, control accuracy, and motion smoothness. In the path planning stage, the average planning time of this method is 61.2 milliseconds, which is about 36% less than the traditional RRT algorithm, demonstrating the efficiency of the improved model in state space search and policy generation. Meanwhile, the average end-effector attitude deviation is only 0.37 degrees, and the end-effector trajectory error is reduced to 0.88 millimeters, showing significant improvement compared to traditional PID control and Q-network control schemes. This indicates that the proposed method achieves smoother, more continuous, and physically consistent motion control during execution.

[0115] In terms of stability and safety, this invention demonstrates particularly outstanding performance. The number of collisions is reduced from 11 per 1,000 tasks in traditional methods to 3, a decrease of over 70%. Simultaneously, the task success rate remains at 97.6% even when the number of obstacles increases to 18, representing improvements of approximately 15.7% and 10.8% compared to RRT and PID control schemes, respectively. This indicates that the improved neural ordinary differential equation model can accurately capture safety margin information in dynamic obstacle scenarios and maintain stable control strategy updates through an improved trust region policy optimization algorithm, exhibiting strong robustness and safety redundancy even under complex operating conditions.

[0116] In terms of energy efficiency and scalability, this invention also demonstrates significant advantages. The average energy consumption per task is 12.5 joules, approximately 17% lower than traditional control methods, and the smoothness index is improved by 42%, indicating a more balanced energy distribution in the control actions and less jitter in the robotic arm during high-speed operations. The model training time is 16 hours, nearly 30% shorter than Q-network control, demonstrating the method's high efficiency in model convergence and computational resource utilization. Overall, the data fully demonstrate that this invention possesses the technical characteristics of high precision, high stability, and high efficiency in complex assembly environments, possessing industrial-grade application and promotion value.

[0117] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A deep learning-based path planning and control method for a robotic arm, characterized in that, include: Collect robotic arm status information and obstacle information from the external environment, preprocess the status information, encode the obstacle information using a distance field, and construct training and testing datasets. The improved divine ordinary differential equation model is trained using the training dataset, continuous-time integral calculation is performed to generate a safe cumulative quantity, and the error is calculated with the test dataset. The model parameters are updated according to the error to obtain the trained improved divine ordinary differential equation model. The improved neural ordinary differential equation model after training is used as the core of the dynamics to construct the simulation environment; In the simulation environment, the trust region threshold and safety weight calculation rules are set, the control strategy of the trust region policy optimization algorithm is initialized, and multiple running trajectories are generated in the simulation environment. A sampling dataset is constructed based on the running trajectories. The true advantage value is calculated based on the sampled dataset. Short-term prediction is performed using the trained improved neural network differential equation model to generate the model advantage value. The true advantage value and the model advantage value are then weighted and fused to obtain the comprehensive advantage value. The safety weights are calculated based on the accumulated safety amount. The relative entropy of the control strategy under each state is summed in weights, and a trust region constraint objective function is constructed. Under the condition that the weighted relative entropy meets the threshold, the control strategy is updated using the comprehensive advantage function as the optimization objective and the conjugate gradient method and the line search method to obtain the target control strategy. The target control strategy is applied to the control of the robotic arm. The current state information is input to obtain the control action. The improved neural constant differential equation model after training is used for short-time prediction. The control action is verified and corrected for safety, and the final control action is obtained to drive the robotic arm to execute.

2. The deep learning-based robotic arm path planning and control method according to claim 1, characterized in that, The preprocessing of the state information includes denoising, alignment, normalization, interpolation, and filtering.

3. The deep learning-based robotic arm path planning and control method according to claim 1, characterized in that, The construction of the training and test datasets includes: The robot arm state information under preset working conditions is collected. The robot arm state information includes the joint angles, joint angular velocities, joint angular accelerations, and the pose data of the end effector in a unified coordinate system. The collected robot arm state information is preprocessed to obtain the preprocessed robot arm state sequence. Information on external environmental obstacles within the robotic arm's workspace is collected. This information includes obstacle point cloud data or grid occupancy information. Interpolation and resampling are performed on the 3D grid. Each grid cell is assigned a distance value corresponding to the distance between the grid cell and the nearest obstacle surface, generating an obstacle distance field code. The preprocessed robotic arm state sequence is then paired with the obstacle distance field code, and training and testing datasets are constructed according to sample partitioning rules.

4. The deep learning-based robotic arm path planning and control method according to claim 1, characterized in that, The obtained improved neural frequent differential equation model after training includes: Read the state information and safety-related information of the robotic arm from the training dataset, set the safety accumulation variable according to the state information and safety-related information, combine the state information and safety accumulation variable to form an extended state vector sequence, and divide the extended state vector sequence according to the preset time length and time step to obtain the extended state training sequence for model training. The extended state training sequence is input into the improved neural ordinary differential equation model. A physical consistency submodule is set up to calculate the physical consistency output of the basic dynamic changes of the robotic arm through the dynamic relationship of the robotic arm. A residual compensation submodule is introduced to generate the residual compensation output, and a safety potential field submodule is introduced to generate the safety potential field output. The physical consistency output, residual compensation output and safety potential field output are combined. Set up event functions to describe joint angle limits and robotic arm end contact with obstacles, monitor the process of extended state training sequence changing over time, record the event occurrence time when the event function meets the preset trigger conditions, divide the continuous time axis into multiple sub-time intervals according to the occurrence time of each event, and perform state reset processing on the extended state training sequence at the occurrence time of the event. The initial extended state vector, control input sequence, and improved neural network differential equation model for each sub-time interval are input into the numerical integrator. Continuous time integration is performed according to the set time step to obtain the extended state prediction sequence within the corresponding sub-time interval. The extended state prediction sequence includes the robotic arm state prediction sequence and the safety accumulation prediction sequence. The extended state prediction sequence and the test dataset are time-aligned, and a loss function containing the robotic arm state deviation term and the safety cumulative deviation term is constructed. A corresponding backpropagation calculation process is established for each sub-time interval. Gradient calculation is performed on the improved God ordinary differential equation model in chronological order within each sub-time interval. At the time of the event, the gradient within the sub-time interval is corrected according to the state reset process. The gradient results obtained from all sub-time intervals are accumulated, and the parameters of the physical consistency submodule, residual compensation submodule and safety potential field submodule are updated simultaneously based on the accumulated gradient to complete the training of the improved God ordinary differential equation model.

5. The deep learning-based robotic arm path planning and control method according to claim 1, characterized in that, The simulation environment to be constructed includes: The trained improved neural ordinary differential equation model is loaded as the dynamic core, and the state vector, control input, simulation time step, simulation start state and simulation termination condition of the simulation environment are set. During the simulation, the state vector and control input of the current simulation time step are fed into the improved neural network constant differential equation model. The state vector and safety accumulation of the next simulation time step are calculated through continuous time integration. The simulation time is iteratively updated until the simulation termination condition is met, thus obtaining the running simulation environment.

6. The deep learning-based robotic arm path planning and control method according to claim 1, characterized in that, The construction of the sampling dataset based on the running trajectory includes: In the simulation environment, the trust region threshold and safety weight calculation function are set, and the control strategy of the trust region policy optimization algorithm is initialized. The safety weight calculation function takes the state information of the robotic arm in the simulation environment and the safety accumulation output by the improved neural network constant differential equation model after training as input to obtain the corresponding safety weight coefficient. In the simulation environment, multiple simulation experiments are performed based on the initialization control strategy. In each simulation experiment, starting from the initial state of the simulation, the current state information is input into the control strategy to obtain the control action. The control action is input into the simulation environment for continuous time integration calculation to obtain the predicted state information and the updated safety accumulation amount at the next moment. The state information, control action, immediate reward, predicted state information and safety accumulation amount of each simulation time step are recorded in time sequence to form the corresponding running trajectory. A dual-advantage path structure of real advantage and model advantage is set up. The state information and control actions in the running trajectory are used as the data required for real advantage calculation, and the predicted state information and safety accumulation in the running trajectory are used as the data required for model advantage calculation. All recorded results obtained from multiple simulation experiments are sorted out to construct a sampled dataset containing real advantage calculation data and model advantage calculation data.

7. The deep learning-based robotic arm path planning and control method according to claim 1, characterized in that, The process of weighting and fusing the actual advantage value and the model advantage value to obtain the comprehensive advantage value includes: The instant rewards are discounted and accumulated to obtain the corresponding discount cumulative return sequence. The state value is calculated for the state at each time step. The state value is subtracted from the discount cumulative return at each time step to obtain the real advantage value sequence corresponding to the control action at each time step. The improved neural network differential equation model after training is invoked, and continuous time integration is performed within a short-time prediction window with the state and control action as initial conditions to obtain the predicted state sequence and the predicted safety cumulative quantity sequence. The model evaluation quantity is obtained based on the predicted state sequence and the predicted safety cumulative quantity sequence. The model evaluation quantity is subtracted from the reference evaluation quantity for the state at each time step to obtain the model advantage value sequence corresponding to the control action at each time step. The true advantage value sequence and the model advantage value sequence at the same time step are linearly combined according to weighting coefficients. The true advantage value is multiplied by the first weighting coefficient, the model advantage value is multiplied by the second weighting coefficient, and the two are summed to obtain the comprehensive advantage value at the time step.

8. The deep learning-based robotic arm path planning and control method according to claim 1, characterized in that, The obtained target control strategy includes: Call the safety weight calculation function, taking the robotic arm state information and safety accumulation as input, to calculate the safety weight coefficient corresponding to each time step; Read the comprehensive advantage value of each time step, calculate the action probability distribution according to the current control strategy, construct candidate control strategies based on the current control strategy, calculate the corresponding action probability distribution, and perform KL divergence calculation on the action probability distribution of the current control strategy and candidate control strategies to obtain the relative entropy value of each time step. The relative entropy value is multiplied by the safety weight coefficient and summed to obtain the weighted relative entropy. The weighted relative entropy is compared with the preset trust region threshold. When the weighted relative entropy meets the trust region threshold condition, the comprehensive advantage value of each time step is multiplied by the action probability distribution of the current control strategy at the time step, and the results of the multiplication are summed to form the optimization objective function. The conjugate gradient method and the line search method are used to determine the parameter update direction and update step size of the control strategy, and the parameters of the control strategy are updated to obtain the target control strategy.

9. The deep learning-based robotic arm path planning and control method according to claim 1, characterized in that, The process of obtaining the final control action and driving the robotic arm to execute it includes: During the online control of the robotic arm, the current state information of the robotic arm and the obstacle distance field encoding information of the external environment are obtained, the target control strategy is input, and the first control action corresponding to the current state is obtained. The current state information of the robotic arm and the first control action are input into the improved neural ordinary differential equation model to obtain the prediction results. The constraint determination results are generated based on the relationship between the prediction results and the joint angle range, joint speed range and obstacle distance constraints. When the constraint determination result meets the preset constraint conditions, the first control action is taken as the final control action. When the constraint determination result does not meet the preset constraint conditions, the first control action is adjusted to obtain the second control action, which is then taken as the final control action and sent to the robotic arm servo drive device for execution.

Images