Carbon fiber automatic laying track planning method of robot and robot

Abstract

The invention relates to the technical field of robot motion track planning, and particularly discloses a carbon fiber automatic laying track planning method of a robot and the robot, the SAC algorithm is deeply combined with redundant degree of freedom robot track planning, and offline training is performed through a deep reinforcement learning framework based on the SAC algorithm; the calculation burden of online optimization in the prior art is avoided, and compared with an online optimization mode in the prior art, the planning time can be greatly shortened, and the calculation efficiency is remarkably improved; meanwhile, the decision-making capability of the SAC algorithm is extended to the redundant degree of freedom of the redundant degree of freedom robot by using a null space optimization method, so that the redundant degree of freedom robot can autonomously perform secondary tasks such as obstacle avoidance, singular point avoidance, configuration optimization and the like by using null space motion while completing a main paving task, and the task efficiency is improved. Therefore, the flexibility and the potential of the redundant degree-of-freedom robot are brought into full play, and the overall laying performance is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot motion trajectory planning, and in particular relates to a carbon fiber automatic laying trajectory planning method of a robot. BACKGROUND

[0002] The carbon fiber automatic laying technology is a core process in modern aerospace, automobile and other high-precision manufacturing industries, which completely changes the traditional manual laying of composite materials and realizes high-efficiency, high-precision, high-quality and low-loss automatic production. The existing trajectory planning methods of robots mainly include sampling-based methods (such as RRT, PRM, etc.), optimization-based methods (such as trajectory optimization, model predictive control, etc.) or learning-based methods. Among them, the sampling-based method can handle high-dimensional space, but it is difficult to guarantee the optimality and smoothness of the trajectory; the optimization-based method can obtain a local optimal solution, but it is sensitive to the initial guess and has high computational cost; the learning-based method mostly uses supervised learning, which requires a large amount of expert teaching data and has limited generalization ability. It can be seen that the trajectory planning methods of robots in the prior art face problems such as path optimization difficulty, limited obstacle avoidance capability and unstable laying quality.

[0003] To solve the foregoing problems, the industry hopes to use a redundant degree of freedom robot to complete the carbon fiber automatic laying operation. The redundant degree of freedom robot has more joints than the minimum number of degrees of freedom required to complete the task, and its additional degrees of freedom can be used for obstacle avoidance, optimization of joint configuration, improvement of flexibility and improvement of laying quality, and it has unique advantages in carbon fiber laying applications. However, the kinematics solving non-uniqueness and trajectory optimization calculation complexity brought by the redundant degree of freedom make it difficult for traditional trajectory planning methods based on analytical methods or numerical optimization to meet the efficiency and optimality requirements of complex surface laying. SUMMARY

[0004] The present application aims to solve the technical problems of low trajectory planning calculation efficiency for complex surface laying of carbon fiber and difficulty in effectively handling the redundant degree of freedom in the prior art, and provides a carbon fiber automatic laying trajectory planning method of a robot and a robot for executing the carbon fiber automatic laying trajectory planning method.

[0005] To solve the above technical problems, the present application provides the following technical solutions:

[0006] The carbon fiber automatic laying trajectory planning method of a robot provided by the present application comprises:

[0007] acquire a current state vector s of a redundant degree of freedom robot with n joints, the state vector s including a joint space state, a task state and an environment interaction state; and input the current state vector s into a pre-trained action model to obtain an optimal executable action output by the action model, so that the redundant degree of freedom robot moves based on the optimal executable action;

[0008] The pre-trained action model is obtained by the following method:

[0009] Construct a state space and an action space of the redundant degree of freedom robot;

[0010] A deep reinforcement learning network framework based on SAC algorithm is designed, which includes a policy network, a double Q network and a target network;

[0011] A multi-objective reward function is designed, including a task reward, a smoothness reward, an energy consumption reward and a constraint reward;

[0012] A null space optimization method is used to process the redundant degree of freedom of the redundant degree of freedom robot;

[0013] A hierarchical control architecture is used to realize the mapping from the task space to the joint space;

[0014] Offline training is performed in a simulation environment to obtain the trained action model.

[0015] The carbon fiber automatic laying trajectory planning method of the robot disclosed in the application combines the SAC algorithm and the redundant degree of freedom robot trajectory planning in depth, performs offline training through the deep reinforcement learning framework based on the SAC algorithm, avoids the calculation burden of online optimization in the prior art, can greatly shorten the planning time compared with the online optimization method in the prior art, and the calculation efficiency is significantly improved. At the same time, the SAC algorithm decision-making ability is extended to the redundant degree of freedom of the redundant degree of freedom robot by using the null space optimization method, which enables the redundant degree of freedom robot to complete the main laying task while autonomously performing obstacle avoidance, singularity avoidance, configuration optimization and other secondary tasks using the null space motion, thereby fully utilizing the flexibility and potential of the redundant degree of freedom robot and improving the overall laying performance.

[0016] Further, the n>6;

[0017] And / or, the state vector S of the state space is defined as:

[0018]

[0019] Wherein, is a joint angle vector; is the joint angular velocity vector; is the end effector position; is the end effector pose, represented by a quaternion; is the target layup point position; is the layup surface normal vector; is the contact force and layup pressure.

[0020] Further, the n > 6;

[0021] Further, the action space is defined as the velocity command of joint space: ; and the velocity command of joint space is limited within the maximum joint velocity range by: .

[0022] Further, the policy network comprises:

[0023] Input layer: state vector s with total dimension of 2n+15;

[0024] Hidden layer: multiple fully connected layers with ReLU activation function;

[0025] Output layer: output the mean value μ∈R n and the log standard deviation log σ∈R n .

[0026] Further, the double Q network comprises and , the structure of which comprises:

[0027] Input layer: state and action splicing, dimension of 3n+15;

[0028] Hidden layer: multiple fully connected layers with ReLU activation function;

[0029] Output layer: output the Q value, which is a scalar.

[0030] Further, the target network and is updated by a soft update mechanism: , wherein is the soft update coefficient.

[0031] Further, the multi-objective reward function is:

[0032]

[0033] wherein, is the task reward, .

[0034] is the smoothness reward, wherein is a penalty on joint acceleration, is a penalty on Cartesian space acceleration, is a pseudo-inverse of Jacobian;

[0035] is an energy consumption reward, where τ is a joint torque vector;

[0036] is a constraint reward, wherein is a joint median, is a safety margin;

[0037] , , , is a weight coefficient.

[0038] Further, the null space optimization method is defined as wherein, output by the SAC policy network, for optimizing robot configuration, obstacle avoidance or optimizing operability.

[0039] Further, the hierarchical control architecture comprises:

[0040] a high-level planner: generating a desired Cartesian trajectory from the SAC policy;

[0041] a middle-level controller: performing inverse kinematics solving and null space optimization;

[0042] a bottom-level controller: performing joint PID control or computing torque control.

[0043] The robot comprises a memory and a processor; the memory is used for storing programs or instructions, and the processor is used for executing the programs or instructions to realize the carbon fiber automatic laying trajectory planning method of the robot; and the robot is a redundant degree of freedom robot.

[0044] The robot according to the present application can extend the decision-making ability of the SAC algorithm to the redundant degrees of freedom of the redundant degree of freedom robot by using the null space optimization method, so that the redundant degree of freedom robot can autonomously perform secondary tasks such as obstacle avoidance, singularity avoidance and configuration optimization by using the null space motion while completing the main laying task, thereby fully exerting the flexibility and potential of the redundant degree of freedom robot and improving the overall laying performance. BRIEF DESCRIPTION OF DRAWINGS

[0045] The above and other objects, features and advantages of the present application will become more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which: Like reference numerals in the figures denote like elements, and the figures are not drawn to scale with the emphasis, instead, being placed upon illustrating the principles of the application.

[0046] Figure 1 The flow chart of the carbon fiber automatic laying trajectory planning method of the robot in the present application. DETAILED DESCRIPTION

[0047] For the purpose of facilitating the understanding of the present application, a more comprehensive description of the present application will be given below with reference to the relevant drawings.

[0048] It should be noted that when an element is referred to as being "connected" to another element, it can be directly connected to the other element or can be connected through intervening elements. As used herein the terms "mounted", "one end", "the other end" and the like are used for the purpose of illustration only and are not intended to limit the present application.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0050] In the present application, an embodiment of a carbon fiber automatic laying trajectory planning method of a robot is specifically provided, which is described with reference to Figure 1, comprising: acquiring a current state vector s of a redundant degree of freedom robot with n joints, the state vector s including a joint space state, a task state and an environment interaction state; and inputting the current state vector s to a pre-trained action model to obtain an optimal executable action output by the action model, so that the redundant degree of freedom robot moves based on the optimal executable action; wherein the pre-trained action model is obtained by training by the following method, which can be understood as being completed by offline training: constructing a state space and an action space of the redundant degree of freedom robot, that is, constructing a simulation environment for deep reinforcement learning training; designing a deep reinforcement learning network framework based on a SAC algorithm (Soft Actor-Critic), the deep reinforcement learning network framework based on the SAC algorithm including a policy network, a double Q network and a target network; designing a multi-objective reward function, including a task reward, a smoothness reward, an energy consumption reward and a constraint reward; using a null space optimization method to process the redundant degree of freedom of the redundant degree of freedom robot; using a hierarchical control architecture to realize mapping from a task space to a joint space; and performing offline training in the simulation environment to obtain the pre-trained action model. The SAC algorithm is deeply combined with trajectory planning of the redundant degree of freedom robot, offline training is performed through the deep reinforcement learning framework based on the SAC algorithm, the calculation burden of online optimization in the prior art is avoided, the planning time is greatly shortened compared to the online optimization mode in the prior art, and the calculation efficiency is significantly improved; at the same time, the decision-making ability of the SAC algorithm is extended to the redundant degree of freedom of the redundant degree of freedom robot by using the null space optimization method, which enables the redundant degree of freedom robot to complete the main paving task while autonomously performing secondary tasks such as obstacle avoidance, singularity avoidance and configuration optimization using zero space motion, thereby fully utilizing the flexibility and potential of the redundant degree of freedom robot and improving the overall paving performance.

[0051] In the preferred embodiment, the number of joints n of the redundant degree of freedom robot is greater than 6, which can be seven, eight, nine, ten or even more, and the number of joints n is greater than 6 according to actual conditions, so that the robot has extra space to realize obstacle avoidance, configuration optimization and other secondary tasks.

[0052] In the preferred embodiment, the state vector S of the state space is defined as:

[0053]

[0054] wherein, is a joint angle vector; is a joint angular velocity vector; is an end effector position; is an end effector attitude, represented by a quaternion; is a target paving point position; for laying the surface normal vector; for contact force and laying pressure. Specifically, the joint angle vector and the joint angular velocity vector provide the configuration and motion state of the robot itself, the end effector position and the end effector pose accurately describe the real-time state of the laying head in space, the target laying point position and the laying surface normal vector clearly the current task target and the perception of the geometric features of the surface, so that the robot not only reaches the target position, but also lays with the correct pose, and the contact force and the laying pressure introduce the key force sensation, so that the action model after pre-training can maintain a constant laying pressure, avoid material damage caused by excessive pressure or poor adhesion caused by insufficient pressure, and directly improve the physical quality of laying. The specific composition and fixed dimension (2n+15) of the state vector S enable the deep neural network to learn stably and efficiently. This structured and comprehensive state input is the key to enabling the model to learn complex rules and generalize well, and is also the prerequisite for the subsequent robot to correctly implement automatic carbon fiber laying.

[0055] In the preferred embodiment, the action space is defined as the velocity command of the joint space: Defining the action space as the velocity command of the joint space can directly output the joint velocity command, so that the trained action model can be seamlessly and low-cost deployed to existing commercial robot controllers without the need for complex intermediate conversion layers or modifications to the underlying controller, making it widely applicable. Compared to directly outputting torque commands, outputting velocity commands has lower accuracy requirements for the robot's dynamics model. Even with some model errors (such as load changes and friction uncertainties), the underlying PID or torque controller can still track the velocity command well, enhancing the robustness and ease of use of the entire system. At the same time, the velocity command of the joint space is limited within the maximum joint speed range by the following formula: By limiting the original action output through this formula, the method can be safer when applied in practice, greatly improving the motion quality; the value range of tanh is (-1, +1), meaning that no matter how large the original value output by the SAC policy network is , the final executed action will be strictly limited within the physical range of [ , ], which fundamentally prevents joint speed overshoot, robot damage, or dangerous situations caused by algorithm output anomalies; through the above formula, the robot's action is stable and the motion is smooth during the automatic laying of carbon fiber, effectively ensuring the quality of automatic laying.

[0056] In the preferred embodiment, the policy network includes: an input layer that inputs a state vector s with a total dimension of 2n+15; a hidden layer that uses multiple fully connected layers with ReLU activation functions; and an output layer that outputs the mean value μ of a Gaussian distribution.n Sum of logarithmic standard deviation log σ∈R n Setting the input layer to a state vector s with a total input dimension of 2n+15 ensures lossless information input, guaranteeing that all key information contained in the state vector s can be completely and without omission fed into the policy network for decision-making. It also enables structured perception, meaning that the fixed input dimension forces the state data to be organized and provided according to this unified structure during the data preprocessing and simulation environment construction stages. This is the cornerstone for stable training and reproduction in the reinforcement learning process. The hidden layers are configured as multi-layer fully connected layers, employing the ReLU activation function. This provides powerful function fitting capabilities for handling high-dimensional and complex state-action mapping relationships. Trajectory planning for a carbon fiber laying robot is a typical high-dimensional, highly nonlinear control problem. The interactions between variables in the state space and the optimal action are extremely complex. The multi-layer fully connected network can gradually abstract and fuse these features through multi-layer nonlinear transformations, thereby learning a mapping strategy from complex states to optimal actions. Compared to activation functions like Sigmoid or Tanh, ReLU activation has advantages such as computational simplicity and gradient non-vanishing properties, making the training process of deep networks faster and more stable, effectively handling high-dimensional inputs of 2n+15 dimensions. The output layer is configured to output a Gaussian distribution with mean μ∈R. n Sum of logarithmic standard deviation log σ∈R n This mechanism enables the network to discover a globally better trajectory strategy with a higher probability, rather than getting stuck in a locally optimal, but possibly not the best, solution. This is especially important for the multi-peak solution space of redundant degree-of-freedom robots. After training, during actual deployment, when faced with unseen work scenarios, model uncertainties, or external disturbances, the randomness of the strategy prevents the robot from always responding rigidly to the exact same state. This allows it to adaptively fine-tune its movements, exhibiting natural robustness to disturbances. For example, when the surface being laid has slight unevenness, this random strategy can automatically adjust the joints to maintain stable contact force. Furthermore, the mean μ and log standard deviation log σ of the output layer are both n-dimensional, meaning that each joint has its own mean and standard deviation. This implies that the policy can independently determine the most likely action (mean) and the magnitude of exploration (standard deviation) for each joint. At the same time, since these outputs are calculated by the same network based on the global state, the actions of all joints are highly coordinated, jointly serving a globally optimal goal. The n-dimensional output is directly mapped to the n-dimensional joint velocity commands defined in the action space, forming a closed loop from perception to decision-making to execution, ensuring the integrity and accuracy of the control commands.

[0057] In a preferred embodiment, the dual-Q network includes and Its structure includes: an input layer (3n+15 dimensions, concatenated with state and action); hidden layers (multiple fully connected layers using ReLU activation); and an output layer (outputting the Q-value, a scalar). Through dual estimation and specific network design, it significantly improves the stability of robot motion and the accuracy of value assessment, thereby preventing policy divergence due to overestimation of value and ultimately converging to a better and more reliable trajectory planning strategy. The input layer concatenates state and action dimensions, using both as inputs, enabling the network to accurately assess the long-term value of performing a specific action in a specific state. The direct output of a scalar clearly represents the quality of the state-action pair, providing a stable and reliable gradient direction for the parameter updates of the policy network.

[0058] In a preferred embodiment, the target network and Update via soft update mechanism: ,in The parameters are soft update coefficients. The target network, as a stable objective for Q-value estimation, changes its parameters slowly, effectively avoiding the moving target problem common in training. If a rapidly updating online Q-network is used directly as the objective, the training objective will constantly and drastically change, making the learning process unstable or even divergent. Through the soft update mechanism, the parameters of the target network smoothly and progressively track the parameters of the online network. This slow change provides a relatively fixed objective for the iterative learning of the Q-function in a short time, ensuring that the entire training process can smoothly converge to an optimal or near-optimal strategy.

[0059] In a preferred embodiment, the multi-objective reward function is: By transforming complex engineering tasks into quantifiable optimization objectives and balancing weight coefficients, the SAC algorithm is guided to autonomously learn a deployment trajectory that simultaneously satisfies high precision, high smoothness, low energy consumption, and safety. In this embodiment, As a reward for the task, It can directly drive the robot end effector to accurately reach the target position and posture, which is the core of ensuring the laying accuracy; For smooth reward, ,in To punish joint acceleration, To punish Cartesian space acceleration, As a Jacobi pseudo-inverse, by penalizing the acceleration of joints and Cartesian space, it effectively suppresses motion jitter, generates a smooth trajectory, and thus improves the laying quality and equipment life; As an energy consumption reward, , where τ is the joint torque vector, which can use the product of joint torque and velocity as cost to guide the algorithm to find more energy-efficient motion modes and optimize energy efficiency; To constrain rewards, ,in In the mid-joint position, For safety margin, actions that approach joint limits are penalized, acting as a built-in safety controller to prevent hardware damage and enhance the system's autonomous safety. , , , These are the weighting coefficients.

[0060] In a preferred embodiment, the zero-space optimization method is defined as follows: ,in, Output from the SAC policy network is used to optimize the robotic arm configuration, obstacle avoidance, or maneuverability. By combining the intelligent decision-making capabilities of the SAC algorithm with the redundancy characteristics of robot kinematics, secondary tasks such as obstacle avoidance, singularity avoidance, and configuration optimization are intelligently utilized without interfering with the main deployment task, thereby comprehensively improving the system's performance and intelligence level.

[0061] In a preferred embodiment, during the model training phase, the null projection is used as part of the action space, and its formula is: This design enables robots to autonomously and efficiently learn how to intelligently utilize redundant degrees of freedom, thereby fully realizing the potential of redundant degree-of-freedom robots.

[0062] In a preferred embodiment, the hierarchical control architecture includes: a high-level planner that generates the desired Cartesian trajectory using a SAC strategy; a mid-level controller that performs inverse kinematics solving and null-space optimization; and a low-level controller that executes joint PID control or computes torque control. Understandably, the high-level planner is responsible for complex intelligent decision-making and generating high-quality desired trajectories. Due to its high computational load, its frequency is reduced to ensure the global optimization quality of the plan. The mid-level controller is responsible for high-frequency inverse kinematics solving and null-space optimization, quickly and accurately converting high-level instructions into joint-space instructions and processing redundant degrees of freedom in real time. The low-level controller executes precise joint-level servo control at the highest frequency, ensuring the system can quickly and stably track instructions and cope with various disturbances.

[0063] In a preferred embodiment, a course-based learning strategy is employed during offline training, including: a first stage of planar tiling tasks; a second stage of cylindrical tiling tasks; and a third stage of freeform surface tiling tasks. By progressively increasing task difficulty, learning efficiency and final performance are improved. The resulting action model can be directly deployed on the robot for trajectory planning. During training, an experience replay buffer, batch size, adaptive learning rate, automatically adjusted temperature parameter α, and discount factor γ are used for optimization. The experience replay buffer employs a large capacity buffer and supports a Priority Experience Replay (PER) mechanism. The batch size is dynamically adjusted based on computational resources and task complexity. The policy network and Q-network use adaptive learning rate adjustment. The temperature parameter α is automatically adjusted, and the target entropy is set to... The discount factor γ is set within a reasonable range according to the characteristics of the task; the offline training environment in this invention uses a physical simulator (such as MuJoCo) for parallel training.

[0064] The automatic carbon fiber laying trajectory planning method for robots in this invention is developed based on the standard ROS framework, supports robots from different brands, and can quickly adapt to new robot platforms through transfer learning, greatly shortening the deployment cycle.

[0065] The present invention also provides a specific embodiment of a robot, including a memory and a processor; the memory is used to store programs or instructions, and the processor is used to implement the above-mentioned automatic carbon fiber laying trajectory planning method of the robot when executing the programs or instructions; the robot is a redundant degree of freedom robot.

[0066] The robot in this invention, by executing the aforementioned automatic carbon fiber laying trajectory planning method, can extend the decision-making capability of the SAC algorithm to the redundant degrees of freedom of the redundant degree-of-freedom robot by utilizing the zero-space optimization method. This enables the redundant degree-of-freedom robot to autonomously perform secondary tasks such as obstacle avoidance, singularity avoidance, and configuration optimization using zero-space motion while completing the main laying task, thereby fully leveraging the flexibility and potential of the redundant degree-of-freedom robot and improving the overall laying performance.

[0067] In this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0068] In the description of this specification, the references to terms such as "preferred embodiment," "another embodiment," "other embodiment," or "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0069] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for automatic carbon fiber laying trajectory planning for a robot, characterized in that, include: Obtain the current state vector s of a redundant degree-of-freedom robot with n joints, the state vector s including joint space state, task state and environmental interaction state; and input the current state vector s into a pre-trained action model to obtain the optimal executable action output by the action model, so that the redundant degree-of-freedom robot moves based on the optimal executable action; The pre-trained action model is obtained through the following method: Construct the state space and motion space of a robot with redundant degrees of freedom; Design a deep reinforcement learning network architecture based on the SAC algorithm, wherein the deep reinforcement learning network architecture based on the SAC algorithm includes a policy network, a double Q network, and a target network; Design a multi-objective reward function, including task reward, smoothness reward, energy consumption reward and constraint reward; The redundant degrees of freedom of a robot with redundant degrees of freedom are handled using the null space optimization method. A hierarchical control architecture is adopted to realize the mapping from task space to joint space; Offline training is performed in a simulation environment to obtain the trained action model.

2. The automatic carbon fiber laying trajectory planning method for robots according to claim 1, characterized in that: Where n>6; And / or, the state vector S of the state space is defined as: in, The joint angle vector; This is the joint angular velocity vector; Position of the end effector; The attitude of the end effector is represented by quaternions; Determine the location of the target deployment point; To lay out the surface normal vector; For contact force and laying pressure.

3. The automatic carbon fiber laying trajectory planning method for robots according to claim 1, characterized in that: Where n > 6; And / or, the motion space is defined as the velocity command in the joint space: ; Furthermore, the velocity commands in the joint space are limited to the maximum joint velocity range by the following formula: .

4. The automatic carbon fiber laying trajectory planning method for robots according to claim 2, characterized in that: The policy network includes: Input layer: Input a state vector s with a total dimension of 2n+15; Hidden layers: Multiple fully connected layers using the ReLU activation function; Output layer: Outputs the mean μ ∈ R of a Gaussian distribution. n Sum of logarithmic standard deviation log σ∈R n .

5. The automatic carbon fiber laying trajectory planning method for robots according to claim 4, characterized in that: The dual-Q network includes and Its structure includes: Input layer: State and action concatenated, with a dimension of 3n+15; Hidden layers: Multiple fully connected layers using the ReLU activation function; Output layer: Outputs the Q value, which is a scalar.

6. The automatic carbon fiber laying trajectory planning method for robots according to claim 4, characterized in that: The target network and Update via soft update mechanism: ,in This is the soft update coefficient.

7. The automatic carbon fiber laying trajectory planning method for robots according to claim 1, characterized in that: The multi-objective reward function is: in, As a reward for the task, ; For smooth reward, ,in To punish joint acceleration, To punish Cartesian space acceleration, For Jacobi's false rebellion; As an energy consumption reward, , where τ is the joint torque vector; To constrain rewards, ,in In the mid-joint position, For safety margin; , , , These are the weighting coefficients.

8. The automatic carbon fiber laying trajectory planning method for robots according to claim 1, characterized in that: The zero-space optimization method is defined as follows: ,in, Outputted by the SAC policy network, it is used to optimize the robotic arm configuration, obstacle avoidance, or maneuverability.

9. The automatic carbon fiber laying trajectory planning method for robots according to claim 1, characterized in that: The hierarchical control architecture includes: High-level planner: Generates the desired Cartesian trajectory using the SAC policy; Mid-level controller: performs inverse kinematics solving and null space optimization; Low-level controller: performs joint PID control or calculates torque control.

10. A robot comprising a memory and a processor; said memory for storing programs or instructions, characterized in that: The processor is used to implement the automatic carbon fiber laying trajectory planning method for the robot as described in any one of claims 1-9 when executing the program or instructions; the robot is a redundant degree-of-freedom robot.

Images