Wafer transfer robot control method, system and device

Abstract

The application provides a wafer transfer robot control method, system and device, and relates to the technical field of automatic control of semiconductor manufacturing equipment. The method comprises the following steps: acquiring joint position and joint angular velocity of a robot; calculating joint position tracking error; inputting the joint position and joint angular velocity into a model identifier to obtain a compensation amount for the uncertainty of a dynamic model of the robot; using an adaptive dynamic programming framework to generate an optimal control strategy in combination with the joint position tracking error, a performance index function and the compensation amount, wherein the compensation amount is used to correct a preliminary control strategy output by the framework; and finally outputting a control torque drive robot motion. Through the cooperation of the model identifier and the adaptive dynamic programming framework, the application effectively reduces the dependence on an accurate mathematical model, can learn and optimize the long-term comprehensive performance of the robot in terms of speed, accuracy and energy consumption online, and improves the efficiency of wafer transfer and the adaptability of the device.

Description

Technical Field

[0001] This invention relates to the field of automation control technology for semiconductor manufacturing equipment, and in particular to a control method, system and equipment for a wafer transfer robotic arm. Background Technology

[0002] Wafer transfer robotic arms are core components of semiconductor manufacturing equipment such as lithography machines, and their control performance directly determines the throughput and wafer processing yield of the equipment.

[0003] Currently, control technologies in this field mainly rely on classical and modern control theories, but significant limitations remain. First, they heavily depend on precise mathematical models. Whether it's classical proportional-integral-derivative (PID) control or modern model predictive control (MPC), both suffer from performance degradation due to model mismatch when faced with the complex nonlinear and time-varying characteristics of robotic arms, as well as performance degradation from long-term operation. This makes it difficult to balance high speed and stability. Second, the control objectives are singular and lack long-term optimization capabilities. Existing methods (such as adaptive control) primarily ensure stability and error convergence, making it difficult to achieve long-term optimal trade-offs among multiple competing objectives such as time, accuracy, and energy consumption.

[0004] Therefore, there is an urgent need for a control scheme that can break free from dependence on accurate models, has online learning capabilities, and can optimize long-term overall performance. Summary of the Invention

[0005] This invention provides a wafer transfer robotic arm control method, system, and device to address the shortcomings of existing wafer transfer robotic arm control systems, which rely too heavily on precise mathematical models and struggle to achieve a long-term optimal trade-off between multiple objectives such as high speed, high precision, and low energy consumption.

[0006] This invention provides a control method for a wafer transfer robotic arm. The wafer transfer robotic arm includes multiple joints and a wafer transfer actuator for transferring wafers in a lithography machine. The method includes the following steps: acquiring the current state information of the wafer transfer robotic arm, including joint positions and joint angular velocities; calculating the joint position tracking error based on the current state information, where the joint position tracking error is the difference between the current joint position and the desired joint position; inputting the current state information into a model identifier to obtain a compensation amount for the uncertainty of the dynamic model of the wafer transfer robotic arm; using an adaptive dynamic programming (ADP) framework, generating an optimal control strategy based on the joint position tracking error, a predefined performance index function, and the compensation amount, where the performance index function is used to optimize the long-term comprehensive performance of the wafer transfer robotic arm, and the compensation amount is used to correct the initial control strategy output by the adaptive dynamic programming framework; and outputting a control torque based on the optimal control strategy to control the movement of the wafer transfer robotic arm.

[0007] According to the wafer transfer robotic arm control method provided by the present invention, the aforementioned model identifier is used to estimate the difference between the dynamic model and the actual dynamics. The dynamic model is a mathematical model describing the dynamic behavior of the wafer transfer robotic arm, and the actual dynamics are the dynamic behavior of the wafer transfer robotic arm in actual operation.

[0008] According to the wafer transfer robotic arm control method provided by the present invention, the aforementioned model identifier is a neural network model identifier. The method of inputting current state information into the model identifier to obtain the compensation amount for the uncertainty of the dynamic model of the wafer transfer robotic arm includes: using the neural network model identifier to estimate the model uncertainty term of the wafer transfer robotic arm based on the current state information, wherein the model uncertainty term refers to the dynamic characteristics not considered in the dynamic model; and calculating and outputting the compensation amount based on the model uncertainty term.

[0009] According to the wafer transfer robotic arm control method provided by the present invention, the aforementioned adaptive dynamic programming framework includes an evaluation neural network and an execution neural network. The optimal control strategy is generated through multiple iterations. The iteration steps include: the evaluation neural network assesses the long-term comprehensive performance of the wafer transfer robotic arm based on the current state information, joint position tracking error, and the current control strategy, and outputs a value function estimate; the execution neural network outputs an iterative control strategy based on the value function estimate; the iterative control strategy is applied to the wafer transfer robotic arm to generate iterative state information and iterative joint position tracking error; based on the iterative state information and iterative joint position tracking error, the weights of the evaluation neural network and the execution neural network are updated simultaneously, so that the value function estimate and iterative control strategy generated in the next iteration cycle are better.

[0010] According to the wafer transfer robotic arm control method provided by the present invention, the weight update process of the above-mentioned evaluation neural network and execution neural network is designed based on Lyapunov stability theory to ensure that the joint position tracking error is ultimately uniformly bounded (UUB).

[0011] According to the wafer transfer robotic arm control method provided by the present invention, the above-mentioned performance index function is a long-term cumulative reward function constructed based on joint position tracking error and utility function; wherein, the long-term cumulative reward function is used to optimize the long-term comprehensive performance of the wafer transfer robotic arm in terms of speed, accuracy and energy consumption.

[0012] According to the wafer transfer robotic arm control method provided by the present invention, the above-mentioned performance index function is: ,in, For performance index functions, For utility function, This is the error fusion function, whose elements include the joint position tracking error and its derivative. For the control strategy, the elements of its output vector are the control torques, and t is the current time.

[0013] According to the wafer transfer robotic arm control method provided by the present invention, the above-mentioned acquisition of the current state information of the wafer transfer robotic arm includes: acquiring the original state signal of the wafer transfer robotic arm; filtering the original state signal and outputting the current state information.

[0014] This invention also provides a wafer transfer robotic arm control system, comprising the following modules: an acquisition module for acquiring the current state information of the wafer transfer robotic arm, the current state information including joint positions and joint angular velocities; a calculation module for calculating the joint position tracking error based on the current state information, the joint position tracking error being the difference between the current joint position and the desired joint position; a generation module for: inputting the current state information into a model identifier to obtain a compensation amount for the uncertainty of the dynamic model of the wafer transfer robotic arm; using an adaptive dynamic programming framework, generating an optimal control strategy based on the joint position tracking error, a predefined performance index function, and the compensation amount, the performance index function being used to optimize the long-term comprehensive performance of the wafer transfer robotic arm, and the compensation amount being used to correct the initial control strategy output by the adaptive dynamic programming framework; and a control module for outputting a control torque based on the optimal control strategy to control the movement of the wafer transfer robotic arm.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the wafer transfer robotic arm control method as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the wafer transfer robotic arm control method as described above.

[0017] The wafer transfer robotic arm control method, system, and device provided by this invention reduce the dependence on precise mathematical models and enhance the robustness of the control system to parameter changes and unmodeled dynamics by introducing a model identifier to estimate and compensate for uncertainties in the dynamic model online. Based on an adaptive dynamic programming framework and a long-term performance index function, it achieves automatic trade-offs and long-term comprehensive optimization among multiple performance indicators such as speed, accuracy, and energy consumption. The entire system has online learning and self-optimization capabilities, enabling it to adapt to performance degradation and load changes during long-term operation of the robotic arm, maintaining optimal performance throughout its entire lifecycle. The weight update process designed using Lyapunov stability theory ensures the stability and reliability of the control system. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts illustrating a wafer transfer robotic arm control method provided by the present invention.

[0020] Figure 2 This is the second flowchart of a wafer transfer robotic arm control method provided by the present invention.

[0021] Figure 3 This is a block diagram of a control strategy for a wafer transfer robotic arm based on adaptive dynamic programming, provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the structure of a wafer transfer robotic arm control system provided by the present invention.

[0023] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] The following is combined with Figures 1 to 3 The present invention describes a wafer transfer robotic arm control method.

[0026] Figure 1 This is one of the flowcharts illustrating a wafer transfer robotic arm control method provided by the present invention, such as... Figure 1 As shown, the method includes the following:

[0027] S101. Obtain the current status information of the wafer transfer robotic arm.

[0028] In this embodiment of the application, the wafer transfer robotic arm includes multiple joints and a wafer transfer execution device for picking up, placing, and transferring wafers.

[0029] In practical applications, wafer transfer robotic arms need to achieve high-speed, high-precision wafer handling while ensuring smooth movement to avoid damaging the wafers.

[0030] For example, a joint refers to the rotatable part of a robotic arm, and each joint typically consists of a motor, a reducer, and an encoder, used to enable the spatial movement of the wafer transfer actuator.

[0031] Specifically, a three-link wafer transfer robotic arm typically includes three main rotary joints: the base joint, the shoulder joint, and the elbow joint.

[0032] For example, a wafer transfer actuator refers to the end effector of a robotic arm, typically employing a mechanical device or a vacuum adsorption device, for gripping, releasing, and transferring wafers.

[0033] Specifically, the design of wafer transfer actuators needs to meet the requirements of being pollution-free and highly reliable, ensuring that no contamination or damage is caused to the wafer surface during the transfer process.

[0034] In this embodiment of the application, the current state information includes joint position and joint angular velocity.

[0035] Among them, joint position refers to the angular displacement of each joint of the robotic arm, expressed in angle values, and is usually directly measured by an encoder installed at the joint; joint angular velocity refers to the angular velocity of each joint of the robotic arm, expressed in angle change rate, and can be obtained by differential calculation of the position signal or by direct measurement by a tachogenerator.

[0036] In some embodiments, the original state signal of the wafer transfer robot arm can be acquired first, and then the original state signal can be filtered to output the current state information.

[0037] In the embodiments of this application, the original state signal may contain noise and interference. By filtering the original state signal, these noises can be filtered out, providing a more accurate state estimate.

[0038] In some embodiments, the kinematic model of a wafer transfer robot arm can be established using the Denavit-Hartenberg (DH) parameter method. The DH parameter method is a standard robot kinematic modeling method that uses four parameters (link length, link twist angle, joint distance, and joint angle) to describe the spatial relationship between adjacent links.

[0039] For example, consider a wafer transfer robotic arm with three links. The links are l1, l2, and l3, and the distance between the wafer transfer actuator and the origin along the Y-axis is d1. The joint angles of the three rotary joints are θ2, θ3, and θ4, and the lateral distance between the first rotary joint and the origin is θ1. According to the DH parameter table, the coordinate transformation matrix between the robotic arm joints is:

[0040] ,

[0041] ,

[0042] ,

[0043] ,

[0044] .

[0045] Where A1 to A5 are transformation matrices between adjacent coordinate systems, and l1, l2, and l3 are the link lengths.

[0046] For example, for a typical wafer transfer robotic arm, the link length may be in the range of 200-500 mm, and the range of motion of the joint angle is usually ±180 degrees.

[0047] For example, multiplying the above transformation matrices yields the pose matrix of the wafer transfer actuator relative to the base coordinates:

[0048] .

[0049] Among them, R θ It is a rotation matrix that describes the orientation of the wafer transfer actuator; P θ It is a position vector that describes the position of the wafer transfer actuator:

[0050] ,

[0051] .

[0052] In practical applications, it is also necessary to consider the definition and transformation of coordinate systems, as well as the consistency of units (such as using radians to represent angles and millimeters to represent lengths).

[0053] Thus, by filtering the original state signal, more accurate current state information can be obtained for subsequent control.

[0054] S102. Calculate the joint position tracking error based on the current state information.

[0055] In this embodiment, the joint position tracking error is the difference between the current joint position and the desired joint position. The joint position tracking error reflects the deviation between the actual motion and the desired trajectory.

[0056] The desired joint position refers to the ideal joint angle trajectory planned according to the wafer transfer task. The generation of the desired trajectory needs to take into account the specific requirements of wafer transfer, such as the starting point, target point, path constraints, and movement speed.

[0057] In practical applications, the desired trajectory is usually generated by trajectory planning algorithms to ensure that the motion process is smooth and shock-free.

[0058] For example, for the i-th arm of a robotic arm, the joint position tracking error can be defined as:

[0059] .

[0060] in, It is the current joint position of the i-th joint. It is the expected joint position of the i-th joint.

[0061] Specifically, since the motion trajectory of the wafer transfer robotic arm can be simplified to planar motion, the joint angles can be solved using planar geometry. During wafer transfer, the robotic arm mainly moves within a plane, with fine-tuning in the vertical direction only performed after positioning is complete. Let the position of the wafer transfer actuator be (x0, y0), then the distance D = The included angles α and β can be calculated using the law of cosines:

[0062] .

[0063] Obtain the joint angle:

[0064] , , .

[0065] Where γ is the angle between the third link and the X-axis. The joint angle is used to calculate the tracking error.

[0066] In actual calculations, issues such as quadrant determination and angle normalization need to be considered.

[0067] For example, the arctan function is often replaced by the atan2 function to avoid quadrant determination errors. Angle values ​​are normalized to the range [-π, π] to ensure the stability of numerical calculations.

[0068] S103. Input the current state information into the model identifier to obtain the compensation amount for the uncertainty of the dynamic model of the wafer transfer robot arm.

[0069] In this embodiment, the model identifier is used to estimate the difference between the dynamic model and the actual dynamics of the wafer transfer robot arm. By estimating these differences online, the control system can adjust its control strategy in a timely manner to adapt to system changes.

[0070] Among them, the dynamic model is a mathematical model describing the dynamic behavior of the wafer transfer robot arm, and the actual dynamics are the dynamic behavior of the wafer transfer robot arm in actual operation.

[0071] For example, the model identifier is based on a neural network to identify the uncertainty terms of the dynamic model. and Compensation will be provided.

[0072] In this embodiment, the compensation amount is calculated based on the output of the neural network, reflecting the magnitude of the model error in the current state.

[0073] Alternatively, the model identifier can be a neural network model identifier.

[0074] In some embodiments, a neural network model identifier can be used to estimate the model uncertainty term of the wafer transfer robot based on the current state information, and then calculate and output the compensation amount based on the model uncertainty term.

[0075] For example, the estimated value of the model identifier is:

[0076] , .

[0077] in, Nonlinear functions in the dynamic model of the robotic arm The estimate; The control input function in the robotic arm dynamics model The estimate; It is the current joint position of the i-th joint; It is the joint angular velocity of the i-th joint; It is the control torque applied to the i-th joint; and It is the activation function of the neural network, and the Gaussian function or the Sigmoid function is usually chosen. and It is the online weight estimation vector of the neural network in the model identifier, obtained through online learning.

[0078] Specifically, the weight update law is as follows:

[0079] , .

[0080] in, and It is the derivative of the online weight estimation vector with respect to time, i.e., the weight update law; and This is the learning rate parameter, which controls the speed at which the weights are updated; here It is the weight update gain, used to adjust the magnitude of the weight update law.

[0081] In this embodiment, the weight update law is designed based on Lyapunov stability theory to ensure the convergence of the estimation error. The learning rate parameter needs to be appropriately selected; too large a parameter will cause system oscillations, while too small a parameter will result in slow convergence.

[0082] In this way, the model identifier estimates uncertainty online and provides compensation. The introduction of compensation enhances the robustness of the control system, enabling it to cope with various uncertainties.

[0083] S104. Using an adaptive dynamic programming framework, the optimal control strategy is generated based on the joint position tracking error, the predefined performance index function, and the compensation amount.

[0084] Among them, the performance index function is used to optimize the long-term comprehensive performance of the wafer transfer robot, and the compensation amount is used to correct the initial control strategy output by the adaptive dynamic programming framework.

[0085] In this embodiment, the ADP framework is an intelligent control method that combines adaptive control and dynamic programming. It can learn the optimal control strategy online and is suitable for complex nonlinear systems. The core idea of ​​the ADP framework is to approximate the optimal solution of dynamic programming through iterative learning, avoiding the difficulty of directly solving the Hamilton-Jacobi-Bellman (HJB) equations.

[0086] The HJB equation is a fundamental equation in optimal control theory, describing the conditions that the optimal value function must satisfy. For continuous-time systems, the HJB equation is a partial differential equation.

[0087] In this embodiment of the application, the ADP framework can be composed of a critic neural network and an actor neural network as its core components.

[0088] The evaluation neural network is used to evaluate the performance of the current strategy, while the execution neural network is used to generate control actions.

[0089] For example, a judgment neural network refers to a neural network used to approximate a performance index function. It typically adopts a feedforward neural network structure, with the system state as the input and the value function estimate as the output.

[0090] For example, the execution neural network refers to a neural network used to generate control strategies, which adopts a feedforward neural network structure, with the system state as the input and the control action as the output.

[0091] For example, the ADP framework solves the optimal control problem through iterative learning, and its core is the HJB equation.

[0092] In this way, the ADP framework can learn and optimize control strategies online to adapt to system changes. This adaptive characteristic enables the control system to cope with uncertainties such as changes in the robot arm's dynamic parameters and external disturbances.

[0093] In this embodiment, the performance index function is a long-term cumulative reward function constructed based on the joint position tracking error and the utility function. The design of the performance index function directly affects the performance of the control system and requires comprehensive consideration of multiple objectives such as tracking accuracy, control energy consumption, and response speed.

[0094] The long-term cumulative reward function is used to optimize the long-term overall performance of the wafer transfer robot arm in terms of speed, accuracy, and energy consumption. Multi-objective optimization ensures that the robot arm can complete the transfer task while taking into account both efficiency and economy.

[0095] For example, the performance metric function is defined as follows: .

[0096] in, For performance index functions, It is a utility function, with an initial state N(0, 0) = 0. This is the error fusion function, where k1 and k2 are the error fusion gain coefficients, which need to be adjusted according to the specific application. It is a joint position tracking error. It is the joint angular velocity tracking error; For the control strategy, the elements of its output vector are the control torques, and t is the current time.

[0097] For example, the utility function typically takes N( )= .

[0098] Here, Q and R are positive definite weight matrices used to balance tracking error and control energy consumption. The Q matrix is ​​usually a diagonal matrix, with diagonal elements representing the degree of importance given to different error components. The R matrix is ​​also a diagonal matrix, with diagonal elements representing the weights for controlling energy consumption.

[0099] In this way, the performance index function transforms the control objective into a long-term optimization problem, ensuring that the control strategy not only considers instantaneous performance but also focuses on long-term comprehensive performance.

[0100] In this embodiment, the optimal control strategy is generated through multiple iterative processes. The number of iterations depends on the complexity of the system and the required convergence accuracy.

[0101] In practical applications, it is usually necessary to balance computational complexity and control performance.

[0102] For example, such as Figure 2 As shown, the iterative steps include:

[0103] S104a: The evaluation neural network evaluates the long-term comprehensive performance of the wafer transfer robot arm based on the current state information, joint position tracking error and current control strategy, and outputs an estimated value function.

[0104] For example, the performance evaluation function for neural network approximation Its form is .

[0105] Among them, X k It is an ideal weight vector. It is an activation function. It is an approximation error.

[0106] In practice, estimated values ​​are used. .

[0107] in, For performance index functions The estimated value; This is an estimate of the ideal weight vector.

[0108] For example, the evaluation neural network is usually a radial basis function (RBF) neural network or a multilayer perceptron (MLP).

[0109] Specifically, the value function estimate is obtained by solving the HJB equation. The HJB equation is defined as follows:

[0110]

[0111] .

[0112] in, It is the Hamiltonian function; It is a performance index function gradient, , It is the time derivative of the error fusion function. ; It is an auxiliary stabilization term, used to introduce the desired acceleration and error derivative to enhance the system's stability and tracking performance. ; and It is an uncertainty term in the dynamic model. It is the expected joint angular acceleration.

[0113] Thus, the value function estimate output by the evaluation neural network is used to assess the long-term performance of the current policy, and the accuracy of the value function estimate directly affects the optimization effect of the control policy.

[0114] S104b: The execution neural network outputs an iterative control strategy based on the estimated value function.

[0115] For example, the neural network generation control strategy is executed. The optimal control law is:

[0116] .

[0117] in, For the optimal control law, the elements of its output vector are the control torques; It is the optimal performance index function The gradient.

[0118] In this embodiment, the structure of the execution neural network is similar to that of the evaluation neural network, but the output layer typically uses a linear activation function to directly output the control quantity. The range of the control quantity needs to be constrained by the physical limitations of the actuator, such as the maximum limit of the motor torque.

[0119] Specifically, due to model uncertainty, after compensation using a model identifier, the final control strategy is adjusted as follows:

[0120] .

[0121] in, This is the approximate optimal control strategy after compensation; Here, the coefficients are constants; To control the gain, it is used to scale the amplitude of the control output; R is the inverse of the positive definite weight matrix; It is the activation function vector The transpose of the gradient; yes The transpose of the estimated value, It is an estimate of the ideal weight vector.

[0122] In the embodiments of this application, the introduction of a model identifier enhances the robustness of the control system to uncertainties. By estimating dynamic parameters online, the control system can adapt to long-term performance degradation such as changes in the robot arm's load and component wear.

[0123] Thus, the neural network outputs an iterative control strategy to drive the robotic arm. The generation of the control strategy needs to meet real-time requirements to ensure the response speed of the control system.

[0124] S104c: Apply the iterative control strategy to the wafer transfer robot arm to generate iterative state information and iterative joint position tracking error.

[0125] In this embodiment, an iterative control strategy is used to output torque to control the movement of the wafer transfer robotic arm, thereby generating the iteration state information and iteration joint position tracking error.

[0126] S104d, based on iterative state information and iterative joint position tracking error, simultaneously updates the weights of the evaluation neural network and the execution neural network, making the value function estimate and iterative control strategy generated in the next iteration cycle better.

[0127] For example, the weight update strategy is based on the error of the approximate HJB equation, and the approximate HJB equation error is defined as follows:

[0128] .

[0129] in, The error is due to the approximation of the Hamilton-Jacobi-Bellman equation; The time derivative of the error fusion function; It is an abbreviation for approximate HJB error.

[0130] The weight update law is:

[0131] .

[0132] in, The ideal weight vector estimate The time derivative; Let the regression vector be defined as follows: , These are the update law parameters.

[0133] In this embodiment, the weight update law is designed based on the gradient descent method, which optimizes the network weights by minimizing the error of the HJB equation.

[0134] It should be noted that the update law parameters An appropriate value needs to be chosen; too large a value may lead to system instability, while too small a value will result in slow convergence.

[0135] Specifically, the dynamics of the weight approximation error are as follows:

[0136] .

[0137] in, It is the weight estimation error vector The time derivative; To update the law parameters; It is the approximation error of the neural network; It is the weight estimation error vector The transpose of ; ξ is the correlation vector.

[0138] In the embodiments of this application, the weight update process of the evaluation neural network and the execution neural network needs to ensure convergence and stability.

[0139] For example, the weight update process can be designed according to Lyapunov stability theory to minimize the joint position tracking error UUB.

[0140] UUB is an important concept in the stability analysis of control systems, representing the state of the system entering and remaining within a bounded range within a finite time.

[0141] Specifically, we choose the Lyapunov function:

[0142] .

[0143] in, It is a Lyapunov function; The optimal performance index function; , It is a weighting error; and It is the ideal weight vector of the neural network in the model identifier; and It is the online weight estimation vector of the neural network in the model identifier.

[0144] Furthermore, through analysis , It is a Lyapunov function The time derivative is used to prove the joint position tracking error UUB.

[0145] In the embodiments of this application, the construction of the Lyapunov function comprehensively considers the tracking error, the value function estimation error, and the model estimation error, reflecting the dynamic characteristics of the system.

[0146] Optionally, the learning process of the ADP framework can be performed offline or online. Offline learning requires a large amount of training data, but the control response is fast after learning is completed; online learning can adapt to changes in the environment, but the computational burden is heavier.

[0147] Through multiple iterations, the ADP framework generates the optimal control strategy, achieving long-term performance optimization. The optimal control strategy can automatically find the best balance among multiple objectives such as speed, accuracy, and energy consumption, achieving comprehensive performance improvements that are difficult to achieve with traditional methods.

[0148] S105. Based on the optimal control strategy, output control torque to control the movement of the wafer transfer robotic arm.

[0149] In this embodiment, the control torque is a physical quantity that acts directly on the joint actuator of the robotic arm, and is achieved through motor torque. The calculation of the control torque needs to take into account the dynamic characteristics and physical limitations of the actuator.

[0150] For example, the control torque u i The joint actuators, acting directly on the robotic arm, enable the robotic arm to track the desired trajectory and control the torque u. i It is calculated from the optimal control strategy. The joint actuator typically uses a servo motor, in conjunction with a high-precision reducer, to provide the required torque and speed.

[0151] Specifically, the dynamic model of the robotic arm is as follows:

[0152] .

[0153] in, It is the inertia matrix, which describes the inertial characteristics of the system; Joint position The second derivative with respect to time represents the joint angular acceleration; It is a matrix of Coriolis force and centrifugal force, describing the effects of Coriolis force and centrifugal force; It is the joint angular velocity; It is the gravity term, which describes the effect of gravity on the system.

[0154] In practical applications, due to variations in the configuration of wafer transfer robotic arms, obtaining accurate dynamic models is difficult. This application reduces the reliance on accurate models through a model identifier and an adaptive control strategy.

[0155] In this way, the robotic arm achieves precise and smooth movement under the control torque. Evaluation indicators of motion performance include positioning accuracy, repeatability, and motion stability.

[0156] Thus, the optimal control strategy, aided by the model identifier, achieves robust control. Experimental results demonstrate that this control strategy can significantly improve the control accuracy and adaptability of the wafer transfer robotic arm.

[0157] Figure 3 This is a block diagram of the control strategy for wafer transfer robotic arm based on adaptive dynamic programming provided by the present invention. The block diagram shows the overall architecture, core components and signal flow of the control system of the present invention.

[0158] like Figure 3 As shown, the entire system achieves a near-optimal control strategy. The closed-loop generation and updating of the policy, specifically, the strategy iteration process generates the optimal control law. After compensation by the model identifier, the control law generates the final approximate optimal control strategy. This constitutes the system's comprehensive control input.

[0159] For example, the control input acts on the controlled object: the wafer transfer robotic arm, whose dynamics are determined by... Description. The system receives a combined input from two parallel control signals: one is the optimal control law. Secondly, it is based on a neural network model identifier for approximate optimal control strategy. .

[0160] For example, the current joint position output by the robotic arm and joint angular velocity , and the desired joint position of the system Desired joint angular velocity At the same time, it is transmitted to the downstream error fusion function unit.

[0161] also, and It is transmitted separately to a neural network model identifier for online learning and to approximate the actual dynamic characteristics of the robotic arm.

[0162] For example, the joint positions of the robotic arm can be received based on a neural network model identifier. and joint angular velocity and control torque This involves online estimation of uncertainties in the dynamics model of a robotic arm. Its mathematical expression is:

[0163] , .

[0164] The output of the neural network model identifier is used to compensate for model uncertainty and improve the robustness of control.

[0165] Furthermore, the output and performance index function of the error fusion function unit Establish a connection. This performance metric function. Connected to the evaluation neural network, the network evaluates the performance index function. By performing function approximation, we obtain , The system is used to solve the approximate HJB equation. Based on the solution, an update law is derived. This update law drives the strategy iteration, ultimately generating and outputting an approximately optimal control strategy for the wafer transfer robot based on adaptive dynamic programming. This completes a full adaptive learning and control closed loop.

[0166] The evaluation neural network serves to approximate the performance index function and is evaluated using an approximate HJB equation. The update law is used to continuously adjust the neural network parameters based on the evaluation results, thereby progressively optimizing the strategy.

[0167] Thus, by deeply integrating the ADP framework with the neural network model identifier, an intelligent control system capable of online learning and self-optimization was constructed, effectively solving the problem of dependence on accurate models and achieving optimization of the long-term comprehensive performance of the wafer transfer robotic arm.

[0168] The wafer transfer robotic arm control system provided by the present invention is described below. The wafer transfer robotic arm control system described below can be referred to in correspondence with the wafer transfer robotic arm control method described above.

[0169] Figure 4 This is a structural diagram of a wafer transfer robotic arm control system provided in an embodiment of this application. The wafer transfer robotic arm control system includes: an acquisition module 401, a calculation module 402, a generation module 403, and a control module 404.

[0170] The system comprises: an acquisition module 401, used to acquire the current state information of the wafer transfer robot arm, including joint position and joint angular velocity; a calculation module 402, used to calculate the joint position tracking error based on the current state information, where the joint position tracking error is the difference between the current joint position and the desired joint position; a generation module 403, used to: input the current state information into the model identifier to obtain the compensation amount for the uncertainty of the dynamic model of the wafer transfer robot arm; and use an adaptive dynamic programming framework to generate an optimal control strategy based on the joint position tracking error, a predefined performance index function, and the compensation amount, where the performance index function is used to optimize the long-term comprehensive performance of the wafer transfer robot arm, and the compensation amount is used to correct the initial control strategy output by the adaptive dynamic programming framework; and a control module 404, used to output a control torque based on the optimal control strategy to control the motion of the wafer transfer robot arm.

[0171] In some embodiments, the model identifier is a neural network model identifier, and the generation module 403 is specifically used to: use the neural network model identifier to estimate the model uncertainty term of the wafer transfer robot based on the current state information, wherein the model uncertainty term refers to the dynamic characteristics not considered in the dynamic model; and calculate and output the compensation amount based on the model uncertainty term.

[0172] In some embodiments, the ADP framework includes an evaluation neural network and an execution neural network. The generation module 403 is specifically used for: evaluating the long-term comprehensive performance of the wafer transfer robot arm based on the current state information, joint position tracking error, and current control strategy, and outputting a value function estimate; outputting an iterative control strategy based on the value function estimate; applying the iterative control strategy to the wafer transfer robot arm to generate iterative state information and iterative joint position tracking error; and updating the weights of the evaluation neural network and the execution neural network based on the iterative state information and iterative joint position tracking error, so that the value function estimate and iterative control strategy generated in the next iteration cycle are better.

[0173] In some embodiments, the acquisition module 401 is specifically used to: acquire the original state signal of the wafer transfer robot arm; filter the original state signal and output the current state information.

[0174] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540. The processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logic instructions in the memory 530 to execute a wafer transfer robotic arm control method. This method includes: acquiring the joint positions and angular velocities of the robotic arm; calculating the joint position tracking error; inputting the joint positions and angular velocities into a model identifier to obtain compensation for the uncertainty of the robotic arm's dynamic model; using an adaptive dynamic programming framework, combining the joint position tracking error, performance index function, and compensation, to generate an optimal control strategy, wherein the compensation is used to correct the initial control strategy output by the framework; and finally outputting a control torque to drive the robotic arm's movement.

[0175] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0176] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the wafer transfer robotic arm control method provided by the above methods. The method includes: acquiring the joint position and joint angular velocity of the robotic arm; calculating the joint position tracking error; inputting the joint position and joint angular velocity into a model identifier to obtain a compensation amount for the uncertainty of the dynamic model of the robotic arm; using an adaptive dynamic programming framework, combining the joint position tracking error, performance index function and compensation amount to generate an optimal control strategy, wherein the compensation amount is used to correct the initial control strategy output by the framework; and finally outputting a control torque to drive the robotic arm to move.

[0177] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0178] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A control method for a wafer transfer robotic arm, characterized in that, The wafer transfer robotic arm is a three-bar linkage, and its kinematic model is established using the DH parameter method. The robotic arm includes a wafer transfer actuator for picking up, placing, and transferring wafers. The method includes: Multiplying the transformation matrices A1, A2, A3, A4, and A5 between multiple adjacent coordinate systems yields the pose matrix of the wafer transfer actuator relative to the base coordinates in the kinematic model. ; Obtain the current state information of the wafer transfer robotic arm, the current state information including joint position and joint angular velocity; Based on the kinematic model, the wafer transfer actuator position is set to (x0, y0), and the corresponding joint angle is calculated through planar geometry as the desired joint position; The difference between the current joint position and the desired joint position is taken as the joint position tracking error; The current state information is input into the model identifier to obtain the compensation amount for the uncertainty of the dynamic model of the wafer transfer robot arm; wherein, the model identifier is used to estimate the difference between the dynamic model and the actual dynamics online, and the difference is characterized by the estimated model uncertainty term, which refers to the dynamic characteristics not considered in the dynamic model, and the actual dynamics are the dynamic behavior of the wafer transfer robot arm in actual operation. Using an adaptive dynamic programming framework, an optimal control strategy is generated based on the joint position tracking error, a predefined performance index function, and the compensation amount. The performance index function is used to optimize the long-term comprehensive performance of the wafer transfer robot in terms of speed, accuracy, and energy consumption. The compensation amount is used to correct the initial control strategy output by the adaptive dynamic programming framework. Based on the optimal control strategy, a control torque is output to control the movement of the wafer transfer robotic arm.

2. The wafer transfer robotic arm control method according to claim 1, characterized in that, The dynamic model is a mathematical model that describes the dynamic behavior of the wafer transfer robot arm.

3. The wafer transfer robotic arm control method according to claim 2, characterized in that, The model identifier is a neural network model identifier. The step of inputting the current state information into the model identifier to obtain compensation for the uncertainty of the dynamic model of the wafer transfer robot arm includes: Using the neural network model identifier, the model uncertainty term of the wafer transfer robot arm is estimated based on the current state information; Based on the uncertainty term of the model, the compensation amount is calculated and output.

4. The wafer transfer robotic arm control method according to claim 1, characterized in that, The adaptive dynamic programming framework includes a judgment neural network and an execution neural network; the generation of the optimal control strategy is achieved through multiple iterative processes, the iterative steps of which include: The evaluation neural network evaluates the long-term comprehensive performance of the wafer transfer robot arm based on the current state information, the joint position tracking error, and the current control strategy, and outputs a value function estimate. The execution neural network outputs an iterative control strategy based on the estimated value function. The iterative control strategy is applied to the wafer transfer robotic arm to generate iterative state information and iterative joint position tracking error; Based on the iterative state information and the iterative joint position tracking error, the weights of the evaluation neural network and the execution neural network are updated simultaneously, so that the value function estimate and iterative control strategy generated in the next iteration cycle are better.

5. The wafer transfer robotic arm control method according to claim 4, characterized in that, The weight update process of the evaluation neural network and the execution neural network is designed according to Lyapunov stability theory to ensure that the joint position tracking error eventually becomes uniformly bounded.

6. The wafer transfer robotic arm control method according to claim 1, characterized in that, The performance index function is a long-term cumulative reward function constructed based on the joint position tracking error and the utility function.

7. The wafer transfer robotic arm control method according to claim 6, characterized in that, The performance index function is: ； in, For performance index functions, For utility function, The error fusion function comprises the joint position tracking error and its derivative. The control strategy is defined by the control torque as the element of its output vector, where t is the current time.

8. The wafer transfer robotic arm control method according to claim 1, characterized in that, The step of obtaining the current status information of the wafer transfer robotic arm includes: Acquire the original state signals of the wafer transfer robotic arm; The original state signal is filtered to output the current state information.

9. A control system for a wafer transfer robotic arm, characterized in that, The wafer transfer robotic arm is a three-bar linkage, and its kinematic model is established using the DH parameter method. The wafer transfer robotic arm includes a wafer transfer actuator for picking up, placing, and transferring wafers. The system includes: The acquisition module is used for: Multiplying the transformation matrices A1, A2, A3, A4, and A5 between multiple adjacent coordinate systems yields the pose matrix of the wafer transfer actuator relative to the base coordinates in the kinematic model. ; Obtain the current state information of the wafer transfer robotic arm, the current state information including joint position and joint angular velocity; The calculation module is used for: Based on the kinematic model, the wafer transfer actuator position is set to (x0, y0), and the corresponding joint angle is calculated through planar geometry as the desired joint position; The difference between the current joint position and the desired joint position is taken as the joint position tracking error; Generate modules for: The current state information is input into the model identifier to obtain the compensation amount for the uncertainty of the dynamic model of the wafer transfer robot arm; wherein, the model identifier is used to estimate the difference between the dynamic model and the actual dynamics online, the actual dynamics being the dynamic behavior of the wafer transfer robot arm in actual operation, and the difference is characterized by the estimated model uncertainty term, which refers to the dynamic characteristics not considered in the dynamic model; Using an adaptive dynamic programming framework, an optimal control strategy is generated based on the joint position tracking error, a predefined performance index function, and the compensation amount. The performance index function is used to optimize the long-term comprehensive performance of the wafer transfer robot in terms of speed, accuracy, and energy consumption. The compensation amount is used to correct the initial control strategy output by the adaptive dynamic programming framework. The control module is used to output a control torque based on the optimal control strategy to control the movement of the wafer transfer robotic arm.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the wafer transfer robotic arm control method as described in any one of claims 1 to 8.

Images