A constraint visual servoing method, system, device and medium based on imitation learning

Abstract

The application discloses a constraint visual servoing method and system based on imitation learning, and a device and a medium, and belongs to the technical field of visual servoing, and comprises the following steps: acquiring target object data, mechanical arm data and camera data, and establishing a target object model, a mechanical arm model and an image feature model; modeling the constraint of the mechanical arm through an imitation learning method, generating demonstration data, and generating demonstration trajectories by combining the demonstration data with an optimized trajectory generation strategy; calculating image feature tracking errors according to the demonstration trajectories, designing a cost function and a Hamilton function of the constraint visual servoing based on the image feature tracking errors, and obtaining an optimal control strategy and optimal control input of the mechanical arm through the Hamilton function; estimating the cost function by using a neural network estimator, obtaining an estimated value of the optimal control strategy, and updating the neural network estimator according to the Hamilton function. The application can generate a mechanical arm trajectory with optimal cost and smoothness without matrix inversion.

Description

Technical Field

[0001] This invention relates to the field of visual servoing technology, specifically to a constraint visual servoing method, system, device, and medium based on imitation learning. Background Technology

[0002] In practical applications of robotic arms, the task space of the robotic arm is dynamic and unstructured. Therefore, visual servo control plays a key role in real-time planning and manipulation. Visual servoing methods can be divided into position-based visual servoing, image-based visual servoing, and a combination of the two. Among them, image-based visual servoing has strong robustness to visual sensor noise and modeling errors.

[0003] Currently, image-based vision servo control is often configured as a nonlinear optimization problem based on numerical and convex solvers. However, numerical solutions are often unreliable and suffer from stability and efficiency issues. It is necessary to ensure the optimal step size to avoid failures due to large step differences. Convex optimization formulas allow the integration of visual and kinematic constraints modeled using rigorous mathematical formulas, but the solutions assume local convexity and may require computation of expensive matrix inversions to start the robotic arm manipulator. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by this invention is: how to treat image-based visual servo control as a nonlinear optimization problem, perform constraint modeling through imitation learning and Hamilton-Jacobi-Bellman equations, use a single lens to form a single demonstration and generalize it into static and moving targets, realize visual and kinematic constraint modeling, and then use Hamilton-Jacobi-Bellman equations to further optimize the model, thereby ensuring the reliability and accuracy of image-based visual servo control.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a constraint visual servoing method based on imitation learning, comprising,

[0007] Data on the target object, robotic arm, and camera are acquired, and models of the target object, robotic arm, and image features are established. Constraints on the robotic arm are modeled using an imitation learning method to generate demonstration data. An optimized trajectory generation strategy is then employed to generate a demonstration trajectory based on this data. Image feature tracking error is calculated based on the demonstration trajectory. A cost function and Hamiltonian function for constrained visual servoing are designed based on this error. The optimal control strategy and optimal control input for the robotic arm are obtained through the Hamiltonian function. A neural network estimator is used to estimate the cost function, obtaining an estimate of the optimal control strategy. The neural network estimator is then updated based on the Hamiltonian function.

[0008] As a preferred embodiment of the constraint visual servoing method based on imitation learning described in this invention, the step of acquiring target object data, robotic arm data, and camera data, and establishing target object model, robotic arm model, and image feature model includes: establishing coordinate systems for the robotic arm base, robotic arm end effector, camera, and target object respectively; establishing a target object model based on the three-dimensional feature points of the target object; establishing a robotic arm model based on the joint angle vectors of the robotic arm manipulator; and establishing an image feature model based on the camera projection image coordinates of the target object feature points.

[0009] As a preferred embodiment of the constraint visual servoing method based on imitation learning described in this invention, the step of modeling the constraints of the robotic arm through imitation learning to generate demonstration data includes: modeling motion constraints through the joint positions and joint velocities of the robotic arm; modeling visual constraints through the field of view constraints of the camera; and guiding the robotic arm toward the target object through kinematic teaching, simultaneously satisfying the motion constraints and the visual constraints, thereby forming demonstration data.

[0010] As a preferred embodiment of the constraint visual servoing method based on imitation learning described in this invention, the step of generating a demonstration trajectory by combining the demonstration data with an optimized trajectory generation strategy includes: extracting image space trajectory and joint space trajectory from the demonstration data; training the image space trajectory and the joint space trajectory using dynamic motion primitives to obtain learned shape parameters; and calculating the demonstration trajectory based on the learned shape parameters using an optimized trajectory generation strategy.

[0011] As a preferred embodiment of the constrained visual servoing method based on imitation learning described in this invention, the method for calculating image feature tracking error based on the demonstration trajectory includes: calculating image feature tracking error based on the current image feature point trajectory and the demonstration trajectory; and calculating the time derivative of image feature tracking error based on the image feature tracking error and the image feature Jacobian matrix.

[0012] As a preferred embodiment of the constrained visual servoing method based on imitation learning described in this invention, the method involves: designing a cost function and a Hamiltonian function for constrained visual servoing based on the image feature tracking error; obtaining the optimal control strategy and optimal control input for the robotic arm through the Hamiltonian function; performing optimal control based on the time derivative of the image feature tracking error; designing a cost function; constructing a Hamiltonian function based on the cost function; and deriving the optimal control strategy and optimal control input for the robotic arm manipulator through the stationarity condition of the Hamiltonian function.

[0013] As a preferred embodiment of the constraint visual servoing method based on imitation learning described in this invention, the method includes: estimating the cost function using a neural network estimator to obtain an estimated value of the optimal control strategy, and updating the neural network estimator according to the Hamiltonian function, comprising: estimating the cost function using a neural network to obtain a cost function of the neural network; calculating the gradient of the cost function of the neural network; obtaining a neural network estimate of the optimal control strategy based on the gradient; calculating the Hamiltonian error between the estimated Hamiltonian function and the ideal Hamiltonian function; and updating the weights of the neural network estimator using an update law, wherein the update law includes minimizing the error function and bounded system error.

[0014] This invention provides a constraint-based visual servoing system based on imitation learning.

[0015] To address the aforementioned technical problems, this invention provides the following technical solution: a constraint visual servoing system based on imitation learning, comprising: a spatial modeling module for acquiring target object data, robotic arm data, and camera data, and establishing target object models, robotic arm models, and image feature models; a constraint learning and trajectory generation module for modeling the constraints of the robotic arm using imitation learning methods, generating demonstration data, and generating a demonstration trajectory using an optimized trajectory generation strategy combined with the demonstration data; a controller design module for calculating image feature tracking error based on the demonstration trajectory, designing the cost function and Hamiltonian function of constraint visual servoing based on the image feature tracking error, and obtaining the optimal control strategy and optimal control input for the robotic arm through the Hamiltonian function; and a weight update module for estimating the cost function using a neural network estimator, obtaining an estimate of the optimal control strategy, and updating the neural network estimator based on the Hamiltonian function.

[0016] The present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the constraint visual servoing method based on imitation learning.

[0017] The present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the constraint visual servoing method based on imitation learning.

[0018] The beneficial effects of this invention are as follows: This invention achieves implicit modeling of constraints and automatic acquisition of expert knowledge through imitation learning, solving the problems of difficult constraint expression and complex parameter adjustment in traditional visual servo control. By combining optimal control theory and neural network estimators, it achieves a unity of theoretical rigor and practical feasibility, possesses dynamic adaptability, and can improve the framework in real time during operation, enhancing flexibility and adaptability. It achieves efficient trajectory optimization, generating cost-optimal and smooth robotic arm trajectories without matrix inversion, solving stability and efficiency issues. It not only eliminates the limitations of human experience dependence and offline parameter tuning in traditional methods, but also achieves continuous optimization of control performance through online learning mechanisms, improving the execution efficiency and environmental adaptability of the robotic arm's visual servo tasks while ensuring control accuracy and stability. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 The present invention provides an overall flowchart of a constraint visual servoing method based on imitation learning, which is an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the structure of a robotic arm base, a robotic arm end effector, a camera, and a target object provided in one embodiment of the present invention.

[0022] Figure 3 This is a schematic diagram of an optimal visual servo control scheme provided in one embodiment of the present invention. Detailed Implementation

[0023] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0024] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0025] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0026] Example 1, referring to Figure 1 As one embodiment of the present invention, this embodiment provides a constrained visual servoing method based on imitation learning, comprising:

[0027] S100: Acquire target object data, robotic arm data, and camera data, and establish target object model, robotic arm model, and image feature model.

[0028] S200: The constraints of the robotic arm are modeled using an imitation learning method to generate demonstration data, and an optimized trajectory generation strategy is used to generate a demonstration trajectory in combination with the demonstration data.

[0029] S300: Calculate the image feature tracking error based on the demonstration trajectory, design the cost function and Hamiltonian function for constrained visual servoing based on the image feature tracking error, and obtain the optimal control strategy and optimal control input of the robotic arm through the Hamiltonian function.

[0030] S400: A neural network estimator is used to estimate the cost function, obtain an estimate of the optimal control strategy, and update the neural network estimator according to the Hamiltonian function.

[0031] It should be noted that image-based visual servoing control methods face complex constraint problems in practical applications. When performing visual servoing tasks, the robotic arm needs to simultaneously satisfy kinematic and visual constraints. Kinematic constraints are mainly reflected in the physical limitations of the robotic arm's joint positions and speeds, while visual constraints are reflected in the limitations of the camera's field of view; the target object must always remain within the camera's effective field of view. Traditional visual servoing control methods often use hard constraints to handle these constraints, which can easily lead to instability in the control system near the constraint boundaries. Furthermore, they lack effective utilization of expert experience, resulting in poor adaptability in real-world environments. In addition, visual servoing control is essentially a nonlinear optimal control problem. Traditional methods typically require a known, accurate system dynamics model and cost function to solve for the optimal control strategy. However, in practical applications, this information is often difficult to obtain or is subject to uncertainty, leading to decreased control performance. Moreover, the robustness of the controller is difficult to guarantee when system parameters change or external disturbances occur.

[0032] Therefore, to address the aforementioned problems of difficulty in constraint modeling, complexity in optimal control solution, and insufficient system robustness, this paper employs an imitation learning method through steps S100-S400 to model kinematic and visual constraints. This avoids the limitations of traditional hard constraint methods and enables the effective transformation and utilization of expert experience. Simultaneously, based on Hamilton-Jacobi-Bellman theory, an optimal control strategy is designed, and an online approximation of the unknown cost function is performed using a neural network estimator. This allows for obtaining the optimal control input without requiring a precise system model, thereby improving the accuracy, stability, and robustness of visual servo control.

[0033] Example 2, refer to Figures 1-3 As one embodiment of the present invention, based on the previous embodiment, a constraint visual servoing method based on imitation learning is provided, comprising:

[0034] In this embodiment of the invention, step S100, which involves acquiring target object data, robotic arm data, and camera data, and establishing a target object model, a robotic arm model, and an image feature model, includes the following steps A1-A4:

[0035] A1: Establish coordinate systems for the robotic arm base, robotic arm end effector, camera, and target object, respectively.

[0036] A2: Establish a model of the target object based on its three-dimensional feature points.

[0037] A3: Establish a robotic arm model based on the joint angle vectors of the robotic arm manipulator.

[0038] A4: Establish an image feature model based on the camera projection image coordinates of the feature points of the target object.

[0039] Specifically, in step A1, establishing coordinate systems for the robotic arm base, the robotic arm end effector, the camera, and the target object means establishing coordinate systems connected to the robotic arm base respectively. Coordinate system of the robotic arm end effector Camera coordinate system Coordinate system of the target object , specifically Figure 2 As shown, where let Indicates from coordinate system coordinates to coordinate system Homogeneous transformation of coordinates, The 1 and 2 in the text represent placeholders, which can be replaced by b, e, c, and o respectively. Here, b represents the robotic arm base, e represents the robotic arm end effector, c represents the camera, and o represents the target object.

[0040] Specifically, in step A2, a model of the target object is built based on its three-dimensional feature points. The specific operations can be as follows:

[0041] The object model consists of many three-dimensional feature points. In coordinate system The coordinates of the three-dimensional feature points in the image. They represent in the coordinate system The coordinates on the horizontal axis, vertical axis, and z-axis have The target object model with these features can be specifically represented as:

[0042] ;

[0043] in, For having A target object model with several features; This is the transpose symbol.

[0044] Specifically, in step A3, the robotic arm model is established based on the joint angle vectors of the robotic arm manipulator. The specific operation can be as follows:

[0045] In vision servo control applications, the internal control loop of the robotic arm is relatively fast, while the external vision loop is slower, allowing the robotic arm model to maintain dynamic stability. Therefore, [the following is a separate, unrelated sentence:] ... The kinematic model of an independently joint-driven robotic arm manipulator, the kinematic model is composed of Given, among which Let be the joint angle vector of the robotic arm manipulator. The pose of the robotic arm's end effector relative to the base. Here, is the kinematic function used to calculate the forward kinematics of the robotic arm, where the kinematic model of the robotic arm manipulator relative to its joint angle vectors is the derivative of the Jacobian matrix of the robotic arm manipulator. Specifically, it can be expressed as:

[0046] ;

[0047] in, and These are the linear and rotational components of the Jacobian matrix of the robotic arm manipulator, respectively.

[0048] Secondly, the control input to the robotic arm manipulator is the joint velocity vector, which can be specifically expressed as:

[0049] ;

[0050] in, This is the joint velocity vector; Let be the time derivative of the joint angle vector of the robotic arm manipulator.

[0051] Specifically, step A4 establishes an image feature model based on the camera projection image coordinates of the target object's feature points, including the following steps A41-A44:

[0052] A41: For visual servoing applications, the camera is mounted on the end effector of the robotic arm in a hand-eye configuration, and the camera's projected image coordinates correspond to the object's feature points. Joint angle vector of the robotic arm manipulator function It can be modeled as:

[0053] ;

[0054] in, for Camera matrix, and These are the normalized focal lengths along the u-axis and v-axis of the image, respectively. is the tilt factor of the coordinate axes. and These are the coordinates of the camera's principal point on the image plane; and The camera projection image coordinates of the object feature points are respectively and ; and They are respectively identity matrix and The zero vector; , and These are the coordinate systems of the robotic arm's end effector. coordinates to camera coordinate system Homogeneous transformation of coordinates, from the coordinate system of the robotic arm base coordinates to the coordinate system of the robotic arm's end effector Homogeneous transformation of coordinates and from the target object's coordinate system coordinates from the coordinate system of the robot arm base Homogeneous transformation of coordinates; For having A target object model with several features.

[0055] A42: Image coordinate vector used for visual servo control in step A41 Specifically, it can be expressed as:

[0056] ;

[0057] in, , and Camera projection image coordinates of object feature points Joint angle vector of the robotic arm manipulator function The abbreviation of .

[0058] A43: Image coordinate vector used for visual servo control At speed level and Having Jacobi relation :

[0059] ;

[0060] in, It is a Jacobian matrix; is the time derivative of the joint angle vector of the robotic arm manipulator; This is the joint velocity vector.

[0061] Furthermore, the Jacobian matrix Specifically, it can be expressed as:

[0062] ;

[0063] in, coordinates In coordinates The rotation matrix of the coordinates in the matrix; For image interaction matrix; and These are the linear and rotational components of the Jacobian matrix of the robotic arm manipulator in the camera coordinate system, respectively. and These are the linear and rotational components of the Jacobian matrix of the robotic arm manipulator, respectively.

[0064] A44: Corresponds to depth Each feature point at submatrix Specifically, it can be expressed as:

[0065] ;

[0066] Among them, the coordinates of the feature points Its image counterpart pass Camera Matrix Related, and Feature points The corresponding actual pixel coordinates, i.e., feature points The pixel positions in the horizontal and vertical directions of the image.

[0067] It should be noted that this invention establishes a complete modeling framework, including establishing standardized coordinate systems for each component and constructing target object models, robotic arm models, and image feature models, thereby achieving a complete mathematical description and geometric relationship establishment for the visual servoing method. Compared with existing technologies that lack unified coordinate system specifications and systematic modeling standards, this invention solves the problems of complex coordinate transformations, insufficient model accuracy, and lack of effective correlation between subsystems in traditional methods through standardized modeling methods. In particular, it uniformly describes the spatial relationships between coordinate systems through homogeneous transformation matrices and accurately establishes the mapping relationship between image space and joint space through Jacobian matrices, ensuring the accuracy and reliability of the mathematical foundation required for subsequent control algorithms. This not only improves the completeness and computational accuracy of modeling but also provides a precise mathematical model foundation for subsequent constraint learning and trajectory generation, thus laying a solid theoretical foundation for the stability and control accuracy of the entire visual servoing control method.

[0068] In this embodiment of the invention, step S200 involves modeling the constraints of the robotic arm using an imitation learning method to generate demonstration data, including the following steps B1-B3:

[0069] B1: Model motion constraints using the joint positions and velocities of the robotic arm.

[0070] B2: Model visual constraints using the camera's field of view constraints.

[0071] B3: Utilizing the knowledge and perception of expert demonstrators, guide the robotic arm toward the target object through kinesthetic teaching, while simultaneously satisfying motion constraints and visual constraints, to generate demonstration data.

[0072] Specifically, modeling motion constraints using the robot arm joint positions and velocities in step B1 refers to using the robot arm joint positions... and joint velocity The upper and lower limits are used to construct motion constraints.

[0073] Specifically, in step B2, modeling visual constraints using the camera's field of view means constructing visual constraints based on the camera's field of view, where the target object has... One characteristic, and It represents the limits in both the horizontal and vertical directions.

[0074] Specifically, in step B3, the knowledge and perception of the expert demonstrator are used to guide the robotic arm toward the target object through kinematic teaching, while simultaneously satisfying kinematic and visual constraints. The resulting demonstration data refers to defining a safe area through learning tasks and using expert human demonstrations or imitation learning methods, implicitly modeling the constraints. The expert demonstrator's knowledge and perception are used to model the kinematic and visual constraints in the visual servo control problem. The expert demonstrator guides the robotic arm toward a general target object through kinematic teaching, while ensuring kinematic and visual constraints, thus obtaining the demonstration data.

[0075] In this embodiment of the invention, step S200 uses an optimized trajectory generation strategy combined with demonstration data to generate a demonstration trajectory, including the following steps C1-C3:

[0076] C1: Extract image spatial trajectory and joint spatial trajectory from the demonstration data.

[0077] C2: Train the image space trajectory and joint space trajectory using dynamic motion primitives to obtain learned shape parameters.

[0078] C3: Based on the learned shape parameters, an optimized trajectory generation strategy is used to calculate and generate a demonstration trajectory.

[0079] Specifically, step C1, extracting image space trajectory and joint space trajectory from the demonstration data, refers to using image space and joint space to extract image space trajectory and joint space trajectory from the demonstration data. The image space trajectory and joint space trajectory are used to train dynamic movement primitives (DMPs).

[0080] Specifically, in step C2, training the image space trajectory and joint space trajectory using dynamic motion primitives to obtain learned shape parameters involves training the dynamic motion primitives using the image space trajectory and joint space trajectory, and using the learned shape parameters from the trained dynamic motion primitives to capture the task and constraints. Within this framework, a modified motion model is used to demonstrate each degree of freedom in the motion. Modeling is performed.

[0081] Furthermore, in step C3, an optimized trajectory generation strategy is used to calculate based on the learned shape parameters. Generating a demonstration trajectory means that in order to generalize the generated image and joint trajectory in the new situation, a single demonstration trajectory is not enough due to the changing contours on different targets. Therefore, a library of multiple demonstration trajectories with decision-making skills is required.

[0082] For example, given a vector Given the new conditions for the start and target configuration of a specific signal, the optimized trajectory generation strategy in step C3... Specifically, it can be expressed as:

[0083] ;

[0084] in, To configure the starting and target signals corresponding to the new conditions. of The first expert presentation One generated trajectory; The design parameters are used; the demonstration trajectory generated by following the optimized trajectory generation strategy is used as the desired image. and joint trajectory .

[0085] It should be noted that this invention achieves implicit modeling of the robotic arm's motion and visual constraints by combining the knowledge and perception of expert demonstrators through imitation learning methods. It also generates high-quality demonstration trajectories through optimized trajectory generation strategies. Compared to existing methods that rely on hard constraint settings and simple trajectory planning, this invention solves the problems of complex constraint modeling, insufficiently smooth trajectory generation, and difficulty in adapting to task scenarios in traditional methods through imitation learning. In particular, the kinematic teaching method using expert demonstrations can naturally integrate kinematic and visual constraints. Through dynamic motion primitive training and optimized trajectory generation strategies, it can generate smoother and more natural motion trajectories while ensuring constraint satisfaction. This not only saves manpower and debugging time but also improves trajectory quality and task adaptability, enabling the robotic arm to perform more precise visual servoing tasks under various constraint conditions, and providing high-quality reference trajectories for subsequent optimal controller design.

[0086] In this embodiment of the invention, step S300, which calculates the image feature tracking error based on the demonstration trajectory, includes the following steps D1-D2:

[0087] D1: Calculate the image feature tracking error based on the current image feature point trajectory and the demonstration trajectory.

[0088] D2: Calculate the time derivative of the image feature tracking error based on the image feature tracking error and the image feature Jacobian matrix.

[0089] Specifically, in step D1, the image feature tracking error is calculated based on the current image feature point trajectory and the demonstration trajectory. Specifically, it can be expressed as:

[0090] ;

[0091] in, The current image feature point trajectory; For the desired image feature point trajectory, For the desired image feature point trajectory In time The set of components decreases to zero over time by following an optimal trajectory that conforms to visual and kinematic constraints.

[0092] Specifically, in step D2, the time derivative of the image feature tracking error is calculated based on the image feature tracking error and the image feature Jacobian matrix. The specific operation can be as follows:

[0093] For tracking purposes, it is assumed that the desired trajectory has dynamics given by the following equation:

[0094] ;

[0095] in, The time derivative of the desired image feature point trajectory; It is a Jacobian matrix; The desired joint velocity; The desired joint velocity.

[0096] The time derivative of the image feature tracking error can be specifically expressed as:

[0097] ;

[0098] in, This is the time derivative of the image feature tracking error; This is the time derivative of the current image feature point trajectory; For optimal control, , This is the joint velocity vector.

[0099] In this embodiment of the invention, step S300 involves designing a cost function and a Hamiltonian function for constrained visual servoing based on image feature tracking error, and obtaining the optimal control strategy and optimal control input for the robotic arm through the Hamiltonian function, including the following steps E1-E3:

[0100] E1: Optimal control is achieved based on the time derivative of image feature tracking error, and a cost function is designed.

[0101] E2: Construct the Hamiltonian function based on the cost function.

[0102] E3: By deriving the stationarity condition of the Hamiltonian function, the optimal control strategy and the optimal control input of the robotic arm manipulator are obtained.

[0103] Specifically, in step E1, optimal control is performed based on the time derivative of the image feature tracking error, and a cost function is designed. The specific operation can be as follows:

[0104] Since the optimal control problem for the time derivative of image feature tracking error is to find the optimal control... To minimize the cost function of the state and control variables, where The joint angle vector of the robotic arm manipulator with respect to time Assume that for the constrained visual servo control problem, there exists a continuous cost function. as follows:

[0105] ;

[0106] in, and It is a positive definite matrix; It is a positive semi-definite function, representing the state cost; It is a positive semi-definite function, representing the control cost; For time derivative.

[0107] Furthermore, for a given continuous cost function Optimal control strategy The following conditions must be met: Optimal control strategy The time derivative of image feature tracking error Stable, and corresponding continuous cost function It is finite; therefore, the optimal cost function Defined as:

[0108] ;

[0109] in, The optimal control strategy; This is a set of candidate control strategies.

[0110] Furthermore, in step E2, the Hamiltonian function is constructed based on the cost function. Specifically, this can be achieved through the following steps:

[0111] make It corresponds to a continuous cost function The Hamiltonian function, and allows for optimal control input. Then the Hamiltonian function Specifically, it can be expressed as:

[0112] ;

[0113] in, It is the Hamiltonian function; The derivative of the cost function. , It is the cost function; This is the time derivative of the image feature tracking error; This is the transpose of the derivative of the cost function.

[0114] Specifically, in step E3, the optimal control strategy and the optimal control input of the robotic arm manipulator are derived using the stationarity condition of the Hamiltonian function. The specific operation can be as follows:

[0115] like Figure 3 The diagram shows the optimal vision servo control scheme. The optimal control input for a given problem is obtained by using the stationarity condition on the Hamiltonian function, and is derived from... Given that the optimal control strategy can be specifically expressed as:

[0116] ;

[0117] in, The optimal control strategy; It is a positive definite matrix; This is the transpose of the Jacobian matrix; This is the optimal cost function.

[0118] Accordingly, the optimal control input of the robotic arm manipulator Specifically, it can be expressed as:

[0119] ;

[0120] in, It is a positive definite matrix; This is the transpose of the Jacobian matrix; Optimal cost function; The desired control input.

[0121] It should be noted that this invention establishes a dynamic model of the time derivative of the image feature tracking error by treating the image feature tracking error as the system state, and designs a cost function and a Hamiltonian function based on this model to achieve the optimal solution for constrained visual servo control. Compared with the traditional feedback control method used in the prior art, this invention solves the problems of low control accuracy, inability to handle constraints, and difficulty in ensuring system stability in traditional methods through optimal control theory. In particular, the optimal control strategy can be rigorously derived through the stationarity condition of the Hamiltonian function, ensuring optimal convergence of the image feature tracking error under the premise of satisfying visual and kinematic constraints. This not only improves the accuracy and stability of visual servo control, but also provides a theoretically rigorous solution for handling multi-constraint optimization problems, enabling the robotic arm to achieve high-precision visual servo tasks in real-world environments, and laying a solid theoretical foundation for the subsequent design of neural network estimators.

[0122] It should be noted that, since visual servo control of a robotic arm manipulator with visual and motion constraints is achieved by providing optimal control input to the robotic arm manipulator, however, the term in the formula corresponding to the above optimal control input... or Normally unavailable, the appropriate control input is found by utilizing neural network-based estimation.

[0123] In this embodiment of the invention, step S400 uses a neural network estimator to estimate the cost function, obtains an estimate of the optimal control strategy, and updates the neural network estimator according to the Hamiltonian function, including the following steps F1-F5:

[0124] F1: The cost function of the neural network is obtained by estimating the cost function through the neural network.

[0125] F2: Calculate the gradient of the cost function of the neural network.

[0126] F3: The neural network estimate of the optimal control strategy is obtained based on the gradient.

[0127] F4: Calculate the Hamiltonian error between the estimated Hamiltonian function and the ideal Hamiltonian function.

[0128] F5: The weights of the neural network estimator are updated using an update law, which includes minimizing the error function and ensuring the boundedness of the systematic error.

[0129] Specifically, in step F1, the cost function is estimated using a neural network. The resulting cost function of the neural network is obtained by using a neural network to estimate the cost function, taking advantage of the network's general approximation properties. Specifically, it can be expressed as:

[0130] ;

[0131] in, The ideal neural network weight vector; It is the transpose symbol; Let them be linearly independent basis vectors, satisfying ; Reconstruction error of the neural network; ideal neural network weight vector and neural network reconstruction error It has an upper bound and satisfies and , and These represent the maximum value of the neural network weight vector and the maximum value of the neural network reconstruction error, respectively.

[0132] Specifically, calculating the gradient of the cost function of the neural network in step F2 refers to assuming that the reconstruction error of the neural network is relative to... The gradient has an upper bound and satisfies ,in The neural network reconstruction error relative to gradient, The neural network reconstruction error relative to The maximum value of the gradient. Therefore, the gradient of the cost function of the neural network can be specifically expressed as:

[0133] ;

[0134] in, The gradient of the cost function of the neural network; The gradient of the cost function of a neural network abbreviation; The derivative of linearly independent basis vectors. ; This is the transpose symbol.

[0135] Specifically, in step F3, the neural network estimate of the optimal control strategy is obtained based on the gradient. The specific operation can be as follows:

[0136] Substituting the gradient of the cost function of the neural network into the optimal control policy yields the rewritten optimal control policy, which can be specifically expressed as:

[0137] ;

[0138] in, The optimal control strategy after rewriting; This is the transpose of the Jacobian matrix.

[0139] By using a neural network estimator to approximate the optimal cost function, an approximate optimal cost function is obtained. Specifically, it can be expressed as:

[0140] ;

[0141] in, For the ideal neural network weight vector The estimate; This is the transpose symbol.

[0142] The neural network estimate of the optimal control strategy is obtained by calculating the rewritten optimal control strategy and the approximate optimal cost function. Specifically, it can be expressed as:

[0143] ;

[0144] in, It is a positive definite matrix; This is the transpose of the Jacobian matrix; For the ideal neural network weight vector The estimate; The derivative of a linearly independent basis vector; This is the transpose symbol.

[0145] Substituting the neural network estimate of the optimal control strategy into the time derivative of the image feature tracking error, we obtain the modified expression, which can be specifically expressed as:

[0146] ;

[0147] in, This is the time derivative of the image feature tracking error; It is a positive definite matrix; It is a Jacobian matrix; For the ideal neural network weight vector The estimate; The derivative of a linearly independent basis vector.

[0148] Specifically, in step F4, the Hamiltonian error between the estimated Hamiltonian function and the ideal Hamiltonian function is calculated. The specific steps can be as follows:

[0149] Note that the cost function and control law are constructed in the form of neural network weight vectors; similarly, the Hamiltonian can be derived from... and The function formed is represented as follows:

[0150] ;

[0151] in, It is the Hamiltonian function; Image feature tracking error; It is the transpose symbol; It is a positive semi-definite function, representing the state cost; The ideal neural network weight vector; The residual error in the neural network expression; It is a composite positive definite matrix. , The derivative of linearly independent basis vectors. For Jacobian matrices, It is a positive definite matrix.

[0152] Furthermore, the residual error of the neural network expression Specifically, it can be expressed as:

[0153] ;

[0154] in, The neural network reconstruction error relative to The gradient; It is the transpose symbol; It is a positive definite matrix.

[0155] Corresponding to the estimated neural network weight vector, the estimated Hamiltonian function Specifically, it can be expressed as:

[0156] ;

[0157] in, It is a positive semi-definite function, representing the state cost; For the ideal neural network weight vector The estimate.

[0158] Based on Hamiltonian function and the estimated Hamiltonian function The Hamiltonian error is calculated and can be expressed as follows:

[0159] ;

[0160] in, For Hamiltonian error; The error between the target weight vector and the estimated weight vector of the neural network; This represents the residual error in the neural network expression.

[0161] Specifically, in step F5, the weights of the neural network estimator are updated using an update law, which includes minimizing the error function and ensuring the boundedness of the systematic error. The specific operation can be as follows:

[0162] By training the comment neural network, the Hamiltonian error function is made... Minimize, specifically, can be expressed as:

[0163]

[0164] ;

[0165] in, For the ideal neural network weight vector The estimate; The error between the target weight vector and the estimated weight vector of the neural network; The learning rate is the critical value. The gain of the stable term; For Hamiltonian error; Hamiltonian error For the ideal neural network weight vector Estimate The gradient; The derivative of a linearly independent basis vector; It is a Jacobian matrix; It is a positive definite matrix; This is the transpose of the Jacobian matrix; It is the transpose symbol; This represents the residual error in the neural network expression.

[0166] It should be noted that the update law in step F5 consists of two parts. The first part makes the Hamiltonian error function... The minimum, the second part guarantees that the system error is bounded, where the critical value guaranteeing the bounded system error is the optimal cost function and the learning parameters. The critical learning rate, This is the gain of the stable term.

[0167] It should be noted that this invention approximates the cost function using a neural network estimator and employs an update law that includes minimizing the error function and ensuring the boundedness of the system error for online learning, thereby achieving effective estimation and continuous optimization of the optimal control strategy. Compared with existing control methods that rely on precise mathematical models or offline training, this invention solves the problems of difficult model uncertainty handling, inability to adapt to environmental changes online, and control performance degradation over time in traditional methods through a neural network estimator. In particular, the continuous improvement of control accuracy is ensured through Hamiltonian error minimization and a dual weight update mechanism. This not only enhances the robustness and adaptability of the visual servo control method but also maintains high-precision control performance in complex and ever-changing working environments, providing a reliable guarantee for long-term stable operation in practical engineering applications.

[0168] In summary, this invention achieves implicit constraint modeling and automatic acquisition of expert knowledge through imitation learning, solving the problems of difficult constraint expression and complex parameter tuning in traditional visual servo control. By combining optimal control theory and neural network estimators, it achieves a unity of theoretical rigor and practical feasibility, possessing dynamic adaptability and the ability to improve the framework in real time during operation, enhancing flexibility and adaptability. It achieves efficient trajectory optimization, generating cost-optimal and smooth robotic arm trajectories without matrix inversion, solving stability and efficiency issues. It not only eliminates the limitations of reliance on manual experience and offline parameter tuning in traditional methods, but also achieves continuous optimization of control performance through an online learning mechanism, improving the execution efficiency and environmental adaptability of the robotic arm's visual servo tasks while ensuring control accuracy and stability.

[0169] Example 3 is an embodiment of the present invention, which provides a constraint visual servoing system based on imitation learning, including: a spatial modeling module, used to acquire target object data, robotic arm data, and camera data, and establish target object model, robotic arm model, and image feature model; a constraint learning and trajectory generation module, used to model the constraints of the robotic arm using imitation learning methods, generate demonstration data, and generate a demonstration trajectory by combining the demonstration data with an optimized trajectory generation strategy; a controller design module, used to calculate the image feature tracking error based on the demonstration trajectory, design the cost function and Hamiltonian function of constraint visual servoing based on the image feature tracking error, and obtain the optimal control strategy and optimal control input of the robotic arm through the Hamiltonian function; and a weight update module, used to estimate the cost function using a neural network estimator, obtain the estimated value of the optimal control strategy, and update the neural network estimator according to the Hamiltonian function.

[0170] This embodiment also provides an electronic device applicable to a constraint visual servoing method based on imitation learning, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the constraint visual servoing method based on imitation learning as proposed in the above embodiment.

[0171] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements a constraint visual servoing method based on imitation learning as proposed in the above embodiments.

[0172] The storage medium proposed in this embodiment and the implementation of a constraint visual servoing method based on imitation learning proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0173] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0174] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A constraint-based visual servoing method based on imitation learning, characterized in that: include, Acquire target object data, robotic arm data, and camera data, and build target object models, robotic arm models, and image feature models; The constraints of the robotic arm are modeled using an imitation learning method to generate demonstration data, and an optimized trajectory generation strategy is used in conjunction with the demonstration data to generate a demonstration trajectory. The process of modeling the constraints of the robotic arm using imitation learning methods and generating demonstration data includes: Motion constraints are modeled using the joint positions and velocities of the robotic arm. Visual constraints are modeled using the camera's field of view constraints; By leveraging the knowledge and perception of expert demonstrators, the robotic arm is guided toward the target object through kinesthetic instruction, simultaneously satisfying the motion constraints and the visual constraints, thus generating demonstration data. The process of generating a demonstration trajectory using an optimized trajectory generation strategy combined with the demonstration data includes: Extract the image spatial trajectory and joint spatial trajectory from the demonstration data; The image spatial trajectory and the joint spatial trajectory are trained using dynamic motion primitives to obtain learned shape parameters; Based on the learned shape parameters, an optimized trajectory generation strategy is used to calculate and generate a demonstration trajectory. The image feature tracking error is calculated based on the demonstration trajectory. The cost function and Hamiltonian function of the constrained visual servoing are designed based on the image feature tracking error. The optimal control strategy and optimal control input of the robotic arm are obtained through the Hamiltonian function. The cost function is estimated using a neural network estimator to obtain an estimate of the optimal control strategy, and the neural network estimator is updated based on the Hamiltonian function.

2. The constraint visual servoing method based on imitation learning as described in claim 1, characterized in that: The process of acquiring target object data, robotic arm data, and camera data, and establishing target object models, robotic arm models, and image feature models includes: Establish coordinate systems for the robotic arm base, robotic arm end effector, camera, and target object, respectively; A target object model is established based on the three-dimensional feature points of the target object; Establish a robotic arm model based on the joint angle vectors of the robotic arm manipulator; An image feature model is established based on the camera projection image coordinates of the feature points of the target object.

3. The constraint visual servoing method based on imitation learning as described in claim 2, characterized in that: Calculating image feature tracking error based on the demonstrated trajectory includes: Calculate the image feature tracking error based on the current image feature point trajectory and the demonstration trajectory; The time derivative of the image feature tracking error is calculated based on the image feature tracking error and the image feature Jacobian matrix.

4. The constraint visual servoing method based on imitation learning as described in claim 3, characterized in that: Based on the image feature tracking error, a cost function and a Hamiltonian function for constrained visual servoing are designed. The optimal control strategy and optimal control input for the robotic arm are obtained through the Hamiltonian function, including: Optimal control is performed based on the time derivative of the image feature tracking error, and a cost function is designed. Construct the Hamiltonian function based on the cost function; By deriving the stationarity condition of the Hamiltonian function, the optimal control strategy and the optimal control input of the robotic arm manipulator are obtained.

5. The constraint visual servoing method based on imitation learning as described in claim 4, characterized in that: The cost function is estimated using a neural network estimator to obtain an estimate of the optimal control strategy, and the neural network estimator is updated based on the Hamiltonian function, including: The cost function of the neural network is obtained by estimating the cost function using a neural network. Calculate the gradient of the cost function of the neural network; The neural network estimate of the optimal control strategy is obtained based on the gradient. Calculate the Hamiltonian error between the estimated Hamiltonian function and the ideal Hamiltonian function; The weights of the neural network estimator are updated using an update law that includes minimizing the error function and ensuring the boundedness of the systematic error.

6. A constraint visual servoing system based on imitation learning, employing the constraint visual servoing method based on imitation learning as described in any one of claims 1 to 5, characterized in that, include: The spatial modeling module is used to acquire target object data, robotic arm data, and camera data, and to build target object models, robotic arm models, and image feature models. The constraint learning and trajectory generation module is used to model the constraints of the robotic arm through imitation learning methods, generate demonstration data, and generate demonstration trajectories by combining the demonstration data with an optimized trajectory generation strategy. The controller design module is used to calculate the image feature tracking error based on the demonstration trajectory, design the cost function and Hamiltonian function of the constrained visual servoing based on the image feature tracking error, and obtain the optimal control strategy and optimal control input of the robotic arm through the Hamiltonian function; The weight update module is used to estimate the cost function using a neural network estimator, obtain an estimate of the optimal control strategy, and update the neural network estimator according to the Hamiltonian function.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the constraint visual servoing method based on imitation learning as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the constraint visual servoing method based on imitation learning as described in any one of claims 1 to 5.

Images