A method and system for constrained visual servoing control of a robotic arm based on a sampling predictor

Abstract

This invention relates to the fields of robotic arm control and computer vision technology, and in particular to a method and system for constrained visual servo control of a robotic arm based on a sampling predictor. The method includes: constructing a kinematic model; converting the kinematic model into a linearized model; constructing a state sampling observer model; constructing a soft measurement system model based on soft measurement values; designing a discrete state observer model; shortening the control cycle by updating the time intervals of the state sampling observer model, the soft measurement system model, and the discrete state observer model; constructing a continuous pixel error model; constructing a predictive performance constraint function model; constructing a standard error model and an image feature error transformation model; constructing a preset performance factor model; constructing a control protocol model; and designing a predictor-based dual-rate integral preset performance controller model to keep the target object within the field of view constraint range. Compared with existing technologies, this invention improves the dynamic response performance of visual servo control under large sampling period control and can prevent the target object from escaping the field of view constraint.

Description

Technical Field

[0001] This invention relates to the fields of robotic arm control and computer vision technology, and in particular to a method and system for constrained visual servo control of a robotic arm based on a sampling predictor. Background Technology

[0002] Visual servoing for robotic arms is widely used in unstructured work scenarios, such as handling, positioning, and grasping tasks in dynamic environments. However, the performance of visual servoing is limited by the camera sampling frequency. As the complexity of image processing algorithms increases, the camera sampling frequency decreases, leading to a mismatch between the system's sampling period and control period, thus reducing the system's dynamic response performance and positioning accuracy. Furthermore, since the camera is typically mounted at the end of the robotic arm, the target can easily be lost from the camera's field of view during dynamic following, resulting in positioning task failure. Based on this, existing technologies typically employ the following measures:

[0003] 1. One solution is to transform the limited field of view problem into an optimization control problem, employing model predictive control (MMC) to achieve visual servoing tasks for robotic arms, as seen in patent (CN116774589A, Robust Nonlinear Model Predictive Control for Visual Servo Target Tracking Control, 2023.07.31) and literature (Liu J, Gao J, Yan W. Lyapunov-based ModelPredictive Visual Servo Control of an Underwater Vehicle-Manipulator System[J].IEEE Transactions on Intelligent Vehicles, 2024). This approach utilizes a dynamic model of the controlled object to predict future states, determining the optimal control quantity based on a performance index function within each control cycle. However, MMC often faces challenges such as high online computational load and low response bandwidth because it requires solving a convex optimization problem within each cycle.

[0004] 2. Another strategy is to constrain the robot's field of view by constructing a preset performance function. This mainly utilizes an error transformation to convert the original constrained model into an equivalent unconstrained model, transforming the performance specifications and field of view constraints into error boundaries, as seen in patent (CN116901061A, A Robotic Arm Trajectory Tracking Control Method Based on Preset Performance, 2023.06.16) and literature (Bechlioulis CP, Heshmati-Alamdari S, Karras GC, et al. Robustimage-based visual servoing with prescribed performance under field of view constraints[J].IEEE Transactions on Robotics, 2019, 35(4):1063-1070). This method is highly valuable in visual servoing applications with high safety requirements.

[0005] However, the performance of the aforementioned techniques drops sharply when the camera sampling frequency is low, because the robotic arm cannot achieve effective control within the camera sampling interval. Therefore, improving the dynamic response performance of visual servoing under long sampling period control while preventing the target object from escaping the field of view remains a challenging engineering application problem. Summary of the Invention

[0006] The purpose of this invention is to overcome the defects of the prior art and provide a robotic arm constraint vision servo control method and system based on a sampling predictor.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] This invention provides a constraint-based visual servo control method for a robotic arm based on a sampling predictor, comprising two stages;

[0009] The first stage, narrowing the control cycle, specifically includes the following steps:

[0010] Step 1: Obtain the actual pixel coordinates of the target object's feature points in the two-dimensional image space, convert the target object information into the camera velocity at the end of the robotic arm, and construct a kinematic model using the actual pixel coordinates and their derivatives.

[0011] Step 2: Represent the kinematic model as a nonlinear model, and then transform the nonlinear model into a linearized model;

[0012] Step 3: Based on the linearized model, design the mixed signal and construct the state sampling observer model;

[0013] Step 4: Predict the soft measurement values ​​for subsequent time steps using the state sampling observer model, and construct a soft measurement system model based on the soft measurement values ​​to obtain virtual feature sampling points;

[0014] Step 5: Design a discrete state observer model to obtain estimated values ​​of feature point information in discrete time.

[0015] Step 6: Update the time interval according to the state sampling observer model, the soft measurement system model, and the discrete state observer model to shorten the control cycle;

[0016] The second stage involves keeping the target object within the field of vision at all times, specifically including the following steps:

[0017] Step 7: Based on the set expected pixel position and the actual pixel position, construct a continuous pixel error model;

[0018] Step 8: Set the visibility constraint of feature point pixel coordinates, and select the prediction performance constraint function model according to the continuous pixel error model;

[0019] Step 9: Based on the continuous pixel error model and the prediction performance constraint function model, construct a standard error model according to the standard error of each feature point on the u and v coordinate axes. Based on the standard error model, transform the image feature error of each feature point on the u and v coordinate axes to obtain the image feature error transformation model.

[0020] Step 10: Calculate the derivative of the image feature error transformation model to obtain the preset performance factor of each feature point on the u and v coordinate axes, and construct the preset performance factor model based on the preset performance factor;

[0021] Step 11: Based on the image feature error transformation model and the preset performance factor model, construct a control protocol model based on the preset performance control protocol of integration;

[0022] Step 12: Based on the control protocol model and the state sampling observer model, design a predictor-based dual-rate integral preset performance controller model, and keep the target object within the field of view constraint range according to the dual-rate integral preset performance controller model.

[0023] The kinematic model is expressed by the following formula:

[0024]

[0025] Among them, S t (t)=[u1,v1,u2,v2,u3,v3,u4,v4] T u i and vi These are the values ​​of the i-th feature point on the u and v axes of the planar coordinate system, respectively. L is the derivative of all visual feature point information over continuous time. s (z,S t (t) is the interaction matrix, y(t) is the pixel position, V(t) is the camera velocity at the end of the robotic arm, and C = I 8×8 .

[0026] The linearized model is expressed by the following formula:

[0027]

[0028] in,

[0029] The mixed signal consists of pixel positions and soft measurement values, and is expressed by the following formula:

[0030]

[0031] Wherein, y(t) k ) represents the pixel position, y m (t k () represents a soft measurement value;

[0032] The state sampling observer model is represented by the following formula:

[0033]

[0034] in, It is S t The estimated value of (t), for The derivative of S t (t) represents all visual feature point information over continuous time, P is the observer gain constant, and V(t) k ) represents the joint velocity in a discrete state. It is a mixed signal.

[0035] The soft measurement system model is represented by the following formula:

[0036]

[0037] Among them, s m (t k ) is a matrix composed of the coordinates of virtual feature sampling points, t k-1 V(t) represents the instantaneous update time of the previous controller. k-1 ) represents the joint velocity in a discrete state. dτ is the differential factor.

[0038] The discrete state observer model is represented by the following formula:

[0039]

[0040] in, It is S t (t k The estimated value of t k-1 Indicates the instantaneous update time of the previous controller.

[0041] The continuous pixel error model is expressed by the following formula:

[0042]

[0043] in, and These are the pixel error values ​​of each feature point on the u and v coordinate axes, respectively. i (t) and v i (t) represents the coordinate values ​​on the u and v axes, respectively. and These are the expected coordinates of each feature point on the u and v axes, respectively, and e(t) is derived from... and Composition, expressed as:

[0044]

[0045] The predictive performance constraint function model is expressed by the following formula:

[0046]

[0047] in, and The prediction performance constraint function for each feature point on the u and v coordinate axes. The max function finds the maximum value of a vector, exp is the exponential function, and ρ ∞ is the steady-state performance parameter, and l is the parameter constrained by the time range.

[0048] The standard error model is expressed by the following formula:

[0049]

[0050] in, and The standard error of each feature point on the u and v coordinate axes;

[0051] The image feature error transformation model is expressed by the following formula:

[0052]

[0053] in, and The image feature error transformation values ​​for each feature point on the u and v coordinate axes after image feature error transformation;

[0054] The preset performance factor model is expressed by the following formula:

[0055]

[0056] in, and All are preset performance factors.

[0057] The control protocol model is represented by the following formula:

[0058]

[0059] Where α0, K1, and K2 are the controller adjustment parameter coefficients. σ(t) is determined by a preset performance factor. and The matrix is ​​composed of superpositions, and its expression is: Ξ(t) is a numerical value derived from image feature error transformation. and The matrix is ​​composed of superpositions, and its expression is:

[0060] The dual-rate integral preset performance controller model is represented by the following formula:

[0061]

[0062] in, and They are The estimated value in, All are about S t The parameters of (t), S t (t) is about u i and v i The parameters will and Replacing each u in i and v i get and

[0063] Secondly, the present invention provides a robotic arm constraint visual servo control system based on a sampling predictor, comprising a visual servo system and a control system, wherein the visual servo system includes a robotic arm and a camera, and the control system includes the model constructed above. The robotic arm constraint visual servo control system based on a sampling predictor is used to implement any of the methods described above.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] 1. Based on a preset performance control design method, this invention quantitatively characterizes the transient response performance of a visual servoing system, improves the dynamic response performance of visual servoing under large sampling period control, and can better solve the problems of field of view constraints and excessive initial error.

[0066] 2. The predictor-based robotic arm constraint vision servo control design method proposed in this invention has higher computational efficiency and dynamic response performance compared with traditional preset performance schemes.

[0067] 3. When there is no output measurement during the entire slow sampling period, the present invention uses a linearized model and a state sampling observer model to generate soft measurement values ​​to realize the prediction step, which improves the response speed and robustness of the robotic arm constraint vision servo control system based on the sampling predictor, and also improves the bandwidth of the robotic arm constraint vision servo control system based on the sampling predictor. Attached Figure Description

[0068] Figure 1 This is a schematic diagram of the steps of the present invention;

[0069] Figure 2 This is a system control framework diagram of the predictor-based vision constraint controller of the present invention;

[0070] Figure 3 This is a comparison chart of the pixel error curve effects of the present invention and a traditional preset performance controller in static visual servoing. Detailed Implementation

[0071] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0072] This embodiment, based on the actual situation of low sampling frequency and high controller frequency in large sampling systems, divides the sampling interval into integer intervals as the controller frequency. Soft measurement values ​​are used as information updates for the observation steps, enhancing the overall system's predictive performance and dynamic response speed. Based on the error between the predicted value and the actual state, constraints are considered in the controller design problem description, and a suitable nominal preset performance controller is designed to theoretically and strictly guarantee that the pre-given constraints are met. The design block diagram is as follows: Figure 2 As shown.

[0073] In this specific example, the accuracy specifications of the RealSense camera used here are shown in Table 1:

[0074] Table 1. Accuracy parameters of RealSense cameras

[0075]

[0076] Based on this, this embodiment provides a constraint-based visual servo control method for a robotic arm using a sampling predictor, such as... Figure 1 As shown, the method includes two stages: shortening the control cycle under a large sampling period; and preventing the target from escaping the field of view constraint.

[0077] The first stage involves shortening the control cycle under a large sampling period, specifically including the following steps:

[0078] Step 1): Select 4 feature points in the camera coordinate system and convert the actual three-dimensional position coordinates of the target object into the actual pixel coordinates in the two-dimensional image space. Introduce the end-effector camera speed and construct the kinematic model Ω1 of the visual servoing (IBVS) system based on the actual pixel coordinates and the end-effector camera speed.

[0079] Specifically, to achieve control over the relative pose of the target object, four feature points are selected. There exists an actual 3D position p in the camera coordinate system. i =(X i ,Y i Z i ) T , i = {1, 2, 3, 4}, X i Y i Z i These are the actual coordinates of the feature point along the x, y, and z axes in space, respectively. Get p i The actual pixel coordinates S in the two-dimensional image space i =(u i ,v i ) T u i and v iThese are the coordinates of each feature point on the u and v axes in the two-dimensional image space, respectively, and f is the camera focal length in pixels. Using S... t (t)=[S1 T S2 T S3 T S4 T ] T S represents all visual feature point information over a continuous time period. i T S represents the transpose of the coordinates of each actual pixel. t (t k ) represents all visual feature point information in discrete time.

[0080] The velocity of the camera at the end of the robotic arm is defined as V(t) = [v cx ,v cy ,v cz ,w cx ,w cy ,w cz ] T v cx v cy v cz It is the instantaneous linear velocity w at the origin of the camera frame within space. cx w cy w cz It is the instantaneous angular velocity of the camera frame origin in space. The interaction matrix of each feature point... The overall interaction matrix of the visual servoing system is obtained by superposition: This represents the transpose of the interaction matrix for each feature point. z = [Z1, Z2, Z3, Z4] T .

[0081] Based on this, the kinematic model Ω1 of the IBVS system is obtained:

[0082]

[0083] Where C = I 8×8 , For S t The derivative of y(t), where y(t) is the pixel position output by the IBVS system.

[0084] Step 2): Based on the kinematic model Ω1 of the IBVS system in Step 1), express it as a nonlinear model Ω2, for the current operating point (S op V op ), S op V represents the pixel coordinates captured at the current working point. opThe camera speed at the current working point is represented by the nonlinear model Ω2, which is used to obtain the linearized model Ω3 corresponding to the current working point. By summarizing the linearized model Ω3, a simplified linearized model Ω4 is obtained.

[0085] The nonlinear model Ω2 is expressed by the following formula:

[0086]

[0087] Here, Θ is a nonlinear function.

[0088] The linearized model Ω3 is expressed by the following formula:

[0089]

[0090] in,

[0091] The linearized model Ω4 is expressed by the following formula:

[0092]

[0093] in,

[0094] Step 3): Based on the simplified linearized model Ω4 obtained in Step 2), design a mixed signal and construct a state sampling observer model Ω5 based on the mixed signal.

[0095] The state sampling observer model Ω5 is represented by the following formula:

[0096]

[0097] in, for The estimated value, It is S t The estimated value of (t), It is S t (t k The estimated value of V(t), where P is the observer gain constant. k ) represents the joint velocity in a discrete state, which occurs in relatively fast time intervals T. c Updates, for example, time series {0,T} c 2T c ,...}. And the pixel position y(t) represents the output of the IBVS system in the discrete state. k Provides a relatively slow update time t at the sampling time. k =k s T s =k s MT c , tk Indicates the current instantaneous update time of the controller, k, k s M and M are positive integers.

[0098] Mixed-signal design

[0099] Among them, y m (t k ) represents the soft measurement value, the actual measured value y(t) at the instant of sampling. k At each sampling instant, k = k s M is obtained from the actual feedback from the camera.

[0100] Step 4): Based on the simplified linearized model Ω4 from Step 2) and the state sampling observer model Ω5 from Step 3), predict the subsequent time step k≠k s Soft measurement value y at time M m (t k Based on soft measurement value y m (t k To obtain virtual feature sampling points, a soft measurement system model Ω6 is constructed.

[0101] The soft measurement system model Ω6 is represented by the following formula:

[0102]

[0103] in, dτ is the differential factor, s m (t k ) is a matrix composed of the coordinates of virtual feature sampling points, t k-1 This indicates the instantaneous update time of the previous controller.

[0104] Step 5): At the actual sampling instant, i.e., k = k s M+1,k s M+2,...(k s When M is +1), the current discrete state observer model Ω7 is obtained to obtain the estimated value of feature point information in discrete time.

[0105] The discrete state observer model Ω7 is represented by the following formula:

[0106]

[0107] in, It is S t (t k The estimated value of t k-1 Indicates the instantaneous update time of the previous controller.

[0108] Step 6): Update the time interval T based on the state sampling observer model Ω5, the soft measurement system model Ω6, and the discrete state observer model Ω7. c This shortens the control cycle.

[0109] The second stage: Based on the performance constraint function model, ensure that the target object is always within the set field of view constraint range, which specifically includes the following steps:

[0110] Step 7): Assuming the QR code coordinate position is continuously measurable, 6. Based on the set expected pixel position and actual pixel position, the continuous pixel error model is Ω8.

[0111] The continuous pixel error model is Ω8:

[0112]

[0113] in, and These are the expected coordinates of each feature point on the u and v axes, respectively. and These are the pixel error values ​​of each feature point on the u and v coordinate axes, respectively. Ω8 can be further expressed as e(t) = S t (t)-S d ,in and

[0114] Step 8): Set u i and v i Subject to visibility constraints: u min <u i <u max v min <v i <v max u max It is the maximum constraint value on the u-axis, u min It is the minimum constraint value on the u-axis, v max It is the maximum constraint value on the v-axis, v min It is the minimum constraint value on the v-axis.

[0115] The prediction performance function model is selected based on the continuous pixel error model Ω8. Definition and The prediction performance constraint function for each feature point on the u and v coordinate axes is expressed as follows:

[0116]

[0117] in, The `max` function finds the maximum value in a given vector, and `exp` is the exponential function. ρ∞ is the steady-state performance parameter, and l is the parameter of the constraint time range designed in this embodiment, which is related to the convergence time to be selected in this invention.

[0118] Step 9): Using the continuous pixel error model Ω8 from Step 7) and the prediction performance constraint function model Ω9 from Step 8), define the standard error of each feature point on the u and v coordinate axes. and Standard error model Ω 10 :

[0119] According to the standard error model Ω 10 The image feature error transformation value after image feature error transformation for each feature point on the u and v coordinate axes is defined as... and The image feature error transformation model Ω is obtained. 11 :

[0120]

[0121] The transformations of each image feature error are superimposed, and defined as a matrix.

[0122] Step 10): For the image feature error transformation model Ω in step 9), 11 Calculate the derivative to obtain the preset performance factor for each feature point on the u and v coordinate axes. and Preset performance factor model Ω 12 :

[0123]

[0124] The determined preset performance factors are superimposed and defined as a matrix:

[0125] Step 11): Based on the image feature error transformation model Ω in step 9). 11 And the preset performance factor model Ω in step 10) 12 The pre-defined control protocol V based on integration * (t) is designed as a control protocol model Ω 13 :

[0126]

[0127] Where α0, K1, and K1 are the controller adjustment parameter coefficients. σ(t) is determined by a preset performance factor. and The matrix is ​​composed of superpositions, and its expression is: Ξ(t) is a numerical value derived from image feature error transformation. and The matrix is ​​composed of superpositions, and its expression is:

[0128] Step 12): Based on the control protocol model Ω in Step 11). 13 And in step 3), design a predictor-based dual-rate integral preset performance controller model Ω5. 14 The dual-rate integral preset performance controller model Ω 14 Used to constrain the field of view of a target object, ensuring that the target object is always within the set field of view constraint range.

[0129] Specifically, the dual-rate integral preset performance controller model Ω 14 The following design is proposed:

[0130]

[0131] in, and They are Specifically, the estimated value in the middle, All are about S t The parameters of (t), It represents all visual feature point information over continuous time, and is also related to the coordinate values ​​u on the u and v axes in the two-dimensional image space. i and v i Therefore, and Replacing each u in i and v i get and K1 and K2 are the controller adjustment parameter coefficients.

[0132] like Figure 2 The diagram shown is a system control framework diagram of the predictor-based field-of-view constraint controller of the present invention; the S is fed back based on the coordinate values ​​of the target object feature points on the u and v axes in the two-dimensional image space. t (t), the sampled signal y(t) obtained through the camera k ) and soft measurement value y m (t k A mixed signal was constructed. Then, a discrete-state observer model Ω7 was established to obtain estimates of feature point information in discrete time. Meanwhile, this embodiment introduces a preset performance function model Ω9 to handle the feature point position constraint problem. To provide a feature point position constraint for the image-based visual servoing (IBVS) system, this embodiment designs a virtual, integral-based preset performance control protocol V. * (t). Finally, based on the control protocol model and state sampling observer model designed above, the dynamic response performance of visual servoing is improved under large sampling period control, and the target object can be prevented from escaping the field of view constraint.

[0133] Furthermore, the control parameter adjustment rules involved in the adjustable-time static target capture controller of the present invention are as follows:

[0134] 1) The predictor gain parameter P is used to adjust the accuracy of the predictor's prediction state, satisfying the constraint P > 0. The observer operates at a relatively long time T. s Update, while the output measurement is only performed at the actual slow sampling instant t=k. s MT c =k s T s Available, where M is an integer. Generally, M should not be too large. The predictor gain parameter P is usually adjusted before adjusting the controller parameters. The value of P should not be too large or too small. The larger the parameter, the faster the estimated state converges, but the estimation accuracy will also decrease, and the initial estimation error will be large, which may damage the dynamic performance of the system. Generally, this parameter needs to be weighed according to the actual application.

[0135] 2) The preset performance control gain parameters K1 and K2 are used to improve the system's response speed, i.e., to adjust the system's bandwidth to satisfy K1 > 0 and K2 > 0, with K1 being relatively large and K2 relatively small. The transient performance parameter l = 0.8 is used to force the exponential convergence command, satisfying l > 0. The selection of l is inversely related to the convergence time; the larger l is, the faster the initial convergence speed. The maximum and minimum constraint values ​​of the u and v coordinate axes of the two-dimensional image plane are u... min =-300,u max =300, v min =-215, v max =215 is used to constrain the camera's field of view, and is usually smaller than the camera's actual field of view. Steady-state performance parameter ρ ∞ =10, when the visual servoing system is in steady state, the pixel error can converge to within 20 pixels, satisfying ρ ∞ >0. It should be noted that when the visual servoing system is in a steady state, the range of constraints it faces is mainly related to ρ. ∞ Related, ρ ∞ The smaller the value of ρ, the narrower the constraint band of the controller when disturbances exist in the steady state. This makes the robotic arm more prone to excessive response due to excessive energy. However, ρ...∞ The smaller the value of ρ, the better the initial control transient performance, so ρ ∞ This parameter needs to be weighed according to the actual application scenario.

[0136] Here, a set of static visual information is provided to illustrate the effectiveness of the method proposed in this embodiment:

[0137] Given the static visual QR code location information S(0) = [4517255-119-240-128-251169] T The desired target position is S. * =[100100100-100-100-100-100100] T .

[0138] Based on the parameter selection rules and the robotic arm used in this example, the control parameter settings for this invention are shown in Table 2:

[0139] Table 2

[0140]

[0141] In addition, to compare the effectiveness of the control scheme, it is compared with the predictive performance controller (PPC) design currently used for applications that limit pixel location. Here, the PPC design is as follows: V1(t) represents the speed of the robotic arm camera under the PPC controller, and K3 is a positive constant.

[0142] The selection of control parameters is shown in Table 3:

[0143] Table 3

[0144]

[0145] Figure 3 In static vision servoing, such as Figure 3 (b) The blue solid line represents the pixel error curve of the method proposed in this invention, and the red dashed line represents the preset performance constraint curve of this invention. The horizontal axis represents time, and the vertical axis represents... and These are the pixel error coordinates on the u and v axes, such as Figure 3 (a) The blue solid line represents the pixel error curve of the predictive performance controller, and the red dashed line represents the preset performance constraint curve of the predictive performance controller. The horizontal axis represents time, and the vertical axis represents... and These are the pixel error coordinates on the u and v axes. The experimental results clearly show that the predictor-based field-of-view constraint controller design method provided by this invention can significantly accelerate static capture speed, improve dynamic response performance, and simultaneously limit errors within a specified range. Specifically, the image feature point error curves show that the predictor module provided by this invention only needs to change one parameter, enabling the observer to fully utilize each sampling position. While saving visual costs, it performs predictions within the sampling interval. Simultaneously, the preset performance indicators effectively limit the convergence time and range of static capture. Compared to a preset performance controller without a predictor, it shows a significant improvement in stability, thereby shortening the convergence time, improving the dynamic response and robustness of the servo system, and simplifying performance tuning.

[0146] Secondly, this embodiment provides a robotic arm constraint visual servo control system based on a sampling predictor, including a visual servo system and a control system. The visual servo system includes a robotic arm and a camera, and the control system includes the model constructed above. The robotic arm constraint visual servo control system based on a sampling predictor is used to implement the method mentioned in this embodiment.

[0147] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A constraint-based visual servo control method for a robotic arm based on a sampling predictor, characterized in that, It includes two stages; The first stage, narrowing the control cycle, specifically includes the following steps: Step 1: Obtain the actual pixel coordinates of the target object's feature points in the two-dimensional image space, convert the target object information into the camera velocity at the end of the robotic arm, and construct a kinematic model using the actual pixel coordinates and their derivatives. Step 2: Represent the kinematic model as a nonlinear model, and then transform the nonlinear model into a linearized model; Step 3: Based on the linearized model, design the mixed signal and construct the state sampling observer model; The mixed signal consists of pixel positions and soft measurement values, and is expressed by the following formula: ； in, For pixel position, These are soft measurement values; The state sampling observer model is represented by the following formula: ； in, yes The estimated value, for The derivative, This represents all visual feature point information over a continuous time period. It is the observer gain constant. Represents the joint velocity in a discrete state. It is a mixed signal; Step 4: Predict the soft measurement values ​​for subsequent time steps using the state sampling observer model, and construct a soft measurement system model based on the soft measurement values ​​to obtain virtual feature sampling points; The soft measurement system model is represented by the following formula: ； in, It is a matrix composed of the coordinates of virtual feature sampling points. Indicates the instantaneous update time of the previous controller. Represents the joint velocity in a discrete state. , , , It is a differential factor; Step 5: Design a discrete state observer model to obtain estimated values ​​of feature point information in discrete time. The discrete state observer model is represented by the following formula: ； in, yes The estimated value, Indicates the instantaneous update time of the previous controller. , , , ; Step 6: Update the time interval according to the state sampling observer model, the soft measurement system model, and the discrete state observer model to shorten the control cycle; The second stage involves keeping the target object within the field of vision at all times, specifically including the following steps: Step 7: Based on the set expected pixel position and the actual pixel position, construct a continuous pixel error model; Step 8: Set the visibility constraint of feature point pixel coordinates, and select the prediction performance constraint function model according to the continuous pixel error model; Step 9: Based on the continuous pixel error model and the prediction performance constraint function model, according to each feature point in... and A standard error model is constructed using the standard errors on the coordinate axes. Based on this standard error model, each feature point is... and The image feature error on the coordinate axis is transformed to obtain the image feature error transformation model; Step 10: Calculate the derivative of the image feature error transformation model to obtain the value of each feature point. and Preset performance factors on the coordinate axes; construct a preset performance factor model based on the preset performance factors. Step 11: Based on the image feature error transformation model and the preset performance factor model, construct a control protocol model based on the preset performance control protocol of integration; Step 12: Based on the control protocol model and the state sampling observer model, design a predictor-based dual-rate integral preset performance controller model, and keep the target object within the field of view constraint range according to the dual-rate integral preset performance controller model. The control protocol model is represented by the following formula: ； in, , and Adjust the parameter coefficients for the controller. , , , Based on preset performance factors and The matrix is ​​composed of superpositions, and its expression is: , Numerical transformation from image feature error and The matrix is ​​composed of superpositions, and its expression is: ; The dual-rate integral preset performance controller model is represented by the following formula: ； in, , and They are , , The estimated value in, , , All are about and The parameters will and Replacing each , , In and get , and .

2. The constraint-based visual servo control method for a robotic arm based on a sampling predictor according to claim 1, characterized in that, The kinematic model is expressed by the following formula: ； in, , and They are the first The feature points in the plane coordinate system and The values ​​of the two coordinate axes, This is the derivative of all visual feature point information over continuous time. The interaction matrix, For pixel position, For the speed of the end-effector camera, .

3. The constraint-based visual servo control method for a robotic arm based on a sampling predictor according to claim 2, characterized in that, The linearized model is expressed by the following formula: ； in, , , For the speed of the end-effector camera, .

4. The constraint-based visual servo control method for a robotic arm based on a sampling predictor according to claim 1, characterized in that, The continuous pixel error model is expressed by the following formula: ； in, and Each feature point is located at and Pixel error values ​​on the coordinate axis and They are and Coordinate values ​​on the coordinate axes and Each feature point is located at and Desired coordinate values ​​on the coordinate axes Depend on and Composition, expressed as: ， ， ； The predictive performance constraint function model is expressed by the following formula: ： ； in, and For each feature point and Prediction performance constraint functions on the coordinate axes , , , The max function finds the maximum value in a vector, and exp is an exponential function. These are steady-state performance parameters. The parameter constrains the time range.

5. The constraint visual servo control method for a robotic arm based on a sampling predictor according to claim 4, characterized in that, The standard error model is expressed by the following formula: ： ， in, and For each feature point and Standard error on the coordinate axes; The image feature error transformation model is expressed by the following formula: ： ， in, and The image feature error transformation values ​​for each feature point on the coordinate axes after image feature error transformation; The preset performance factor model is expressed by the following formula: ： ， in, and All are preset performance factors.

6. A constraint-based visual servo control system for a robotic arm based on a sampling predictor, characterized in that, The system includes a visual servoing system and a control system, wherein the visual servoing system includes a robotic arm and a camera, and the control system includes the model constructed above. The robotic arm constrained visual servoing control system based on the sampling predictor is used to implement the method described in any one of claims 1-5.

Images