A robot visual servo predictive control method based on neurodynamics

Abstract

The application relates to a kind of robot vision servo prediction control methods based on neurodynamics, construct vision servo error model based on neurodynamics;By comparing the current actual coordinate value and reference coordinate value of feature point at k time, the error of current position feature point coordinate and expected feature point coordinate is obtained through coordinate transformation, is brought into vision servo error model, and the processed error signal vector is obtained by neurodynamics model processing, the objective function of vision servo error model is set, the vision servo prediction model is constructed, the optimal solution of control signal is solved, and the first column element of optimal solution is used to robot control;Repeat until robot reaches target point.The controller of the application can not only drive robot to reach target point under the condition of effectively processing constraint, but also can produce relatively smooth continuous control signal, so that the control amount generated will not produce mutation, thereby improving the control performance of system, and making the control effect more excellent.

Description

Technical Field

[0001] This invention relates to the field of robot visual servo control technology, and in particular to a robot visual servo predictive control method based on neurodynamics. Background Technology

[0002] In recent years, as incomplete wheeled mobile robots have been widely used in various industries, their inherent limitations have attracted attention. For example, incomplete wheeled mobile robots can only move along the direction of the wheels and cannot rotate along the axis. In contrast, omnidirectional mobile robots can achieve movement in multiple directions and can rotate around their own center of gravity with zero radius. Therefore, they have gradually become an emerging research field in wheeled mobile robots.

[0003] Currently, control methods for wheeled mobile robots include sliding mode control, inverse stepping control, and adaptive control. However, due to the abrupt changes in initial speed and torque, it is difficult for robot drive devices such as motors to provide sufficient torque to allow for a significant change in speed within a short period of time, resulting in these methods not being ideal in practical applications.

[0004] In robot control, visual servo control is a common method. However, traditional visual servo control systems are not very adept at handling constraints of robot hardware and environment. Summary of the Invention

[0005] This invention addresses the problems existing in the prior art and provides a robot visual servoing predictive control method based on neurodynamics. It uses a neurodynamic model to process the jumps in error signals during robot motion, establishes a visual servoing error model based on neurodynamics, and uses model predictive control for visual servoing control.

[0006] The technical solution adopted in this invention is a robot visual servoing predictive control method based on neurodynamics. This method involves constructing a visual servoing error model based on neurodynamics; comparing the current actual coordinates and expected coordinates of feature point P at time k, and after coordinate transformation, obtaining the error between the current and expected feature point coordinates, which is then substituted into the visual servoing error model. A neurodynamics model is then introduced to obtain a processed error signal vector. Based on this neurodynamically processed error signal vector, an objective function for the visual servoing error model is set, a visual servoing prediction model is constructed, and the optimal solution for the control signal is solved. The first column of the optimal solution is then applied to the robot's control. These steps are repeated until the robot finally reaches the target point.

[0007] Preferably, constructing a visual servoing error model based on neurodynamics includes the following steps:

[0008] Step 1.1: Provide the visual servoing error model for the omnidirectional mobile robot. The error signal vector in the omnidirectional mobile robot visual servoing error model is e = [e1 e2 e3]. T ;

[0009] Step 1.2: Introduce a neurodynamic model to process the jumps in the error signal, resulting in a new error signal vector E = [E1 E2 E3] after processing by the neurodynamic model. T ;

[0010] Step 1.3: Use the new error signal vector E = [E1 E2 E3] T Replace the error signal vector e = [e1 e2 e3] in the visual servoing error model. T This yields a visual servoing error model based on neurodynamics. In the above steps, T represents the transpose of the vector.

[0011] Preferably, in step 1, an error signal vector e = [e1 e2 e3] is introduced. T satisfy

[0012]

[0013] Where f is the focal length of the monocular camera, θ is the angle between the world coordinate system and the robot coordinate system, called the heading angle, with counterclockwise as positive, (x p ,y p (x) represents the coordinates of the current feature point P in the image's physical coordinate system. pd ,y pd θ represents the desired coordinates of feature point P in the physical coordinate system of the image; d Let θ be the robot's desired heading angle. e This represents the error between the actual heading angle and the desired heading angle, and satisfies θ. e =θ-θ d .

[0014] Preferably, in step 1, the error signal vector e = [e1 e2 e3] T Satisfying the visual servo error model

[0015]

[0016] in, Let α be the derivative of the error signal vector, and let α be the height coefficient, satisfying... Z c Let feature point P be the coordinates (X, Y, X) of the camera coordinate system. c ,Y c Z cThe value along the Zc axis represents the height of feature point P; [v x v y ω] T v is the velocity vector of the omnidirectional mobile robot. x and v y Here, ω represents the lateral and longitudinal linear velocities of the omnidirectional mobile robot, respectively; ω represents the angular velocity of the omnidirectional mobile robot's rotation; and d represents the origin O of the camera coordinate system. c And the origin O of the robot coordinate system r The distance.

[0017] To address the issue of speed jumps affecting control performance during the control process, a visual servoing error model based on neurodynamics is proposed, where the error signal vector e = [e1 e2 e3] T A conventional neurodynamic model is introduced to process the error signal vector, reducing abrupt changes in the error signal and generating a smooth and continuous control signal. The conventional neurodynamic model is described as follows:

[0018]

[0019] Among them, C m It is the cell membrane capacitance, V m It is the transmembrane voltage, E Na and E K These are the saturation potentials of sodium and potassium ions, E, respectively. p It is the passive leakage current in the membrane, g Na g K and g p These are the conductances of sodium ions, potassium ions, and the passive channel, respectively.

[0020] In this invention, a neurodynamic model is used to handle the problem of error signal jumps, and the processed error signal vector is then used for visual servo stabilization.

[0021] Preferably, in step 1.2, for equation (3), let C m =1,V m =E i E p =0,E Na =B i E k =D i ,g p =A i ,g Na =f(e i ),g K =g(e i ), where e = [e1 e2 e3] TThis represents the error signal vector in the original visual error model, E = [E1E2 E3]. T Let represent the error signal vector after neurodynamic processing, where i = 1, 2, 3. The error signal vector e = [e1 e2 e3] T By introducing a general neurodynamic model, we obtain

[0022]

[0023]

[0024] Among them, A i e represents the corresponding error signal vector i passive decay rate, B i and D i These represent the corresponding error signal vectors e i The upper and lower bounds, A i B i and D i All are greater than 0; f(e) i ) and g(e i ) represent the corresponding error signal vectors e i The upper and lower bounds of the threshold linear function are defined; through the processing of the above model, the jump of each error is limited to the [-D] of the corresponding error signal vector. i B i Within the range.

[0025] Preferably, the error signal vector E = [E1 E2 E3] processed by the neurodynamic model is used. T Replace the error signal vector e = [e1 e2 e3] T In step 1.3, the visual servoing error model based on neurodynamics is as follows:

[0026]

[0027] in This is the derivative of the processed error signal vector E. The original error signal vector e is transformed into a new error signal vector E through the neurodynamic model, and then the processed error signal vector E is used for subsequent control.

[0028] Preferably, in order to effectively handle constraints and enable the visual servoing task to be executed efficiently, this invention considers using model predictive control for controller design; constructing the visual servoing prediction model includes the following steps:

[0029] Step 2.1: Perform Euler discretization on the neurodynamic-based visual servoing error model to obtain the corresponding discrete-time visual servoing error model;

[0030] Step 2.2: Define the objective function of the visual servoing error model of the omnidirectional mobile robot to obtain the visual servoing prediction model of the omnidirectional mobile robot based on neurodynamics;

[0031] Step 2.3: Obtain the visual servo prediction controller for the omnidirectional mobile robot based on the visual servo prediction model.

[0032] Preferably, in step 2.1, the discrete-time visual servoing error model based on neurodynamics satisfies the following:

[0033]

[0034] Among them, T s Sampling time;

[0035] Distinguishing between the error signal vector E and the control input vector u, we obtain E(k+1) = A(k)E(k) + B(k)u(k), where,

[0036]

[0037]

[0038] Here, u(k) represents the control input vector, i.e., u(k) = [v x (k) v y (k) ω(k)] T By rewriting the above formula in a more compact form, we get...

[0039] E(k+1)=h(E(k),u(k)) (10)

[0040] E(k+1) is the value of the error signal vector E(k) at the next sampling time after processing by the neurodynamic model, and h(·) represents the corresponding function mapping relationship.

[0041] Preferably, the objective function of the omnidirectional mobile robot visual servo error model is,

[0042]

[0043] Where Q and R are weight matrices, the former reflecting the system's ability to track feature points, and the latter reflecting the system's requirements for control constraints; N c and N pLet E(k+i|k) represent the control time domain and the prediction time domain, respectively. Their magnitudes affect the system's response speed. E(k+i|k) represents the predicted value of the error signal vector E at time k to time k+i after neurodynamic processing, and u(k+i-1|k) represents the predicted value of the control input u(k) at time k to time k+i-1.

[0044] A visual servoing prediction model based on neurodynamics for omnidirectional mobile robots.

[0045] E(k+i|k)=h(E(k+i-1|k),u(k+i-1|k)) (12)

[0046] Preferably, considering the robot's hardware constraints and visual visibility constraints, the omnidirectional mobile robot's visual servo predictive controller is...

[0047]

[0048] stE(k|k)=E0 (13b)

[0049] E(k+i|k)=h(E(k+i-1|k),u(k+i-1|k)),i=[1,Np] (13c)

[0050] u pmin ≤u p ≤u pmax ,v pmin ≤v p ≤v pmax (13d)

[0051] v min ≤v≤v max ,ω min ≤ω≤ω max (13e)

[0052] Where E0 represents the initial time-of-flight error signal vector after neurodynamic processing, u * (k) represents the optimal solution to the optimization problem; u p and v p Let (13d) represent the coordinates of the feature point in the physical coordinate system of the image, and let (13d) represent the visual visibility constraint of the mobile robot; v and ω represent the linear velocity and angular velocity of the mobile robot, respectively, and let (13e) represent the velocity constraint of the mobile robot; where the subscripts min and max represent the minimum constraint and maximum constraint of the corresponding data, respectively.

[0053] This invention relates to a robot visual servoing predictive control method based on neurodynamics. It constructs a visual servoing error model based on neurodynamics. By comparing the current actual coordinates and reference coordinates of feature point P at time k, and performing coordinate transformation, the error between the current position feature point coordinates and the desired feature point coordinates is obtained. This error is then substituted into the visual servoing error model and processed by the neurodynamics model to obtain the processed error signal vector E. Based on the error signal vector E, the objective function J(k) of the visual servoing error model is set, a visual servoing prediction model is constructed, and the optimal solution u of the control signal is solved. * (k), with the optimal solution u * The first column of (k) controls the robot; repeat the above steps until the robot finally reaches the target point.

[0054] Through this invention, an omnidirectional mobile robot moves towards a target point. Under the influence of the neurodynamic model, the controller can generate relatively continuous and smooth control signals, enabling the omnidirectional mobile robot to reach the target point smoothly.

[0055] The beneficial effects of this invention are that, based on the visual servoing error model of an omnidirectional mobile robot, a neurodynamic-based visual servoing error model is established. On this basis, a controller is designed in conjunction with a model predictive control strategy. This controller can not only drive the robot to the target point while effectively handling constraints, but also generate a relatively smooth and continuous control signal so that the control quantity generated does not produce abrupt changes, thereby improving the control performance of the system and making the control effect more excellent. Attached Figure Description

[0056] Figure 1 This is a top view schematic diagram of the omnidirectional mobile robot in this invention;

[0057] Figure 2 This is a schematic diagram of the omnidirectional mobile robot of the present invention in a coordinate system;

[0058] Figure 3 This is a control flow diagram of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] This embodiment discloses a visual servoing predictive control method for an omnidirectional mobile robot, the specific steps of which are as follows:

[0061] When an omnidirectional mobile robot moves in a plane, its kinematic equations are as follows:

[0062]

[0063] Where (x, y) represents the centroid of the omnidirectional mobile robot in the world coordinate system X. w -Y w In planar coordinates, θ represents the angle between the world coordinate system and the robot coordinate system, called the heading angle, with counterclockwise being positive. x and v y ω represents the longitudinal and lateral linear velocities of the omnidirectional mobile robot, and ω represents the angular velocity of the omnidirectional mobile robot.

[0064] First, consider the transformation between the world coordinate system and the robot coordinate system. Essentially, the changes in the robot's coordinate system during subsequent motion are simply changes around the Z-axis of the world coordinate system. c Since the axes rotate and translate in the plane, we can write the transformation equations between the world coordinate system and the robot coordinate system in the following form.

[0065]

[0066] Where (X) W ,Y W Z W (X) represents the robot's coordinates in the world coordinate system. r ,Y r Z r () represents the robot's coordinates in the robot's coordinate system. Simultaneously, we consider the transformation between the robot's coordinate system and the camera's coordinate system.

[0067]

[0068] Where (X) c ,Y c Z c ) represents the coordinates of the current feature point in the camera coordinate system, and d represents the origin O of the camera coordinate system. c And the origin O of the robot coordinate system r The distance.

[0069] Considering the transformation between the camera coordinate system and the image physical coordinate system, we have the following formula:

[0070]

[0071] Where f is the focal length of the camera, (x p ,yp ) represents the coordinates of the feature point P at the current location in the image's physical coordinate system.

[0072] Based on the above results, a mobile robot-camera-feature point model was established.

[0073] The mobile robot acquires feature point information of the surrounding environment through a monocular camera mounted on top. By comparing the acquired information with stored feature point information, the error is calculated to control the robot's movement. Therefore, it is first necessary to establish the relationship between the mobile robot and the monocular camera. Combining equations (2) and (3), differentiating and simplifying them yields...

[0074]

[0075] Combining the kinematic model of the mobile robot (1), we can obtain

[0076]

[0077] The new error signal vector is introduced as follows:

[0078]

[0079] (x pd ,y pd θ represents the desired coordinates of feature point P in the physical coordinate system of the image. d θ represents the robot's expected heading angle. e Let θ represent the error of the robot's heading angle, and satisfy θ e =θ-θ d This represents the error between the expected heading angle and the actual heading angle.

[0080] Differentiating equation (7) and combining it with the kinematic model (1), the visual servoing error model of the omnidirectional mobile robot is described as follows:

[0081]

[0082] like Figure 2 As shown, where This represents the altitude coefficient, Z. c This represents the coordinates (X, Y) of feature point P in the camera coordinate system. c ,Y c Z c (Z) Central axis c The numerical value of the direction represents the height of the feature point P. e = [e1 e2 e3] T The error signal vector, T represents the derivative of the error signal vector, where T represents the transpose of the matrix.

[0083] First, preprocess the image read by the camera at the current moment to obtain the coordinates (x, y, y) of feature point P in the physical coordinate system of the image. p ,y p ), then the obtained (x p ,y p ) and the coordinates (x) of feature point P in the desired image pd ,y pd The error vector is obtained by performing calculations. For the heading angle error, it can be obtained by estimating the robot's current heading angle and comparing it with the desired heading angle.

[0084] Introduction of neurodynamic models:

[0085] Considering that speed jumps can affect control performance, and error jumps can lead to speed jumps, this invention introduces a neurodynamic model to process the error signal obtained in the above process, thereby generating a smoother control signal.

[0086] The error signal vector is defined as e = [e1 e2 e3]. T Introducing neurodynamics, we can obtain the following equation.

[0087]

[0088] in

[0089]

[0090] A i Represents the corresponding error signal vector e i passive decay rate, B i and D i Represents the corresponding error signal vector e i The upper and lower bounds, A i B i and D i All are greater than 0. f(e) i ) and g(e i ) are the corresponding error signal vectors e i The upper and lower bounds of the threshold linear function are defined. Error jumps will be limited to the corresponding error signal vector e. i [-D] i B i Within the range of ], where i = 1, 2, 3.

[0091] Next, we use the new error signal vector E = [E1 E2 E3] T Replace the error signal vector e = [e1 e2 e3] in the visual servoing error model (8). T Then the visual servoing error model can be transformed as follows:

[0092]

[0093] in It is the derivative of the processed error signal vector E.

[0094] Visual servo predictive controller design

[0095] Model predictive control is a good approach to effectively handle the constraints imposed by hardware in control systems.

[0096] The sampling time is T. s By discretizing equation (11) using Euler, we can obtain the corresponding discrete-time visual servo error system:

[0097]

[0098] Separating the error signal vector E(k) and the control input vector u(k) yields:

[0099] E(k+1)=A(k)E(k)+B(k)u(k) (13)

[0100] in:

[0101]

[0102]

[0103] Equation (13) can be written in a more compact form, namely:

[0104] E(k+1)=h(E(k),u(k)) (16)

[0105] Where E(k+1) is the error signal vector processed by the neurodynamic model, and u(k) is the control input vector, where u(k) = [v x (k) v y (k) ω(k)] T h(·) represents the corresponding function mapping relationship.

[0106] Considering the omnidirectional mobile robot visual servoing error system (16) after processing by the neurodynamic model, in order to effectively handle the constraints of robot hardware and visual visibility, the objective function of the omnidirectional mobile robot visual servoing error model is defined as follows:

[0107]

[0108] Where Q and R are weight matrices, the first term reflects the system's ability to track the desired point, and the second term reflects the requirements for control constraints. c and Np Let E(k+i|k) represent the control time domain and the prediction time domain, respectively. The magnitude of both will affect the speed of the control response. E(k+i|k) represents the predicted value of the error signal vector E at time k to time k+i after neurodynamic processing, and u(k+i-1|k) represents the predicted value of the control input vector u at time k to time k+i-1.

[0109] The quadratic objective function (17) of the omnidirectional mobile robot yields the following visual servoing prediction model based on neurodynamics:

[0110] E(k+i|k)=h(E(k+i-1|k),u(k+i-1|k)) (18)

[0111] Considering the hardware constraints of the robot itself, the visual servo predictive controller for the omnidirectional mobile robot is...

[0112]

[0113] stE(k|k)=E0 (19b)

[0114] E(k+i|k)=h(E(k+i-1|k),u(k+i-1|k)),i=[1,Np] (19c)

[0115] u pmin ≤u p ≤u pmax ,v pmin ≤v p ≤v pmax (19d)

[0116] v min ≤v≤v max ,ω min ≤ω≤ω max (19e)

[0117] Where E0 represents the initial time-time error signal vector after neurodynamic processing, u * (k) represents the optimal solution to the optimization problem; (19b)~(19e) are the constraints that the optimization problem must satisfy, where u p and v p Let (19d) represent the coordinates of the feature point in the physical coordinate system of the image, and let v and ω represent the linear velocity and angular velocity of the mobile robot, respectively. Let (19e) represent the velocity constraint of the mobile robot. The subscripts min and max represent the minimum and maximum constraints of the corresponding data, respectively.

[0118] like Figure 3As shown in the given control block diagram, the control flow of this invention is as follows: First, the coordinate values ​​of the target feature point and the target feature point P in the image captured by the monocular camera are compared. After a series of coordinate transformations, the visual servo error e = [e1 e2 e3] is obtained. T Secondly, since error jumps are a major factor causing velocity jumps, a neurodynamic model is used to process the error, resulting in an error signal vector E = [E1 E2 E3] processed by the neurodynamic model. T Next, based on the error signal vector E = [E1 E2 E3] obtained above... T Substitute the input into the model predictive controller to solve for the optimal solution of the control signal u(k); then convert the optimal solution u... * The first column of (k) acts on the robot's control process. The speeds of the three omnidirectional wheels are solved by the robot's inverse kinematics model and the robot's kinematics model to control the robot. All of the above work is repeated in every subsequent sampling cycle until the robot can finally settle at the target point.

Claims

1. A robot visual servoing predictive control method based on neurodynamics, characterized in that: By comparing the current actual coordinates and reference coordinates of feature point P at time k, and performing coordinate transformation, the error between the current feature point coordinates and the desired feature point coordinates is obtained, thus constructing a visual servoing error model. Then, a neurodynamic model is used to process the error signal vector in the visual servoing error model, resulting in a processed error signal vector. The construction of a neurodynamic-based visual servoing error model includes the following steps: Step 1.1: Obtain the visual servoing error model of the omnidirectional mobile robot. The error signal vector in the visual servoing error model of the omnidirectional mobile robot is... ; Step 1.2: Introduce the error signal vector into neurodynamics to obtain a new error signal vector. , ， ， Among them, A i e represents the corresponding error signal vector i passive decay rate, B i and D i These represent the corresponding error signal vectors e i The upper and lower bounds, A i B i and D i All are greater than 0; f(e) i ) and g(e i ) represent the corresponding error signal vectors e i The upper and lower bounds of the threshold linear function are given, where i = 1, 2, 3; each error jump is restricted to the corresponding... Within the range; Step 1.3: With the new error signal vector Replace the error signal vector in the visual servoing error model Thus, a visual servoing error model based on neurodynamics is obtained, where T represents the transpose of the matrix in the above steps; Based on the error signal vector, the objective function of the visual servoing error model is set, and a visual servoing prediction model is constructed. The construction of the visual servoing prediction model includes the following steps: Step 2.1: Perform Euler discretization on the neurodynamic-based visual servoing error model to obtain the corresponding discrete-time visual servoing error model; Step 2.2: Define the objective function of the visual servoing error model based on neurodynamics for the omnidirectional mobile robot. ， Where Q and R are weight matrices, N c and N p Let E(k+i|k) represent the control time domain and the prediction time domain, respectively. E(k+i|k) represents the predicted value of the error signal vector E at time k+i after neurodynamic processing, and u(k+i-1|k) represents the predicted value of the control input at time k+i-1. A visual servoing prediction model based on neurodynamics was obtained for the omnidirectional mobile robot; Step 2.3: Obtain the visual servo prediction controller for the omnidirectional mobile robot based on the visual servo prediction model; Solve for the optimal solution of the control signal, and apply the first column of the optimal solution to the robot's control; repeat until the robot finally reaches the target point.

2. The robot visual servoing predictive control method based on neurodynamics according to claim 1, characterized in that: In step 1.1, the error signal vector satisfy, ， Where f is the focal length of the monocular camera, and θ is the angle between the world coordinate system and the robot coordinate system, i.e., the heading angle, with counterclockwise as positive. Let P be the coordinates of feature point P in the physical coordinate system of the image. Let P be the desired coordinates of feature point P in the physical coordinate system of the image; Let the robot's desired heading angle be... Let be the robot's heading angle error, and satisfy . .

3. The robot visual servoing predictive control method based on neurodynamics according to claim 1, characterized in that: In step 1.1, the visual servoing error model is as follows: ， For height coefficient, satisfying Z c The coordinates of feature point P in the camera coordinate system Central axis Z c The numerical value of the direction; v is the velocity vector of the omnidirectional mobile robot. x and v y These are the lateral and longitudinal linear velocities of the omnidirectional mobile robot, respectively, and ω is the angular velocity of the omnidirectional mobile robot. The variable d is the origin O of the camera coordinate system. c And the origin O of the robot coordinate system r The distance.

4. The robot visual servoing predictive control method based on neurodynamics according to claim 3, characterized in that: In step 1.3, the visual servoing error model based on neurodynamics is as follows: 。 5. The robot visual servoing predictive control method based on neurodynamics according to claim 3, characterized in that: In step 2.1, the discrete-time visual servo error model satisfies the following: ， Where T s Sampling time; Distinguish between the error signal vector E and the control input vector u, and obtain , in, ， ， get E(k+1) is the value of the error signal vector E(k) after processing by the neurodynamic model at the next sampling time, and u(k) is the control input vector, satisfying... , Here, T represents the transpose of the matrix, and h(·) represents the corresponding function mapping relationship.

6. The robot visual servoing predictive control method based on neurodynamics according to claim 1, characterized in that: The visual servoing prediction model for omnidirectional mobile robots based on neurodynamics is as follows: 。 7. The robot visual servoing predictive control method based on neurodynamics according to claim 6, characterized in that: The omnidirectional mobile robot visual servo predictive controller is, ， ， ， ， ， Where E0 represents the initial time-space error signal vector after neurodynamic processing. Represents the optimal solution to the optimization problem; u p and v p Let v be the coordinates of the feature point in the physical coordinate system of the image, representing the visual visibility constraint of the mobile robot; v and ω are the linear velocity and angular velocity of the mobile robot, respectively, representing the velocity constraint of the mobile robot; where the subscripts min and max represent the minimum and maximum constraints of the corresponding data, respectively.

Images