Visual servo intelligent robust control method of mobile mechanical arm for complex operation tasks

Abstract

The invention belongs to the technical field of robot control, and discloses a visual servo intelligent robust control method for a mobile mechanical arm for a complex operation task, which comprises the following steps of: 1, acquiring a working scene image, and calculating a space coordinate of a target feature point in real time through a projection transformation model; step 2, dynamically inhibiting visual measurement noise by adopting adaptive Kalman filtering, generating a feedforward compensation signal through an integral sliding mode observer, and decoupling chassis slippage disturbance; step 3, inputting the pose signal into a radial RBF neural network, designing an RBF gain scheduler, and adjusting an output proportion-integral gain in real time to suppress external time-varying disturbance; 4, adopting a hybrid visual servo mode switching mechanism, and generating a mobile platform control instruction through an integral sliding mode surface in a position servo mode; and 5, designing a three-order composite controller to drive the mechanical arm so as to realize accurate grabbing. According to the method, the grabbing deviation caused by kinematics uncertainty of the mobile platform and dynamic environment disturbance is eliminated.

Description

Technical Field

[0001] This invention belongs to the field of robot control technology, specifically relating to a visual servo intelligent robust control method for mobile robotic arms performing complex tasks. Background Technology

[0002] With the accelerated advancement of Industry 4.0, flexible and intelligent manufacturing equipment is gradually becoming the core driving force for improving production line efficiency. Among them, omnidirectional robotic arms, with their dual advantages of the all-domain mobility of mobile platforms and the precision operation capabilities of robotic arms, have shown great application potential in highly dynamic and complex scenarios such as automobile manufacturing, including multi-model door panel gripping and transfer on body welding lines and precise insertion of seat assemblies via overhead conveyor belts. However, existing technologies still face problems such as defects in kinematic compensation mechanisms, reliance on multi-source sensors, and bottlenecks in dynamic disturbance suppression. Current mainstream improvement schemes exhibit two major limitations: sensor fusion schemes, while improving positioning accuracy, do not address the time-varying characteristics of the slip coefficient; model enhancement schemes, while strengthening disturbance rejection, increase the risk of singular configurations due to neglecting the cooperative control of servo mode switching. The uncertainties of such systems make their control problems a difficult point in the field of control research.

[0003] In existing technologies, such as the paper "Position Vision Servoing of a 6-RSS Parallel Robot Based on Adaptive Sliding Mode Control" (N. Zhu et al., ISA Transactions, 2019), a sliding mode control method combining adaptive Kalman filtering and RBF neural network is proposed. The method estimates the end-effector pose of the parallel robot in real time through a grating sensor and uses a neural network to dynamically optimize the control gain. This method has two limitations: (1) it relies on a high-precision parallel platform (6-RSS structure) with a fixed base and does not consider the kinematic uncertainties caused by the moving chassis; (2) the experiment only verifies the trajectory tracking performance in a structured environment (maximum position error 1.05 mm) and does not involve the dynamic target interaction and obstacle avoidance requirements in actual grasping tasks.

[0004] The paper "Robust Hybrid Vision Servoing for Omnidirectional Mobile Arms with Kinematic Uncertainty" (K. Jo et al., IEEE Transactions on Cybernetics, 2023) designed a collaborative framework of Integral Sliding Mode Observer (ISMO) and hybrid vision servoing. It estimates chassis slip perturbation through a monocular camera and combines position-image hybrid servoing to avoid singular configurations of the robotic arm. This method still has shortcomings: (1) The experiment only verifies the pose tracking accuracy (feature point error < 3 pixels) and does not extend to the force control and object deformation problems in actual grasping operations; (2) It does not solve the problem of target feature point occlusion in dense and cluttered scenes and relies on the prior calibrated camera intrinsic parameter matrix.

[0005] Overall, existing technologies are still unable to effectively overcome the grasping deviation problems caused by the kinematic uncertainties of the mobile platform and dynamic environmental disturbances in robot control technology. Summary of the Invention The purpose of this invention is to provide a vision servo intelligent robust control method for mobile robotic arms performing complex tasks, in order to solve the technical problem of grasping deviation caused by kinematic uncertainties of the mobile platform and dynamic environmental disturbances in the prior art.

[0006] The vision servo intelligent robust control method for mobile robotic arms performing complex tasks includes the following steps.

[0007] Step 1: Use a monocular vision camera to acquire images of the work scene, and calculate the spatial coordinates of the target feature points in real time through a projection transformation model.

[0008] Step 2: Based on the output of Step 1, an adaptive Kalman filter is used to dynamically suppress visual measurement noise, and a feedforward compensation signal is generated through an integral sliding mode observer to decouple chassis slip disturbance.

[0009] Step 3: Input the pose signal processed in Step 2 into the RBF (Radial Basis Function) neural network. Based on the gradient descent rule and Lyapunov stability constraints, design the RBF gain scheduler to adjust the output proportional-integral gain in real time to suppress external time-varying disturbances.

[0010] Step 4: Based on the optimized control parameters in Step 3, a hybrid vision servo mode switching mechanism is adopted. In position servo mode, control commands for the moving platform are generated through the integral sliding surface. When the target enters the robotic arm operation area, the system automatically switches to image servo mode to precisely control the robotic arm end effector.

[0011] Step 5: Design a third-order composite controller to drive the robotic arm, so as to enable the mobile robotic arm to accurately grasp the target object in a real complex environment.

[0012] Preferably, in step 1, images of the work scene are first acquired at a certain frequency using a monocular vision camera deployed at the end of the robotic arm. This invention employs an online self-calibration strategy, iteratively updating the camera intrinsic parameter matrix online based on the correspondence of feature points in the image sequence. After obtaining the accurate intrinsic parameter matrix, the system uses a projection transformation model to convert the pixel coordinates of the two-dimensional target feature points in the image. , Solve the three-dimensional control coordinates in the camera coordinate system , , .

[0013] Preferably, the projection transformation model used is:

[0014] In the formula, , For image plane pixel coordinates, , Focal length , Principal point coordinates , These are the rotation matrix and translation vector in the camera's extrinsic parameters, respectively. , , The target point has 3D coordinates in the camera coordinate system.

[0015] Preferably, in step 2, an adaptive Kalman filter is used to process the output original pose signal; the adaptive Kalman filter updates the process slip covariance matrix in real time based on the historical state estimate and the system input within a sliding window of a preset length N; the dynamic update rule for the process noise covariance matrix is ​​as follows.

[0016] In the formula, Let k be the process slip covariance matrix at time k. The state estimate at time k. Let k be the mean of the states at time k. The system input at time k, The value represents the size of the sliding window, and the T in the upper right corner indicates the matrix transpose.

[0017] Preferably, in step 2, a matrix containing the unknown slip coefficients is established. The kinematic perturbation model of the omnidirectional platform is as follows: The actual velocity of the platform is decomposed into the sum of a measurable velocity vector and an unknown bounded perturbation term derived from the slip parameters, expressed as follows:

[0018] in, , The measured slip coefficient; For the platform velocity vector, For the gear train configuration matrix, The radius of the drive wheel, The gear ratio of the reducer. This is the motor speed vector. It is an identity matrix.

[0019] Preferably, in step 2, for the unknown disturbance term, an integral sliding mode observer based on the dynamic equation of image feature points is designed, and an integral sliding mode surface is constructed. .in, This represents the observer's integral sliding surface. For feature point estimation error, This is the integral term of the feature point estimation error; based on this sliding surface, an observer update law is designed to generate a feedforward compensation signal for the uncertainty term, calculated as follows:

[0020] In the formula, This indicates a kinematic uncertainty compensation signal. The dynamic slip compensation amount is calculated using the following formula: ; This term is used to suppress high-frequency components of measurement noise. , For observer parameters, Represents a time variable. The gain is the observer gain; this feedforward compensation signal is then sent to the main controller to actively counteract the impact of slip disturbances on the system control accuracy, thereby achieving precise decoupling of the disturbances.

[0021] Preferably, in step 3, the preprocessed pose signal forms a sliding surface, which is then input into the RBF neural network to convert the Gaussian radial basis function... The proportional gain adjustment is directly output as the activation function. ;in, Let be the activation function of the hidden layer of the RBF neural network. This represents the index number of the hidden layer neuron in the RBF neural network. This represents the input vector of the RBF neural network. and They represent the first The center vector and width of a Gaussian radial basis function.

[0022] Preferably, in step 3, the RBF neural network uses gradient descent to update and learn its internal parameters online, so as to minimize the performance index function between the actual output of the system and the network's predicted output. The internal parameters such as the weights, center, and width of the network are updated online, thereby achieving adaptive adjustment of the controller gain. Based on Lyapunov stability theory, the convergence of the update mechanism is ensured, and the network outputs a set of optimal proportional-integral gain adjustment every 0.1 seconds. The RBF neural network adjusts the gain online over time.

[0023] Preferably, in step 4, when the distance between the end effector of the robotic arm and the moving platform is detected to meet the following conditions... The conditions, among which, The system automatically switches to servo mode to determine the maximum working radius of the robotic arm, as shown in the following expression:

[0024] in, Represents the integral sliding surface. Indicates pose error. This represents the initial pose error. This represents the finite-time convergence term of PBVS. Represents the integral variable. This indicates the speed command of the robotic arm platform. This represents the speed of the mobile platform in the camera coordinate system. This represents the finite-time convergent term of IBVS. Represents the image servo sliding surface. Indicates the finite-time convergence exponent. Represents the IBVS gain matrix. Represents a symbolic function. It is a state-dependent gain matrix, consisting of a constant diagonal matrix. and constitute, Represents the interaction matrix. Represents the pseudo-inverse of the interaction matrix; when At that time, the system uses PBVS to control global localization. The expression for driving the motion of the mobile platform is as follows:

[0025] in, This is an estimate of the kinematic uncertainty. That is, positive definite gain. This represents the reference target pose set for the mobile platform. Represents the inverse of the rotation matrix. This is a platform speed command.

[0026] Preferably, in step 5, the third-order composite controller has the following expression:

[0027] in , representing the adaptive PI control term optimized online by the RBF neural network. Indicates the sliding surface of the controller. The integral term controls the gain. This is a robust term for sliding mode control, used to suppress unmodeled dynamics and residual errors, and is typically combined with a saturation function. To reduce high-frequency chattering, The boundary layer thickness is the saturation function. Indicates the robust term control gain. For the total control torque, , , For the standard dynamic model, where, The inertia matrix, For the centrifugal force matrix, For gravity, This represents the generalized position coordinate vector of the robotic arm's end effector. This represents the generalized position and velocity vector at the end of the robotic arm.

[0028] The technical advantages of this invention are as follows: 1) This invention combines precise compensation for kinematic uncertainties, adaptive suppression of visual noise, and intelligent optimization of control parameters to achieve ultra-high precision grasping of 0.38mm in industrial-grade high-dynamic scenarios, which is far superior to existing solutions.

[0029] 2) In this invention, the integrated integral sliding mode observer and adaptive sliding mode controller enable the system to have a strong ability to suppress external disturbances, ensuring operational stability in complex environments.

[0030] 3) This invention designs a hybrid visual servoing strategy to actively avoid singular configurations and ensure that the target does not leave the field of view, significantly improving the success rate of complex tasks.

[0031] 4) Since the entire system of the present invention relies only on a monocular camera and does not require fusion of multiple source sensors such as IMU, the hardware cost is greatly reduced and it is easy to deploy and maintain. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the overall process of the visual servo intelligent robust control method for a mobile robotic arm performing complex tasks according to the present invention. Figure 2 This is a schematic diagram of the motion trajectory of a mobile robotic arm in three-dimensional space.

[0033] Figure 3 This is a graph showing the tracking performance of the sliding mode observer (ISMO) for kinematic uncertainties (true values ​​vs. estimated values).

[0034] Figure 4 This is a graph showing the convergence of the estimation error of the sliding mode observer (ISMO) over time.

[0035] Figure 5 For online dynamic adjustment of controller gain in RBF neural network ( (Process diagram) Figure 6 This is a curve showing the convergence of the distance error between the robot's end effector and the target over time.

[0036] Figure 7 This is a diagram of the robot's divergent motion trajectory due to instability.

[0037] Figure 8 This is a graph showing the failure of visual feature point errors to converge. Detailed Implementation

[0038] The following detailed description of the embodiments, with reference to the accompanying drawings, will further illustrate the specific implementation of the present invention, in order to help those skilled in the art to have a more complete, accurate, and in-depth understanding of the inventive concept and technical solution of the present invention.

[0039] Table 1 is a parameter description table for this manual.

[0040] Table 1: Parameter Description Table.

[0041]

[0042]

[0043]

[0044] like Figures 1-8 As shown, the present invention provides a vision servo intelligent robust control method for a mobile robotic arm performing complex operations, comprising the following steps.

[0045] Step 1: Use a monocular vision camera to acquire images of the work scene, and calculate the spatial coordinates of the target feature points in real time through a projection transformation model.

[0046] In this example, after the control system is started, it first acquires images of the working scene at a frequency of 30Hz using a monocular vision camera deployed at the end of the robotic arm. To avoid the cumulative error introduced by traditional methods that rely on pre-calibrated camera intrinsic parameters, this invention employs an online self-calibration strategy, which iteratively updates the camera intrinsic parameter matrix online based on the correspondence of feature points in the image sequence.

[0047] Furthermore, after obtaining the accurate intrinsic parameter matrix, the system uses a projection transformation model to convert the pixel coordinates of the two-dimensional target feature points in the image. , Solve the three-dimensional control coordinates in the camera coordinate system , , This invention combines the solvePnP algorithm to further solve for the camera pose.

[0048] The projection transformation model is as follows:

[0049] In the formula, , For image plane pixel coordinates, , for x , y Focal length of direction, , Principal point coordinates , Here are the camera extrinsic parameters, and here are the rotation matrix and translation vector, respectively. , , These are the 3D coordinates of the target point in the camera coordinate system. For example... Figure 2 As shown, the feature point error can quickly converge to zero under the action of the designed controller, achieving high-precision positioning.

[0050] Step 2: Based on the output of Step 1, a robust control strategy integrating feedforward compensation and filtering is designed. An adaptive Kalman filter is used to dynamically suppress visual measurement noise, while an integral sliding mode observer is used to generate a feedforward compensation signal to decouple chassis slip disturbance.

[0051] This step uses an adaptive Kalman filter (AKF) to process the output raw pose signal. This adaptive Kalman filter (AKF) is designed to update the process slip covariance matrix in real time within a sliding window of a preset length N, based on historical state estimates and system input.

[0052] The dynamic update rule for the process noise covariance matrix is ​​as follows: In the formula, Let k be the process slip covariance matrix at time k. The state estimate at time k. Let k be the mean of the states at time k. The system input at time k, The value represents the size of the sliding window, and the T in the upper right corner indicates the matrix transpose.

[0053] In addition, this step establishes a matrix containing unknown slip coefficients. The kinematic perturbation model of the omnidirectional platform is presented. The actual velocity of the platform is decomposed into the sum of a measurable velocity vector and an unknown bounded perturbation term derived from the slip parameters, as expressed below:

[0054] in, , The measured slip coefficient; For the platform velocity vector, For the gear train configuration matrix, The radius of the drive wheel, The gear ratio of the reducer. This is the motor speed vector. It is an identity matrix.

[0055] This step involves designing an integral sliding mode observer (ISMO) based on the dynamic equations of image feature points for the unknown perturbation term. Its core is the construction of an integral sliding surface. .in, This represents the observer's integral sliding surface. For feature point estimation error, This is the integral term of the feature point estimation error.

[0056] Based on this sliding mode surface, an observer update law is designed to generate a feedforward compensation signal for the uncertainty term, and the calculation formula is as follows:

[0057] In the formula, This indicates a kinematic uncertainty compensation signal. The dynamic slip compensation amount is calculated using the following formula: ; This term is used to suppress high-frequency components of measurement noise. , For observer parameters, Represents a time variable. This is the observer gain. The feedforward compensation signal is then sent to the main controller to actively counteract the impact of slip disturbances on the system control accuracy, thereby achieving precise decoupling of the disturbances. Figure 3 The sliding mode observer based on AKF filtering shows a small estimation magnitude of uncertainty. Figure 4 As shown, the ISMO estimate (red dashed line) can quickly and accurately track the true value of the disturbance (black solid line).

[0058] Step 3: Input the pose signal processed in Step 2 into the RBF (Radial Basis Function) neural network. Based on the gradient descent rule and Lyapunov stability constraints, design the RBF gain scheduler to adjust the output proportional-integral gain in real time to suppress external time-varying disturbances.

[0059] In this step, the preprocessed pose signal forms the sliding surface (i.e. The sliding surface is input into the radial basis function (RBF) neural network, and the Gaussian radial basis function is... The proportional gain adjustment is directly output as the activation function. .in, Let be the activation function of the hidden layer of the RBF neural network. This represents the index number of the hidden layer neuron in the RBF neural network. This represents the input vector of the RBF neural network. and They represent the first The Gaussian radial basis functions (RBFs) are defined by their center vectors and widths. These RBFs map the low-dimensional pose error signal to a high-dimensional feature space, enhancing the network's ability to approximate the nonlinear dynamics of the system.

[0060] Furthermore, the RBF neural network employs gradient descent to update and learn its internal parameters online, minimizing the performance index function between the actual system output and the network's predicted output. This online updating of internal parameters such as weights, center, and width enables adaptive adjustment of the controller gain. Specifically, the update rule for its width parameter is as follows:

[0061] in, This is the gain adjustment amount. For learning rate, The momentum coefficient, This represents the pose error of the nth position. This represents the sliding surface value of the nth degree of freedom. Indicates the actual output of the system. This represents the change in control input. Based on Lyapunov stability theory, the convergence of the update mechanism is ensured, and the network outputs a set of optimal proportional-integral (PI) gain adjustments every 0.1 seconds. The RBF neural network adjusts the gain online over time. Figure 5 As shown, controller gain It is not a fixed value, but is dynamically adjusted according to the real-time status of the system. In the early stages of a task, the gain is rapidly increased to achieve a fast response; as the error decreases, the gain adjustment tends to be gradual to ensure the stability of the system.

[0062] Step 4: Based on the optimized control parameters from Step 3, a hybrid vision servo (HVS) mode switching mechanism is adopted. In position servo (PBVS) mode, control commands for the moving platform are generated through the integral sliding surface. When the target enters the robotic arm's operating area, it automatically switches to image servo (IBVS) mode to precisely control the robotic arm's end effector.

[0063] Specifically, this step designs a hybrid vision servo mode switching mechanism, which switches when the distance between the robotic arm end effector and the moving platform meets the specified conditions. The conditions, among which, The system automatically switches to servo mode to determine the maximum working radius of the robotic arm, as shown in the following expression:

[0064] in, Represents the integral sliding surface. Indicates pose error. This represents the initial pose error. This represents the finite-time convergence term of PBVS. Represents the integral variable. This indicates the speed command of the robotic arm platform. This represents the speed of the mobile platform in the camera coordinate system. This represents the finite-time convergent term of IBVS. Represents the image servo sliding surface. Indicates the finite-time convergence exponent. Represents the IBVS gain matrix. Represents a symbolic function. It is a state-dependent gain matrix, consisting of a constant diagonal matrix. and constitute, Represents the interaction matrix. This represents the pseudo-inverse of the interaction matrix.

[0065] when ( When the maximum working radius of the robotic arm is reached, the system uses PBVS-dominated global positioning. The expression for driving the motion of the mobile platform is as follows:

[0066] in, This is an estimate of the kinematic uncertainty. That is, positive definite gain. This represents the reference target pose set for the mobile platform. Represents the inverse of the rotation matrix. For platform speed commands; Figure 6 The display shows the distance error between the robotic arm and the target based on the hybrid vision servo mode. The distance error between the robot end and the target decreases rapidly after startup and converges to near zero after about 8 seconds, with no overshoot or oscillation throughout the process.

[0067] Step 5: Design a third-order composite controller to drive the robotic arm to achieve millimeter-level positioning accuracy of the end effector, so as to enable the mobile robotic arm to accurately grasp target objects in real complex environments.

[0068] In this step, to suppress high-frequency chattering from the output, a third-order composite controller integrating feedforward compensation, neural network-optimized gain, and sliding mode robustness is designed, with the following expression:

[0069] in , representing the adaptive PI control term optimized online by the RBF neural network. Indicates the sliding surface of the controller. The integral term controls the gain. This is a robust term for sliding mode control, used to suppress unmodeled dynamics and residual errors, and is typically combined with a saturation function. To reduce high-frequency jitter, The boundary layer thickness is a saturation function. Indicates the robust term control gain. For the total control torque, , , For the standard dynamic model, where, The inertia matrix, For the centrifugal force matrix, For gravity, This represents the generalized position coordinate vector of the robotic arm's end effector. This represents the generalized position and velocity vector at the end of the robotic arm.

[0070] In summary, this invention provides a visual servoing intelligent robust control method for a mobile robotic arm performing complex tasks. First, a monocular camera is used to acquire real-time images of the working scene and perform pose estimation and self-correction. Then, an adaptive Kalman filter and an integral sliding mode observer are used to compensate for and suppress kinematic uncertainties and visual noise. To address external time-varying disturbances, a gain scheduler based on an RBF neural network is designed to adjust controller parameters in real time. Furthermore, a hybrid visual servoing mode switching mechanism is used to avoid singular configurations. Finally, a third-order composite controller integrating feedforward compensation, neural network optimization, and robust terms is implemented through software programming. This invention effectively compensates for system uncertainties and significantly suppresses external disturbances without requiring multi-source sensor fusion, and the pose tracking error converges within a finite time, thereby improving the motion tracking accuracy of the mobile robotic arm.

[0071] The above-mentioned conventional control method is applied to the same simulation task. This control method only includes a hybrid visual servo and a fixed-gain sliding mode controller, does not consider kinematic uncertainty compensation, and does not estimate and compensate for chassis slip.

[0072] A monocular camera is used to acquire images of the working scene in real time, and feature points of the target object are extracted from them to perform real-time pose estimation and self-correction.

[0073] In this example, after the control system is started, it first acquires images of the working scene at a frequency of 30Hz using a monocular vision camera deployed at the end of the robotic arm. This invention employs an online self-calibration strategy, iteratively updating the camera intrinsic parameter matrix online based on the correspondence of feature points in the image sequence.

[0074] Furthermore, after obtaining the accurate intrinsic parameter matrix, the system uses a projection transformation model to convert the pixel coordinates of the two-dimensional target feature points in the image. , Solve the three-dimensional control coordinates in the camera coordinate system , , This invention combines the solvePnP algorithm to further solve for the camera pose.

[0075] The projection transformation model used is:

[0076] In the formula, , For image plane pixel coordinates, , Focal length , Principal point coordinates , These are the rotation matrix and translation vector in the camera's extrinsic parameters, respectively. , , These are the 3D coordinates of the target point in the camera coordinate system. For example... Figure 7 As shown, the end effector of the robotic arm (blue trajectory) exhibited violent oscillations and disordered movement in the early stages of its motion. Uncompensated slip disturbances led to system control instability, causing the robotic arm to enter a singular region of its workspace while attempting to track the target, ultimately resulting in control divergence and mission failure.

[0077] A hybrid vision servo (HVS) mode switching mechanism is adopted. In position servo (PBVS) mode, control commands for the moving platform are generated through an integral sliding surface. When the target enters the robotic arm's operating area, it automatically switches to image servo (IBVS) mode to precisely control the robotic arm's end effector.

[0078] Specifically, a hybrid vision servo mode switching mechanism is designed, which switches when the distance between the robotic arm end effector and the moving platform meets the specified conditions. ( (Maximum working radius of the robotic arm), automatically switches servo mode:

[0079] in, Represents the integral sliding surface. This represents the image servo sliding surface.

[0080] For PBVS-dominated global positioning, adopt Drive the movement of the mobile platform;

[0081] in, To obtain the required sliding mode observer output value, This is the set gain. Figure 8 As shown, without effective compensation, the visual feature point error of the system exhibits complex dynamics. While the error value of the traditional method (red dashed line) decreases somewhat in the later stages of the task, it suffers from severe oscillations throughout and fails to converge stably. In contrast, the method of this invention (blue solid line) can suppress the error within a stable range without showing any divergence, demonstrating the robustness and stability of the control framework in the face of severe disturbances.

[0082] The comparison of the above application cases demonstrates that in real-world conditions with kinematic uncertainties (such as slippage), without effective estimation and compensation, and adaptive adjustment of the controller gain, traditional hybrid vision servoing methods will suffer from singularity problems due to system instability, ultimately leading to control divergence and task failure. Simultaneously, it proves that the method proposed in this invention, through effective observation and compensation of kinematic uncertainties, intelligent optimization of controller parameters, and active avoidance of singular configurations, can significantly improve the accuracy and robustness of visual servoing control of mobile robotic arms in complex environments.

[0083] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other occasions without modification, are all within the protection scope of the present invention.

Claims

1. A vision-servoing intelligent robust control method for a mobile robotic arm performing complex tasks, characterized by: Includes the following steps: Step 1: Use a monocular vision camera to acquire images of the work scene, and calculate the spatial coordinates of the target feature points in real time using a projection transformation model; Step 2: Based on the output of Step 1, an adaptive Kalman filter is used to dynamically suppress visual measurement noise, and a feedforward compensation signal is generated through an integral sliding mode observer to decouple chassis slip disturbance. Step 3: Input the pose signal processed in Step 2 into the RBF neural network. Based on the gradient descent rule and Lyapunov stability constraints, design the RBF gain scheduler to adjust the output proportional-integral gain in real time to suppress external time-varying disturbances. Step 4: Based on the optimized control parameters in Step 3, a hybrid vision servo mode switching mechanism is adopted. In position servo mode, control commands for the moving platform are generated through the integral sliding surface. When the target enters the robotic arm operation area, the system automatically switches to image servo mode to precisely control the robotic arm end effector. Step 5: Design a third-order composite controller to drive the robotic arm, so as to enable the mobile robotic arm to accurately grasp the target object in a real complex environment.

2. The vision servo intelligent robust control method for a mobile robotic arm performing complex operations according to claim 1, characterized in that: In step 1, images of the work scene are first acquired at a certain frequency using a monocular vision camera deployed at the end of the robotic arm. This invention employs an online self-calibration strategy, iteratively updating the camera intrinsic parameter matrix online based on the correspondence of feature points in the image sequence. After obtaining the accurate intrinsic parameter matrix, the system uses a projection transformation model to convert the pixel coordinates of the two-dimensional target feature points in the image. , Solve the three-dimensional control coordinates in the camera coordinate system , , .

3. The vision servo intelligent robust control method for a mobile robotic arm performing complex operations according to claim 2, characterized in that: The projection transformation model used is: In the formula, , For image plane pixel coordinates, , Focal length , Principal point coordinates , These are the rotation matrix and translation vector in the camera's extrinsic parameters, respectively. , , The target point has 3D coordinates in the camera coordinate system.

4. The vision servo intelligent robust control method for a mobile robotic arm performing complex operations according to claim 1, characterized in that: In step 2, an adaptive Kalman filter is used to process the output raw pose signal. Within a sliding window of preset length N, the adaptive Kalman filter updates the process slip covariance matrix in real time based on historical state estimates and system input. The dynamic update rule for the process noise covariance matrix is ​​as follows: In the formula, Let k be the process slip covariance matrix at time k. The state estimate at time k. Let k be the mean of the states at time k. The system input at time k, The value represents the size of the sliding window, and the T in the upper right corner indicates the matrix transpose.

5. The vision servo intelligent robust control method for a mobile robotic arm performing complex operations according to claim 4, characterized in that: In step 2, a matrix containing the unknown slip coefficients is established. The kinematic perturbation model of the omnidirectional platform is as follows: The actual velocity of the platform is decomposed into the sum of a measurable velocity vector and an unknown bounded perturbation term derived from the slip parameters, as expressed below: in, , The measured slip coefficient; For the platform velocity vector, For the gear train configuration matrix, The radius of the drive wheel, The gear ratio of the reducer. This is the motor speed vector. It is an identity matrix.

6. The vision servo intelligent robust control method for a mobile robotic arm performing complex operations according to claim 5, characterized in that: In step 2, for the unknown disturbance term, an integral sliding mode observer based on the dynamic equation of image feature points is designed to construct an integral sliding surface. ;in, This represents the observer's integral sliding surface. For feature point estimation error, This is the integral term of the feature point estimation error; based on this sliding surface, an observer update law is designed to generate a feedforward compensation signal for the uncertainty term, calculated as follows: In the formula, This indicates a kinematic uncertainty compensation signal. The dynamic slip compensation amount is calculated using the following formula: ; The term is used to suppress high-frequency components of measurement noise. , For observer parameters, Represents a time variable. The gain is the observer gain; this feedforward compensation signal is then sent to the main controller to actively counteract the impact of slip disturbances on the system control accuracy, thereby achieving precise decoupling of the disturbances.

7. The vision servo intelligent robust control method for a mobile robotic arm performing complex operations according to claim 1, characterized in that: In step 3, the preprocessed pose signal forms a sliding surface, which is then input into the RBF neural network to process the Gaussian radial basis functions. The proportional gain adjustment is directly output as the activation function. ;in, Let be the activation function of the hidden layer of the RBF neural network. This represents the index number of the hidden layer neuron in the RBF neural network. This represents the input vector of the RBF neural network. and They represent the first The center vector and width of a Gaussian radial basis function.

8. The vision servo intelligent robust control method for a mobile robotic arm performing complex operations according to claim 7, characterized in that: In step 3, the RBF neural network uses gradient descent to update and learn its internal parameters online, minimizing the performance index function between the actual system output and the network's predicted output. It updates the network's internal parameters, such as weights, center, and width, online to achieve adaptive adjustment of the controller gain. Based on Lyapunov stability theory, the convergence of the update mechanism is ensured, and the network outputs a set of optimal proportional-integral gain adjustments every 0.1 seconds. The RBF neural network adjusts its gain online over time.

9. The vision servo intelligent robust control method for a mobile robotic arm performing complex operations according to claim 1, characterized in that: In step 4, when the distance between the end effector of the robotic arm and the moving platform is detected to meet the condition... The conditions, among which, The system automatically switches to servo mode to determine the maximum working radius of the robotic arm, as shown in the following expression: in, Represents the integral sliding surface. Indicates pose error. This represents the initial pose error. This represents the finite-time convergence term of PBVS. Represents the integral variable. Indicates the speed command of the robotic arm platform. This represents the speed of the mobile platform in the camera coordinate system. This represents the finite-time convergent term of IBVS. Represents the image servo sliding surface. Indicates the finite-time convergence exponent. Represents the IBVS gain matrix. Represents a symbolic function. It is a state-dependent gain matrix, consisting of a constant diagonal matrix. and constitute, Represents the interaction matrix. Represents the pseudo-inverse of the interaction matrix; Among them, the system's PBVS is the primary global localization method, employing... The expression for driving the motion of the mobile platform is as follows: in, This is an estimate of the kinematic uncertainty. That is, positive definite gain. The reference target pose set for the mobile platform. Represents the inverse of the rotation matrix. This is a platform speed command.

10. The vision servo intelligent robust control method for a mobile robotic arm performing complex operations according to claim 1, characterized in that: In step 5, the expression for the third-order composite controller is as follows: in , representing the adaptive PI control term optimized online by the RBF neural network. Indicates the sliding surface of the controller. The integral term controls the gain; This is a robust term for sliding mode control, used to suppress unmodeled dynamics and residual errors, and is typically combined with a saturation function. To reduce high-frequency chattering, The boundary layer thickness is the saturation function. Indicates the robust term control gain. For the total control torque, , , For the standard dynamic model, where, The inertia matrix, For the centrifugal force matrix, For gravity, This represents the generalized position coordinate vector of the robotic arm's end effector. This represents the generalized position and velocity vector at the end of the robotic arm.

Images