Visual servo method for transformer substation drainage wire disconnecting and connecting robot

Abstract

The invention discloses a visual servo method for a transformer substation drainage wire disconnecting and connecting robot, and the method comprises the following steps: building a mapping relation between a mechanical arm tail end camera and a mechanical arm kinematic model, and building an image-based visual servo system IBVS; acquiring a drainage wire clamp image in real time, and extracting image feature points of a drainage wire clamp bolt by using the recognition model; the model prediction controller carries out rolling optimization based on the deviation between the extracted current image features and the expected image features, estimates and compensates external disturbance and unmodeled dynamics borne by the system in real time, and calculates an optimal, smooth and safe mechanical arm motion instruction; and the mechanical arm movement instruction is sent to a mechanical arm movement control unit to drive the mechanical arm to move. According to the method, the pose control of the drainage wire disconnecting robot can be realized, the error convergence in the visual servo process is rapid, the steady-state error is small, the dynamic response is timely, and the requirements of the drainage wire disconnecting operation on the precision, the stability and the real-time performance are fully met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot control, and in particular to a visual servo method for a substation lead wire disconnection robot. BACKGROUND

[0002] With the progress of science and technology, the robot-related industry and technology are booming, making great contributions to the development of human society in many fields such as economic development and social services.

[0003] As an important public infrastructure, substations play an important role in ensuring power supply, supporting economic and social development, and improving people's livelihood. With the advancement of new-type power system construction and the progress of industrial automation and intelligent manufacturing technology, higher requirements are put forward for the operation and maintenance service level of substations.

[0004] With the advancement of the construction of new-generation intelligent substations, the power grid puts forward higher requirements for the operation safety and intelligent level of operation and maintenance of substations. Among them, live disconnection of lead wires as a key high-risk operation link of substation equipment condition-based maintenance directly affects the reliable operation of the power grid. The traditional manual live operation mode has problems such as dependence on manual experience, high labor intensity and high safety risk, and it is urgent to realize intelligent upgrading of operation mode through technical means.

[0005] However, there is a lack of intelligent operation equipment in the prior art, which leads to the fact that this type of operation still mainly relies on manual maintenance, making it difficult to meet the digital transformation needs of current power grid intelligent operation and maintenance and safe live operation. Under this background, using an aerial work manipulator to replace manual operation to perform live disconnection tasks has become an inevitable choice to improve the safety and automation level of operation. Therefore, it is necessary to use an aerial work manipulator to live disconnect lead wires to replace manual operation to complete this dangerous task.

[0006] Visual servo control is an artificial intelligence technology that uses image information and visual signals to control the movement of robots. In visual servo control, a robot obtains real-time target image information through one or more cameras, and in the controller, image feature parameters are used as feedback signals to calculate control deviations, and control instructions are generated through coordinate transformation to drive the manipulator to dynamically adjust from the current pose to the desired pose. However, the visual servo control in the prior art is limited to laboratory scenarios, and in actual applications, actual external interference factors such as high-altitude wind load are often ignored, resulting in inaccurate predictions. SUMMARY

[0007] The present application aims to solve the problems in the prior art.

[0008] The technical scheme adopted by the present application to solve its technical problems is: a visual servo method for a transformer substation drainage line disconnection robot is provided, a model predictive control algorithm is adopted, and a disturbance observer is introduced to enhance the robustness of the system; the method comprises the following steps:

[0009] A system establishment step is provided to establish a mapping relationship between the camera at the end of the mechanical arm and the kinematic model of the mechanical arm, and to build an image-based visual servo system IBVS; a deep learning model is trained and inference acceleration is performed to obtain an identification model for real-time detection of drainage line clamp bolts and deploy the model into the visual servo system IBVS;

[0010] A feature extraction step is provided to collect drainage line clamp images in real time through the camera at the end of the mechanical arm, and to extract image feature points of the drainage line clamp bolts by using the identification model;

[0011] An instruction calculation step is provided, in which a model predictive controller performs rolling optimization based on the deviation between the current image feature points and the expected image feature points of the drainage line clamp bolts; during the optimization process, the motion speed of the camera at the end of the mechanical arm is used as a constraint condition to calculate optimal, smooth and safe mechanical arm motion instructions;

[0012] A mechanical arm driving step is provided to send the mechanical arm motion instructions to a mechanical arm motion control unit to drive the mechanical arm to move; the feature extraction step is returned until the drainage line disconnection work is accurately completed.

[0013] Preferably, the mapping relationship between the camera at the end of the mechanical arm and the kinematic model of the mechanical arm is established, and an image-based visual servo system IBVS is built, comprising the following steps:

[0014] The three-dimensional coordinates of the image feature points on the surface of the target object are obtained by using a binocular stereo camera; and the three-dimensional coordinates of the image feature points on the surface of the target object are mapped to a two-dimensional image plane by using a perspective projection model;

[0015] The pose mapping relationship between the image feature points and the end effector of the mechanical arm is established, and the motion trajectory of the end effector of the mechanical arm is calculated.

[0016] Preferably, the three-dimensional coordinates of the image feature points on the surface of the target object are mapped to a two-dimensional image plane by using a perspective projection model, which is represented as:

[0017] ;

[0018] wherein the three-dimensional space coordinates are , the two-dimensional image plane coordinates are , , i represents the pixel index, and n represents the total number of pixel points; f represents the camera focal length, and respectively denote the image coordinate system axes and scale factor of the axes.

[0019] Preferably, the mapping relationship between the image feature points and the pose of the end effector of the robot arm is established, and the motion speed of the end effector of the robot arm is calculated, including the following steps:

[0020] The mapping relationship between the image feature and the pose of the end effector of the robot arm is established, and is expressed as:

[0021] ;

[0022] ;

[0023] wherein, denotes the image feature change rate; denoted as the image Jacobian matrix, denotes the speed of the end effector;

[0024] In the image-based visual servoing system IBVS, the control error is expressed as

[0025] ;

[0026] wherein, denotes the current feature point pose, denotes the desired feature point pose

[0027] The change rate of the control error is expressed as:

[0028] ;

[0029] wherein, is obtained from ;

[0030] The speed of the end effector is expressed as

[0031] ;

[0032] wherein, denotes the visual servoing gain, denotes the Moore-Penrose pseudo-inverse of the image Jacobian matrix .

[0033] Preferably, the deep learning model is trained and inference acceleration is performed to obtain a model for real-time detection of a drainage clamp bolt and is deployed into the visual servoing system IBVS, including the following steps:

[0034] Image acquisition of the bolt on the drainage wire clip from multiple angles, construction of a data set, annotation of the data set, and training of a YOLOv11 inference model using the data set;

[0035] The trained YOLOv11 inference model is deployed to a TensorRT inference architecture to generate a final efficient inference engine, i.e., the model for real-time detection of drainage wire clip bolts.

[0036] Preferably, the model predictive controller is established by the following method:

[0037] The IBVS model is discretized in time using the Newton-Euler method to obtain a discrete-time model, denoted as:

[0038] ;

[0039] wherein, is the image feature at the next time, is the current image feature, is the sampling time;

[0040] An expression of the time-dependent visual servoing model is established, denoted as:

[0041] ;

[0042] wherein s(k) is a state vector representing the system state at a single discrete time point k, represents the image Jacobian matrix at a single discrete time point k;

[0043] A prediction model of the model predictive controller is established, denoted as:

[0044] ;

[0045] wherein, is the state prediction vector at step k, is the image Jacobian matrix at step k, is the predicted control sequence at step k, and the parameter is the dimension value of the state, is the dimension of the input control quantity, is composed of an identity matrix. Preferably, the optimization objective function of the model predictive controller is:

[0046] ;

[0047] ;

[0048] wherein, ​​Let be the expected vector of the target features, and let be the state error weighting matrix. Let be the control sequence, and be the weighting matrix of the control inputs. The objective of the controller is to find a function that satisfies the constraints. The smallest control sequence.

[0049] Preferably, the constraint condition, which uses the movement speed of the end-effector camera, is expressed as:

[0050] ;

[0051] in, Indicates the maximum speed of the end effector. This indicates the minimum speed of the end effector.

[0052] Preferably, the model controller further incorporates a disturbance observer to enhance the robustness of the system; including the following steps:

[0053] Considering a visual servoing system with external disturbances and unmodeled dynamics, the discrete-time model is extended as follows:

[0054] ;

[0055] in, for The state at any given moment, for The state at any given moment, for The total perturbation vector at time t; express Speed ​​of the end-effector camera at all times;

[0056] Expanding the total disturbance vector into a new system state yields the expanded state model, which is represented as follows:

[0057] ;

[0058] Where I represents the identity matrix;

[0059] Based on the aforementioned extended state model, a discrete-time extended state observer is designed. The observer employs... Predict using data from different times, and then utilize... The measured value at time is corrected and expressed as:

[0060] ;

[0061] ;

[0062] in, for priori estimation of the state at time k, is the estimation of the state at time k, is the state at time k, is the velocity at time k, is the disturbance estimation at time k, is the observation error, is the state at time k, is the estimation of the state, is the estimation of the disturbance, is the priori estimation of the disturbance at time k, is the estimation gain matrix of the state, is the estimation gain matrix of the disturbance, is the sampling time;

[0063] integrating the disturbance estimated by the observer into the MPC framework of the model controller, and designing a velocity feedforward compensation term based on the disturbance estimation to offset the effect of the disturbance estimation on the system, denoted as:

[0064] ;

[0065] wherein, is a regularization parameter to avoid numerical instability when the Jacobian matrix is close to singular;

[0066] ultimately obtaining the control quantity composed of the nominal control quantity optimized by the MPC and the compensation quantity of the ESO, denoted as:

[0067] ;

[0068] wherein, denotes the final control quantity, denotes the nominal control quantity optimized by the MPC, denotes the velocity feedforward compensation term at time k.

[0069] Preferably, the integrating the disturbance estimated by the ESO into the MPC framework of the model controller also adjusts the constraint condition to ensure that the control quantity satisfies the velocity constraint, and the adjusted constraint condition is denoted as:

[0070] ;

[0071] wherein, is the disturbance compensation quantity at the current time.

[0072] The present application has the following beneficial effects: the present application can realize pose control of the drainage line disconnection robot, the visual servoing process error converges rapidly and the steady-state error is small, the dynamic response is timely, and the requirements of the drainage line disconnection operation on precision, stability and real-time are fully met, and the verification experiment proves the effectiveness and efficiency of the method.

[0073] The present application will be further described in detail below in combination with the drawings and embodiments, but the present application is not limited to the embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1 The method step diagram of the embodiment of the present application;

[0075] Figure 2 The system overall framework of the embodiment of the present application;

[0076] Figure 3 The ESO-based MPC closed-loop control system structure diagram of the embodiment of the present application;

[0077] Figure 4 The identification model training and testing process of the embodiment of the present application;

[0078] Figure 5 The visual servoing experimental platform used for the identification model training and testing of the embodiment of the present application;

[0079] Figure 6 The experimental result schematic diagram of the verification experiment of the embodiment of the present application. DETAILED DESCRIPTION

[0080] Referring to Figure 1 The method step diagram of the embodiment of the present application is shown, a model predictive control algorithm is adopted, and a disturbance observer is introduced to enhance the robustness of the system; including the following steps:

[0081] S101, a system establishment step, a mapping relationship between a camera at the end of a mechanical arm and a kinematics model of the mechanical arm is established, an image-based visual servoing system IBVS is built; a deep learning model is trained and inference acceleration is performed, an identification model for real-time detection of drainage line clamp bolts is obtained and deployed into the visual system;

[0082] S102, a feature extraction step, a drainage line clamp image is collected in real time by a camera at the end of a mechanical arm, and image feature points of the drainage line clamp bolt are extracted by using the identification model;

[0083] S103, an instruction calculation step, a model predictive controller performs rolling optimization based on the deviation of the extracted current image feature and the expected image feature; the movement speed of the camera at the end of the mechanical arm is taken as a constraint condition in the optimization process, and the optimal, smooth and safe mechanical arm movement instruction is calculated;

[0084] S104, a mechanical arm driving step, sending a mechanical arm motion instruction to a mechanical arm motion control unit to drive the mechanical arm to move; returning to the feature extraction step until the drainage line disconnection operation is accurately completed.

[0085] Specifically, in the robot vision servo, the mechanical arm vision servo system realizes real-time acquisition, processing and feedback of image information by establishing a dynamic coupling mechanism of the vision system and the mechanical arm motion control unit, and then accurately guides the mechanical arm to complete target positioning and trajectory execution. In the robot vision servo, the extraction of image features has a very key influence on the performance of the vision controller. The commonly used image features are point features, optical flow fields, kernel sampling, brightness, mutual information, image matrices, etc. The selection of point features directly affects the tracking, recognition ability of the system to the target object, and the accuracy and stability of the control system. Among them, point features are the simplest and most widely existing image features on objects, which are easier to detect and track, and the difficulty of extracting point features is relatively small, and the computing resources required for processing point features are relatively small. Therefore, the center point feature of the bolt is used as the marker feature of the image plane in the embodiment of the application. The center point feature is used as a feedback signal to adjust the servo control loop. The control logic flow of IBVS is shown in Figure 2 .

[0086] Specifically, the IBVS control model derivation process is as follows:

[0087] The perspective projection model is a classic mathematical model for describing the projection relationship of a three-dimensional space object to a two-dimensional image plane. This model maps the three-dimensional coordinates of the image feature points on the surface of the target object to the corresponding two-dimensional pixel coordinates by setting the projection center; the mapping relationship between the three-dimensional coordinates , and the corresponding two-dimensional pixel coordinates can be expressed as:

[0088]

[0089] wherein, represents the focal length of the camera, and respectively represent the scale factors of the image coordinate system axis and axis.

[0090] In order to establish the pose mapping relationship between the image features and the mechanical arm end effector, the differential transformation is performed on both ends of formula (1-1), and the following formula (2-1) is obtained:

[0091]

[0092] wherein the definitions

[0093]

[0094] Then, equation (1-1) can be written as

[0095]

[0096] wherein is called the image Jacobian matrix. This matrix is used to describe the approximate linear transformation relationship between the image feature space velocity and the end-effector motion velocity of the robot arm, so as to establish an approximate linear mapping relationship between the end-effector operation space and the image feature space.

[0097] In the image-based visual servoing (IBVS) system, the control error is expressed as

[0098]

[0099] wherein represents the current feature point pose, represents the desired feature point pose. By combining equation (1-4) and equation (1-5), we can obtain

[0100]

[0101] wherein is obtained from in equation (1-3).

[0102] The velocity of the end-effector is expressed as

[0103]

[0104] wherein represents the visual servoing gain, represents the Moore-Penrose pseudo-inverse of the image Jacobian matrix Thus, the relationship between the image feature and the camera velocity is established. Based on the above relationship, the control law is designed, and by updating the motion velocity of the end-effector in real time, the robot arm can be driven to move in the three-dimensional space, so that the error between the two-dimensional image feature point and the desired pose gradually converges, and finally the tracking and positioning of the bolt on the drainage wire clamp are realized.

[0105] The above analysis establishes a continuous model of visual servoing based on image feature error. In order to apply this model to the model predictive control framework, it is first necessary to transform it into discrete time form in order to facilitate the prediction of system states and outputs between sample instants.

[0106] Accordingly, the IBVS model is discretized in time using the Newton-Euler method according to equation (1-6) expressed as

[0107]

[0108] where is the image feature at the next instant, is the current image feature, is the sampling time. During the servoing process, the image feature coordinates change and so does the interaction matrix value, thus leading to the following time-dependent visual servoing model expression:

[0109]

[0110] According to the visual servoing model equation (1-9), in the case of step prediction, its prediction model is expressed as

[0111]

[0112] where is the step state prediction vector, is the step predicted control sequence, the parameter n is the dimension value of the state, and m is the dimension of the input control quantity, is composed of the identity matrix.

[0113] In the current visual servoing control scheme of the robotic arm, the control signal is the spatial velocity of the camera at the end of the robotic arm , and the camera is installed at the end of the robotic arm. Since the movement speed of each joint of the robotic arm has a maximum speed limit, and the robotic arm end is equipped with an electric wrench, it is necessary to constrain the camera speed to avoid impact on the load-bearing robotic arm due to sudden changes in the speed command. Then the control sequence , and its boundary constraint is

[0114]

[0115] According to the motion state of the image feature and the predicted sequence of the control quantity, the objective function is constructed as

[0116]

[0117] in Let be the expected vector of the target features, 𝑄 be the state error weighting matrix, and 𝑅 be the control quantity weighting matrix.

[0118] In each control cycle, the optimal control sequence is obtained by solving the above-mentioned constrained optimization problem. A rolling optimization strategy is adopted, taking only the first element of the sequence as the nominal control variable. .

[0119] However, given the strong environmental disturbances such as high-altitude wind loads encountered in this application, relying solely on the constraint capabilities of MPC is insufficient to fully guarantee the tracking accuracy and robustness of the guide wire assembly and disassembly robot system, leading to inaccurate model-based predictions and consequently severely impacting the final performance of visual servoing.

[0120] To directly address this issue, an Extended State Observer (ESO) is introduced. The ESO expands the total disturbance experienced by the system (including unmodeled dynamics and external disturbances) into a new state variable and performs real-time estimation and observation. Subsequently, the observed disturbance value is fed forward to the MPC controller for real-time compensation. This proactively suppresses the impact of disturbances on visual servo accuracy while the MPC performs constraint and trajectory planning, thereby enhancing the overall system performance. The specific control scheme is as follows: Figure 3 As shown. Considering a visual servoing system with external disturbances and unmodeled dynamics, the original discrete-time model (1-8) is extended to...

[0121]

[0122] in for The total disturbance vector at time t. To estimate and compensate for this disturbance in real time, it is expanded into a new system state:

[0123]

[0124] Based on the above extended state model, a discrete-time extended state observer is designed. Since there is a time delay in acquiring the Jacobian matrix and camera velocity in the actual system, the observer employs... Predict using data from different times, and then utilize... The measured values ​​at time are corrected, where

[0125] Prediction step:

[0126]

[0127] Calibration step:

[0128]

[0129] where is the state estimation value, is the disturbance estimation value, is the state estimation gain matrix, is the disturbance estimation gain matrix.

[0130] The disturbance estimated by the ESO is integrated into the MPC framework to form the disturbance-resistant visual servoing control. To counteract the influence of the estimated disturbance on the system, a velocity feedforward compensation term based on the disturbance estimation is designed:

[0131]

[0132] where is a regularization parameter used to avoid numerical instability when the Jacobian matrix is close to singular. The final control quantity is composed of the nominal control quantity optimized by the MPC and the ESO compensation quantity:

[0133]

[0134] To ensure that the final control quantity satisfies the velocity constraint, the constraint on the nominal control sequence is adjusted accordingly in the MPC optimization

[0135]

[0136] where is the disturbance compensation quantity at the current time. By equation (1-19), it can be ensured that

[0137]

[0138] That is, the finally implemented control quantity strictly satisfies the velocity limit of the manipulator.

[0139] This design enables the system to complete constraint optimization and trajectory planning by the MPC while estimating and compensating the total disturbance in real time by the ESO, significantly enhancing the robustness and control accuracy of visual servoing in a high-altitude disturbance environment.

[0140] Specifically, in the S101, to realize detection of the bolts on the drainage wire clamp, a data set needs to be prepared to train the model, and the specific process is as follows: Figure 4The experimental platform is built in the laboratory. An Intel RealSense D435i depth camera is used to capture the bolts on the drainage wire clamp from multiple angles, and a total of 1322 pictures are obtained as a dataset. The visual servoing experimental platform is shown in Figure 5 It is composed of a mechanical arm, a depth camera, a bus, and a drainage wire clamp. The trained model is named main_graph, and its input is an images tensor with a dimension of float32 [1,3,1280,1280] (corresponding to a single 3-channel input image with a resolution of 1280x1280), and its output is an output0 tensor with a dimension of float32 [1,5,33600], and the total number of model parameters is 2750388.

[0141] To achieve efficient recognition of point features, the trained Yolo11 model is deployed to the TensorRT inference architecture. TensorRT, as a core C++ library inference acceleration framework, integrates inference optimizers and runtime environments, aiming to make the trained deep learning model run with higher throughput and lower latency, and help achieve high-performance inference on NVIDIA GPUs.

[0142] The deployed model is verified on the experimental platform, and the hardware and software environment configuration of the experimental platform is as follows: the operating system is Ubuntu22.04, and the graphics processing unit is NVIDIA RTX2060. The experimental results show that the model inference speed is about 25 frames per second on average, and the minimum frame rate is stable at more than 20 frames per second. Given that the data acquisition device uses an Intel RealSense D435i depth camera with a maximum frame rate of 30 frames per second, the above inference speed indicators have met the real-time processing requirements of matching the frame rate of the front-end device.

[0143] To verify the efficient visual servoing method for the drainage wire disconnection robot designed by the embodiment of the application, a physical experiment is performed. The error and speed data during the system operation are collected through the physical experiment, and the results are shown in Figure 6 The experimental results show that the visual servoing method designed by the embodiment of the application can realize the pose control of the drainage wire disconnection robot, the visual servoing process error converges rapidly and the steady-state error is small, the dynamic response is timely, fully meets the requirements of precision, stability and real-time of the drainage wire disconnection operation, and verifies the effectiveness and efficiency of the method.

[0144] It can be seen that the example of the present application realizes real-time solution and control law design of the motion speed of the camera at the end of the mechanical arm by establishing the mapping model of the three-dimensional space bolt center feature point and the two-dimensional image pixel coordinates, and constructing the image Jacobian matrix combined with differential transformation. In addition, in order to realize accurate identification of the bolt center feature point on the drainage wire clamp, the present application uses the Intel RealSense D435i depth camera to collect images of the bolt on the drainage wire clamp from multiple angles to obtain a total of 1322 images as a data set by using the experimental platform built in the laboratory. Through labeling and training of the above data set, combined with Yolo11 inference model, inference acceleration is realized by using TensorRT, and efficient identification of the bolt center feature point is realized, which provides accurate target feature point for image-based visual servoing of the mechanical arm.

[0145] The above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A visual servoing method for a substation lead line disconnection robot, characterized in that, Adopting model predictive control algorithm, and introducing disturbance observer to enhance the robustness of the system; Including the following steps: The system establishment step, the mapping relationship between the camera at the end of the mechanical arm and the kinematic model of the mechanical arm is established, and an image-based visual servo system IBVS is built; The deep learning model is trained and inference acceleration is performed, an identification model for real-time detection of drainage clamp bolts is obtained, and is deployed to the visual servo system IBVS; The feature extraction step, the drainage clamp image is collected in real time through the camera at the end of the mechanical arm, and the image feature points of the drainage clamp bolt are extracted by using the identification model; The instruction calculation step, the model predictive controller performs rolling optimization based on the deviation of the current image feature points and the expected image feature points of the drainage clamp bolt; In the optimization process, the motion speed of the camera at the end of the mechanical arm is taken as the constraint condition, and the optimal, smooth and safe mechanical arm motion instruction is calculated; The mechanical arm driving step, the mechanical arm motion instruction is sent to the mechanical arm motion control unit to drive the mechanical arm to move; Return to the feature extraction step until the drainage line disconnection work is accurately completed.

2. The visual servoing method for substation-oriented de-termination robot according to claim 1, wherein, The mapping relationship between the camera at the end of the mechanical arm and the kinematic model of the mechanical arm is established, and an image-based visual servo system IBVS is built, including the following steps: The three-dimensional coordinates of the image feature points on the surface of the target object are obtained by using a binocular stereo camera; And the three-dimensional coordinates of the image feature points on the surface of the target object are mapped to the two-dimensional image plane by using the perspective projection model; The pose mapping relationship between the image feature points and the end effector of the mechanical arm is established, and the motion trajectory of the end effector of the mechanical arm is calculated.

3. The visual servoing method for substation-oriented de-termination robot according to claim 2, wherein, The three-dimensional coordinates of the image feature points on the surface of the target object are mapped to the two-dimensional image plane by using the perspective projection model, which is represented as: ； wherein the three-dimensional space coordinates are , the two-dimensional image plane coordinates are , , i represents a pixel index, and n represents the total number of pixel points; represents a camera focal length, and respectively represent scale factors of the image coordinate system axis and axis.

4. The visual servoing method for substation-oriented de-termination robot according to claim 3, wherein, The pose mapping relationship between the image feature points and the end effector of the mechanical arm is established, and the motion speed of the end effector of the mechanical arm is calculated, including the following steps: The pose mapping relationship between the image feature points and the end effector of the mechanical arm is established, and the motion speed of the end effector of the mechanical arm is calculated, including the following steps: ； ； wherein, denotes a rate of change of image features; is called the image Jacobian matrix, denotes a velocity of the end effector; In an image-based visual servoing system IBVS, a control error is expressed as ； wherein, represents the current feature point pose, represents the desired feature point pose The pose mapping relationship between the image feature points and the end effector of the mechanical arm is established, and the motion speed of the end effector of the mechanical arm is calculated, including the following steps: ； wherein obtained by obtained; The pose mapping relationship between the image feature points and the end effector of the mechanical arm is established, and the motion speed of the end effector of the mechanical arm is calculated, including the following steps: ； wherein represents a visual servoing gain, represents a Moore-Penrose pseudo-inverse of the image Jacobian .

5. The visual servoing method for substation-oriented de-termination robot according to claim 1, wherein, The three-dimensional coordinates of the image feature points on the surface of the target object are obtained by using a binocular stereo camera; And the three-dimensional coordinates of the image feature points on the surface of the target object are mapped to the two-dimensional image plane by using the perspective projection model; The three-dimensional coordinates of the image feature points on the surface of the target object are obtained by using a binocular stereo camera; And the three-dimensional coordinates of the image feature points on the surface of the target object are mapped to the two-dimensional image plane by using the perspective projection model; The model predictive controller is established by the following way:

6. The visual servoing method for substation-oriented de-termination robot according to claim 1, wherein, The IBVS model is discretized by using Newton-Euler method to obtain a discrete time model, which is represented as: The expression of the time-dependent visual servo model is established, which is represented as: ； wherein, is the image feature of the next time instant, is the current image feature, is the sampling time; The prediction model of the model predictive controller is established, which is represented as: ； where s(k) is the state vector, representing the system state at a single discrete time point k, represents the image Jacobian matrix at a single discrete time point k; The optimization objective function of the model predictive controller is: ； wherein is step state prediction vector, is step image Jacobian matrix, is step predicted control sequence, parameter is a dimension value of the state, is a dimension of the input control quantity, is composed of an identity matrix.

7. The visual servoing method for substation-oriented de-termination robot according to claim 6, wherein, The motion speed of the camera at the end of the mechanical arm is taken as the constraint condition, which is represented as: ； wherein, is the desired vector of features, Q is a state error weighting matrix, is the control sequence, R is a control weighting matrix; the objective of the controller is to find the control sequence that minimizes the objective function subject to the constraints.

8. The visual servoing method for substation-oriented de-termination robot according to claim 6, wherein, The model controller also introduces a disturbance observer to enhance the robustness of the system; Including the following steps: ； wherein, represents the maximum velocity of the end effector, represents the minimum velocity of the end effector.

9. The visual servoing method for substation-oriented de-termination robot according to claim 6, wherein, ​ Considering the visual servoing system with external disturbance and unmodeled dynamics, the discrete-time model is extended as ； wherein, is the state at time is the state at time is the total disturbance vector at time denotes the camera velocity at the end of the robot arm at time The total disturbance vector is expanded to the new system state to obtain the extended state model as ； where I represents the identity matrix; A discrete-time extended state observer is designed based on the extended state model, which adopts The predicted value is corrected by the measured value at the time instant, expressed as: The predicted value is corrected by the measured value at the time instant, expressed as: ； ； wherein is a prior estimate of the state at time is an estimate of the state at time is the state at time is the velocity at time is a disturbance estimate at time is an observation error, is the state at time is a state estimate, is a disturbance estimate, is a prior estimate of the disturbance at time is a state estimation gain matrix, is a disturbance estimation gain matrix, is a sampling time; The disturbance estimated by the observer is integrated into the MPC framework of the model controller, and a velocity feedforward compensation term based on the disturbance estimation is designed to offset the effect of the disturbance estimation on the system as ； wherein is a regularization parameter used to avoid numerical instability problems when the Jacobian matrix is close to singular. Finally, the control quantity composed of the nominal control quantity optimized by the MPC and the ESO compensation quantity is obtained as ； wherein denotes the final control quantity, denotes the nominal control quantity of the MPC optimization, denotes the speed feedforward compensation term at the k time point.

10. The visual servoing method for a substation-oriented de-termination robot according to claim 9, wherein, The disturbance estimated by the ESO is integrated into the MPC framework of the model controller, and the constraint condition is adjusted to ensure that the control quantity satisfies the speed constraint, and the adjusted constraint condition is expressed as: ； wherein, is the disturbance compensation quantity for the current time instant.

Images