A method, apparatus, robot, and storage medium for operating robots in substations.

Abstract

This application relates to the field of robotics and provides a method, apparatus, robot, and storage medium for substation robot operations. The method includes: capturing real-time image information using a camera and extracting image features from the real-time image information during the movement of the robotic arm; inputting the desired image features and the extracted image features into a visual servo control model deployed on the robot, and outputting a robotic arm control signal; obtaining a preliminary motion trajectory of the robotic arm based on the angle control quantities of each joint of the end effector; inputting the desired speed signal of the end effector into an active disturbance rejection controller deployed on the robot, and performing disturbance compensation on the preliminary motion trajectory through the active disturbance rejection controller to obtain a target motion trajectory; and controlling the end effector to perform operations on the target object according to the target motion trajectory. By utilizing visual servoing and active disturbance rejection control, the robot has autonomous operation and anti-interference capabilities, ensuring the stability of the robot's operation.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a substation robot operation method, a substation robot operation device, a corresponding robot, and a corresponding computer-readable storage medium. Background Technology

[0002] Substations have complex operating environments and limited working space. When performing live-line maintenance on power equipment with voltage levels of 220kV and above, they are usually subject to various constraints, such as strict requirements on weather conditions, safety protection and insulation tools. Furthermore, due to the confined space, equipotential work poses significant safety risks and places high demands on the skill level and proficiency of the operators.

[0003] In related technologies, live-line working robots are mainly used in the power transmission and distribution field, while the application of related technologies and equipment in the substation field is still blank. In particular, in operation scenarios such as automatic disassembly and assembly of the lead wires and bolt tightening of 220kV disconnecting circuit breakers, due to the lack of intelligent working equipment, the current main reliance is still on traditional manual maintenance methods. However, traditional retrieval methods are no longer suitable for the new requirements of intelligent operation and maintenance of the power grid and safe live-line work. Therefore, there is a need to develop intelligent and safe live-line working equipment.

[0004] However, most robots in the field of substation robots are used for non-contact tasks such as inspection. Although some substation operation robots can perform contact operations, they cannot perform the work of connecting and disconnecting guide wires. A small number of guide wire connecting and disconnecting robots can achieve the corresponding functions, but they require human-machine collaborative operation, remote control or teleoperation, and have weak autonomous operation capabilities. At the same time, they cannot resist factors that affect safe operation, such as high-altitude wind disturbance. Summary of the Invention

[0005] This application provides a substation robot operation method, device, robot, and storage medium. By utilizing visual servoing and active disturbance rejection control, the robot can achieve autonomous operation and anti-interference capabilities, ensuring the stability of operation in high-voltage working environments and contributing to the safe and stable operation of power equipment.

[0006] In one aspect, embodiments of this application provide a substation robot operation method, wherein the robot has a working robotic arm, the working robotic arm includes an end effector and a camera, and the method includes:

[0007] During the movement of the robotic arm, image features are extracted from the real-time image information captured by the camera.

[0008] The desired image features and the extracted image features are input into the visual servo control model deployed on the robot, and the output is the robotic arm control signal; wherein, the image Jacobian matrix in the visual servo control model is used to indicate the mapping relationship between the rate of change of image feature points and the motion speed of the end effector, and the robotic arm control signal includes the angle control amount of each joint of the end effector generated based on the mapping relationship;

[0009] The initial motion trajectory of the robotic arm is obtained based on the angle control values ​​of each joint of the end effector;

[0010] The desired velocity signal of the end effector is input to the active disturbance rejection controller deployed on the robot. The active disturbance rejection controller performs disturbance compensation on the preliminary motion trajectory to obtain the target motion trajectory.

[0011] The end effector is controlled to perform operations on the target work object according to the target motion trajectory.

[0012] In another aspect, embodiments of this application provide a substation robot operation device applied to the robot, the robot having a working robotic arm, the working robotic arm including an end effector and a camera, the device comprising:

[0013] The image feature extraction module is used to extract image features from the real-time image information captured by the camera during the movement of the robotic arm.

[0014] The control signal generation module is used to input the desired image features and the extracted image features into the visual servo control model deployed on the robot, and output the robotic arm control signal; wherein, the image Jacobian matrix in the visual servo control model is used to indicate the mapping relationship between the rate of change of image feature points and the motion speed of the end effector, and the robotic arm control signal includes the angle control amount of each joint of the end effector generated based on the mapping relationship;

[0015] The motion trajectory planning module is used to obtain the preliminary motion trajectory of the working robot arm based on the angle control values ​​of each joint of the end effector;

[0016] The disturbance compensation module is used to input the desired velocity signal of the end effector to the active disturbance rejection controller deployed on the robot, and to perform disturbance compensation on the preliminary motion trajectory through the active disturbance rejection controller to obtain the target motion trajectory;

[0017] The job control module is used to control the end effector to perform work on the target object according to the target motion trajectory.

[0018] In another aspect, embodiments of this application also provide a robot, including: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements any of the substation robot operation methods described above.

[0019] In another aspect, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the substation robot operation methods described above.

[0020] In another aspect, embodiments of this application also provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the substation robot operation method described in the above aspects.

[0021] The substation robot operation method, apparatus, robot, and storage medium provided in this application embodiment capture real-time image information and extract image features based on a camera during the movement of the robot's working manipulator. The desired image features and the extracted image features are input to a visual servo control model deployed on the robot. Based on the mapping relationship between the changes in image feature points indicated by the image Jacobian matrix in the visual servo control model and the movement speed of the end effector, the angle control quantities of each joint of the end effector are generated based on the mapping relationship, and the manipulator control signal is obtained. At this time, the initial motion trajectory of the working manipulator can be planned based on the aforementioned angle control quantities of each joint of the end effector. Then, the desired speed signal of the end effector can be input to the active disturbance rejection controller deployed on the robot. The aforementioned active disturbance rejection controller performs disturbance compensation on the initial motion trajectory to obtain the target motion trajectory, and then controls the end effector to perform operations on the target work object according to the target motion trajectory. By deploying a visual servo model and an active disturbance rejection controller on the robot, the visual servo control maps the motion speed of the end effector to the feature point speed on the image plane, reducing the error of the end effector's control signal on the image plane. Furthermore, the active disturbance rejection control compensates for the uncertainty of the visual servo model, achieving stable control against disturbances. This enables the robot to operate autonomously and resist interference, ensuring its operational stability in high-voltage electrical environments and contributing to the safe and stable operation of power equipment. Attached Figure Description

[0022] Figure 1 This is a diagram illustrating the operational scenario of the substation robot provided in this application embodiment;

[0023] Figure 2 This is a schematic diagram of the structure of the robotic arm provided in the embodiments of this application;

[0024] Figure 3This is a flowchart illustrating the steps of a substation robot operation method provided in an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of the control flow of the visual servoing system provided in the embodiments of this application;

[0026] Figure 5 This is a schematic diagram of the structure of the visual servoing and active disturbance rejection control algorithm provided in the embodiments of this application;

[0027] Figure 6 This is a schematic diagram of the tuning process of the control parameters of the active disturbance rejection controller provided in the embodiments of this application;

[0028] Figure 7 This is a simulation trajectory result diagram provided in the embodiments of this application;

[0029] Figure 8 This is a structural block diagram of a substation robot operation device according to an embodiment of this application;

[0030] Figure 9 This is a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0031] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0032] Traditional substation derailment robots are mostly human-machine collaborative, remotely controlled or teleoperated, and lack strong autonomous operation capabilities. At the same time, in the limited space, high altitude, complex and changeable environment of substations, the trajectory of the end effector of traditional substation derailment robots is easily affected by environmental changes such as wind disturbance, and its trajectory may deviate, which may lead to serious consequences.

[0033] This application's embodiments utilize a visual servoing model and an active disturbance rejection controller (ADRC) deployed on a robot. Visual servoing control maps the motion speed of the end effector to the feature point velocity on the image plane, reducing errors by converting the end effector's control signal into an image plane value. ADRC also compensates for uncertainties in the visual servoing model, achieving stable control against disturbances. This enables the robot to operate autonomously and resist interference, ensuring operational stability in high-voltage power environments such as substations where workspaces are limited, high-altitude, complex, and variable, thus contributing to the safe and stable operation of power equipment. Specifically, the image Jacobian matrix representing the mapping relationship between the robot's image space and task space, along with the uncertainties of the visual servoing system, can be lumped into the same state equation. A nonlinear extended state observer is introduced to estimate the lumped uncertainty of the visual servoing system online. This design provides a visual servoing and ADRC scheme for substation robots, improving their operational control accuracy and anti-interference capabilities.

[0034] Specifically, refer to Figure 1 The diagram illustrates the working scenario of the substation robot provided in this application embodiment. In the substation working scenario, the robot is involved in the disassembly and reassembly of the substation guide wires. During the disassembly and reassembly of the guide wires, the disassembly of bolts and the clamping of the guide wires are involved.

[0035] The robot for substations can be a substation lead wire disconnection and connection robot, or an equipotential maintenance robot, or any robot capable of disconnecting and connecting substation lead wires. This application does not limit this.

[0036] For example, such as Figure 1 As shown, the substation robot can be fixedly connected by three parts: a working robotic arm 1, a lifting frame 2, and a traveling chassis 3. The working robotic arm 1 can be an equipotential working robotic arm, and the traveling chassis 3 can be a ground potential traveling chassis. The substation robot composed of the aforementioned parts can perform operations on the target work object 4, such as disassembling bolts and clamping guide wires. That is, the target work object 4 can include bolts and guide wires.

[0037] Optionally, the operation process of the substation robot, specifically the disassembly and reassembly of the guide wire, can be described as follows: the traveling chassis 3 moves to the area below the target work object 4, allowing the substation robot to first perform disassembly and reassembly operations on the middle line of the three-phase power line. Specifically, the lifting frame 2 lifts the robotic arm 1 to a preset distance from the target work object 4, allowing the equipotential robotic arm to perform disassembly and reassembly operations on the middle line of the three-phase power line. For example, assuming the target work object 4 is a guide wire 1.5m away from the robotic arm 1, the lifting frame 2 can lift the robotic arm 1 to a position 1m below the guide wire. This embodiment does not impose any limitations on this.

[0038] In some embodiments of this application, in order to ensure the precise position of the lifting frame 2 and the precise position of the robotic arm 1 during disassembly and assembly operations, the robotic arm can be driven based on visual positioning information.

[0039] Specifically, such as Figure 2 As shown, the robotic arm 1 may include an end effector and a camera. The end effector may include a disassembly arm 101 and a gripping arm 104. The camera used may be an RGB-D camera (Red-Green-Blue-Depth Camera, which refers to a visual sensor that can capture color information and depth information simultaneously). It may include a first RGB-D camera 102 located on the same robotic arm as the disassembly arm 101, a second RGB-D camera 103 located on the same robotic arm as the gripping arm 104, and a third RGB-D camera 105 located in the middle of the two robotic arms, the disassembly arm 101 and the gripping arm 104.

[0040] Optionally, the third RGB-D camera 105 in the middle section can capture the working images of the robotic arm where the disassembly arm 101 and the gripping arm 104 are located. Specifically, the lifting frame 2 can drive the working robotic arm 1 to be lifted according to the visual positioning information of the third RGB-D camera 105. The first RGB-D camera 102 can capture the working images of the disassembly arm 101. Specifically, the disassembly arm 101 can perform the work according to the visual positioning information of the first RGB-D camera 102. The second RGB-D camera 103 can capture the working images of the gripping arm 104. Specifically, the gripping arm 104 can perform the work according to the visual positioning information of the second RGB-D camera 103.

[0041] It should be noted that for the specific operation process of the robotic arm 1, please refer to the embodiment of the substation robot operation method provided in this application. The embodiments of this application will not be repeated here.

[0042] In some embodiments of this application, after the disassembly arm 101 and the clamping arm 104 have completed the bolt and guide wire operations in the current work area, for example, the clamping arm 104 clamps the guide wire and the disassembly arm 101 performs bolt disassembly operations. After visual inspection confirms that there are no abnormalities, the guide wire is reconnected to the original circuit. After all three phase wires in the current area have been operated, the lifting frame 2 can be controlled to descend, and the traveling chassis 3 can be controlled to travel to the next work area, for example, to travel to the middle phase of the next guide wire disassembly and connection work area, so as to control the working robot arm 1 to perform operations on the target work object 4 in the next work area. That is, the above work process is repeated until the disassembly and connection of guide wires in all work areas of the substation are completed.

[0043] In this application embodiment, for the live-line work of dismantling and installing the lead wires of 220KV isolating circuit breakers, an equipotential maintenance robot and a work solution for substation robots are provided, which are committed to solving the technical problems of live-line maintenance on site and eliminating the pain points of concern to power users.

[0044] Reference Figure 3 This document illustrates a flowchart of a substation robot operation method according to an embodiment of this application. The substation robot operation method is described as follows: Figure 2 The robot shown performs and is applied to, for example, Figure 1 The work scenario shown may specifically include the following steps:

[0045] Step S301: During the movement of the robotic arm, real-time image information is captured by the camera, and image features of the real-time image information are extracted.

[0046] The movement of the robotic arm relies on visual positioning information. Visual positioning information mainly refers to the real-time image information captured by its camera during the movement of the robotic arm. This real-time image information may include image features of the object to be identified. In the operation scenario of this application embodiment, the extracted image features usually refer to the feature information of the feature point of the target object.

[0047] In some embodiments of this application, the movement of the robotic arm can be adjusted in real time by tracking the aforementioned feature points, so as to ensure that the feature points within the camera's field of view can be guided to the predetermined target position, thereby enabling the robotic arm to perform disassembly and assembly operations on the target work object.

[0048] Step S302: Input the desired image features and the extracted image features into the visual servo control model deployed on the robot, and output the robotic arm control signal.

[0049] Optionally, to ensure that feature points within the camera's field of view can be guided to the predetermined target position, an image-based visual servoing (IBVS) system can be used to accurately identify and track feature points in the image, thereby manipulating the robotic arm using real-time image information captured by the camera. It should be noted that deploying an image-based visual servoing system in the robot provided in this embodiment helps ensure the robot's autonomous operation.

[0050] Visual servo control is a control method based on visual feedback, which can be applied to the control of mobile robots and underwater robots. In this embodiment, it is applied to the control of a robot facing a substation.

[0051] In a visual servoing system, the required control action can be calculated by comparing the differences between detected feature points in an image and pre-defined target features. In some embodiments of this application, the image features extracted from real-time image information are the detected feature points in the image, and the desired image features are the pre-defined target features. The desired image features and the extracted image features can be input into a visual servoing control model deployed on the robot, and the robot arm control signal can be obtained by outputting the visual servoing control model.

[0052] Specifically, the difference between the desired image features and the extracted image features is mainly quantified through the image Jacobian matrix in the visual servo control model. This matrix can describe the geometric relationship between the changes in image feature points and the motion of the end effector of the robotic arm. Specifically, it can be used to indicate the mapping relationship between the rate of change of image feature points and the motion speed of the end effector. Based on the image Jacobian matrix, which can encode information about the motion of image pixels, the motion in the feature pixels is associated with the motion of the end effector of the robotic arm, thereby deriving the control signal for the robotic arm.

[0053] Among them, the rate of change of image feature points refers to the rate of change of image features captured by the camera, and the motion speed of the end effector is actually the motion speed of the camera in three-dimensional space. The mapping relationship of image Jacobian matrix quantization can associate the motion in feature pixels with the operation of the end effector, and map the motion speed of the end effector to the feature point speed on the image plane. The control signal of the working robot is derived with the goal of reducing image deviation.

[0054] In some embodiments of this application, visual servoing control can be manifested as defining a robot kinematic error function based on the difference between the robot's current pose and the desired pose obtained from the vision system. Error convergence is continuously achieved as the robot moves towards the target work object, converting the control commands from the end effector into error reduction on the image plane. It should be noted that the specific process of deriving the control signal is not limited in the embodiments of this application.

[0055] Step S303: The initial motion trajectory of the robotic arm is obtained based on the angle control values ​​of each joint of the end effector.

[0056] The robotic arm control signals may include control quantities generated based on mapping relationships, which are primarily generated with the goal of reducing image deviation.

[0057] Optionally, the generated control quantity can refer to the angle control quantity of each joint of the end effector.

[0058] In some embodiments of this application, in order to manipulate the robotic arm, a preliminary motion trajectory of the robotic arm can be initially planned based on the control quantity generated with the goal of reducing image deviation. The generated preliminary motion trajectory is the motion trajectory under ideal conditions.

[0059] It should be noted that in the substation operation scenario, the robotic arm performs the disassembly and assembly of the guide wire, which is specifically implemented by the end effector. The end effector includes a disassembly arm and a clamping arm. The target objects of the operation include bolts and guide wires. That is, the disassembly and assembly of the guide wires involves the disassembly arm disassembling the bolts and the clamping arm clamping the guide wires. The preliminary planned motion trajectory can include the preliminary motion trajectory of the disassembly arm disassembling the bolts and the preliminary motion trajectory of the clamping arm clamping the guide wires.

[0060] Step S304: The desired speed signal of the end effector is input to the active disturbance rejection controller deployed on the robot. The active disturbance rejection controller performs disturbance compensation on the initial motion trajectory to obtain the target motion trajectory.

[0061] In some embodiments of this application, the motion trajectory of the robotic arm is usually subject to internal and external disturbances. Internal parameter disturbances can refer to the influence of control parameters inside the robotic arm; external disturbances can refer to the disturbances generated by the characteristics of the robotic arm itself or its components during operation, such as joint friction / damping, parameter uncertainty, actuator dynamics, vibration during operation, etc. In this case, the desired speed signal of the end effector can be input to the active disturbance rejection controller deployed on the robot, and the active disturbance rejection controller can perform disturbance compensation on the initial motion trajectory.

[0062] Among them, the active disturbance rejection controller is deployed on the robot. The active disturbance rejection control algorithm is beneficial to improving the robot's anti-interference ability, that is, it can improve the robot's robustness.

[0063] Specifically, the motion trajectory is usually directly affected by parameter tuning. In this embodiment, the control parameters of the active disturbance rejection controller can be tuned so that the initial motion trajectory is affected by the tuned control parameters, thereby achieving disturbance compensation for the initial motion trajectory and obtaining the target motion trajectory after parameter tuning.

[0064] It should be noted that in the substation operation scenario, the robotic arm performs the disassembly and assembly of the guide wire, which is specifically implemented by the end effector. The end effector includes a disassembly arm and a clamping arm. The target objects of the operation include bolts and guide wires. That is, the disassembly and assembly of the guide wires involves the disassembly arm disassembling the bolts and the clamping arm clamping the guide wires. The target motion trajectory after parameter tuning can include the target motion trajectory of the disassembly arm disassembling the bolts and the target motion trajectory of the clamping arm clamping the guide wires.

[0065] Active Disturbance Rejection Control (ADRC) is a technique that draws on the model-independent advantage of PID (Proportional-Integral-Derivative) control. Its core idea is to aggregate the internal uncertainties and external disturbances of the visual servoing system into a generalized total disturbance, and then introduce a nonlinear extended state observer (ESO) to estimate it. Simultaneously, a tracking differentiator (TD) is added at the external signal input to manage the transition process of the input signal, enabling ADRC to have a fast response and good anti-interference capability. The specific implementation process of ADRC is not limited in the embodiments of this application.

[0066] Step S305: Control the end effector to perform operations on the target work object according to the target motion trajectory.

[0067] The target motion trajectory can be understood as the motion trajectory after overcoming internal and external interference. A robotic arm performing disassembly and assembly operations according to the aforementioned motion trajectory possesses anti-interference capabilities. In some embodiments of this application, the end effector can be controlled to perform operations on the target object according to the target motion trajectory, thereby ensuring the stability of robot operations in a high-voltage working environment.

[0068] In some embodiments of this application, after the end effector performs work on the target object according to the target motion trajectory, the actual motion trajectory of the end effector can be obtained and fed back to the visual servo control model to form closed-loop control, so as to correct errors in real time, improve motion accuracy and enhance system robustness.

[0069] In a preferred embodiment of this application, after the disassembly arm and clamping arm have completed the bolt and guide wire operations in the current work area, for example, the clamping arm clamps the guide wire and the disassembly arm removes the bolts, and after visual inspection confirms no abnormalities, the guide wire is reconnected to the original circuit. After all three phase wires in the current area have been operated, the lifting frame can be controlled to descend, and the traveling chassis can be controlled to travel to the next work area, for example, to travel to the middle phase of the next guide wire disassembly and connection work area, so as to control the working robotic arm to perform operations on the target work object in the next work area. That is, the above work process is repeated until the disassembly and connection of guide wires in all work areas of the substation are completed. This embodiment of the application does not limit this.

[0070] In this embodiment, during the movement of the robot's working arm, real-time image information is captured by a camera and image features are extracted. The desired image features and the extracted image features are input to a visual servo control model deployed on the robot. Based on the mapping relationship between the changes in image feature points indicated by the image Jacobian matrix in the visual servo control model and the movement speed of the end effector, a control quantity is generated based on the mapping relationship to obtain the robot arm control signal. At this time, the initial movement trajectory of the working robot arm can be planned based on the aforementioned control quantity. Then, the desired speed signal of the end effector can be input to the active disturbance rejection controller deployed on the robot. The aforementioned active disturbance rejection controller performs disturbance compensation on the initial movement trajectory to obtain the target movement trajectory, and then controls the end effector to perform operations on the target work object according to the target movement trajectory. By deploying a visual servo model and an active disturbance rejection controller on the robot, the visual servo control maps the motion speed of the end effector to the feature point speed on the image plane, reducing the error of the end effector's control signal on the image plane. Furthermore, the active disturbance rejection control compensates for the uncertainty of the visual servo model, achieving stable control against disturbances. This enables the robot to operate autonomously and resist interference, ensuring its operational stability in high-voltage electrical environments and contributing to the safe and stable operation of power equipment.

[0071] In some embodiments of this application, the control process for visual servoing may specifically include the following steps:

[0072] Sub-step S21 involves constructing the kinematic error function of the robotic arm based on the extracted image features and desired image features using a visual servo control model.

[0073] In a visual servoing system, the required control actions can be calculated by comparing the differences between detected feature points in an image and pre-defined target features. The visual servoing control process involves image processing techniques, namely feature extraction and feature matching. Once feature points are identified, the visual servoing system can calculate the deviation between the feature points and the target position. This deviation is used to adjust the robotic arm's movement to reduce errors and achieve precise positioning. In other words, feature points can be used as feedback signals to adjust the servo control loop.

[0074] In some embodiments of the present application, Figure 4 As shown, an image-based vision controller generates control signals for the robotic arm targeting the joint angle controller. Responding to these control signals, the robotic arm is manipulated to perform corresponding movements. During the robotic arm's movement, images are acquired via an end-effector camera. Image features are then extracted from these end-effector images, identifying the detected feature points and the desired image features as pre-defined target features. These desired and extracted features are input to a visual servo control model deployed on the robot. The model outputs the robotic arm control signals, thus generating the control signals for the joint angle controller. The robot then manipulates its current movement according to these output signals. This process of end-effector image acquisition, image feature extraction, desired image feature input, and image-based vision control is repeated, achieving a closed-loop servo control circuit.

[0075] Specifically, visual servo control can be expressed as defining a robot kinematic error function based on the difference between the robot's current pose and the desired pose obtained from the vision system. As the robot moves to the target work object, error convergence is continuously achieved to reduce the error in the image plane by converting the control commands of the end effector into the image plane.

[0076] Optionally, the visual information captured by the camera can be used to extract features through a feature extraction network, that is, to detect the target objects bolts and guide lines. The features extracted from the visual information and the desired image features are then input into the visual servo (IBVS) control model. Based on the extracted image features and the desired image features, the visual servo control model constructs the kinematic error function of the robotic arm, so as to perform error convergence on the aforementioned constructed kinematic error function of the robotic arm.

[0077] The extracted image features are used to indicate the actual visual feature state, while the desired image features are used to indicate the desired visual feature state.

[0078] For example, the kinematic error function of the robotic arm can refer to the difference between the current pose of the robotic arm and the desired pose obtained from the vision system. As the robotic arm moves to the bolt and the guide line, the error convergence is continuously performed. The goal of visual servo control is to minimize the error defined by the following formula.

[0079] The constructed kinematic error function e(t) of the robotic arm can be expressed by the following formula:

[0080]

[0081] In the formula, It can refer to the set of regions of interest in an image. This can refer to a set of intrinsic and extrinsic camera parameters, including but not limited to focal length, pixel size, and camera pose; function sum function The current state and the desired state can be defined separately, and the function... sum function The specific formula can be seen as follows:

[0082]

[0083] In the formula, This represents the actual visual features, namely the coordinates of the bolts and guide lines in the image, i.e., two-dimensional image coordinates. , It is the camera's focal length; This represents the desired visual feature state, where (m*, n*) are the corresponding two-dimensional image coordinates. Here, is assumed to be the spatial point of the bolt to be disassembled in the world coordinate system. Its coordinates relative to the camera coordinate system are At this point, the coordinates of the work object relative to the camera coordinate system can be converted into two-dimensional image coordinates mapped onto the image plane based on the camera's focal length.

[0084] Sub-step S22: Establish an image Jacobian matrix based on the rate of change of image feature points of the robot's work object feature points and the motion speed of the end effector.

[0085] The rate of change of image feature points can be determined based on the motion speed of the feature points of the working object relative to the camera coordinate system, and the motion speed of the end effector can be determined based on the motion speed of the camera in the world coordinate system.

[0086] To establish the mapping relationship between the camera's 3D spatial motion velocity and the rate of change of image point features at a point in world coordinates, we can use... This represents the velocity movement of a camera in a world coordinate system as a rigid body. Let's assume we select a spatial point in the world coordinate system where a bolt to be disassembled / reassembled is located. Its coordinates relative to the camera coordinate system are Therefore, the velocity of point P relative to the camera coordinate system can be expressed as follows:

[0087]

[0088] Right now:

[0089] In the formula, v is a linear velocity vector that can be used to represent the camera's translational velocity in the world coordinate system. x v y and v z These can be represented as the camera's translational velocity along the x, y, and z axes, respectively. ω is the angular velocity vector, which can be used to represent the camera's rotational rate about each axis. x ω y ω z These can represent the camera's rotational speed around the x, y, and z axes, respectively. This refers to the spatial point in the world coordinate system where the bolts to be disassembled / reassembled are located. Let P be the spatial point in the camera coordinate system. These are the three-dimensional coordinates of spatial point P after it moves at a speed based on the camera coordinate system.

[0090] The key to designing a visual servo controller lies in image feature selection and kinematic modeling of these features. The kinematic model of image features refers to the mapping relationship between the rate of change of image features captured by the camera and the camera's velocity in three-dimensional space, which can be represented by the following formula:

[0091]

[0092] In the formula, It could refer to the rate of change of a characteristic; (This refers to the three-dimensional spatial motion velocity in the current camera coordinate system, which is the motion velocity of the robotic arm's end effector relative to the polar coordinate system). It is the camera's focal length; These are the corresponding two-dimensional image coordinates; Let be the Jacobian matrix of the image.

[0093] The visual servo control of the visual servo control system relies on the image Jacobian matrix, which can be used to represent the relationship between feature changes and the motion speed of the end effector of the robotic arm. In the embodiments of this application, it is implemented based on the image Jacobian matrix. The image Jacobian matrix can describe the relationship between the changes of feature points in the image and the camera motion, and associate the camera speed with the feature speed in the normalized image coordinates. Specifically, it illustrates the mapping relationship between the point feature transformation rate and the camera's motion speed in three-dimensional space.

[0094] In this embodiment of the application, for a guide line target that is constantly moving due to wind disturbance at high altitude, the image Jacobian matrix can help predict and compensate for the movement of the guide line, thereby achieving dynamic tracking.

[0095] For example, to ensure the matrix dimension requirement of d=6, the number of image features should exceed 6. To control the robotic arm of an equipotential work robot with 6 degrees of freedom, at least 3 feature points are required. When the feature vector consists of 3 feature points, the image Jacobian matrix can be expressed as follows:

[0096]

[0097] Where, L s For the constructed image Jacobian matrix, L p1 L p2 L p3 These are feature vectors representing three feature points. Through the above processing, an image Jacobian that encodes information about the motion of image pixels can be used to correlate the motion in the feature pixels with the motion of the end effector, facilitating the subsequent generation of robot arm control signals for the working robot arm.

[0098] Sub-step S23: Based on the image Jacobian matrix and the mechanical arm kinematic error function, error convergence is performed to obtain the motion speed of the end effector.

[0099] In visual servo control, the velocity of the camera end effector can be mapped to the velocity of feature points on the image plane based on the image Jacobian matrix. This mapping is achieved by considering the kinematic model of the camera and the geometric relationship between the image feature points. In other words, the control commands of the end effector can be converted into errors on the image plane through the image Jacobian matrix, thereby achieving accurate tracking of the target.

[0100] Specifically, based on the mapping relationship between the motion velocity of the feature points of the work object relative to the camera coordinate system and the motion velocity of the camera in the world coordinate system, the kinematic error function of the robotic arm can be converged. Specifically, the convergence is aimed at minimizing the error value, reducing the error between the actual visual feature state and the desired visual feature state, thereby realizing the conversion of the control commands of the end effector into the image plane with reduced error, and obtaining the desired feature point velocity of the work object feature points. Since the mapping relationship maps the motion velocity of the end effector to the feature point velocity on the image plane, the desired feature point velocity can be obtained by combining the aforementioned mapping relationship during the error convergence process, and then the motion velocity of the end effector can be obtained. Specifically, it can be expressed as obtaining the motion velocity of the end effector by mapping the desired feature point velocity based on the mapping relationship.

[0101] For example, the error convergence of the kinematic error function of the robotic arm can be represented by the following equation:

[0102]

[0103] In the formula, e represents the error. Scaling factor This is the pseudo-inverse of the desired image Jacobian matrix, which is derived from the features. The selection is used for calculation. The speed of the end effector.

[0104] Sub-step S24: Generate robotic arm control signals based on the motion speed of the end effector;

[0105] The movement speed of the end effector can be used to determine the control quantity of the end effector; the control quantity can refer to the angle control quantity of each joint of the end effector. At this time, the robot control signal can be generated based on the movement speed of the end effector. The robot control signal is used to manipulate the angle of each joint of the end effector. That is, the robot control signal for the joint angle controller can be generated based on the image vision controller.

[0106] Optionally, the speed of camera movement, i.e., the speed of end effector movement, can be used. As a control variable, the angle control variables of each joint are then calculated using the kinematics of the robotic arm, thereby generating the robotic arm control signal.

[0107] In some embodiments of this application, the motion speed of the end effector can be mapped to the joint angular velocity of each joint of the end effector based on the pseudo-inverse matrix of the Jacobian matrix of the robotic arm. Then, the joint angular velocity of each joint of the end effector is integrated to obtain the angle control quantity of each joint, and then the robotic arm control signal is generated based on the angle control quantity of each joint.

[0108] The Jacobian matrix of the robotic arm can be used to indicate the mapping relationship between the joint angular velocity of each joint of the end effector and the motion speed of the end effector.

[0109] For example, the calculation formula for the angle control amount of each joint can be as follows:

[0110]

[0111] In the formula, This refers to joint angular velocity. This refers to the speed of movement of the end effector. This refers to the pseudo-inverse of the Jacobian matrix of the robotic arm. This refers to the angle control values ​​of each joint. The Jacobian matrix of the robotic arm is as follows:

[0112]

[0113] In the formula, Let be the unit vector of the rotation axis of the i-th joint. Let i be the position of the origin of the coordinate system for the i-th joint. Let be the position of the end effector, where assuming the end effector has n joints, the i-th joint can be one of the n joints, i.e., i = 1, 2, ..., n (n is a positive integer).

[0114] In this embodiment, by deploying a visual servo model on the robot, the motion speed of the end effector is mapped to the feature point speed on the image plane using visual servo control. This reduces the error in converting the control signal of the end effector into an image plane, enabling the robot facing the substation to have autonomous operation capabilities.

[0115] In some embodiments of this application, in order to manipulate the robotic arm, a preliminary motion trajectory of the robotic arm can be initially planned based on the control quantity generated with the goal of reducing image deviation. The generated preliminary motion trajectory is the motion trajectory under ideal conditions.

[0116] Specifically, the positions of the positioning bolts and the clamping positions of the guide wires can be detected based on visual information fused from multiple sensors. Then, based on the obtained three-dimensional coordinate information, the rotation angle of each joint, i.e., the angle control quantity of each joint, can be obtained by solving the inverse kinematics of the robotic arm. The rotation angles of each joint are combined together, i.e., when they rotate simultaneously, to form the overall motion trajectory of the robotic arm. In practical applications, the measured positions of the positioning bolts and the clamping positions of the guide wires can be understood as real-time image information captured by the camera during the movement of the robotic arm. The rotation angle of each joint can be obtained based on the angle control quantities of each joint generated by the above mapping relationship. That is, in an ideal case, the target trajectory can be a preliminary motion trajectory obtained based on the angle control quantities of each joint.

[0117] It should be noted that in the substation operation scenario, the robotic arm performs the disassembly and assembly of the guide wire, which is specifically implemented by the end effector. The end effector includes a disassembly arm and a clamping arm. The target objects of the operation include bolts and guide wires. That is, the disassembly and assembly of the guide wires involves the disassembly arm disassembling the bolts and the clamping arm clamping the guide wires. The preliminary planned motion trajectory can include the preliminary motion trajectory of the disassembly arm disassembling the bolts and the preliminary motion trajectory of the clamping arm clamping the guide wires.

[0118] Typically, the motion trajectory of a robotic arm will be affected by internal and external disturbances. Internal parameter disturbances can refer to the influence of the control parameters inside the robotic arm; external disturbances can refer to the disturbances generated by the characteristics of the robotic arm itself or its components during operation, such as joint friction / damping, parameter uncertainty, actuator dynamics, vibration during operation, etc. In this case, the desired speed signal of the end effector can be input to the active disturbance rejection controller deployed on the robot, and the active disturbance rejection controller can perform disturbance compensation on the motion trajectory obtained from the above preliminary planning.

[0119] Specifically, this can be achieved by inputting the initial motion trajectory of the disassembly arm removing bolts and the initial motion trajectory of the clamping arm clamping the guide line into the IBVS control model, so that the IBVS control model is combined with the active disturbance rejection controller to design a vision servo and active disturbance rejection control scheme for substation robots, thereby improving the operation control accuracy and anti-interference capability of substation robots.

[0120] The active disturbance rejection control process combined with visual servo control can specifically include the following steps:

[0121] In sub-step S41, the tracking differentiator of the active disturbance rejection controller outputs the dynamic characteristics of the working robot based on the input desired speed signal.

[0122] Optionally, the IBVS control model can transmit the desired velocity signal to the Active Disturbance Rejection Controller (ADRC). The ADRC integrates the image Jacobian matrix, which represents the mapping relationship between the robot's image space and the task space, and the uncertainty terms of the visual servoing system into the same channel's state equation.

[0123] Reference Figure 5 The diagram shows a schematic of the structure of the visual servoing and active disturbance rejection control algorithm provided in the embodiment of this application.

[0124] In some embodiments of this application, the input desired speed signal may refer to the operating speed of the two robotic arms, the disassembly arm and the gripping arm. It can be determined based on the desired trajectory generated by the tracking differentiator TD in the Active Disturbance Rejection Controller (ADRC). Specifically, the desired speed signal can be transmitted to the tracking differentiator TD, which can output the dynamic characteristics caused by internal parameter disturbances and external disturbances. The dynamic characteristics are then transmitted to the extended observer ESO in the ADRC, enabling the extended observer ESO to perform state estimation.

[0125] The tracking differentiator (TD) can arrange the transition process for a given step signal to resolve the contradiction between system speed and overshoot.

[0126] For example, the tracking differentiator TD can extract the differential signal from an input signal containing random noise and suppress the noise. The extracted differential signal can also be used as a low-pass filter to effectively suppress high-frequency components and output its tracking signal. First-order differential signal and second-order differential signal The basic principle of the TD tracking differentiator is to suppress noise by using a pure differentiating element followed by a first-order low-pass filter in series. This structure allows the system to reduce the impact of noise while extracting the differentiating signal.

[0127] For example, the formula for a tracking differentiator can be as follows:

[0128]

[0129] In the formula, It could refer to the integration step size; It could refer to the filter factor; It could refer to the first The input signal at any given moment; z(k) can refer to the velocity factor, which is mainly used to determine the tracking speed; z(k) can refer to the state variables inside the tracking differentiator, which is mainly used to smooth the input signal and extract its differential signal. Among them, z1(k) can refer to the smoothed tracking value of the input signal, which is used to represent the joint position; z2(k) can refer to the first-order differential signal, which is used to represent the joint velocity; z3(k) can refer to the second-order differential signal, which is used to represent the joint acceleration. This is the fast optimal control synthesis function.

[0130] In some embodiments of this application, the dynamic characteristics of the tracking differentiator TD output can specifically be manifested as the acquisition of the robot's generalized perturbation terms, including model uncertainties and internal and external perturbations.

[0131] For example, suppose the dynamic model design for the robotic arm of an equipotential work robot is as follows:

[0132]

[0133] In the formula, τ can refer to the input torque, which is usually an n×1 vector, where n is the number of joints in the robotic arm; This can refer to the robot's generalized perturbation term, including model uncertainties and internal and external perturbations; , and The position, velocity, and acceleration of a joint can be represented separately. This could refer to the inertia matrix of the robotic arm; This represents the Coriolis force and the centrifugal matrix; This represents the gravity matrix.

[0134] Assumption definition , Its state-space expression can be expressed as follows:

[0135]

[0136] In the formula, x1 can refer to the angle state of the robotic arm joint, that is, the position of the robotic arm joint; x2 can refer to the motion state of the robotic arm joint, that is, the speed of the robotic arm joint; y refers to its measurement output, which can be mainly used for feedback control.

[0137] Optionally, the robotic arm possesses complex dynamic characteristics caused by internal parameter disturbances and external disturbances (i.e., joint friction, actuator dynamics, vibrations during operation, etc.). Specifically, based on the tracking signal and first-order differential signal output by the desired trajectory generated by the aforementioned differential tracker TD, and the actual states x1 of the robotic arm joint angles and x2 of the robotic arm joint angular velocities, the ADRC controller can be driven to generate disturbance rejection control quantities. The tracking signal output by the TD-generated desired trajectory can be used to indicate the desired position, the first-order differential signal generated by the TD-generated desired velocity can be used to indicate the desired velocity, the actual state x1 of the robotic arm joint angles can be used to indicate the actual position, and the actual state x2 of the robotic arm joint angular velocities represents the actual velocity. The desired position and the actual position, as well as the desired velocity and the actual velocity, can then be compared, and the ADRC controller can be driven to generate disturbance rejection control quantities based on the comparison results.

[0138] In sub-step S42, the state of the working robot arm is estimated by the extended observer of the active disturbance rejection controller when it moves according to the initial motion trajectory, and the state estimation result is obtained.

[0139] Optionally, a nonlinear extended state observer can be introduced to estimate the lumped uncertainty of the visual servo system online, and a visual servo and active disturbance rejection control scheme for substation robots can be designed to improve the operation control accuracy and anti-interference capability of substation robots.

[0140] Specifically, an Extended Observer (ESO) can be designed to estimate the joint angles, angular velocities, and angular accelerations of an equipotential robot, as well as to estimate unknown uncertainties such as internal and external disturbances. The ESO can provide accurate estimates of the system's internal state, even when the system model has uncertainties, and can estimate internal and external disturbances in real time. This is crucial for maintaining system performance in environments with unknown disturbances.

[0141] In some embodiments of this application, the state of the working robot arm when it moves according to the initial motion trajectory can be estimated by the extended observer (ESO) of the active disturbance rejection controller to obtain the state estimation result, so as to expand the disturbance that can affect the controlled output into a new state variable, while ensuring that the expanded new state variable is in the control law.

[0142] For example, the formula for the extended observer ESO can be as follows:

[0143]

[0144] In the formula, y and u are the inputs; , , , Track y, y', y'', and the expanded portion separately; , , , This is the gain coefficient; , , It is a nonlinear factor; The interval length of the linear segment; As a compensation factor; As a nonlinear function, its expression can be shown in the following formula:

[0145]

[0146] In the formula, The threshold value can be used to distinguish whether the error e is in the linear or nonlinear region; the nonlinear exponent... It can be used to control the slope in nonlinear regions.

[0147] Sub-step S43 involves nonlinearly combining the deviation between the dynamic characteristics of the robotic arm and the state estimation results to obtain the state control quantity.

[0148] Nonlinear feedback control can dynamically adjust the control gain according to the system error, achieving the effect of large gain for small errors and small gain for large errors, thereby improving the accuracy and response speed of the control system.

[0149] Specifically, ADRC controllers, through nonlinear feedback, can more effectively compensate for disturbances in the system, including unmodeled dynamic and external disturbances. This feedback mechanism enables the system to maintain stability and performance in the face of uncertainties and disturbances, helping to improve the system's robustness. Even with changes in parameters or external conditions, the performance of the control system can be maintained, which helps to reduce overshoot, especially in scenarios such as substation conductor disconnection and connection robot operations where fast response and high precision are required.

[0150] Sub-step S44: Use preset state observation values ​​to perform disturbance compensation on the state control quantity of the working robot arm when it moves according to the initial motion trajectory.

[0151] For example, the process of nonlinear combination and disturbance compensation can be achieved by the following formula:

[0152]

[0153] As shown in the above equation, the output signal of the tracking differentiator TD can be used to... , , State estimation with extended state observer ESO , , The deviations between them are nonlinearly combined to obtain the control quantity. Then, through the observed values To compensate for disturbances, the total output of the active disturbance rejection controller is obtained. .

[0154] Sub-step S45: The controller parameters of the active disturbance rejection controller are tuned based on the feedback error of the disturbance compensation result.

[0155] The ADRC algorithm model has many parameters, and the tuning of the controller parameters can directly affect the performance of the controller and the stability of the entire system. Tuning the controller parameters can help optimize the performance of the disturbance observer and the compensator, thereby more accurately estimating and compensating for internal and external disturbances in the system.

[0156] In practical applications, ADRC parameter tuning usually needs to take into account multiple indicators such as the system's dynamic characteristics, desired response time, overshoot, steady-state error, and sensitivity to disturbances.

[0157] For example, a genetic algorithm can be used to tune the controller parameters, and its objective function can be as follows:

[0158]

[0159] In the formula, For feedback error, To adjust the time, For overshoot, , , For weights.

[0160] The primary function of the objective function is to quantify the performance indicators of the control system into comparable numerical values, guiding the algorithm towards a better solution. Specifically, this can be achieved by substituting the feedback error, settling time, and overshoot obtained from different parameter combinations into the objective function, thereby improving the objective function. The smaller the better. So, at this point... The combination of parameters is the optimized controller parameters.

[0161] like Figure 6 As shown, the feedback error can be... Substituting the objective function, the ADRC parameters are optimized using a genetic algorithm under the condition of the input desired velocity signal v(t), thereby realizing the controller parameters of the active disturbance rejection controller. This facilitates the subsequent use of the tuned controller parameters to compensate for the initial motion trajectory, resulting in the target motion trajectory y(t) for the substation diversion line disconnection robot.

[0162] It should be noted that once the objective function is defined, the parameter optimization process of the genetic algorithm is initiated. In each generation, the newly generated population is evaluated and tested in the ADRC environment to identify the best-performing individuals. This process continues until a preset number of generations is reached, and finally, the optimal parameter set is selected from all generations. This embodiment of the application does not impose any limitations on this process.

[0163] Sub-step S46: Based on the tuned controller parameters, compensate for the initial motion trajectory to obtain the target motion trajectory.

[0164] The initial motion trajectory is the motion trajectory under ideal conditions, while the actual running trajectory is usually directly affected by the ADRC parameter tuning.

[0165] In some embodiments of this application, the Active Disturbance Rejection Controller (ADRC) can smooth the step-planned trajectory into a actually trackable reference signal through the tracking differentiator (TD), and the Extended State Observer (ESO) can dynamically adjust the control quantity in real time to estimate disturbances such as wind disturbance and model error, thereby compensating for disturbances and correcting trajectory deviations to obtain the target motion trajectory.

[0166] The target motion trajectory can be understood as the motion trajectory after overcoming internal and external interference. The robotic arm that performs disassembly and assembly operations according to the aforementioned motion trajectory has anti-interference capabilities.

[0167] In practical applications, the end effector can be controlled to perform tasks on the target object according to the target motion trajectory, thereby ensuring the stability of robot operation in a high-voltage working environment.

[0168] In some embodiments of this application, after the end effector performs work on the target object according to the target motion trajectory, the actual motion trajectory of the end effector can be obtained and fed back to the visual servo control model to form closed-loop control, so as to correct errors in real time, improve motion accuracy and enhance system robustness.

[0169] In a preferred embodiment of this application, after the disassembly arm and clamping arm have completed the bolt and guide wire operations in the current work area, for example, the clamping arm clamps the guide wire and the disassembly arm removes the bolts, and after visual inspection confirms no abnormalities, the guide wire is reconnected to the original circuit. After all three phase wires in the current area have been operated, the lifting frame can be controlled to descend, and the traveling chassis can be controlled to travel to the next work area, for example, to travel to the middle phase of the next guide wire disassembly and connection work area, so as to control the working robotic arm to perform operations on the target work object in the next work area. That is, the above work process is repeated until the disassembly and connection of guide wires in all work areas of the substation are completed. This embodiment of the application does not limit this.

[0170] Furthermore, to verify the vision servoing and active disturbance rejection control method for the substation feeder disconnection robot designed in this application embodiment, simulation studies can be conducted. Simultaneously, this control method can be compared with vision servoing combined with PID control to better verify the advantages of this control method. Specifically, the simulation trajectory results can be shown as follows: Figure 7 As shown, in Figure 7 In this context, q1 represents the desired trajectory, ADRC represents the trajectory generated by the visual servoing and active disturbance rejection control method in this embodiment, and PID represents the contrast control method. Figure 7 As can be seen, the control algorithm of this application embodiment can track the desired trajectory well and reduce the impact of external interference.

[0171] In this embodiment, by deploying a visual servo model and an active disturbance rejection controller on the robot, the visual servo control maps the motion speed of the end effector to the feature point speed on the image plane, converts the control signal of the end effector into error reduction on the image plane, and uses active disturbance rejection control to compensate for the uncertainty of the visual servo model, thereby achieving stable control against disturbances. This enables the robot to have autonomous operation and anti-interference capabilities. In high-voltage working environments, such as substations with limited working space, high altitude, complex and changeable environments, the stability of its operation can be guaranteed, which is conducive to ensuring the safe and stable operation of power equipment.

[0172] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0173] Reference Figure 8 This diagram illustrates a structural block diagram of a substation robot operation device according to an embodiment of this application. Applied to a robot, the robot has a working robotic arm, which includes an end effector and a camera. Specifically, it may include the following modules:

[0174] The image feature extraction module 801 is used to extract image features from real-time image information captured by a camera during the movement of the robotic arm.

[0175] The control signal generation module 802 is used to input the desired image features and the extracted image features into the visual servo control model deployed on the robot, and output the robotic arm control signal; wherein, the image Jacobian matrix in the visual servo control model is used to indicate the mapping relationship between the rate of change of image feature points and the motion speed of the end effector. The robotic arm control signal includes the angle control amount of each joint of the end effector generated based on the mapping relationship.

[0176] The motion trajectory planning module 803 is used to obtain the initial motion trajectory of the working robot arm based on the angle control values ​​of each joint of the end effector.

[0177] The disturbance compensation module 804 is used to input the desired speed signal of the end effector to the active disturbance rejection controller deployed on the robot, and to perform disturbance compensation on the initial motion trajectory through the active disturbance rejection controller to obtain the target motion trajectory;

[0178] The job control module 805 is used to control the end effector to perform work on the target object according to the target motion trajectory.

[0179] In some embodiments of this application, the control signal generation module 802 may include the following sub-modules:

[0180] The control signal generation submodule is used to construct the kinematic error function of the robotic arm based on the extracted image features and the desired image features through the visual servo control model; establish the image Jacobian matrix based on the rate of change of image feature points of the robot's working object feature points and the motion speed of the end effector; perform error convergence based on the image Jacobian matrix and the kinematic error function of the robotic arm to obtain the motion speed of the end effector; and generate the robotic arm control signal based on the motion speed of the end effector.

[0181] In some embodiments of this application, the rate of change of image feature points in the image Jacobian matrix is ​​determined based on the motion velocity of the object feature points relative to the camera coordinate system, and the motion velocity of the end effector in the image Jacobian matrix is ​​determined based on the motion velocity of the camera in the world coordinate system; the control signal generation submodule may include the following units:

[0182] The motion speed determination unit is used to perform error convergence on the kinematic error function of the robotic arm based on the mapping relationship between the motion speed of the feature points of the work object relative to the camera coordinate system and the motion speed of the camera in the world coordinate system, so as to obtain the expected feature point speed of the feature points of the work object; based on the mapping relationship, the motion speed of the end effector is obtained by mapping the expected feature point speed.

[0183] In some embodiments of this application, the control signal generation submodule may include the following units:

[0184] The control signal generation unit is used to acquire the Jacobian matrix of the robotic arm, and based on the pseudo-inverse matrix of the Jacobian matrix, to map the motion velocity of the end effector to the joint angular velocity of each joint of the end effector; wherein, the Jacobian matrix of the robotic arm is used to indicate the mapping relationship between the joint angular velocity of each joint of the end effector and the motion velocity of the end effector; the joint angular velocities of each joint of the end effector are integrated to obtain the angle control quantity of each joint, and the robotic arm control signal is generated based on the angle control quantity of each joint.

[0185] In some embodiments of this application, the disturbance compensation module 804 may include the following sub-modules:

[0186] The disturbance compensation submodule is used to output the dynamic characteristics of the working robot based on the input desired velocity signal through the tracking differentiator of the active disturbance rejection controller; to estimate the state of the working robot when moving according to the initial motion trajectory through the extended observer of the active disturbance rejection controller, and to obtain the state estimation result; to nonlinearly combine the deviation between the dynamic characteristics of the working robot and the state estimation result to obtain the state control quantity; to perform disturbance compensation on the state control quantity of the working robot when moving according to the initial motion trajectory using preset state observation values; to tune the controller parameters of the active disturbance rejection controller based on the feedback error of the disturbance compensation result; and to compensate the initial motion trajectory based on the tuned controller parameters to obtain the target motion trajectory.

[0187] In some embodiments of this application, after the end effector performs operations on the target work object according to the target motion trajectory, the apparatus provided in this application may further include the following modules:

[0188] The closed-loop control module is used to acquire the actual motion trajectory of the end effector and feed the actual motion trajectory back to the visual servo control model. The end effector includes a disassembly arm and a clamping arm. The target work objects include bolts and guide lines. The motion trajectory includes the motion trajectory of the disassembly arm disassembling the bolts and the motion trajectory of the clamping arm clamping the guide lines.

[0189] In some embodiments of this application, the robot also has a lifting frame and a traveling chassis. The operation control module 805 is also used to control the lifting frame to descend and control the traveling chassis to travel to the next operation area after the disassembly arm and clamping arm have completed the bolt and guide line work in the current operation area, and to control the operation robot arm to perform work on the target operation object in the next operation area.

[0190] In this embodiment, during the movement of the robot's working arm, real-time image information is captured by a camera and image features are extracted. The desired image features and the extracted image features are input to a visual servo control model deployed on the robot. Based on the mapping relationship between the changes in image feature points indicated by the image Jacobian matrix in the visual servo control model and the movement speed of the end effector, a control quantity is generated based on the mapping relationship to obtain the robot arm control signal. At this time, the initial movement trajectory of the working robot arm can be planned based on the aforementioned control quantity. Then, the desired speed signal of the end effector can be input to the active disturbance rejection controller deployed on the robot. The aforementioned active disturbance rejection controller performs disturbance compensation on the initial movement trajectory to obtain the target movement trajectory, and then controls the end effector to perform operations on the target work object according to the target movement trajectory. By deploying a visual servo model and an active disturbance rejection controller on the robot, the visual servo control maps the motion speed of the end effector to the feature point speed on the image plane, reducing the error of the end effector's control signal on the image plane. Furthermore, the active disturbance rejection control compensates for the uncertainty of the visual servo model, achieving stable control against disturbances. This enables the robot to operate autonomously and resist interference, ensuring its operational stability in high-voltage electrical environments and contributing to the safe and stable operation of power equipment.

[0191] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0192] The robot provided in this application embodiment may also include a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described substation robot operation method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0193] This application also provides a computer-readable storage medium, see embodiments thereof. Figure 9 The computer-readable storage medium 900 provided stores a computer program 91. When the computer program 91 is executed by the processor, it implements the various processes of the above-described substation robot operation method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0194] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0195] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of the embodiments of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The division of modules in the embodiments of this application is merely a logical division; in actual applications, there may be other division methods. For example, multiple modules may be combined into or integrated into another system, or some features may be ignored or not performed. Additionally, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interface, and the indirect coupling or communication connection between modules may be electrical or other similar forms, none of which are limited in the embodiments of this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed among multiple circuit modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of this application.

[0196] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0197] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0198] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or modules, and may be electrical, mechanical, or other forms.

[0199] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0200] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0201] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0202] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0203] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0204] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 a process or multiple processes and / or boxes Figure 1 The functions specified in one or more boxes; these computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0205] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0206] Finally, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0207] The technical solutions provided in the embodiments of this application have been described in detail above. Specific examples have been used in the embodiments of this application to illustrate the principles and implementation methods of the embodiments of this application. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.

Claims

1. A method for robot operation in a substation, characterized in that, The robot has a robotic arm, which includes an end effector and a camera; the method includes: During the movement of the robotic arm, image features are extracted from the real-time image information captured by the camera. The desired image features and the extracted image features are input into the visual servo control model deployed on the robot, and the output is a robotic arm control signal. The image Jacobian matrix in the visual servo control model indicates the mapping relationship between the rate of change of image feature points and the motion speed of the end effector. The robotic arm control signal includes angle control values ​​for each joint of the end effector generated based on the mapping relationship. The rate of change of image feature points in the image Jacobian matrix is ​​determined based on the motion speed of the robot's work object feature points relative to the camera coordinate system, and the motion speed of the end effector in the image Jacobian matrix is ​​determined based on the motion speed of the camera in the world coordinate system. Specifically... The steps include: constructing a robotic arm kinematic error function based on the extracted image features and the desired image features using the visual servo control model; establishing an image Jacobian matrix based on the rate of change of image feature points of the work object and the motion speed of the end effector; performing error convergence on the robotic arm kinematic error function based on the mapping relationship between the motion speed of the work object feature points relative to the camera coordinate system and the motion speed of the camera in the world coordinate system to obtain the desired feature point speed of the work object; mapping the desired feature point speed to obtain the motion speed of the end effector based on the mapping relationship; and generating a robotic arm control signal based on the motion speed of the end effector. The initial motion trajectory of the robotic arm is obtained based on the angle control values ​​of each joint of the end effector; The desired velocity signal of the end effector is input to the active disturbance rejection controller deployed on the robot. The active disturbance rejection controller performs disturbance compensation on the preliminary motion trajectory to obtain the target motion trajectory. The end effector is controlled to perform operations on the target work object according to the target motion trajectory.

2. The method according to claim 1, characterized in that, The generation of robotic arm control signals based on the motion speed of the end effector includes: Obtain the Jacobian matrix of the robotic arm, and based on the pseudo-inverse matrix of the Jacobian matrix, map the motion velocity of the end effector to the joint angular velocity of each joint of the end effector; wherein, the Jacobian matrix of the robotic arm is used to indicate the mapping relationship between the joint angular velocity of each joint of the end effector and the motion velocity of the end effector. The joint angular velocities of each joint of the end effector are integrated to obtain the angle control values ​​of each joint, and the control signals of the robotic arm are generated based on the angle control values ​​of each joint.

3. The method according to claim 1, characterized in that, The step of inputting the desired velocity signal of the end effector to the active disturbance rejection controller deployed on the robot, and performing disturbance compensation on the preliminary motion trajectory through the active disturbance rejection controller to obtain the target motion trajectory includes: The tracking differentiator of the active disturbance rejection controller outputs the dynamic characteristics of the robotic arm based on the input desired speed signal; The state of the robotic arm running according to the initial motion trajectory is estimated by the extended observer of the active disturbance rejection controller, and the state estimation result is obtained. The deviation between the dynamic characteristics of the robotic arm and the state estimation result is nonlinearly combined to obtain the state control quantity; The state control quantity of the robotic arm is perturbed and compensated by using preset state observation values ​​when it moves according to the initial motion trajectory. The controller parameters of the active disturbance rejection controller are tuned based on the feedback error of the disturbance compensation result; The initial motion trajectory is compensated based on the tuned controller parameters to obtain the target motion trajectory.

4. The method according to claim 1 or 3, characterized in that, After controlling the end effector to perform operations on the target work object according to the target motion trajectory, the method further includes: The actual motion trajectory of the end effector is obtained and fed back to the visual servo control model. The end effector includes a disassembly arm and a clamping arm. The target work object includes a bolt and a guide line. The motion trajectory includes the motion trajectory of the disassembly arm disassembling the bolt and the motion trajectory of the clamping arm clamping the guide line.

5. The method according to claim 4, characterized in that, The robot also has a lifting frame and a traveling chassis, and the method further includes: After the assembly / disassembly arm and the clamping arm have completed their work on the bolts and guide lines in the current work area, the lifting frame is controlled to descend, and the traveling chassis is controlled to move to the next work area. The robotic arm is then controlled to perform work on the target object in the next work area.

6. A substation robot operation device for implementing the substation robot operation method as described in claim 1, characterized in that, Applied to the robot, the robot having a working robotic arm, the working robotic arm including an end effector and a camera, the device includes: The image feature extraction module is used to extract image features from the real-time image information captured by the camera during the movement of the robotic arm. The control signal generation module is used to input the desired image features and the extracted image features into the visual servo control model deployed on the robot, and output the robotic arm control signal; wherein, the image Jacobian matrix in the visual servo control model is used to indicate the mapping relationship between the rate of change of image feature points and the motion speed of the end effector, and the robotic arm control signal includes the angle control amount of each joint of the end effector generated based on the mapping relationship; The motion trajectory planning module is used to obtain the preliminary motion trajectory of the working robot arm based on the angle control values ​​of each joint of the end effector; The disturbance compensation module is used to input the desired velocity signal of the end effector to the active disturbance rejection controller deployed on the robot, and to perform disturbance compensation on the preliminary motion trajectory through the active disturbance rejection controller to obtain the target motion trajectory; The job control module is used to control the end effector to perform work on the target object according to the target motion trajectory.

7. A robot, characterized in that, include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the substation robot operation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the substation robot operation method as described in any one of claims 1 to 5.

Images