Substation robot operation method and device, robot and storage medium

Abstract

The embodiment of the invention relates to the field of robots, and provides a substation robot operation method and device, a robot and a storage medium, and the method comprises the steps: capturing real-time image information based on a camera in the movement process of an operation mechanical arm, and extracting the image features of the real-time image information; the expected image features and the extracted image features are input into a visual servo control model deployed on the robot, and a mechanical arm control signal is output and obtained; based on the angle control quantity of each joint of the end effector, obtaining a preliminary movement track of the operation mechanical arm; inputting an expected speed signal of the end effector into an active-disturbance-rejection controller deployed on the robot, and performing disturbance compensation on the initial movement track through the active-disturbance-rejection controller to obtain a target movement track; and controlling the end effector to operate the target operation object according to the target movement track. Through visual servo and active disturbance rejection control, the robot has autonomous operation and anti-interference capabilities, and the operation stability of the robot is ensured.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of robotics technology, and in particular, to a substation robot operating method, a substation robot operating device, a corresponding robot, and a corresponding computer-readable storage medium. Background Art

[0002] The working environment of substations is complex and the working space is limited. When conducting live maintenance work on power equipment with a voltage level of 220kV and above, it is usually restricted by various factors, such as strict requirements on climatic conditions, safety protection and insulating tools. In addition, due to the small space, there are greater safety risks in equipotential operations, and there are also high requirements for the skill level and proficiency of the operators.

[0003] Among the relevant technologies, live-working robots are mainly used in the field of power transmission and distribution, while the application of relevant technical equipment in the field of power transformation is still blank. Especially in the operation scenarios such as automatic disassembly and assembly of the drainage line and bolt tightening of the 220kV isolating circuit breaker, due to the lack of intelligent operation equipment, it still mainly relies on traditional manual maintenance methods. However, the traditional retrieval method is no longer suitable for the new needs of the current intelligent operation and maintenance of the power grid and safe live-working. Therefore, there is a need to develop intelligent and safe live-working equipment.

[0004] However, in the field of substation robots, most robots are used in non-contact operations such as inspections. Although some substation operation robots can perform contact operations, they cannot perform the work of disconnecting and connecting the guide wires. A small number of guide wire disconnection robots can achieve the corresponding functions, but they require human-machine collaborative operation, remote remote control or teleoperation, and have weak autonomous operation capabilities. At the same time, they cannot resist factors that affect safe operations such as high-altitude wind disturbances. Summary of the invention

[0005] The embodiments of the present application provide a substation robot operation method, device, robot and storage medium, which can enable the robot to have autonomous operation and anti-interference capabilities by utilizing visual servoing and self-anti-disturbance control, ensure the stability of operation in a high-voltage electrical working environment, and help ensure the safe and stable operation of power equipment.

[0006] In one aspect, an embodiment of the present application provides a substation robot operation method, wherein the robot has an operation robot arm, and the operation robot arm includes an end effector and a camera, and the method includes:

[0007] During the movement of the operating robot arm, based on the real-time image information captured by the camera, image features of the real-time image information are extracted;

[0008] The desired image features and the extracted image features are input into a visual servo control model deployed on the robot, and a manipulator control signal is output; wherein the image Jacobian matrix in the visual servo control model is used to indicate a mapping relationship between a rate of change of image feature points and a movement speed of the end effector, and the manipulator control signal includes an angle control amount of each joint of the end effector generated based on the mapping relationship;

[0009] Obtaining a preliminary motion trajectory of the operating robot arm based on the angle control amount of each joint of the end effector;

[0010] Inputting the desired velocity signal of the end effector into an active disturbance rejection controller deployed on the robot, and performing disturbance compensation on the preliminary motion trajectory through the active disturbance rejection controller to obtain a target motion trajectory;

[0011] The end effector is controlled to operate on the target operating object according to the target motion trajectory.

[0012] On the other hand, an embodiment of the present application provides a substation robot operation device, which is applied to the robot, the robot has an operation robot arm, the operation robot arm includes an end effector and a camera, and the device includes:

[0013] An image feature extraction module is used to extract image features of the real-time image information based on the real-time image information captured by the camera during the movement of the operating robot arm;

[0014] A control signal generation module, used for inputting the expected image features and the extracted image features into a visual servo control model deployed on the robot, and outputting a manipulator control signal; wherein the image Jacobian matrix in the visual servo control model is used to indicate the mapping relationship between the rate of change of the image feature points and the movement speed of the end effector, and the manipulator control signal includes the angle control amount of each joint of the end effector generated based on the mapping relationship;

[0015] A motion trajectory planning module, used to obtain a preliminary motion trajectory of the operating robot arm based on the angle control amount of each joint of the end effector;

[0016] A disturbance compensation module, used for inputting the desired velocity signal of the end effector into an active disturbance rejection controller deployed on the robot, and performing disturbance compensation on the preliminary motion trajectory through the active disturbance rejection controller to obtain a target motion trajectory;

[0017] The operation control module is used to control the end effector to operate on the target operation object according to the target motion trajectory.

[0018] On the other hand, an embodiment of the present application further provides a robot, comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, any one of the substation robot operation methods described is implemented.

[0019] On another aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, any one of the substation robot operation methods described above is implemented.

[0020] On the other hand, an embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the substation robot operation method described in the above aspects.

[0021] The substation robot operation method, device, robot and storage medium provided in the embodiments of the present application capture real-time image information and extract image features based on a camera during the movement of the robot's operating manipulator arm, input the expected image features and the extracted image features into a visual servo control model deployed on the robot, and generate angle control quantities of each joint of the end effector based on the mapping relationship between the changes in image feature points indicated by the image Jacobian matrix in the visual servo control model and the movement speed of the end effector to obtain a manipulator arm control signal. At this time, a preliminary motion trajectory of the operating manipulator arm can be obtained based on the angle control quantity planning of each joint of the end effector, and then the expected speed signal of the end effector can be input into an anti-disturbance rejection controller deployed on the robot, and the preliminary motion trajectory is compensated for by the anti-disturbance rejection controller deployed above to obtain a target motion trajectory, thereby controlling the end effector to operate on the target operating object according to the target motion trajectory. By deploying a visual servo model and an anti-disturbance control on the robot, the movement speed of the end effector is mapped to the feature point speed on the image plane using visual servo control, the control signal of the end effector is converted into error reduction on the image plane, and the uncertainty of the visual servo model is compensated for using anti-disturbance control to achieve stable anti-disturbance control. This enables the robot to have autonomous operation and anti-interference capabilities, and can ensure the stability of its operation in a high-voltage electrical working environment, which is beneficial to ensuring the safe and stable operation of power equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a working scene diagram of a substation robot provided in an embodiment of the present application;

[0023] Figure 2 is a structural schematic diagram of the operating mechanical arm provided in an embodiment of the present application;

[0024] Figure 3It is a flowchart of the steps of a substation robot operation method provided by an embodiment of the present application;

[0025] Figure 4 is a control flow diagram of a visual servoing system provided in an embodiment of the present application;

[0026] Figure 5 It is a structural diagram of a visual servoing and anti-disturbance control algorithm provided in an embodiment of the present application;

[0027] Figure 6 It is a schematic diagram of a flow chart of setting control parameters of an active disturbance rejection controller provided in an embodiment of the present application;

[0028] Figure 7 It is a simulation trajectory result diagram provided by an embodiment of the present application;

[0029] Figure 8 It is a structural block diagram of a substation robot operation device according to an embodiment of the present application;

[0030] Fig. 9 It is a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0032] Traditional substation guide line disconnecting robots are mostly human-machine collaborative, remotely controlled or teleoperated, and have weak autonomous operation capabilities. At the same time, in the substation's limited operating space, high altitude, complex and changeable environment, the trajectory of the end effector of the traditional substation guide line disconnecting robot is easily affected by environmental changes such as wind disturbance, and its trajectory may deviate, leading to serious consequences.

[0033] The embodiment of the present application is to deploy a visual servo model and an anti-disturbance control on the robot, map the motion speed of the end effector to the speed of the feature points on the image plane by using the visual servo control, convert the control signal of the end effector into the error reduction on the image plane, and use the anti-disturbance control to compensate for the uncertainty of the visual servo model, so as to achieve stable anti-disturbance control, so that the robot has autonomous operation and anti-disturbance capabilities, and can ensure the stability of its operation in a high-voltage electrical working environment, such as a substation where the operating space is limited, high-altitude, complex, and changeable, which is conducive to ensuring the safe and stable operation of power equipment. Among them, the image Jacobian matrix representing the mapping relationship between the robot image space and the task space and the uncertainty terms of the visual servo system can be lumped into the state equation of the same channel, and a nonlinear extended state observer can be introduced to estimate the lumped uncertainty of the visual servo system online, so as to design a visual servo and anti-disturbance control scheme for substation robots to improve the operation control accuracy and anti-disturbance capability of substation robots.

[0034] Specifically, refer to Figure 1 , showing an operation scenario diagram of the substation robot provided in an embodiment of the present application. In the operation scenario of the substation, the robot facing the substation is involved in the disconnection and connection of the substation guide wire. In the process of disconnecting and connecting the guide wire, it involves the removal of bolts and the clamping of the guide wire.

[0035] Among them, the robot facing the substation can be a substation drainage line disconnection robot, or it can be an equipotential maintenance robot, or it can be any robot that can disconnect and connect the substation drainage line, and the embodiments of the present application are not limited to this.

[0036] For example, Figure 1 As shown, the substation robot can be composed of three parts, namely, an operating robot arm 1, a lifting frame 2 and a traveling chassis 3, which are fixedly connected. The operating robot arm 1 can be an equipotential operating robot arm, and the traveling chassis 3 can be a ground potential traveling chassis. The substation robot composed of the aforementioned parts can operate on the target operating object 4, such as disassembly of bolts and clamping of guide wires, that is, the target operating object 4 can include bolts and guide wires.

[0037] Optionally, the operation process of the substation robot, that is, the process of disconnecting and connecting the guide wire of the substation robot, can be expressed as the traveling chassis 3 traveling to the lower part of the waiting operation area of ​​the target operation object 4, so that the substation robot first disconnects and connects the middle line of the three-phase wire, which is specifically expressed as the lifting frame 2 driving the operation robot arm 1 to be lifted to the preset distance from the target operation object 4, so that the equipotential operation robot arm disconnects and connects the middle line of the three-phase wire. For example, assuming that the target operation object 4 is a guide wire with a distance of 1.5m from the operation robot arm 1, the lifting frame 2 can drive the operation robot arm 1 to be lifted to 1m below the guide wire, and the embodiment of the present application does not limit this.

[0038] In some embodiments of the present application, in order to ensure the accurate position driven by the lifting frame 2 and the accurate position of the operating robot arm 1 during the disassembly operation, the driving of the operating robot arm can be achieved based on the visual positioning information.

[0039] Specifically, Figure 2 As shown, the operating robot arm 1 may include an end effector and a camera, and its end effector may include a disassembly arm 101 and a clamping arm 104; the camera used may be an RGB-D camera (Red-Green-Blue-Depth Camera, which refers to a visual sensor that can simultaneously capture color information and depth information), which may include a first RGB-D camera 102 located on the same robot arm as the disassembly arm 101, a second RGB-D camera 103 located on the same robot arm as the clamping arm 104, and a third RGB-D camera 105 located in the middle of the disassembly arm 101 and the clamping arm 104.

[0040] Optionally, the third RGB-D camera 105 in the middle position can fully capture the operating images of the robotic arms where the disassembly arm 101 and the clamping arm 104 are located, and the lifting frame 2 can specifically drive the operating robotic arm 1 to lift according to the visual positioning information of the third RGB-D camera 105; the first RGB-D camera 102 can capture the operating image of the disassembly arm 101, and the disassembly arm 101 can specifically operate according to the visual positioning information of the first RGB-D camera 102; the second RGB-D camera 103 can capture the operating image of the clamping arm 104, and the clamping arm 104 can specifically operate according to the visual positioning information of the second RGB-D camera 103.

[0041] It should be noted that for the specific operation process of the operating robot arm 1, please refer to the embodiment of the substation robot operation method provided in this application, and the embodiment of this application will not be described in detail here.

[0042] In some embodiments of the present application, after the disassembly and assembly arm 101 and the clamping arm 104 complete the bolt and guide wire operations in the current operating area, for example, the clamping arm 104 clamps the guide wire and the disassembly and assembly arm 101 performs the bolt removal operation, and the guide wire is connected back to the original circuit after visual inspection and no abnormalities are found, after the three-phase wires in the current area are operated, the lifting frame 2 can be controlled to descend, and the traveling chassis 3 can be controlled to move to the next operating area, for example, to below the middle phase of the next area to be operated for disconnecting the guide wire, so as to control the operating robot arm 1 to operate on the target operating object 4 in the next operating area, that is, repeat the above-mentioned operating process until the disconnection of the guide wires in all operating areas of the substation is completed.

[0043] In the embodiments of the present application, an equipotential maintenance robot and an operation plan for a substation robot are provided for the live disassembly and assembly of the 220KV isolating circuit breaker drain line, which is dedicated to solving the technical difficulties of on-site live maintenance and eliminating the pain points of concern to power users.

[0044] Reference Figure 3 , shows a flowchart of a substation robot operation method provided by an embodiment of the present application, the substation robot operation method is as follows Figure 2 The robot shown is executed and applied to Figure 1 The operation scenario shown may specifically include the following steps:

[0045] Step S301, during the movement of the operating robot arm, real-time image information is captured based on a camera, and image features of the real-time image information are extracted.

[0046] The movement of the working robot arm depends on the visual positioning information. The visual positioning information mainly refers to the real-time image information captured by the camera of the working robot arm during its movement. The real-time image information may include image features of the object to be identified. In the working scene of the embodiment of the present application, the extracted image features usually refer to the feature information of the feature point of the target working object.

[0047] In some embodiments of the present application, the above-mentioned feature points can be tracked to achieve real-time adjustment of the movement of the robotic arm to ensure that the feature points within the camera's field of view can be guided to the predetermined target position, thereby enabling the robotic arm to perform disassembly and connection operations on the target work object.

[0048] Step S302 , inputting the expected image features and the extracted image features into a visual servo control model deployed on the robot, and outputting a robot arm control signal.

[0049] Optionally, in order to ensure that the feature points within the camera's field of view can be guided to the predetermined target position, an image-based visual servoing system (IBVS) can be used to accurately identify and track the feature points in the image, so as to use the real-time image information captured by the camera to manipulate the working robot arm. It should be noted that the image-based visual servoing system is deployed in the robot provided in the embodiment of the present application, which is conducive to ensuring the autonomous operation of the robot.

[0050] Visual servo control is a control method based on visual feedback, which can generally be applied to the control of mobile robots and underwater robots. In the embodiment of the present application, it is applied to the control of robots facing substations.

[0051] In a visual servo system, the required control action can be calculated by comparing the difference between the feature points detected in the image and the pre-set target features. In some embodiments of the present application, the image features extracted from the real-time image information are the feature points detected in the image, and the expected image features are the pre-set target features. The expected image features and the extracted image features can be input into a visual servo control model deployed on the robot, and the robot arm control signal is obtained through the output of the visual servo control model.

[0052] Specifically, the difference between the expected image features and the extracted image features is mainly quantified through the image Jacobian matrix in the visual servo control model. The matrix can describe the geometric relationship between the change of image feature points and the movement of the end effector of the robot arm. Specifically, it can be used to indicate the mapping relationship between the change rate of image feature points and the movement speed of the end effector. Based on the image Jacobian matrix that can encode information about the movement of image pixels, the movement in the feature pixels is associated with the movement of the end effector of the working robot arm, and then the control signal for the robot working robot arm is derived.

[0053] Among them, the rate of change of image feature points refers to the rate of change of image features captured by the camera. The movement speed of the end effector is actually the movement speed of the camera in three-dimensional space. The mapping relationship quantified by the image Jacobian matrix can associate the movement in the feature pixels with the operation of the end effector, and map the movement speed of the end effector to the feature point speed on the image plane, so as to derive the control signal of the operating robot arm with the goal of reducing image deviation.

[0054] In some embodiments of the present application, visual servo control can be expressed as defining a robot kinematic error function based on the difference between the robot's current posture and the desired posture obtained from the visual system, continuously performing error convergence in the process of the robot moving to the target work object, and converting the control command of the end effector into error reduction on the image plane. It should be noted that the embodiments of the present application do not limit the specific process of deriving the control signal.

[0055] Step S303 , obtaining a preliminary motion trajectory of the working robot arm based on the angle control amount of each joint of the end effector.

[0056] The robot arm control signal may include a control amount generated based on a mapping relationship, and the control amount is mainly generated with the goal of reducing image deviation.

[0057] Optionally, the generated control amount may refer to the angle control amount of each joint of the end effector.

[0058] In some embodiments of the present application, in order to realize the manipulation of the operating robot arm, the preliminary motion trajectory of the operating robot arm can be preliminarily planned based on the control quantity generated with the goal of reducing image deviation, and the generated preliminary motion trajectory is the motion trajectory under ideal conditions.

[0059] It should be noted that in the operation scenario of the substation, the operating robot arm performs the disconnection and connection of the guide wire, which is specifically implemented by the end effector. The end effector includes a disassembly arm and a clamping arm. The target operation objects include bolts and guide wires. That is, the disconnection and connection operation of the guide wire involves the disassembly arm disassembling the bolts and the clamping arm clamping the guide wire. The preliminary planned preliminary motion trajectory may include the preliminary motion trajectory of the disassembly arm disassembling the bolts and the preliminary motion trajectory of the clamping arm clamping the guide wire.

[0060] Step S304: input the desired velocity signal of the end effector to an active disturbance rejection controller deployed on the robot, and perform disturbance compensation on the preliminary motion trajectory through the active disturbance rejection controller to obtain a target motion trajectory.

[0061] In some embodiments of the present application, the motion trajectory of the operating robot arm will usually be affected by internal and external interferences. The internal parameter disturbance may refer to the influence of the control parameters inside the operating robot arm; the external disturbance may refer to the interference caused by the operating robot arm's own characteristics or components during operation, such as joint friction / damping, parameter uncertainty, actuator dynamics, vibration during operation, etc. At this time, the desired speed signal of the end effector can be input into the anti-disturbance rejection controller deployed on the robot, and the preliminary motion trajectory can be compensated for disturbances through the anti-disturbance rejection controller.

[0062] Among them, the self-disturbance rejection controller is deployed on the robot, and the self-disturbance rejection control algorithm is beneficial to improving the robot's anti-interference ability, that is, it can improve the robot's robustness.

[0063] Specifically, the motion trajectory is usually directly affected by parameter setting. In the embodiment of the present application, the control parameters of the anti-disturbance controller can be adjusted so that the initial motion trajectory is affected by the adjusted control parameters, thereby achieving disturbance compensation for the initial motion trajectory and obtaining the target motion trajectory after parameter adjustment.

[0064] It should be noted that in the operation scenario of the substation, the operating robot arm performs the disconnection and connection of the guide wire, which is specifically implemented by the end effector. The end effector includes a disassembly arm and a clamping arm. The target operation objects include bolts and guide wires. That is, the disconnection and connection operation of the guide wire involves the disassembly arm disassembling the bolts and the clamping arm clamping the guide wire. The target motion trajectory after parameter setting may include the target motion trajectory of the disassembly arm disassembling the bolts and the target motion trajectory of the clamping arm clamping the guide wire.

[0065] Active Disturbance Rejection Control (ADRC) is proposed by learning from the advantages of PID control (Proportional-Integral-Derivative Control) that it does not rely on models. Its core idea is to aggregate the internal uncertainty and external disturbance of the visual servo system into a generalized total disturbance, and introduce a nonlinear extended state observer (ESO) to estimate it; at the same time, a tracking differentiator (TD) is added to the external signal input to arrange a transition process for the input signal, so that the ADRC has a fast response process and good anti-interference ability. The specific implementation process of the ADRC is not limited in the embodiments of the present application.

[0066] Step S305 , controlling the end effector to operate on the target operating object according to the target motion trajectory.

[0067] The target motion trajectory can be understood as the motion trajectory after overcoming internal and external interferences. The operating robot arm that performs disassembly and connection operations according to the aforementioned motion trajectory has anti-interference capabilities. In some embodiments of the present application, the end effector can be controlled to operate the target operation object according to the target motion trajectory, so as to achieve the purpose of ensuring the stability of the robot operation in a high-voltage working environment.

[0068] In some embodiments of the present application, after controlling the end effector to operate on the target work object according to the target motion trajectory, the actual motion trajectory of the end effector can also be obtained, and the actual motion trajectory can be fed back to the visual servo control model to form a closed-loop control to correct errors in real time, improve motion accuracy and enhance system robustness.

[0069] In a preferred embodiment of the present application, after the disassembly and assembly arm and the clamping arm complete the operation on the bolts and guide wires in the current operating area, for example, the clamping arm clamps the guide wires and the disassembly and assembly arm performs the operation of removing the bolts, and the guide wires are connected back to the original circuit after visual inspection and no abnormalities are found, after the operation on the three-phase wires in the current area is completed, the lifting frame can be controlled to descend, and the traveling chassis can be controlled to move to the next operating area, for example, to below the middle phase of the next area to be operated for disconnecting the guide wires, so as to control the operating robot arm to operate on the target operating object in the next operating area, that is, repeat the above-mentioned operating process until the disconnection of the guide wires in all operating areas of the substation is completed, and the embodiment of the present application is not limited to this.

[0070] In an embodiment of the present application, during the movement of the robot's operating manipulator arm, real-time image information is captured based on a camera and image features are extracted, and the desired image features and the extracted image features are input into a visual servo control model deployed on the robot. Based on the mapping relationship between the changes in image feature points indicated by the image Jacobian matrix in the visual servo control model and the movement speed of the end effector, a control quantity is generated based on the mapping relationship to obtain a manipulator arm control signal. At this time, a preliminary motion trajectory of the operating manipulator arm can be obtained based on the aforementioned control quantity planning, and then the desired speed signal of the end effector can be input into an anti-disturbance rejection controller deployed on the robot. The preliminary motion trajectory is compensated for by the aforementioned deployed anti-disturbance rejection controller to obtain a target motion trajectory, and then the end effector is controlled to operate on the target working object according to the target motion trajectory. By deploying a visual servo model and an anti-disturbance control on the robot, the movement speed of the end effector is mapped to the feature point speed on the image plane using visual servo control, the control signal of the end effector is converted into error reduction on the image plane, and the uncertainty of the visual servo model is compensated for using anti-disturbance control to achieve stable anti-disturbance control. This enables the robot to have autonomous operation and anti-interference capabilities, and can ensure the stability of its operation in a high-voltage electrical working environment, which is beneficial to ensuring the safe and stable operation of power equipment.

[0071] In some embodiments of the present application, the control process of visual servoing may specifically include the following steps:

[0072] Sub-step S21, constructing a robot arm kinematic error function based on the extracted image features and expected image features through a visual servo control model.

[0073] In a visual servo system, the required control action can be calculated by comparing the difference between the feature points detected in the image and the pre-set target features. The visual servo control process involves image processing technology, namely feature extraction and feature matching. Once the feature points are identified, the visual servo system can calculate the deviation between the feature points and the target position. The aforementioned deviation is used to adjust the robot movement to reduce errors and achieve precise positioning, that is, the feature points can be used as feedback signals to adjust the servo control loop.

[0074] In some embodiments of the present application, Figure 4 As shown, the image-based visual controller generates a robot arm control signal for the joint angle controller, and responds to the robot arm control signal to manipulate the robot arm to perform corresponding movements. During the movement of the operating robot arm, images can be collected by the camera at the end of the robot arm, and then image features can be extracted from the camera image at the end of the robot arm. The extracted image features are the feature points detected in the image, and the expected image features are the pre-set target features. The expected image features and the extracted image features are input into the visual servo control model deployed on the robot, and the robot arm control signal is obtained through the output of the visual servo control model, that is, the image-based visual controller generates a robot arm control signal for the joint angle controller, and then the current movement of the robot arm is manipulated according to the currently output robot arm control signal, and the above-mentioned end camera image collection, image feature extraction, expected image feature input and image-based visual control operation process are repeated to realize the closed loop of the servo control loop.

[0075] Specifically, visual servo control can be expressed as defining the robot kinematic error function based on the difference between the robot's current posture and the expected posture obtained from the visual system, and continuously performing error convergence in the process of the robot moving to the target work object to convert the control command of the end effector into error reduction on the image plane.

[0076] Optionally, features can be extracted from the visual information collected by the camera through a feature extraction network, that is, the bolts and guide lines of the working targets can be detected, and the features extracted from the visual information and the expected image features can be input into the visual servoing (IBVS) control model together. The visual servoing control model can be used to construct a robot arm kinematic error function based on the extracted image features and the expected image features, so as to perform error convergence on the robot arm kinematic error function constructed above.

[0077] The extracted image features are used to indicate the actual visual feature state, and the expected image features are used to indicate the expected visual feature state.

[0078] Exemplarily, the robot arm kinematic error function may refer to the difference between the current posture of the robot arm and the desired posture obtained from the visual system. The error converges continuously during the movement of the robot arm to the bolt and the guide line. The goal of visual servo control is to minimize the error defined by the following formula.

[0079] The constructed robot arm kinematic error function e(t) can be expressed as follows:

[0080]

[0081] In the formula, Can refer to the set of regions of interest in an image, Can refer to a collection of intrinsic and extrinsic parameters of a camera, including but not limited to focal length, pixel size, camera pose, etc.; function and function Can be defined as the current state and the desired state, function and function The specific formula can be shown as follows:

[0082]

[0083] In the formula, It represents the actual visual features, that is, the coordinates of the bolts and guide lines in the image, that is, the two-dimensional image coordinates , is the focal length of the camera; represents the desired visual feature state, and (m*, n*) is the corresponding 2D image coordinate. Among them, it is assumed that the spatial point of the bolt to be disassembled in the world coordinate system , whose coordinates relative to the camera coordinate system are At this time, the coordinates of the working object relative to the camera coordinate system can be converted into two-dimensional image coordinates mapped on the image plane based on the focal length of the camera.

[0084] Sub-step S22, establishing an image Jacobian matrix based on the image feature point change rate of the robot's work object feature points and the movement speed of the end effector.

[0085] The rate of change of the image feature points can be determined based on the movement speed of the feature points of the work object relative to the camera coordinate system, and the movement speed of the end effector can be determined based on the movement speed of the camera in the world coordinate system.

[0086] In order to establish the mapping relationship between the camera's three-dimensional space movement speed and the rate of change of the image point feature at a certain point in the world coordinates, we can use Indicates that a camera moves at a speed in a rigid body motion in the world coordinate system. Suppose a space point of the bolt to be disassembled in the world coordinate system is selected. , whose coordinates relative to the camera coordinate system are , then the speed of point P relative to the camera coordinate system can be expressed as follows:

[0087]

[0088] Right now:

[0089] In the formula, is a linear velocity vector, which can be used to represent the translation speed of the camera in the world coordinate system, v x 、v y and v z It can represent the translation speed of the camera in the x, y, and z axis directions respectively; is the angular velocity vector, which can be used to represent the rotation rate of the camera around each axis, ω x ,ω y ,ω z It can represent the rotation rate of the camera around the x, y, and z axes respectively; is the spatial point of the bolt to be removed in the world coordinate system, is the spatial point P in the camera coordinate system, They are respectively the three-dimensional coordinates of the spatial point P after moving at the moving speed based on the camera coordinate system.

[0090] The key to designing a visual servo controller is image feature selection and kinematic modeling of image features. The kinematic model of image features refers to the mapping relationship between the rate of change of image features captured by the camera and the speed of the camera in three-dimensional space, which can be expressed as follows:

[0091]

[0092] In the formula, It can refer to the rate of change of a characteristic; is the three-dimensional space motion speed in the current camera coordinate system, that is, the motion speed of the robot end effector relative to the polar coordinate system); is the focal length of the camera; are the corresponding 2D image coordinates; is the image Jacobian matrix.

[0093] The visual servo control of the visual servo control system relies on the image Jacobian matrix, which can be used to represent the relationship between feature changes and the movement speed of the end effector of the robot arm. In the embodiment of the present application, it is implemented based on the image Jacobian matrix. The image Jacobian matrix can describe the relationship between the changes of feature points in the image and the camera movement, and link the camera speed with the feature speed in the normalized image coordinates, and specifically explain the mapping relationship between the point feature transformation rate and the camera movement speed in three-dimensional space.

[0094] In an embodiment of the present application, for a guide line target that is constantly moving due to wind disturbance at high altitude, the image Jacobian matrix can help predict and compensate for the movement of the guide line and achieve dynamic tracking.

[0095] For example, in order to ensure the matrix dimension requirement, so that the matrix dimension satisfies d=6, the number of image features should be more than 6. In order to control the manipulator of an equipotential working robot with 6 degrees of freedom, at least 3 feature points are required. When the feature vector is composed of 3 feature points, the image Jacobian matrix can be shown as follows:

[0096]

[0097] Where, L s is the constructed image Jacobian matrix, L p1 , L p2 , L p3 The image Jacobian that can encode information about the motion of image pixels can be used to associate the motion in the feature pixels with the motion of the end effector, so as to facilitate the subsequent generation of the robot control signal for the working robot arm.

[0098] Sub-step S23, performing error convergence based on the image Jacobian matrix and the robot arm kinematic error function to obtain the motion speed of the end effector.

[0099] In visual servo control, the velocity of the camera end effector can be mapped to the velocity of the feature points on the image plane based on the image Jacobian matrix. This mapping is achieved by considering the kinematic model of the camera and the geometric relationship between the image feature points. That is, through the image Jacobian matrix, the control command of the end effector can be converted into error reduction on the image plane, thereby achieving accurate tracking of the target.

[0100] Specifically, based on the mapping relationship between the movement speed of the feature point of the working object relative to the camera coordinate system and the movement speed of the camera in the world coordinate system, the error convergence of the kinematic error function of the robot arm can be performed. Specifically, the convergence is performed with the goal of minimizing the error value, thereby reducing the error between the actual visual feature state and the expected visual feature state, and converting the control command of the end effector into the error reduction on the image plane to obtain the expected feature point speed of the feature point of the working object; since the mapping relationship maps the movement speed of the end effector to the feature point speed on the image plane, the expected feature point speed can be obtained in combination with the aforementioned mapping relationship during the error convergence process, and then the movement speed of the end effector can be obtained. Specifically, based on the mapping relationship, the expected feature point speed is mapped to obtain the movement speed of the end effector.

[0101] Exemplarily, the error convergence of the robot arm kinematic error function can be expressed as follows:

[0102]

[0103] In the formula, e is the error, is the scaling factor, is the pseudo-inverse matrix of the required image Jacobian matrix, and the Jacobian matrix is ​​based on the characteristics The selection is calculated based on is the movement speed of the end effector.

[0104] Sub-step S24, generating a robot arm control signal based on the movement speed of the end effector;

[0105] The movement speed of the end effector can be used to determine the control amount of the end effector; the control amount may refer to the angle control amount of each joint of the end effector. At this time, a robotic arm control signal can be generated based on the movement speed of the end effector. The robotic arm control signal is used to manipulate the angles of each joint of the end effector, that is, a robotic arm control signal for a joint angle controller can be generated based on an image-based visual controller.

[0106] Optionally, the camera's movement speed can be used as the end effector's movement speed. As the control quantity, the angle control quantity of each joint is then calculated using the kinematics of the robot arm to generate the robot arm control signal.

[0107] In some embodiments of the present application, the movement speed of the end effector can be mapped to the joint angular velocity of each joint of the end effector based on the pseudo-inverse matrix of the Jacobian matrix of the robot arm, and then the joint angular velocity of each joint of the end effector is integrated to obtain the angle control amount of each joint, and then the robot arm control signal is generated based on the angle control amount of each joint.

[0108] The Jacobian matrix of the robot arm can be used to indicate the mapping relationship between the joint angular velocity of each joint of the end effector and the movement speed of the end effector.

[0109] For example, the calculation formula for the angle control amount of each joint can be as follows:

[0110]

[0111] In the formula, is the joint angular velocity, Refers to the movement speed of the end effector, refers to the pseudo-inverse matrix of the Jacobian matrix of the robot arm, Refers to the angle control amount of each joint. Among them, the Jacobian matrix of the robot arm is:

[0112]

[0113] In the formula, is the unit vector of the i-th joint rotation axis, is the origin position of the i-th joint coordinate system, is the position of the end effector, where, assuming that the end effector has n joints, the i-th joint can be one of the n joints, that is, i=1,2,…,n (n is a positive integer).

[0114] In an embodiment of the present application, by deploying a visual servo model on the robot, visual servo control is used to map the movement speed of the end effector to the feature point speed on the image plane, and the control signal of the end effector is converted into error reduction on the image plane, so that the robot for the substation has the ability to operate autonomously.

[0115] In some embodiments of the present application, in order to realize the manipulation of the operating robot arm, the preliminary motion trajectory of the operating robot arm can be preliminarily planned based on the control quantity generated with the goal of reducing image deviation, and the generated preliminary motion trajectory is the motion trajectory under ideal conditions.

[0116] Specifically, the position of the positioning bolt and the guide wire clamping position can be detected based on the visual information of multi-sensor fusion, and then the rotation angle of each joint, that is, the angle control amount of each joint, can be obtained through the inverse kinematics of the robot arm based on the obtained three-dimensional coordinate information; the rotation angles of each joint are combined together, that is, when they rotate simultaneously, they can form the motion trajectory of the entire robot arm. In practical applications, the measured positioning bolt position and guide wire clamping position can be understood as the real-time image information captured by the camera during the movement of the operating robot arm, and each joint rotation angle can be obtained based on the angle control amount of each joint generated by the above mapping relationship, that is, the target trajectory in an ideal situation can be the preliminary motion trajectory obtained based on the angle control amount of each joint.

[0117] It should be noted that in the operation scenario of the substation, the operating robot arm performs the disconnection and connection of the guide wire, which is specifically implemented by the end effector. The end effector includes a disassembly arm and a clamping arm. The target operation objects include bolts and guide wires. That is, the disconnection and connection operation of the guide wire involves the disassembly arm disassembling the bolts and the clamping arm clamping the guide wire. The preliminary planned preliminary motion trajectory may include the preliminary motion trajectory of the disassembly arm disassembling the bolts and the preliminary motion trajectory of the clamping arm clamping the guide wire.

[0118] Usually, the motion trajectory of the operating robot arm will be affected by internal and external disturbances. Internal parameter disturbances can refer to the influence of control parameters inside the operating robot arm; external disturbances can refer to the interference caused by the operating robot arm’s own characteristics or components during operation, such as joint friction / damping, parameter uncertainty, actuator dynamics, vibration during operation, etc. At this time, the desired speed signal of the end effector can be input into the anti-disturbance rejection controller deployed on the robot, and the motion trajectory obtained by the above preliminary planning can be compensated for disturbances through the anti-disturbance rejection controller.

[0119] Specifically, the preliminary motion trajectory of the disassembly arm for removing the bolt and the preliminary motion trajectory of the clamping arm for clamping the guide wire are input into the IBVS control model, so that the IBVS control model is combined with the anti-disturbance controller to design a visual servoing and anti-disturbance control scheme for the substation robot, so as to improve the operation control accuracy and anti-interference ability of the substation robot.

[0120] The process of the anti-disturbance control combined with visual servo control may specifically include the following steps:

[0121] Sub-step S41: outputting the dynamic characteristics of the working robot arm based on the input desired speed signal through the tracking differentiator of the active disturbance rejection controller.

[0122] Optionally, the IBVS control model can pass the desired speed signal to the ADRC, which aggregates the image Jacobian matrix representing the mapping relationship between the robot image space and the task space and the uncertainty items of the visual servo system into the state equation of the same channel.

[0123] Reference Figure 5 , showing a structural schematic diagram of a visual servoing and anti-disturbance control algorithm provided in an embodiment of the present application.

[0124] In some embodiments of the present application, the input expected speed signal may refer to the operating speed of two robotic arms including a disassembly arm and a clamping arm, which can be determined based on the expected trajectory generated by the tracking differentiator TD in the active disturbance rejection controller ADRC. Specifically, the expected speed signal can be transmitted to the tracking differentiator TD, and the tracking differentiator TD can output the dynamic characteristics caused by internal parameter disturbances and external disturbances, so as to transmit the dynamic characteristics to the extended observer ESO in the active disturbance rejection controller ADRC, so that the extended observer ESO performs state estimation.

[0125] The tracking differentiator TD can arrange the transition process for a given step signal to resolve the contradiction between the rapidity and overshoot of the system.

[0126] Exemplarily, the tracking differentiator TD can extract a differential signal from an input signal containing random noise and suppress the noise. The extracted differential signal can also be used as a low-pass filter to effectively suppress high-frequency components and output its tracking signal. , first-order differential signal And the second-order differential signal Among them, the basic principle of the TD tracking differentiator is to suppress noise through a pure differential link and then connect a first-order low-pass filter in series. This structure allows the system to reduce the impact of noise while extracting the differential signal.

[0127] Exemplarily, the formula for the tracking differentiator may be as follows:

[0128]

[0129] In the formula, It can refer to the integration step size; Can refer to the filter factor; It can refer to The input signal at the time; It can refer to the speed factor, which is mainly used to determine the parameter of tracking speed; z(k) can refer to the state variable inside the tracking differentiator, which is mainly used to smooth the input signal and extract its differential signal. 1 (k) can refer to the smoothed tracking value of the input signal, which is used to represent the position of the joint; z 2 (k) may refer to the first-order differential signal, which is used to represent the velocity of the joint; z 3 (k) can refer to the second-order differential signal, which is used to represent the acceleration of the joint; It is a comprehensive function for fast optimal control.

[0130] In some embodiments of the present application, the dynamic characteristics of the output of the differentiator TD are tracked, which can be specifically manifested as obtaining the generalized disturbance term of the robot, including model uncertainty and internal and external disturbances.

[0131] For example, it is assumed that the dynamic model design of the mechanical arm of the equipotential working robot is as follows:

[0132]

[0133] Where τ can refer to the input torque, which is usually an n×1 vector, where n is the number of joints of the robot arm; It can refer to the generalized disturbance term of the robot, including model uncertainty and internal and external disturbances; , and Can represent the position, velocity and acceleration of the joint respectively; It can refer to the inertia matrix of the robot arm; represents the Coriolis force and the centrifugal matrix; represents the gravity matrix.

[0134] Hypothesis Definition , , its state space expression can be shown as follows:

[0135]

[0136] In the formula, x 1 It can refer to the angle state of the robot arm joint, that is, the position of the robot arm joint; x 2 It can refer to the motion state of the robot joint, that is, the speed of the robot joint; y refers to its measured output, which can be mainly used for feedback control.

[0137] Optionally, the robot has complex dynamic characteristics caused by internal parameter disturbances and external disturbances (i.e., joint friction, actuator dynamics, vibration during operation, etc.). Specifically, the tracking signal and the first-order differential signal of the desired trajectory output generated by the differential tracker TD can be compared with the actual state x of the robot joint angle. 1 and the actual state x of the robot joint angular velocity 2 , driving the ADRC controller to generate the anti-disturbance control quantity. Among them, the tracking signal of the desired trajectory output generated by TD can be used to indicate the desired position, the first-order differential signal generated by TD can be used to indicate the desired speed, and the actual state of the robot arm joint angle x 1 Can be used to indicate the actual position, the actual state of the angular velocity of the robot joint x 2 It represents the actual speed. At this time, the expected position can be compared with the actual position, and the expected speed can be compared with the actual speed. Based on the comparison results, the ADRC controller is driven to generate the anti-disturbance control amount.

[0138] Sub-step S42, estimating the state of the working robot arm when it moves according to the preliminary motion trajectory through the extended observer of the ADRC, and obtaining a state estimation result.

[0139] Optionally, a nonlinear extended state observer can be introduced to perform online estimation of the lumped uncertainty of the visual servo system, and a visual servo and anti-disturbance control scheme for the substation robot can be designed to improve the operation control accuracy and anti-disturbance capability of the substation robot.

[0140] Specifically, the extended observer ESO can be designed to estimate the joint angle, angular velocity, and angular acceleration of the equipotential operation robot and estimate the unknown uncertain internal and external disturbances. The extended observer ESO can provide an accurate estimate of the internal state of the system, and can estimate the internal and external disturbances in the system in real time even when there is uncertainty in the system model, which is crucial to maintaining system performance in an environment with unknown disturbances.

[0141] In some embodiments of the present application, the state of the operating robot arm when it moves according to the preliminary motion trajectory can be estimated by the extended observer ESO of the self-disturbance rejection controller to obtain a state estimation result, so as to expand the disturbance effect that can affect the controlled output into a new state variable, while ensuring that the expanded new state variable is within the control rate.

[0142] For example, the formula of the extended observer ESO can be shown as follows:

[0143]

[0144] Where y and u are inputs; , , , Track y, y', y'' and the expanded part separately; , , , is the gain coefficient; , , is a nonlinear factor; is the interval length of the linear segment; is the compensation factor; It is a nonlinear function, and its expression can be shown as follows:

[0145]

[0146] In the formula, is the threshold value, which can be used to distinguish whether the error e is in the linear region or the nonlinear region; the nonlinear index , which can be used to control the slope of the nonlinear region.

[0147] Sub-step S43, performing nonlinear combination on the deviation between the dynamic characteristics of the working robot arm and the state estimation result to obtain the state control quantity.

[0148] Nonlinear feedback control can dynamically adjust the control gain according to the system error, achieving the effect of large gain for small errors and small gain for large errors, thereby improving the accuracy and response speed of the control system.

[0149] Specifically, the ADRC controller can more effectively compensate for disturbances in the system, including unmodeled dynamics and external disturbances, through nonlinear feedback. This feedback mechanism can enable the system to maintain stability and performance in the face of uncertainty and disturbances, help improve the robustness of the system, maintain the performance of the control system even when parameters change or external conditions change, and help reduce overshoot, especially in the scenario of substation conductor disconnection robot operation with fast response and high precision requirements.

[0150] Sub-step S44, using preset state observation values ​​to perform disturbance compensation on the state control quantity of the working robot arm when it moves according to the preliminary motion trajectory.

[0151] Exemplarily, the process of nonlinear combination and disturbance compensation can be implemented by the following formula:

[0152]

[0153] As shown in the above formula, the output signal of the tracking differentiator TD can be , , State Estimation with Extended State Observer (ESO) , , The deviations between them are nonlinearly combined to obtain the control quantity ; Then, through the observed observations Realize disturbance compensation and obtain the total output of the active disturbance rejection controller .

[0154] Sub-step S45, adjusting controller parameters of the active disturbance rejection controller based on the feedback error of the disturbance compensation result.

[0155] There are many parameters in the ADRC algorithm model, and the tuning of the controller parameters can directly affect the performance of the controller and the stability of the entire system. At this time, the controller parameters can be tuned to help optimize the performance of the disturbance observer and compensator, thereby more accurately estimating and compensating for internal and external disturbances in the system.

[0156] In practical applications, ADRC parameter tuning usually needs to take into account multiple indicators such as the system's dynamic characteristics, expected response time, overshoot, steady-state error, and sensitivity to disturbances.

[0157] Exemplarily, a genetic algorithm may be used to tune the controller parameters, and its objective function may be as follows:

[0158]

[0159] In the formula, is the feedback error, To adjust the time, is the overshoot, , , is the weight.

[0160] The objective function is mainly used to quantify the performance indicators of the control system into comparable values, and guide the algorithm to search for a better solution. Specifically, it can be expressed as the feedback error, adjustment time, and overshoot obtained by combining different parameters into this objective function, so that the objective function The smaller the better. Then, at this time The parameter combination is the optimized controller parameters.

[0161] like Figure 6 As shown, the feedback error can be Substitute the objective function, and then use the genetic algorithm to optimize the ADRC parameters when the expected speed signal v(t) is input, so as to realize the controller parameters of the active disturbance rejection controller, so as to facilitate the subsequent use of the tuned controller parameters to compensate the initial motion trajectory and obtain the target motion trajectory y(t) for the substation drainage line disconnection robot.

[0162] It should be noted that once the objective function is defined, the parameter optimization process of the genetic algorithm is started. In each generation, the newly generated population will be evaluated and tested in the ADRC environment to identify the individuals with the best performance. This process will continue until the preset number of generations is reached, and the optimal parameter set will be finally selected from all generations. This is not limited in the embodiments of the present application.

[0163] Sub-step S46, compensating the preliminary motion trajectory based on the adjusted controller parameters to obtain the target motion trajectory.

[0164] The preliminary motion trajectory is the motion trajectory under ideal conditions, while the actual operation trajectory is usually directly affected by the ADRC parameter setting.

[0165] In some embodiments of the present application, the active disturbance rejection controller ADRC can smooth the step-type planning trajectory into an actual trackable reference signal through the tracking differentiator TD, and the extended state observer ESO can estimate disturbances such as wind disturbances and model errors in real time to dynamically adjust the control quantity, compensate for disturbances and correct trajectory deviations, and obtain the target motion trajectory.

[0166] The target motion trajectory can be understood as the motion trajectory after overcoming internal and external interferences. The operating robot arm that performs disassembly and connection operations according to the aforementioned motion trajectory has anti-interference capabilities.

[0167] In practical applications, the end effector can be controlled to operate on the target object according to the target motion trajectory, so as to ensure the stability of the robot operation in a high-voltage working environment.

[0168] In some embodiments of the present application, after controlling the end effector to operate on the target work object according to the target motion trajectory, the actual motion trajectory of the end effector can also be obtained, and the actual motion trajectory can be fed back to the visual servo control model to form a closed-loop control to correct errors in real time, improve motion accuracy and enhance system robustness.

[0169] In a preferred embodiment of the present application, after the disassembly and assembly arm and the clamping arm complete the operation on the bolts and guide wires in the current operating area, for example, the clamping arm clamps the guide wires and the disassembly and assembly arm performs the operation of removing the bolts, and the guide wires are connected back to the original circuit after visual inspection and no abnormalities are found, after the operation on the three-phase wires in the current area is completed, the lifting frame can be controlled to descend, and the traveling chassis can be controlled to move to the next operating area, for example, to below the middle phase of the next area to be operated for disconnecting the guide wires, so as to control the operating robot arm to operate on the target operating object in the next operating area, that is, repeat the above-mentioned operating process until the disconnection of the guide wires in all operating areas of the substation is completed, and the embodiment of the present application is not limited to this.

[0170] Furthermore, in order to verify the visual servoing and anti-disturbance control method for the substation drain wire disconnection robot designed in the embodiment of the present application, a simulation study can be conducted, and the control method can also be compared with the visual servoing combined with PID control to better verify the advantages of the control method. Specifically, the simulation trajectory result diagram can be as follows Figure 7 As shown, in Figure 7 Where q1 is the desired trajectory, ADRC is the trajectory generated by the visual servoing and anti-disturbance control method in the embodiment of the present application, and PID is a comparative control method. Figure 7 It can be seen that the control algorithm of the embodiment of the present application can better track the desired trajectory and reduce the impact of external interference.

[0171] In an embodiment of the present application, a visual servoing model and an anti-disturbance rejection controller are deployed on the robot, and visual servo control is used to map the movement speed of the end effector to the feature point speed on the image plane, and the control signal of the end effector is converted into error reduction on the image plane. The uncertainty of the visual servo model is compensated for by anti-disturbance rejection control to achieve stable anti-disturbance control, so that the robot has autonomous operation and anti-interference capabilities, and can ensure the stability of its operation in high-voltage electrical working environments, such as substations with limited operating space, high altitudes, complex and changeable environments, which is conducive to ensuring the safe and stable operation of power equipment.

[0172] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0173] Reference Figure 8 , shows a structural block diagram of a substation robot operation device provided by an embodiment of the present application, which is applied to a robot, the robot has an operation robot arm, the operation robot arm includes an end effector and a camera, and specifically may include the following modules:

[0174] The image feature extraction module 801 is used to extract image features of the real-time image information captured by the camera during the movement of the operating robot arm;

[0175] The control signal generation module 802 is used to input the desired image features and the extracted image features into the visual servo control model deployed on the robot, and output a manipulator control signal; wherein the image Jacobian matrix in the visual servo control model is used to indicate the mapping relationship between the rate of change of the image feature points and the movement speed of the end effector. The manipulator control signal includes the angle control amount of each joint of the end effector generated based on the mapping relationship;

[0176] The motion trajectory planning module 803 is used to obtain the preliminary motion trajectory of the working robot arm based on the angle control amount of each joint of the end effector;

[0177] The disturbance compensation module 804 is used to input the desired velocity signal of the end effector into the active disturbance rejection controller deployed on the robot, and perform disturbance compensation on the preliminary motion trajectory through the active disturbance rejection controller to obtain the target motion trajectory;

[0178] The operation control module 805 is used to control the end effector to operate on the target operation object according to the target motion trajectory.

[0179] In some embodiments of the present application, the control signal generating module 802 may include the following submodules:

[0180] The control signal generation submodule is used to construct a robot arm kinematic error function based on the extracted image features and expected image features through a visual servo control model; establish an image Jacobian matrix based on the image feature point change rate of the robot's work object feature points and the movement speed of the end effector; perform error convergence based on the image Jacobian matrix and the robot arm kinematic error function to obtain the movement speed of the end effector; and generate a robot arm control signal based on the movement speed of the end effector.

[0181] In some embodiments of the present application, the rate of change of the image feature points in the image Jacobian matrix is ​​determined based on the movement speed of the feature points of the work object relative to the camera coordinate system, and the movement speed of the end effector in the image Jacobian matrix is ​​determined based on the movement speed of the camera in the world coordinate system; the control signal generation submodule may include the following units:

[0182] The motion speed determination unit is used to perform error convergence on the kinematic error function of the robot arm based on the mapping relationship between the motion speed of the feature point of the work object relative to the camera coordinate system and the motion speed of the camera in the world coordinate system, so as to obtain the expected feature point speed of the feature point of the work object; based on the mapping relationship, the expected feature point speed is mapped to obtain the motion speed of the end effector.

[0183] In some embodiments of the present application, the control signal generation submodule may include the following units:

[0184] A control signal generating unit is used to obtain the Jacobian matrix of the robot arm, and based on the pseudo-inverse matrix of the Jacobian matrix of the robot arm, map the motion speed of the end effector to the joint angular velocity of each joint of the end effector; wherein the Jacobian matrix of the robot arm is used to indicate the mapping relationship between the joint angular velocity of each joint of the end effector and the motion speed of the end effector; integrate the joint angular velocity of each joint of the end effector to obtain the angle control amount of each joint, and generate the robot arm control signal based on the angle control amount of each joint

[0185] In some embodiments of the present application, the disturbance compensation module 804 may include the following submodules:

[0186] The disturbance compensation submodule is used to output the dynamic characteristics of the working robot arm based on the input desired speed signal through the tracking differentiator of the self-disturbance rejection controller; estimate the state of the working robot arm when it moves according to the preliminary motion trajectory through the extended observer of the self-disturbance rejection controller to obtain the state estimation result; nonlinearly combine the deviation between the dynamic characteristics of the working robot arm and the state estimation result to obtain the state control quantity; use the preset state observation value to perform disturbance compensation on the state control quantity of the working robot arm when it moves according to the preliminary motion trajectory; adjust the controller parameters of the self-disturbance rejection controller based on the feedback error of the disturbance compensation result; compensate the preliminary motion trajectory based on the adjusted controller parameters to obtain the target motion trajectory.

[0187] In some embodiments of the present application, after controlling the end effector to operate the target operation object according to the target motion trajectory, the device provided by the embodiment of the present application may further include the following modules:

[0188] The closed-loop control module is used to obtain the actual motion trajectory of the end effector and feed the actual motion trajectory back to the visual servo control model; wherein the end effector includes a disassembly arm and a clamping arm, the target working object includes bolts and guide wires, and the motion trajectory includes the motion trajectory of the disassembly arm removing the bolts and the motion trajectory of the clamping arm clamping the guide wire.

[0189] In some embodiments of the present application, the robot also has a lifting frame and a traveling chassis, and the operation control module 805 is also used to control the lifting frame to descend after the disassembly arm and the clamping arm complete the operation on the bolts and guide wires in the current operation area, and control the traveling chassis to move to the next operation area, and control the operation robot arm to operate on the target operation object in the next operation area.

[0190] In an embodiment of the present application, during the movement of the robot's operating manipulator arm, real-time image information is captured based on a camera and image features are extracted, and the desired image features and the extracted image features are input into a visual servo control model deployed on the robot. Based on the mapping relationship between the changes in image feature points indicated by the image Jacobian matrix in the visual servo control model and the movement speed of the end effector, a control quantity is generated based on the mapping relationship to obtain a manipulator arm control signal. At this time, a preliminary motion trajectory of the operating manipulator arm can be obtained based on the aforementioned control quantity planning, and then the desired speed signal of the end effector can be input into an anti-disturbance rejection controller deployed on the robot. The preliminary motion trajectory is compensated for by the aforementioned deployed anti-disturbance rejection controller to obtain a target motion trajectory, and then the end effector is controlled to operate on the target working object according to the target motion trajectory. By deploying a visual servo model and an anti-disturbance control on the robot, the movement speed of the end effector is mapped to the feature point speed on the image plane using visual servo control, the control signal of the end effector is converted into error reduction on the image plane, and the uncertainty of the visual servo model is compensated for using anti-disturbance control to achieve stable anti-disturbance control. This enables the robot to have autonomous operation and anti-interference capabilities, and can ensure the stability of its operation in a high-voltage electrical working environment, which is beneficial to ensuring the safe and stable operation of power equipment.

[0191] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0192] The robot provided in the embodiment of the present application may also include a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned substation robot operation method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0193] The present application also provides a computer-readable storage medium. Fig. 9 The computer readable storage medium 900 provided stores a computer program 91. When the computer program 91 is executed by the processor, the various processes of the above-mentioned substation robot operation method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0194] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0195] It should be noted that the terms "first", "second", etc. in the specification and claims of the embodiments of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described here can be implemented in an order other than the content illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device including a series of steps or modules need not be limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The division of the modules that appear in the embodiments of the present application is only a logical division. There may be other division methods when implemented in practical applications, such as multiple modules can be combined or integrated into another system, or some features can be ignored, or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in the embodiments of the present application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules, and some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0196] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0197] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0198] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0199] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0200] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0201] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0202] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0203] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0204] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded into a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0205] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0206] Finally, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0207] The technical solutions provided in the embodiments of the present application are introduced in detail above. The principles and implementation methods of the embodiments of the present application are explained by using specific examples in the embodiments of the present application. The description of the above embodiments is only used to help understand the methods and core ideas of the embodiments of the present application. At the same time, for those skilled in the art, according to the ideas of the embodiments of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the embodiments of the present application.

Claims

1. A substation robot operation method, characterized in that: The robot has a working mechanical arm, the working mechanical arm includes an end effector and a camera, and the method includes: During the movement of the operating robot arm, based on the real-time image information captured by the camera, image features of the real-time image information are extracted; The desired image features and the extracted image features are input into a visual servo control model deployed on the robot, and a manipulator control signal is output; wherein the image Jacobian matrix in the visual servo control model is used to indicate a mapping relationship between a rate of change of image feature points and a movement speed of the end effector, and the manipulator control signal includes an angle control amount of each joint of the end effector generated based on the mapping relationship; Obtaining a preliminary motion trajectory of the operating robot arm based on the angle control amount of each joint of the end effector; Inputting the desired velocity signal of the end effector into an active disturbance rejection controller deployed on the robot, and performing disturbance compensation on the preliminary motion trajectory through the active disturbance rejection controller to obtain a target motion trajectory; The end effector is controlled to operate on the target operating object according to the target motion trajectory.

2. The method according to claim 1, characterized in that The step of inputting the expected image features and the extracted image features into a visual servo control model deployed on the robot and outputting a robot arm control signal comprises: Constructing a kinematic error function of the robot arm based on the extracted image features and the expected image features through the visual servo control model; Establishing an image Jacobian matrix based on the image feature point change rate of the working object feature point of the robot and the movement speed of the end effector; Perform error convergence based on the image Jacobian matrix and the robot arm kinematic error function to obtain the motion speed of the end effector; A robot arm control signal is generated based on the movement speed of the end effector.

3. The method according to claim 2, characterized in that The change rate of the image feature point in the image Jacobian matrix is ​​determined based on the movement speed of the work object feature point relative to the camera coordinate system, and the movement speed of the end effector in the image Jacobian matrix is ​​determined based on the movement speed of the camera in the world coordinate system; The performing error convergence based on the image Jacobian matrix and the robot arm kinematic error function to obtain the motion speed of the end effector includes: Based on the mapping relationship between the motion speed of the feature point of the work object relative to the camera coordinate system and the motion speed of the camera in the world coordinate system, performing error convergence on the kinematic error function of the robot arm to obtain the expected feature point speed of the feature point of the work object; Based on the mapping relationship, the expected feature point velocity is mapped to obtain the movement velocity of the end effector.

4. The method according to claim 2, characterized in that: The generating of the robot arm control signal based on the movement speed of the end effector comprises: Obtaining a Jacobian matrix of the robot arm, and mapping the motion speed of the end effector to the joint angular velocity of each joint of the end effector based on a pseudo-inverse matrix of the Jacobian matrix of the robot arm; wherein the Jacobian matrix of the robot arm is used to indicate a mapping relationship between the joint angular velocity of each joint of the end effector and the motion speed of the end effector; The joint angular velocity of each joint of the end effector is integrated to obtain the angle control amount of each joint, and the robot arm control signal is generated based on the angle control amount of each joint.

5. The method according to claim 1, characterized in that The step of inputting the desired speed signal of the end effector into an anti-disturbance rejection controller deployed on the robot, and performing disturbance compensation on the preliminary motion trajectory by the anti-disturbance rejection controller to obtain a target motion trajectory includes: Outputting the dynamic characteristics of the working robot arm based on the input desired speed signal through the tracking differentiator of the active disturbance rejection controller; By using the extended observer of the active disturbance rejection controller, the state of the operating robot arm when it is operating according to the preliminary motion trajectory is estimated to obtain a state estimation result; Performing nonlinear combination on the deviation between the dynamic characteristics of the working robot arm and the state estimation result to obtain a state control quantity; Using a preset state observation value to perform disturbance compensation on a state control amount of the operating robot arm when the operating robot arm moves according to the preliminary motion trajectory; Adjusting controller parameters of the active disturbance rejection controller based on a feedback error of a disturbance compensation result; The preliminary motion trajectory is compensated based on the tuned controller parameters to obtain the target motion trajectory.

6. The method according to claim 1 or 5, characterized in that: After controlling the end effector to operate the target operating object according to the target motion trajectory, the method further includes: Acquiring an actual motion trajectory of the end effector, and feeding the actual motion trajectory back to the visual servo control model; Among them, the end effector includes a disassembly arm and a clamping arm, the target working object includes a bolt and a guide wire, and the motion trajectory includes the motion trajectory of the disassembly arm disassembling the bolt and the motion trajectory of the clamping arm clamping the guide wire.

7. The method according to claim 6, characterized in that The robot also has a lifting frame and a traveling chassis, and the method further comprises: After the disassembly and assembly arm and the clamping arm complete the operation on the bolts and guide wires in the current operation area, the lifting frame is controlled to descend, and the traveling chassis is controlled to move to the next operation area, and the operation robot arm is controlled to operate on the target operation object in the next operation area.

8. A substation robot operation device, characterized in that: Applied to the robot, the robot has an operating mechanical arm, the operating mechanical arm includes an end effector and a camera, and the device includes: An image feature extraction module is used to extract image features of the real-time image information based on the real-time image information captured by the camera during the movement of the operating robot arm; A control signal generation module, used for inputting the expected image features and the extracted image features into a visual servo control model deployed on the robot, and outputting a manipulator control signal; wherein the image Jacobian matrix in the visual servo control model is used to indicate the mapping relationship between the rate of change of the image feature points and the movement speed of the end effector, and the manipulator control signal includes the angle control amount of each joint of the end effector generated based on the mapping relationship; A motion trajectory planning module, used to obtain a preliminary motion trajectory of the operating robot arm based on the angle control amount of each joint of the end effector; A disturbance compensation module, used for inputting the desired velocity signal of the end effector into an active disturbance rejection controller deployed on the robot, and performing disturbance compensation on the preliminary motion trajectory through the active disturbance rejection controller to obtain a target motion trajectory; The operation control module is used to control the end effector to operate on the target operation object according to the target motion trajectory.

9. A robot, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the substation robot operation method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the substation robot operation method according to any one of claims 1 to 7 is implemented.

Images