Visual servo control method and system based on robot wire hanging operation

Abstract

The invention relates to the technical field of power system hot-line work, and discloses a visual servo control method and system based on robot wire hanging work, and the method comprises the steps: starting a robot to approach a power transmission line, and carrying out the detection and execution of potential transfer operation through an equipotential arm; the distance between the robot and the power transmission line is obtained in real time through a binocular camera, and whether the robot reaches the preset position of line hanging operation or not is judged; based on a DeepLabV3 + semantic segmentation algorithm, a gap between the robot and the power transmission line is identified, and it is ensured that walking wheels of the robot are accurately aligned with the line hanging position of the power transmission line; and a PID controller is used for adjusting the contraction action of a walking arm of the robot, and when it is detected that the gap between the walking wheel and the power transmission line is zero, jacking operation is executed, and line hanging operation is completed. The high-precision identification of the gap between the robot and the power transmission line is realized, the real-time adjustment capability of the contraction action of the walking arm is enhanced, and the response speed and the operation precision of the system are improved. And electric field distortion and air breakdown risks are effectively reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of live-line operation in electric power systems, and in particular to a visual servo control method and system based on robot line hanging operation. Background Art

[0002] With the continuous growth of global electricity demand, the maintenance and management of high-voltage transmission lines have become key tasks for the stable operation of power systems. Traditional high-voltage line maintenance operations usually rely on manual operation, and operators need to wear insulating equipment to detect and maintain the lines under live conditions. This type of operation is not only dangerous, but also difficult for operators to operate due to the influence of high-voltage electric fields, and prone to errors. In recent years, with the rapid development of robotics technology, especially the progress of automatic control and visual recognition technology, the power industry has gradually begun to introduce live-line operation robots to perform complex and dangerous operations. Replacing manual operation with robots has greatly improved the safety and efficiency of operations. Among them, visual servo control technology, as one of the core technologies of live-line operation robots, can monitor the transmission line environment in real time and provide accurate position information and action instructions for robot operations. At present, vision-based automatic control technology has been applied to many fields, but robot control in high-voltage live environments still faces many challenges.

[0003] Although the existing live-line working robot technology has been able to achieve a certain degree of automated operation, there are still some key technical difficulties. First, the electric field distortion and air breakdown risks during the potential transfer process have not been effectively controlled. The existing robot system is easily affected by the change of the electric field during the equipotential operation, resulting in arc discharge and even equipment damage. Secondly, in the transmission line operation, most of the existing technologies rely on monocular cameras or simple sensor systems for distance measurement and target recognition, resulting in limited measurement accuracy and unable to meet the precise control requirements in complex working environments. Especially when the robot is hanging the line, the existing control system often finds it difficult to accurately identify and adjust the gap between the walking wheel and the transmission line, resulting in low efficiency and even safety hazards in the hanging line operation. In contrast, our invention has achieved high-precision recognition of the gap between the robot and the transmission line by introducing a binocular camera and the DeepLabV3+ semantic segmentation algorithm, and combined with the PID controller to adjust the walking arm in real time to ensure the accuracy and stability of the hanging line operation. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problems solved by the present invention are: equipment damage caused by electric field distortion and air breakdown risk during potential transfer; the existing ranging technology has insufficient measurement accuracy in complex environments and cannot accurately judge the gap between the robot and the transmission line; and the control system has a response delay when adjusting the walking arm retraction action in real time, affecting the safety and efficiency of the line hanging operation.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a visual servo control method based on robot hanging line operation, comprising:

[0007] The robot is started to approach the power transmission line and detects and performs potential transfer operations using the equipotential arm;

[0008] The binocular camera is used to obtain the distance between the robot and the power transmission line in real time to determine whether the robot has reached the predetermined position for the line hanging operation;

[0009] Based on the DeepLabV3+ semantic segmentation algorithm, the gap between the robot and the transmission line is identified to ensure that the robot's walking wheels are accurately aligned with the hanging position of the transmission line;

[0010] A PID controller is used to adjust the retraction action of the robot's walking arm. When it is detected that the gap between the walking wheel and the transmission line is zero, the tightening operation is performed to complete the line hanging operation.

[0011] As a preferred solution of the visual servo control method based on robot wire hanging operation described in the present invention, the potential transfer operation includes: when the robot rises to the equipotential safety distance of the transmission line, the equipotential arm rises, and the operating system analyzes the change of electric field strength in real time to ensure that the equipotential operation is completed before the air breaks down;

[0012] When the system detects that the electric field strength is close to the critical breakdown value, the system immediately issues an alarm and the operator needs to take prompt action.

[0013] After air breakdown occurs, the equipotential arm forms an electrical connection with the transmission line to ensure that the robot and the line reach an equipotential state and eliminate the potential difference.

[0014] As a preferred solution of the visual servo control method based on the robot wire hanging operation described in the present invention, wherein: judging whether the robot has reached the predetermined position of the wire hanging operation includes, during the robot's ascent, the binocular camera captures the image of the power transmission line in real time, and calculates the vertical distance between the robot and the power transmission line through parallax;

[0015] The vertical distance is compared with the set threshold to determine whether the predetermined position for the hanging line operation has been reached.

[0016] As a preferred solution of the visual servo control method based on robot line hanging operation described in the present invention, wherein: identifying the gap between the robot and the transmission line includes model building, model training, model evaluation and identification processes;

[0017] The model building includes using the DeepLabV3+ architecture, including the Xception backbone network and the dilated spatial pyramid pooling module to build the DeepLabV3+ model;

[0018] The training model includes enhancing and preprocessing the transmission line image captured in real time by the binocular camera, and marking the running wheels, transmission lines and the gap between them in the image to form a data set;

[0019] Divide the dataset into training set, test set and validation set;

[0020] Use the training set to train the DeepLabV3+ model, optimize the model parameters, select the loss function to evaluate the gap between the model output and the actual label, and perform gradient updates;

[0021] Model evaluation includes evaluating the trained DeepLabV3+ model on the test set to ensure that the DeepLabV3+ model has high accuracy in identifying gaps; and comparing the differences between the DeepLabV3+ model prediction results and the actual markings to analyze the performance of the DeepLabV3+ model in gap identification.

[0022] As a preferred solution of the visual servo control method based on robot hanging line operation described in the present invention, the recognition process includes: inputting the real-time captured image into the trained DeepLabV3+ model; the DeepLabV3+ model extracts the high-level features of the input image through the encoder part, and generates a segmentation output of the same size as the input image through the decoder part;

[0023] In the DeepLabV3+ model output, the areas of the running wheels, transmission lines and their gaps are marked, and the actual gap size is reflected through pixel-level segmentation results.

[0024] As a preferred solution of the visual servo control method based on robot hanging line operation described in the present invention, wherein: the PID controller adjusts the contraction action of the robot walking arm including adjusting the gap and top line operation;

[0025] The gap adjustment includes: a PID controller receiving the actual gap fed back by the visual inspection system in real time, and adjusting the deviation between the expected gap and the actual gap to gradually reduce the actual gap until the gap is detected to be zero.

[0026] As a preferred solution of the visual servo control method based on robot wire hanging operation described in the present invention, wherein: the wire lifting operation includes, when the gap is detected to be zero, the tension wheel is gradually raised under the drive of the motor to ensure that the walking arm is completely pressed on the transmission line;

[0027] During the tightening process, the photoelectric sensor continuously monitors the pressure between the travel wheel and the transmission line to ensure that no abnormal situation occurs;

[0028] When the tensioning wheel is successfully tightened, the lifting mechanism will immediately lift the rope mechanism and leave the transmission line. The system confirms the tightening status and the stability of the walking arm through sensors to ensure the smooth completion of the line hanging operation.

[0029] A visual servo control system based on robot hanging line operation, comprising:

[0030] The visual inspection module uses a binocular camera to perform real-time distance measurement to obtain the distance between the robot and the transmission line, and uses the DeepLabV3+ algorithm to identify the gap, thereby achieving accurate detection of the gap between the walking wheel and the transmission line.

[0031] Potential transfer module, which handles the potential transfer process between the live working robot and the transmission line to avoid the risks of air breakdown and arc discharge;

[0032] Binocular camera ranging module: The binocular camera calculates the distance between the robot and the transmission line through the principle of parallax and triangulation, ensuring the accuracy of equipotential operation and line hanging operation.

[0033] The DeepLabV3+ semantic segmentation module uses deep convolutional neural networks and dilated convolution technology to perform semantic segmentation on the gap between the running wheels and the transmission lines, ensuring that the status of the running wheels can be accurately judged during line hanging operations.

[0034] PID control module, based on the gap data of visual feedback, the PID controller adjusts the retraction operation of the walking arm in real time;

[0035] The tightening and line hanging operation module, when the PID controller confirms that the gap is zero, performs the tightening operation, presses the walking arm firmly on the transmission line, completes the line hanging operation, and starts the jacking mechanism to lift the rope mechanism to prepare for subsequent operations.

[0036] The sensor feedback module uses photoelectric sensors to monitor the status of the tension wheel and key parameters in the wire hanging process in real time to ensure the safety and accuracy of the operation.

[0037] A computer device comprises: a memory and a processor; the memory stores a computer program, wherein the processor implements the steps of any one of the methods of the present invention when executing the computer program.

[0038] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of any one of the methods of the present invention.

[0039] Beneficial effects of the present invention: The visual servo control method based on robot wire hanging operation provided by the present invention realizes high-precision recognition of the gap between the robot and the transmission line through a binocular camera and DeepLabV3+ algorithm, ensuring that the walking wheel is accurately aligned with the wire hanging position. Secondly, the introduction of the PID controller enhances the real-time adjustment capability of the walking arm contraction action, improves the system response speed and operation accuracy. Thirdly, the application of the equipotential arm effectively reduces the risk of electric field distortion and air breakdown, ensuring the safe operation of the robot in a live environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0041] Figure 1 An overall flow chart of a visual servo control method based on robot wire hanging operation provided by the first embodiment of the present invention;

[0042] Figure 2 A servo control flow chart of a visual servo control method for a robot wire hanging operation provided in a second embodiment of the present invention;

[0043] Figure 3 An equivalent schematic diagram of a potential transfer process of a visual servo control method based on a robot wire hanging operation provided by a second embodiment of the present invention;

[0044] Figure 4 A diagram of an image coordinate system and a pixel coordinate system of a visual servo control method based on a robot wire hanging operation provided by a second embodiment of the present invention;

[0045] Figure 5 A schematic diagram of binocular camera ranging based on a visual servo control method for robot wire hanging operation provided in the second embodiment of the present invention;

[0046] Figure 6 A network structure diagram of DeepLabV3+ based on a visual servo control method for robot hanging line operation provided by the second embodiment of the present invention;

[0047] Figure 7A deep separable convolution process diagram of a visual servo control method based on robot wire hanging operation provided by the second embodiment of the present invention;

[0048] Figure 8 A hollow space pyramid pooling diagram of a visual servo control method based on robot wire hanging operation provided by the second embodiment of the present invention;

[0049] Fig. 9 A PID control diagram based on a visual system for a visual servo control method for a robot hanging line operation provided in a second embodiment of the present invention;

[0050] Fig.10 A PID simulation model diagram of gap control based on a visual servo control method for robot hanging line operation provided by the second embodiment of the present invention;

[0051] Fig.11 A PID controller step response result diagram of a visual servo control method based on robot wire hanging operation provided by the second embodiment of the present invention;

[0052] Fig.12 A camera imaging principle diagram of a visual servo control method based on robot hanging line operation provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0054] Example 1, reference Figure 1 , as an embodiment of the present invention, provides a visual servo control method based on robot hanging line operation, comprising:

[0055] S1: Start the robot to approach the transmission line and use the equipotential arm to detect and perform potential transfer operations.

[0056] When the live working robot rises to approach the transmission line, it needs to use a binocular camera to calculate the distance between the robot and the transmission line in order to determine when it reaches the equipotential working distance; after completing the equipotential, it rises again, stops rising when it reaches the line hanging working distance, starts closing the robot arm, and uses a photoelectric sensor to determine whether the walking wheel is directly above the transmission line; then the DeepLabV3+ algorithm is used to identify the gap between the walking wheel groove and the transmission line, and when the gap is zero, the tensioning wheel is raised and tightened to ensure that the robot is safely hanging on the transmission line; finally, the lifting mechanism is controlled to rise and the rope mechanism is lifted up to leave the transmission line, which is convenient for the robot to walk and work later. The visual servo process of the robot hanging line operation is as follows: Figure 2 shown.

[0057] Since the live transmission line operation robot needs to work on the line with power, the robot must undergo a potential transfer process when it rises from the ground to the transmission line. This transfer process will cause the electric field to be distorted when the robot approaches the transmission line. As the distance between the robot and the line decreases, the electric field distortion gradually intensifies, thereby increasing the risk of air breakdown. To ensure the safe and stable operation of the robot during this process, the equipotential operation must be completed before air breakdown occurs. Air breakdown will cause arcing and release energy, which may cause damage to the internal components of the robot. Moreover, as the breakdown process gradually intensifies, it will initially be corona discharge, then evolve into gap arc discharge, and may eventually form continuous arc discharge. To ensure the safety and stability of the robot, the operator needs to pay close attention to the changes in various parameters during the potential transfer process and take necessary measures in a timely manner to deal with potential risks.

[0058] The transfer current generated during the potential transfer process will induce electromagnetic waves, which will couple with the heat dissipation holes of the robot chassis, resulting in cavity resonance, which may damage the control and communication panels inside the chassis. Therefore, the equipotential arm method is used in the process to achieve the potential transfer between the robot and the transmission line. The end clamp wheel of the equipotential arm is made of copper material. It is necessary to analyze the equipotential process to determine the safe distance of the equipotential and the key parameters of the equipotential arm.

[0059] The potential transfer process between the robot and the transmission line can be divided into the following three stages:

[0060] Pre-potential transfer stage: This stage refers to the process of the robot rising from the ground to the safe distance of equal potential. In this process, the robot is far away from the transmission line, so the electromagnetic field generated by the transmission line has no obvious effect on the robot. In order to improve safety, the safe distance of equal potential is usually increased in actual projects;

[0061] Transition stage: This stage refers to the process in which the operator drives the motor to raise the equipotential arm after the robot reaches the safe distance of equipotential and before air breakdown occurs. Once the robot reaches the safe distance of equipotential, the operator will control the equipotential arm to rise. As the equipotential arm rises, the degree of distortion of the surrounding electromagnetic field will also increase;

[0062] Potential transfer stage: This stage refers to the process from the time when air breakdown occurs between the equipotential arm and the transmission line until the completion of the equipotential connection with the target transmission line.

[0063] Furthermore, by constructing an equivalent model of the robot's potential transfer, the parameters of the robot during the potential transfer process can be analyzed and modeled. In the equivalent circuit model, the electric field inhomogeneity of the rod-plate gap is taken into account, and the electric field environment of the transmission line and the operating robot is equivalent to a rod-plate gap with uneven field strength.

[0064] The specific analysis process is as follows: First, use the fitting function to calculate the capacitance value related to the distance between the target transmission line and the live working robot in the equivalent circuit model. The fitting function can be determined based on the actual situation and experimental data. Then, compare the distance between the live working robot and the transmission line with the critical breakdown field strength to calculate the safe distance for the potential transfer operation. This safe distance can be used as a reference for the research report on navigation, positioning and control technology for UAV-robot collaborative operations during the robot's on-line and off-line processes to ensure that the potential difference between the robot and the transmission line is always within a safe range to avoid air breakdown. Finally, by establishing an equivalent circuit model, using simulation software for simulation, and calculating the equipotential safety distance, an analysis and evaluation method for the potential transfer state of the live working robot during its on-line and off-line processes can be provided, which helps to ensure the safety of the live working robot and avoid potential risks caused by the electric field. The equivalent circuit is as follows: Figure 3 shown.

[0065] exist Figure 3 In the example, the arcing and arcing process of the air gap between the robot and the transmission line is simulated by using switches. a , R a are the equivalent capacitance and insulation resistance between the robot and the transmission line respectively; C b , R b are the equivalent capacitance and resistance between the robot and the tower, the ground and other transmission lines; C c , R c are the equivalent capacitance and resistance between the transmission line and the ground respectively; I a is the transfer current; R xis the resistance of the arc; L is the equivalent inductance. Calculate the equivalent capacitance parameters: The equivalent capacitance between the robot and the transmission line, as well as the capacitance between the robot and the ground, the tower and other transmission lines, can first be calculated using the electromagnetic field finite element method to find out the distribution of their potential, and then calculated based on the definition of capacitance. The specific calculation steps are as follows: Apply voltage to the transmission line, assuming that the robot is not charged, and calculate the voltage between the robot and the transmission line and the voltage between the robot and the ground, as shown in formula (6.1):

[0066] C a du1+C b du1=0(6.1)

[0067] Without changing the voltage of the transmission line, a small amount of charge is applied to the robot, and the voltage between the robot and the transmission line and the voltage between the robot and the ground are calculated, as shown in formula (6.2):

[0068] C a du2+C b du2=dQ(6.2)

[0069] Solving equations (6.1) and (6.2) together, we get the equivalent capacitance C a and C c According to the average breakdown field strength formula of the rod-plate gap (6.3):

[0070]

[0071] The calculation formula for the average field strength of 220kV transmission line can be obtained (6.4):

[0072]

[0073] According to formula (6.3), when the radius of the transmission line is given, the average breakdown field strength is related to the distance between the robot and the transmission line. In the 220kV transmission line environment, the average breakdown field strength is 0.65kV / mm, and the equipotential safety distance can be calculated by formula (6.4).

[0074] Furthermore, when operating in a live environment, the robot must be at the same potential as the power transmission line to prevent air breakdown or arc discharge caused by electric field distortion. Through the design of the equipotential arm, the robot can accurately sense the electric field strength when approaching the power transmission line and safely complete the potential transfer operation by adjusting its posture in real time. This process effectively reduces the electrical risks in the operation and ensures the safety of the robot's operation.

[0075] S2: The distance between the robot and the power transmission line is obtained in real time through the binocular camera to determine whether the robot has reached the predetermined position for the line hanging operation.

[0076] The principle of camera pinhole imaging refers to the use of the linear propagation of light and the projection principle of geometric optics to limit external light inside the camera through a small hole to form an inverted real image. The function of the photosensitive element (such as film or image sensor) in the camera is to convert light signals into electrical signals and ultimately generate digital images. The element converts light into charge or voltage signals, which are converted into digital signals by an analog-to-digital converter (ADC), and then image processing and digital storage are performed to ultimately generate digital image files. The image after imaging is the opposite of the real object, and its image is relatively large. After image reduction and rotation, an equivalent imaging plane with the same size as the real object and the opposite image can be obtained.

[0077] During the manufacturing and installation process of the camera, manufacturing and installation errors often occur. These errors will cause the camera to distort when shooting objects and no longer conform to the pinhole imaging method. Therefore, when using a binocular camera for measurement, the camera distortion problem must be solved first to improve the measurement accuracy.

[0078] (1) Radial distortion is caused by the change in the optical path of the original light when it passes through the lens due to the processing error of the lens itself during the lens manufacturing process, thus causing image distortion. Radial distortion is usually divided into barrel distortion and pincushion distortion: barrel distortion causes the center of the image to shrink relative to the edge, so that even if the straight lines in the image are straight in the real world, they will appear to bend inward in the image; pincushion distortion causes the center of the image to enlarge relative to the edge, so that even if the straight lines in the image are straight in the real world, they will appear to bend outward in the image. The radial distortion correction formula is shown in (6.5)

[0079]

[0080] In the formula, r represents the distance from the point to the center of the image, (x, y) represents the coordinates of the distortion point, and x r ,y r ) represents the coordinates of the point after distortion correction. When the distortion is not very serious, g is usually used. i and g2 are sufficient to solve the distortion problem of the camera lens, but when the distortion is large, it is necessary to introduce the distortion coefficient g3 for correction. (2) Tangential distortion is generated when the lens is assembled, which makes the optical axis of the lens not completely perpendicular to the image plane, thus causing image distortion. Tangential distortion is usually divided into horizontal tangential distortion and vertical tangential distortion: horizontal tangential distortion makes the horizontal lines in the image appear to be bent to the left or right in the image; while vertical tangential distortion makes the vertical lines in the image appear to be bent upward or downward in the image. The tangential distortion correction formula is shown in (6.6).

[0081]

[0082] The intrinsic parameters of a camera refer to the parameters related to the internal characteristics and geometric structure of the camera itself, such as focal length and aperture. They are usually inherent properties of the camera and do not change with changes in the external environment. The extrinsic parameters of a camera refer to the parameters that describe the position and posture of the camera in three-dimensional space, including the camera's translation vector Euler angle, etc., which are used to correspond and transform the camera coordinate system to the world coordinate system and need to be obtained through camera calibration or calculation. The two define the transformation relationship between the camera coordinate system and the world coordinate system, and are used to convert the pixel coordinates in the image into the actual object coordinates in three-dimensional space. In order to accurately describe the camera imaging model, four coordinate systems are introduced to describe different geometric relationships. World coordinate system (O w X w Y w Z w ) represents the coordinates of the actual object in three-dimensional space; the camera coordinate system ( c X c Y c Z c ) represents the position and posture of the camera; the image coordinate system (o-xy) is used to describe the two-dimensional coordinates in the image; the pixel coordinate system (o-uv) represents the pixel coordinates in the image. Figure 4 As shown, O0 and O1 are the origins of the pixel coordinate system and the image coordinate system respectively, dx and dy represent the unit pixel size of the camera. Based on this information, the relationship between the image coordinate system and the pixel coordinate system can be obtained, as shown in formula (6.7).

[0083] x=(u-u0)dx, y=(v-v0)dy(6.7)

[0084] Rewrite formula (6.7) into a homogeneous matrix form as formula (6.8), which is the expression for converting any point in the pixel coordinate system to the image coordinate system.

[0085]

[0086] exist Fig.12 In the figure, o is the center of the camera optical axis, M is the camera imaging plane, and N is the actual object plane.

[0087] A point in the robot's workspace coordinate system In the camera coordinate system (O c X c Y c Z c ) is denoted by Q = [X c Y c Z c ] T , the coordinates in the image coordinate system (o-xy) are Q = [xy] TAs shown in formula (6.9), according to the principle of similar triangles, we can get The coordinate values ​​in the three coordinate systems are in geometric proportion:

[0088]

[0089] Rewrite formula (6.9) into a homogeneous matrix form as formula (6.10), which is the expression for converting any point in the image coordinate system to the camera coordinate system.

[0090]

[0091] The camera coordinate system and the world coordinate system both exist in three-dimensional space, the difference lies in the choice of origin and direction. Therefore, only rigid body transformation, such as translation and rotation, is needed to obtain the conversion relationship between them. Take a point Q = [X w Y w Z w ] T , expressed as Q = [X c Y c Z c ] T , the conversion formula is shown in formula (6.11).

[0092]

[0093] Rewritten as a homogeneous matrix as formula (6.12):

[0094]

[0095] Where M1 is the camera external parameter matrix, which is obtained by camera calibration. Combining formula (6.8) and formula (6.10), the expression for converting pixel coordinates to camera coordinates is shown in formula (6.13):

[0096]

[0097] Where M2 is the camera intrinsic parameter matrix.

[0098] In order to obtain the internal and external parameters of the camera, the binocular camera is calibrated.

[0099] Binocular camera ranging is a three-dimensional measurement method based on binocular vision. It uses two cameras to simulate the way human eyes observe scenes to obtain depth information. The principle of binocular camera ranging uses the concepts of parallax and triangulation. By calculating the parallax between pixels and combining camera parameters and baseline length, the distance measurement of objects in the scene is achieved. The principle diagram is as follows Figure 5 As shown:

[0100] M is the target object, Oι , O r are the optical centers of the left and right cameras, D is the distance between the left and right optical centers, f is the focal length of the camera, and M ι 、M r are the coordinates of the target object, M in the left and right camera image coordinate systems, and L is the vertical distance from the target object to the camera.

[0101] According to the principle of similar triangles, ΔMM l M r ~ΔMO l O r , we can get formula (6.14):

[0102]

[0103] Where: D-(X l -X r )=M l M r ,X l and X r are the horizontal coordinates of the pixel points of the target object M in the left and right images respectively.

[0104] Then the vertical distance between the target object and the camera is obtained as shown in formula (6.15):

[0105]

[0106] Where: d = X l -X r Indicates the parallax of the left and right cameras.

[0107] When the robot is online, the binocular camera at the bottom of the walking arm at one end will extract the information of the transmission line in real time, calculate the vertical distance between the robot and the transmission line, and compare the distance with the set threshold to determine whether equipotential operation and walking arm closing operation can be performed.

[0108] In order to verify the accuracy of binocular camera ranging, ten groups of samples were randomly selected and the results are shown in Table 1.

[0109] Table 1 is a comparison table of the binocular camera measurement values ​​and actual values.

[0110]

[0111]

[0112] The measurement accuracy of the experimental samples in the table was evaluated, and the results showed that the maximum error was 5.96 cm and the minimum error was 0.28 cm, which indicated that the proposed distance measurement method had high precision and accuracy. The measurement errors of all samples were controlled within 5%, meeting the requirements of measuring the equipotential working safety distance and the closing walking arm safety distance during the ascent of the live working robot.

[0113] Furthermore, real-time distance measurement is performed through binocular cameras to ensure that the robot can accurately locate when approaching the power transmission line. The binocular camera uses the principle of parallax and triangulation to calculate the distance. Compared with traditional monocular cameras or laser ranging technology, the binocular camera is more stable in complex electric fields and environments. This design not only improves the accuracy of measurement, but also enhances the environmental adaptability of the system, ensuring that the robot can accurately reach the predetermined position of the line hanging operation, thus laying the foundation for subsequent operations.

[0114] S3: Based on the DeepLabV3+ semantic segmentation algorithm, the gap between the robot and the transmission line is identified to ensure that the robot's walking wheels are accurately aligned with the hanging position of the transmission line.

[0115] The DeepLabv3+ semantic segmentation algorithm achieves efficient and accurate pixel-level image segmentation by combining deep convolutional neural networks and dilated convolution (or expanded convolution) technology. DeepLabv3+ adopts an encoder-decoder structure, in which the encoder part uses a powerful convolutional neural network for feature extraction. Figure 7 As shown in the figure, it consists of the Xception backbone network and the ASPP module. The Xception network uses convolution and pooling techniques to extract image features and generate high semantic feature outputs. This output is sent to the ASPP module for multi-scale hole convolution sampling. The ASPP module consists of hole convolutions with different hole rates and a global average pooling layer. These feature maps are spliced ​​and fused in the channel dimension, and 1×1 convolution is used for channel dimensionality reduction to capture contextual information of multiple scales of the image.

[0116] The decoder is used to process low-semantic feature maps, which have higher resolution and richer edge information. The decoder first uses 1×1 convolution to reduce the dimension of the low-semantic feature map extracted from the DCNN module, and then matches the reduced-dimensional feature map with the encoder output feature map that has been upsampled 4 times to perform image splicing and fusion. Finally, by using 3×3 convolution and 4 times upsampling operations, the goal of generating a prediction result map of the same size as the input image is achieved. Research Report on Navigation, Positioning and Control Technology for UAV-Robot Collaborative Operation

[0117] Depthwise separable convolution is a commonly used convolution operation in convolutional neural networks. It consists of two steps: depthwise convolution and pointwise convolution. Depthwise convolution applies an independent convolution kernel to each channel of the input data to perform convolution operations on the channel. This operation can effectively capture the spatial correlation between channels. The function of pointwise convolution is to linearly combine the output feature maps of depthwise convolution to generate the final output feature map. This operation can reduce the amount of calculation and the number of parameters. Depthwise separable convolution has fewer parameters, higher computational efficiency and better representation ability than traditional convolution.

[0118] In order to achieve the refinement of the convolution operation, the depthwise separable convolution adopts the method of performing depthwise convolution first and then point-by-point convolution. Its advantage is that it reduces the number of parameters and reduces the computational complexity, and improves the work efficiency while ensuring high expression ability. The specific process is as follows: Figure 8 as shown.

[0119] The depth-wise separable convolution formulas are shown in Equations (6.16), (6.17), and (6.18).

[0120]

[0121] SepConv(W p ,W d ,y) (i,j) =Point wiseConv (i,j) (W p ,Depth wiseConv (i,j) (W d ,y))(6.18)Wherein, DepthwiseCom(W,y) (i,j) For the depth convolution operation, PointwiseConv(W,y) (i,j) For point-by-point convolution operations, SepConv(W p ,W d ,y) (i,j) is a depth-wise separable convolution operation. w is the convolution kernel, W p is point-wise convolution, W d It is a channel-by-channel convolution, y is the input feature map, i, j are the rows and columns during convolution, k, l, m are the convolution kernel sizes, and M is the number of feature map channels.

[0122] Dilated convolution, also known as expanded convolution or dilated convolution, is a convolution operation commonly used in neural networks. Compared with traditional convolution operations, dilated convolution introduces one or more dilated parameters to introduce spaced sampling points within the receptive field, capture a wider range of contextual information, and reduce the spatial resolution loss of feature maps. In dilated convolution, spaces or holes are introduced between the elements of the convolution kernel. By inserting a certain number of zero values ​​between the elements of the convolution kernel, the sampling interval of the convolution kernel on the input feature map becomes larger. By increasing the interval of the convolution kernel, dilated convolution can expand the receptive field of each convolution kernel, that is, it can "see" farther pixels, which can better capture the long-distance contextual information in the image, which is very effective for the recognition and segmentation of large-scale objects or global features.

[0123] In order to solve the problem that traditional DeepLabV3 has difficulty in processing multi-scale objects or detail segmentation, DeepLabV3+ introduces the ASPP module, which constructs a dilated spatial convolution pooling pyramid by using multiple dilated convolutions and global pooling operations with different dilation rates in parallel. By introducing the ASPP module, DeepLabV3+ has obvious advantages in multi-scale context modeling and detail capture, can better process images with objects of different scales, and can maintain high segmentation accuracy and boundary clarity.

[0124] ASPP structure Fig. 9 As shown in the figure, the input feature map undergoes a 1×1 convolution operation, which can reduce the number of channels and generate a lower-dimensional feature map. A 3×3 convolution kernel with three different dilation rates (6, 12, and 18) is used for dilated convolution operations, and convolution operations are performed on the input feature map respectively. The purpose of these three dilated convolution operations is to obtain contextual information and spatial position information at different scales. Each dilated convolution operation will generate a corresponding feature map. The output feature map includes the original low-dimensional feature map and feature maps at three different dilation rates, for a total of four feature maps. The dilated convolution calculation process is shown in Formula (6.19), where i represents the input feature, y represents the output feature value, s represents the dilation rate, and w[k] is a convolution kernel of size k×k.

[0125] y[i]=Σ k [i+sk]w[k](6.19)

[0126] The last step of the ASPP module is to introduce a global average pooling operation. The specific steps are as follows: first, the input feature map is pooled to reduce the dimension and then restored using a 1×1 convolution; then, the feature values ​​of each channel are summed up to calculate the feature value corresponding to each channel; finally, the five output feature maps are integrated and the convolution operation is used to further reduce the dimension.

[0127] Furthermore, the DeepLabV3+ semantic segmentation algorithm was introduced to improve the robot's ability to identify subtle gaps in complex environments. Through the deep learning algorithm, the system can accurately identify the gap between the running wheels and the transmission line, ensuring that the robot can safely and accurately perform the hanging line operation on the high-voltage transmission line. This algorithm not only improves the accuracy of gap recognition, but also has the ability to efficiently process multi-scale objects, providing the robot with more intelligent operation control.

[0128] S4: Use the PID controller to adjust the retraction action of the robot's walking arm. When it is detected that the gap between the walking wheel and the transmission line is zero, perform the tightening operation to complete the line hanging operation.

[0129] In order to ensure that the four walking arms of the robot are completely pressed on the transmission line, a vision-based PID control is proposed, such as Fig.10 As shown in the figure, it includes modules such as visual inspection, image analysis and processing, and motor control. The visual inspection system uses image algorithms to obtain target information, image analysis and processing to obtain the gap size, and the PID controller is used to determine the position at the next moment, to achieve real-time tracking of the gap, and to complete the robot's walking arm retraction operation by controlling the motor. When the gap is 0, the tensioning wheel is raised and tightened, and the lifting mechanism lifts the rope bracket to complete the robot's hanging operation.

[0130]

[0131] The deviation e between the expected gap and the actual gap is adjusted by formula (6.20). Where e(t is the input of the PID controller, U(t) is the output of the PID controller, and K p is the proportionality coefficient, K i is the integration coefficient, K d is the differential coefficient. The system feeds back the gap size detected by the image to the PID in real time to control the working motor and obtain the desired gap size. The controller Simulink simulation model is as follows Fig.11 shown.

[0132] Use the step signal as the input signal to simulate the PID controller. The results are as follows Fig.12 As shown in the figure, when the system receives a step signal, a small amplitude oscillation is generated, but the overshoot is small, and the system can return to a stable state in just 2.6 seconds, so the PID controller can achieve precise control of the gap size. In addition, the PID controller has the characteristics of good stability and easy operation, and is suitable for use in practical applications of robots.

[0133] Furthermore, PID control can provide real-time feedback and adjustment of the movement of the walking arm to ensure that the gap between the walking wheel and the transmission line is accurately zero. This dynamic control method ensures the reliability of the tightening operation, avoids errors caused by human operation, greatly improves the automation level and efficiency of the operation, and also ensures the safety of the hanging line operation.

[0134] On the other hand, this embodiment also provides a visual servo control system based on robot hanging line operation, which includes:

[0135] The visual inspection module uses a binocular camera to perform real-time distance measurement to obtain the distance between the robot and the transmission line, and uses the DeepLabV3+ algorithm to identify the gap, thereby achieving accurate detection of the gap between the walking wheel and the transmission line.

[0136] Potential transfer module, which handles the potential transfer process between the live working robot and the transmission line to avoid the risks of air breakdown and arc discharge;

[0137] Binocular camera ranging module: The binocular camera calculates the distance between the robot and the transmission line through the principle of parallax and triangulation, ensuring the accuracy of equipotential operation and line hanging operation.

[0138] The DeepLabV3+ semantic segmentation module uses deep convolutional neural networks and dilated convolution technology to perform semantic segmentation on the gap between the running wheels and the transmission lines, ensuring that the status of the running wheels can be accurately judged during line hanging operations.

[0139] PID control module, based on the gap data of visual feedback, the PID controller adjusts the retraction operation of the walking arm in real time;

[0140] The tightening and line hanging operation module, when the PID controller confirms that the gap is zero, performs the tightening operation, presses the walking arm firmly on the transmission line, completes the line hanging operation, and starts the jacking mechanism to lift the rope mechanism to prepare for subsequent operations.

[0141] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0142] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0143] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0144] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0145] Example 2, in order to verify the effectiveness of the visual servo control method based on the robot hanging line operation, this experiment selected 220kV transmission line as the test object. The experimental site simulates the working environment of the actual transmission line, and is equipped with an intelligent robot that can realize binocular camera ranging, DeepLabV3+ algorithm gap recognition, and PID control. First, the robot is in the initial state and approaches the transmission line. Through the automatic control system of the equipotential arm, the robot rises to the equipotential safety distance, and monitors the change of electric field strength during the potential transfer process to ensure that air breakdown does not occur. While approaching the transmission line, the binocular camera begins to obtain the distance between the robot and the transmission line in real time. Using the parallax and triangulation principle of the binocular camera, the robot can accurately calculate the vertical distance between itself and the transmission line. When the distance reaches the set threshold, the robot stops rising and enters the hanging line operation stage.

[0146] During the line hanging process, the DeepLabV3+ semantic segmentation algorithm recognizes the gap between the robot's walking wheels and the transmission line in real time. By performing deep separable convolution and dilated convolution processing on the image data obtained by the binocular camera, the DeepLabV3+ model can extract high-level features and generate segmentation output. In this process, the model uses an encoding-decoding structure and combines multi-scale contextual information to ensure the accuracy of gap recognition. Next, the PID controller receives the actual gap data fed back by the visual system and gradually reduces the actual gap by adjusting the contraction action of the walking arm. The PID controller adjusts the deviation between the expected gap and the actual gap in real time to ensure the smooth operation of the robot. When the gap is detected to be zero, the system triggers the tightening operation, and the tightening wheel rises under the drive of the motor to ensure that the walking arm is firmly pressed on the transmission line. Subsequently, the lifting mechanism is started, the rope lifting mechanism is lifted up, and it leaves the transmission line, and the line hanging operation is successfully completed.

[0147]

[0148] From the data in the table, it can be seen that the robot can accurately perform the wire hanging operation under different initial positions. The ranging accuracy of the binocular camera is always maintained within the range of 0.3-0.7cm, which fully verifies the stability and reliability of the visual system in high-voltage live environments. The gap recognition accuracy of the DeepLabV3+ algorithm is between 90.9% and 95.5%, showing extremely high image segmentation capabilities, especially when dealing with the fine gap between the robot's walking wheels and the transmission lines. High-precision recognition can be achieved. The number of adjustments to the PID controller varies with the initial gap, but the overall performance is stable, and the number of control adjustments is between 2-5 times, ensuring the response speed of the system.

[0149] By comparing the experimental data, it can be found that the present invention is superior to the prior art in many aspects. First, the combination of binocular camera and DeepLabV3+ algorithm greatly improves the accuracy of gap detection. It is difficult for existing monocular cameras or simple sensor systems to achieve such a high resolution, especially in the line hanging operation in complex environments. The accuracy of visual feedback directly affects the safety and efficiency of the operation. Secondly, the application of PID controller ensures the real-time adjustment of the walking arm contraction during the operation of the robot, avoiding the errors and delays that may be caused by traditional manual operation or non-adaptive control. The control system of the present invention can complete multiple precise adjustments in a short period of time, significantly improving the efficiency of the line hanging operation. Finally, the design of the equipotential arm solves the potential risks of electric field distortion and air breakdown, enabling the robot to safely perform operations in a high-voltage environment, enhancing the reliability and practicality of the system.

[0150] In summary, the present invention significantly improves the line hanging accuracy and operating efficiency of the robot in live operations through the innovative design of the visual servo control system, combined with deep learning algorithm and PID control, and has obvious innovation and advantages.

[0151] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A visual servo control method based on robot hanging line operation, characterized in that: include: The robot is started to approach the power transmission line and detects and performs potential transfer operations using the equipotential arm; The binocular camera is used to obtain the distance between the robot and the power transmission line in real time to determine whether the robot has reached the predetermined position for the line hanging operation; Based on the DeepLabV3+ semantic segmentation algorithm, the gap between the robot and the transmission line is identified to ensure that the robot's walking wheels are accurately aligned with the hanging position of the transmission line; A PID controller is used to adjust the retraction action of the robot's walking arm. When it is detected that the gap between the walking wheel and the transmission line is zero, the tightening operation is performed to complete the line hanging operation.

2. The visual servo control method based on robot hanging line operation according to claim 1, characterized in that: The potential transfer operation includes: when the robot rises to the equipotential safety distance of the transmission line, the equipotential arm rises, and the operating system analyzes the change in electric field strength in real time to ensure that the equipotential operation is completed before the air breaks down; When the system detects that the electric field strength is close to the critical breakdown value, the system immediately issues an alarm and the operator needs to take prompt action. After air breakdown occurs, the equipotential arm forms an electrical connection with the transmission line to ensure that the robot and the line reach an equipotential state and eliminate the potential difference.

3. The visual servo control method based on robot hanging line operation as claimed in claim 2, characterized in that: Determining whether the robot has reached the predetermined position for the line hanging operation includes: during the robot's ascent, the binocular camera captures the image of the transmission line in real time, and calculates the vertical distance between the robot and the transmission line through parallax; The vertical distance is compared with the set threshold to determine whether the predetermined position for the hanging line operation has been reached.

4. The visual servo control method based on robot hanging line operation as claimed in claim 3, characterized in that: Identifying the gap between the robot and the transmission line includes model building, model training, model evaluation and identification processes; The model building includes using the DeepLabV3+ architecture, including the Xception backbone network and the dilated spatial pyramid pooling module to build the DeepLabV3+ model; The training model includes enhancing and preprocessing the transmission line image captured in real time by the binocular camera, and marking the running wheels, transmission lines and the gap between them in the image to form a data set; Divide the dataset into training set, test set and validation set; Use the training set to train the DeepLabV3+ model, optimize the model parameters, select the loss function to evaluate the gap between the model output and the actual label, and perform gradient updates; Model evaluation includes evaluating the trained DeepLabV3+ model on the test set to ensure that the DeepLabV3+ model has high accuracy in identifying gaps; The differences between the prediction results of the DeepLabV3+ model and the actual markings are compared, and the performance of the DeepLabV3+ model in gap recognition is analyzed.

5. The visual servo control method based on robot hanging line operation according to claim 4, characterized in that: The recognition process includes inputting the real-time captured image into the trained DeepLabV3+ model; the DeepLabV3+ model extracts high-level features of the input image through the encoder part and generates a segmentation output of the same size as the input image through the decoder part; In the DeepLabV3+ model output, the areas of the running wheels, transmission lines and their gaps are marked, and the actual gap size is reflected through pixel-level segmentation results.

6. The visual servo control method based on robot hanging line operation according to claim 5, characterized in that: The PID controller regulates the retraction action of the robot's walking arm, including adjusting the gap and top line operation; The gap adjustment includes: a PID controller receiving the actual gap fed back by the visual inspection system in real time, and adjusting the deviation between the expected gap and the actual gap to gradually reduce the actual gap until the gap is detected to be zero.

7. The visual servo control method based on robot hanging line operation according to claim 6, characterized in that: The line-jacking operation includes, when the gap is detected to be zero, the tension wheel is gradually raised under the drive of the motor to ensure that the walking arm is completely pressed on the transmission line; During the tightening process, the photoelectric sensor continuously monitors the pressure between the travel wheel and the transmission line to ensure that no abnormal situation occurs; When the tensioning wheel is successfully tightened, the lifting mechanism will immediately lift the rope mechanism and leave the transmission line. The system confirms the tightening status and the stability of the walking arm through sensors to ensure the smooth completion of the line hanging operation.

8. A visual servo control system based on robot hanging line operation using the method according to any one of claims 1 to 7, characterized in that: The visual inspection module uses a binocular camera to perform real-time distance measurement to obtain the distance between the robot and the transmission line, and uses the DeepLabV3+ algorithm to identify the gap, thereby achieving accurate detection of the gap between the walking wheel and the transmission line. Potential transfer module, which handles the potential transfer process between the live working robot and the transmission line to avoid the risks of air breakdown and arc discharge; Binocular camera ranging module: The binocular camera calculates the distance between the robot and the transmission line through the principle of parallax and triangulation, ensuring the accuracy of equipotential operation and line hanging operation. The DeepLabV3+ semantic segmentation module uses deep convolutional neural networks and dilated convolution technology to perform semantic segmentation on the gap between the running wheels and the transmission lines, ensuring that the status of the running wheels can be accurately judged during line hanging operations. PID control module, based on the gap data of visual feedback, the PID controller adjusts the retraction operation of the walking arm in real time; The tightening and line hanging operation module, when the PID controller confirms that the gap is zero, performs the tightening operation, presses the walking arm firmly on the transmission line, completes the line hanging operation, and starts the jacking mechanism to lift the rope mechanism to prepare for subsequent operations. The sensor feedback module uses photoelectric sensors to monitor the status of the tension wheel and key parameters in the wire hanging process in real time to ensure the safety and accuracy of the operation.

9. A computer device comprising: Memory and processor; The memory stores a computer program, and is characterized in that: when the processor executes the computer program, the steps of the visual servo control method based on the robot hanging line operation are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the visual servo control method based on robot hanging line operation are implemented.

Images