A method and system for a drone-coordinated robot onboarding

Abstract

The application belongs to the technical field of power grid, and the method comprises the following steps: constructing a visual servo control framework based on double-layer structure predictive control to realize the motion control of the unmanned aerial vehicle, wherein the upper-layer control is used for realizing the trajectory planning of the motion trajectory of the unmanned aerial vehicle, and the lower-layer control is used for realizing the trajectory tracking of the motion trajectory of the unmanned aerial vehicle; determining the release control parameter of the robot released by the unmanned aerial vehicle and the wire grabbing control parameter of the robot according to the visual servo control framework; and the unmanned aerial vehicle executes the operation of releasing the robot according to the release control parameter, and the robot executes the operation of wire grabbing according to the wire grabbing control parameter. Through the construction of the visual servo control framework based on the trajectory planning and the trajectory tracking, the linkage cooperation between the robot and the unmanned aerial vehicle is realized, the operation efficiency is improved, the risk of adverse linkage is reduced, and the accuracy of the operation of the robot wire grabbing and the separation of the unmanned aerial vehicle is improved.

Description

Technical Field

[0001] This invention belongs to the field of power grid technology and relates to a method and system for deploying robots in collaboration with unmanned aerial vehicles (UAVs). Background Technology

[0002] Most power transmission lines are located outdoors, and complex weather conditions and the passage of time can damage them, creating safety hazards. Currently, the maintenance and inspection of power transmission lines must be carried out manually under high-risk conditions of high altitude, high voltage, and harsh environments. However, manual work is relatively inefficient, and with the widespread use of ultra-high voltage power grids and multi-circuit towers, voltage levels are increasing and phase-to-phase distances are getting closer, making live-line work extremely dangerous. There is an urgent need to develop methods to replace manual labor for safe and efficient live-line work. Now, robots can be used to replace manual labor in power transmission line-related tasks, offering high efficiency and low risk.

[0003] At the same time, since the power transmission lines cover a very large area, it is impossible to set up robots for operation in every location. Therefore, in reality, there is a need for a method to quickly transport robots to designated locations. As an unmanned vehicle, drones can lift objects of a certain weight to meet the needs of robot transportation. Their high speed can quickly transport robots to designated locations, and their large flight range can reduce the number of robots and drones that need to be configured.

[0004] However, drones and robots are not a single unit. Therefore, when drones transport robots to designated locations and deploy them to enable them to perform online operations, problems may arise where drones and robots are not coordinated and cannot properly grip the cable. This problem can lead to operation failure and damage to the robot or the cable itself. Summary of the Invention

[0005] In view of the aforementioned existing problems, this invention is proposed. By sensing the movement trajectory of the UAV through a robot and constructing a servo control framework based on trajectory planning and trajectory tracking, the efficiency of the two working together during subsequent release and line grabbing is improved. The movement of the UAV and the separation process of the robot can be separated, reducing the potential for undesirable linkage between the two. Based on the visual servo control framework, the parameters for the UAV release robot and the robot's line grabbing are determined, which can improve the transportation and separation of the UAV.

[0006] Therefore, a method for deploying robots in collaboration with drones is provided.

[0007] To address the aforementioned technical problems, this invention provides the following technical solution: a method for deploying robots in collaboration with unmanned aerial vehicles (UAVs), comprising:

[0008] A visual servo control framework based on two-layer predictive control is constructed to realize the motion control of a UAV. The upper layer control is used to realize trajectory planning of the UAV's motion trajectory, and the lower layer control is used to realize trajectory tracking of the UAV's motion trajectory. Based on the visual servo control framework, the release control parameters of the UAV releasing the robot and the gripping control parameters of the robot gripping the line are determined. The UAV executes the operation of releasing the robot according to the release control parameters, and the robot executes the operation of gripping the line according to the gripping control parameters.

[0009] As a preferred embodiment of the robot deployment method for drone collaboration described in this invention, the robot's perception of the drone's motion trajectory includes the following two cases: the robot carrying a motion sensing element obtains the drone's motion trajectory, and the drone carrying a motion sensing element acquires the drone's motion trajectory.

[0010] The robot, equipped with motion sensing elements, obtains the drone's motion trajectory by sensing changes in the robot's own motion state. The drone's motion trajectory is then calculated using a preprocessing formula, expressed as follows:

[0011]

[0012] The location of the drone is X. Q =[x Q ,y Q ,z Q ] T The robot's position is X L =[x L ,y L ,z L ] T l is the distance between the drone and the robot, θ is the robot's angle parameter, and φ is the angle parameter between the drone and the robot.

[0013] The acquisition of the drone's trajectory by the motion sensing element carried by the drone includes data interaction between the drone and the robot. The robot does not carry a motion sensing element, but learns the data output by the motion sensing element carried by the drone through interaction with the drone, and then calculates the trajectory according to the above preprocessing formula to obtain the drone's trajectory.

[0014] As a preferred embodiment of the drone-assisted robot deployment method described in this invention, the servo control framework includes a visual servo framework based on a two-layer predictive control structure. The motion control process consists of two layers: trajectory planning and trajectory tracking. The upper layer performs optimization processing for vibration suppression, disturbance resistance, and constraint handling in trajectory planning, while the lower layer only performs tracking control of the target trajectory. This allows the lower layer to complete the control of the drone releasing the robot and the robot grasping the line without making significant changes.

[0015] Among them, the constraint processing during the motion process includes mechanical constraints, visibility constraints, and control quantity constraints.

[0016] As a preferred embodiment of the drone-assisted robot deployment method described in this invention, the trajectory planning includes: visually identifying the target corresponding to the power transmission line; performing vibration suppression, anti-disturbance, and constraint processing based on the drone's motion trajectory; using the visual sensors mounted on the drone to identify the power transmission line and obtain its position and shape information; after visually identifying the power transmission line, calculating the distance d and angle parameter θ between the drone and the power transmission line by comparing the visually identified power transmission line with the drone's current position; performing trajectory planning based on the relative positional relationship between the drone and the power transmission line and the drone's motion trajectory; and outputting the planned trajectory data to the lower-level trajectory tracking layer after obtaining the desired trajectory.

[0017] The motion trajectory includes motion parameters, current environmental parameters, and preset task movement path parameters.

[0018] The trajectory planning data includes parameters for movement and specified constraints, which are the data required for the control process to release the robot and control the robot to grasp the line, thus realizing special operations.

[0019] As a preferred embodiment of the drone-assisted robot deployment method described in this invention, the trajectory tracking includes determining the parameters for the drone to release the robot and the robot to grasp the line based on the trajectory planning data and under motion constraints. After receiving the trajectory planning data transmitted by the trajectory planning layer, the trajectory tracking layer calculates the control input through a predictive control algorithm and sends it to the control systems of the drone and the robot to achieve tracking of the desired trajectory.

[0020] The control model of the drone is represented as follows:

[0021] u = [u1, u2, u3] 2

[0022] Based on the current state and the system model, the predicted state value for a future period is calculated as follows:

[0023] x p red(t+1) = f(x(t), u(t))

[0024] The predictive control algorithm is as follows:

[0025]

[0026] Where u1, u2, and u3 represent the throttle, servo, and pitch control of the UAV, f represents the dynamic model of the system, and xp red(t+1) is the predicted state value at a future time, x(t) is the state at the current time, u(t) is the control input at the current time, K1 and K2 are the controller parameters, and n is the size of the control window.

[0027] The motion constraints include: angle restrictions on robot joints, ensuring that the target corresponding to the power transmission line is always located in the image space, and speed restrictions on the actuators.

[0028] As a preferred embodiment of the drone-assisted robot deployment method described in this invention, the visual SLAM algorithm includes the following specific steps to solve the localization and map building problem of the robot when moving in an unknown environment: When the robot starts, a sensor data fusion process begins. After the data fusion of multiple sensors, including LiDAR and IMU, is completed, feature extraction is performed. The SIFT algorithm is used to extract feature points, resulting in feature points A1, A2, A3 and B1, B2, B3. The extracted feature data is matched with feature points in the map to establish data association and solve the robot's localization problem in an unknown environment. Taking A1 and B1 as an example, the distance d and angle difference Δθ between them are calculated.

[0029] d = sqrt((a2-a1)) 2 +(b2-b1) 2 )

[0030]

[0031] Where (a1,b1) and (a2,b2) are the coordinates of A1 and B1, respectively. The calculated distance and angle difference are compared with the set thresholds: when d≤X pixels and Δθ≤Y°, the matching relationship between A1 and B1 is considered reliable. If the matching relationship between A1 and B1 is taken as a reliable matching relationship, the output data is correlated. When d>X pixels and Δθ>Y°, the feature point relationship is considered mismatched, and the feature distance is adjusted. If the error is within an acceptable range, the matching relationship is output as the corrected parameters, and the matching ends.

[0032] Once the data association is established, motion estimation is performed to estimate the robot's trajectory in the environment, as well as its relative position and posture. The feature points and associated information extracted from the trajectory are integrated into the map to construct a map of the robot's environment for loop closure detection. When the robot loses its localization, it is relocated to re-determine its position on the map.

[0033] As a preferred embodiment of the drone-assisted robot deployment method described in this invention, the parameters for the robot to grasp the line include: after the visual servo control framework completes data analysis and control, the drone executes the robot release operation according to the output control parameter u(t) and outputs the corresponding control parameter P. S To ensure the robot safely detaches from the drone, after the robot is released by the drone, the robot will operate according to the control parameters P output by the visual servo control framework. S After adjusting its posture and position, the robotic arm grasps the power transmission line at a specified time, position, and angle. Once successfully grasped, the robotic arm adjusts its gripping force and angle based on control parameters output by the visual servo control framework to ensure stable operation on the power line. Following successful deployment, the visual servo control framework continues to generate corresponding control parameters P. S1 P S2 P S3 …P Sn This ensures that the robot can move stably on the power transmission line, and that the operation of adjusting its position and posture enables the robot to successfully complete its task.

[0034] Another objective of this invention is to provide a system for a drone-assisted robot deployment method. Through the servo control framework of the trajectory planning module and the trajectory tracking module, the movement of the drone and the separation process from the robot can be separated, reducing the possible undesirable linkage between the two; it increases the vibration suppression and disturbance resistance during the movement of the robotic arm; and it avoids the impact of adding constraints on the system's tracking accuracy.

[0035] A drone-assisted robot deployment system is characterized by comprising a construction module, a determination module, a release module, and a line-grabbing module.

[0036] The aforementioned construction module is used to build a visual servo control framework based on a two-layer predictive control structure to achieve motion control of the UAV.

[0037] The determining module is used to determine the release control parameters of the drone-released robot and the gripping control parameters of the robot gripping the line based on the visual servo control framework.

[0038] The release module is used to control the drone to perform the operation of releasing the robot according to the release control parameters.

[0039] The wire-grabbing module is used to control the robot to perform wire-grabbing operations according to the wire-grabbing control parameters.

[0040] A computer device includes a memory and a processor, the memory storing a computer program, characterized in that the processor executes the computer program to implement the steps of a method for deploying a drone-coordinated robot online.

[0041] A computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of a method for deploying a drone-coordinated robot online.

[0042] The beneficial effects of this invention are as follows: By sensing the movement trajectory of the drone through the robot and constructing a servo control framework based on trajectory planning and trajectory tracking, the movement states of the robot and the drone can be linked, improving the efficiency of their coordinated operation during subsequent release and line grabbing. The servo control framework based on trajectory planning and trajectory tracking allows the movement of the drone and the separation process from the robot to be separated, reducing potential adverse linkages between the two. Determining the parameters for drone release and robot line grabbing based on the visual servo control framework can improve the accuracy of drone transportation / separation and robot line grabbing operations, avoiding accidents caused by abnormal line grabbing. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0044] Figure 1 This is a flowchart illustrating a method for deploying robots in collaboration with drones, as provided in one embodiment of the present invention.

[0045] Figure 2 This is a schematic diagram of the workflow of a drone-coordinated robot deployment system according to an embodiment of the present invention. Detailed Implementation

[0046] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0047] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0048] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it an embodiment that is mutually exclusive, either alone or selectively, with other embodiments.

[0049] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0050] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0051] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0052] Example 1

[0053] Reference Figure 1 This is the first embodiment of the present invention, which provides a method for deploying robots in collaboration with unmanned aerial vehicles (UAVs), including:

[0054] S1: Construct a visual servo control framework based on two-layer predictive control to realize the motion control of the UAV. The upper layer control is used to realize the trajectory planning of the UAV's motion trajectory, and the lower layer control is used to realize the trajectory tracking of the UAV's motion trajectory.

[0055] Further, the robot is a specially designed automated apparatus or automated device for performing designated on-line operations. The on-line operations refer to operations performed by the robot on a power transmission line or suspended under the line on the power transmission line or other related power transmission equipment / facilities. In addition to various mechanisms and circuits for performing on-line operations, the robot is also provided with motion sensing elements for sensing the unmanned aerial vehicle, such as a three-axis sensor, an acceleration sensor, a speed sensor, a gyroscope, etc. The change of the motion state of the robot itself can be sensed through the motion sensing elements.

[0056] It should be noted that the motion trajectory of the unmanned aerial vehicle sensed by the robot includes the following two cases: the motion trajectory of the unmanned aerial vehicle obtained by the robot carrying the motion sensing element, and the motion trajectory of the unmanned aerial vehicle acquired by the motion sensing element carried by the unmanned aerial vehicle; that the motion trajectory of the unmanned aerial vehicle obtained by the robot carrying the motion sensing element includes: sensing, by the motion sensing element, the change of the motion state of the robot itself by the unmanned aerial vehicle, and calculating according to a preprocessing formula to obtain the motion trajectory of the unmanned aerial vehicle, which is expressed as:

[0057]

[0058] Wherein, the position of the unmanned aerial vehicle is X Q =[x Q ,y Q ,z Q T , the position of the robot is X L =[x L ,y L ,z L T , l is the distance between the unmanned aerial vehicle and the robot, θ is an angle parameter of the robot, and φ is an angle parameter between the unmanned aerial vehicle and the robot; that the motion trajectory of the unmanned aerial vehicle acquired by the motion sensing element carried by the unmanned aerial vehicle includes: data interaction exists between the unmanned aerial vehicle and the robot, the robot does not carry the motion sensing element, and knows the data output by the motion sensing element carried by the unmanned aerial vehicle through interaction with the unmanned aerial vehicle, so that the motion trajectory is calculated according to the preprocessing formula to obtain the motion trajectory of the unmanned aerial vehicle.

[0059] It should also be noted that the on-line operation of the robot generally includes two important links: motion control, and robot releasing and wire-grasping control. If the two stages are processed at the same time, the data processing will have very high requirements, and the linkage effect will not be good. To this end, a servo control framework based on trajectory planning and trajectory tracking is constructed, which is divided into two stages for separate processing, so that the motion states between the robot and the unmanned aerial vehicle can be linked, the efficiency of the cooperative operation of the two during subsequent releasing and wire-grasping is improved, and through the servo control framework of trajectory planning and trajectory tracking, the motion of the unmanned aerial vehicle and the process of separating the robot can be separated, so as to reduce possible adverse linkage between the two.​​

[0060] It should also be noted that the servo control framework includes a visual servo framework based on a two-layer predictive control structure. The motion control process consists of two layers: trajectory planning and trajectory tracking. The upper layer performs optimization processing for vibration suppression, disturbance rejection, and constraint handling in trajectory planning, while the lower layer only performs tracking control of the target trajectory, enabling the lower layer to complete the motion control of the robotic arm without making significant changes. Its motion control process consists of two layers: trajectory planning and trajectory tracking. Each layer has different control targets, planners, and controllers to achieve fast and stable tracking of the target trajectory by the robotic arm. The main advantages of adopting a layered structure are: firstly, it increases the vibration suppression and disturbance rejection capabilities during the robotic arm's movement; secondly, it avoids the impact of adding constraints on the system's tracking accuracy.

[0061] The constraints during motion include mechanical constraints, visibility constraints, and control constraints. Mechanical constraints are the angle limitations of the joints; visibility constraints ensure that the target remains in the image space; and control constraints limit the speed of the actuators. In the two-layer structure design, vibration suppression, disturbance rejection, and constraint processing are optimized in the upper layer, while the lower layer only performs tracking control of the target trajectory. This allows the lower layer to complete the motion control of the robotic arm without making significant changes.

[0062] In the design of the tracking controller, since factors such as torque and target position have already been considered in the upper-level trajectory planner, and a reference trajectory that meets the constraints and control requirements has been generated, these factors can be ignored for the time being. Only accurate tracking and improved calculation speed need to be considered. The two layers of controllers each have their own focus and are interconnected, working together to determine the optimal timing for the separation of the robot and the drone, and jointly ensuring that the robotic arm moves quickly and stably to the target position to complete the line grabbing and loading.

[0063] S2: Determine the release control parameters of the drone releasing the robot and the gripping control parameters of the robot gripping the line based on the visual servo control framework.

[0064] Furthermore, once the visual servo control framework completes data analysis and control, and outputs corresponding control parameters to control the release of the robot from the drone and the robot's line-grabbing operation, it can improve the accuracy of drone transportation / separation and robot line-grabbing operations, and avoid accidents caused by abnormal line-grabbing.

[0065] Furthermore, different frameworks have specific processing structures, which consist of processors, sensors, and auxiliary circuits. Trajectory planning involves visually identifying targets corresponding to power transmission lines. Obtaining targets via vision is a mature technology that can reduce the cost of practical application. Considering that the electromagnetic environment associated with power transmission lines can interfere with identification methods such as target detection using reflected waves, visual identification is the most reliable approach. The targets on the power transmission lines are determined based on the tasks to be performed at that time.

[0066] The trajectory planning includes visually identifying the target corresponding to the power transmission line, and performing vibration suppression, anti-disturbance, and constraint processing based on the drone's motion trajectory: using the visual sensors on the drone to identify the power transmission line and obtain its position and shape information; after visually identifying the power transmission line, comparing the visually identified power transmission line with the drone's current position to calculate the distance d and angle parameter θ between the drone and the power transmission line; and performing trajectory planning based on the relative positional relationship between the drone and the power transmission line and the drone's motion trajectory to obtain the desired trajectory. After obtaining the planned trajectory data, the planned trajectory data is output to the lower-level trajectory tracking layer.

[0067] The motion trajectory includes motion parameters, current environmental parameters, and preset operational path parameters. The operational path parameters reveal the original motion plan and allow comparison with existing parameters to determine if the drone will fly along the preset path. Current environmental parameters identify various factors affecting drone flight, such as wind speed, wind direction, and humidity. Based on these parameters and the motion parameters, corresponding motion parameters can be modified to suppress vibrations during flight and resist disturbances caused by external factors. The motion parameters and specified constraints enable the control processes required for specific operations. In this scheme, the data for releasing the robot and controlling its gripping of the cable, thus enabling the control processes required for specific operations, constitutes the trajectory planning data.

[0068] It should be noted that the trajectory tracking includes determining the parameters for the UAV to release the robot and the robot to grasp the line based on the trajectory planning data and under motion constraints. After receiving the trajectory planning data transmitted by the trajectory planning layer, the trajectory tracking layer calculates the control input through a predictive control algorithm and sends it to the control systems of the UAV and the robot to achieve tracking of the desired trajectory.

[0069] The control model of the drone is represented as follows:

[0070] u = [u1, u2, u3] 2

[0071] Based on the current state and the system model, the predicted state value for a future period is calculated as follows:

[0072] x p red(t+1) = f(x(t), u(t))

[0073] The predictive control algorithm is as follows:

[0074]

[0075] Where u1, u2, and u3 represent the throttle, servo, and pitch control of the UAV, f represents the dynamic model of the system, and x p red(t+1) is the predicted state value at a future time, x(t) is the state at the current time, u(t) is the control input at the current time, K1 and K2 are the controller parameters, and n is the size of the control window.

[0076] The motion constraints include: angle restrictions on robot joints, ensuring that the target corresponding to the power transmission line is always located in the image space, and speed restrictions on the actuators.

[0077] It should also be noted that after the visual servo control framework completes data analysis and control, the UAV executes the robot release operation based on the output control parameter u(t) and outputs the corresponding control parameter P. S To ensure the robot safely detaches from the drone, after the robot is released by the drone, the robot will operate according to the control parameters P output by the visual servo control framework. S After adjusting its posture and position, the robotic arm grasps the power transmission line at a specified time, position, and angle. Once successfully grasped, the robotic arm adjusts its gripping force and angle based on control parameters output by the visual servo control framework to ensure stable operation on the power line. Following successful deployment, the visual servo control framework continues to generate corresponding control parameters P. S1 P S2 P S3 …P Sn This ensures that the robot can move stably on the power transmission line, and that the operation of adjusting its position and posture enables the robot to successfully complete its task.

[0078] S3: The drone executes the release robot operation according to the release control parameters, and the robot executes the wire grabbing operation according to the wire grabbing control parameters.

[0079] Furthermore, SLAM is an abbreviation for Simultaneous localization and mapping, which mainly solves the problem of robot localization and mapping when moving in unknown environments by involving steps such as sensor data fusion, feature extraction, data association, motion estimation, optimization, map building, loop closure detection, and relocalization.

[0080] It should be noted that the visual SLAM algorithm addresses the localization and mapping problem of robots moving in unknown environments through the following specific steps: When the robot starts, a sensor data fusion process begins. After data fusion from multiple sensors, including LiDAR and IMU, is completed, feature extraction is performed. The SIFT algorithm is used to extract feature points, resulting in feature points A1, A2, A3 and B1, B2, B3. The extracted feature data is then matched with feature points in the map to establish data associations, thus solving the robot's localization problem in unknown environments. Taking A1 and B1 as an example, the distance d and angle difference Δθ between them are calculated.

[0081] d = sqrt((a2-a1)) 2 +(b2-b1) 2 )

[0082]

[0083] Where (a1,b1) and (a2,b2) are the coordinates of A1 and B1, respectively. The calculated distance and angle difference are compared with the set thresholds: when d≤X pixels and Δθ≤Y°, the matching relationship between A1 and B1 is considered reliable. If the matching relationship between A1 and B1 is taken as a reliable matching relationship, the output data is correlated. When d>X pixels and Δθ>Y°, the feature point relationship is considered mismatched, and the feature distance is adjusted. If the error is within an acceptable range, the matching relationship is output as the corrected parameters, and the matching ends.

[0084] Once the data association is established, motion estimation is performed to estimate the robot's trajectory in the environment, as well as its relative position and posture. The feature points and associated information extracted from the trajectory are integrated into the map to construct a map of the robot's environment for loop closure detection. When the robot loses its localization, it is relocated to re-determine its position on the map.

[0085] Example 2

[0086] Reference Figure 1 As an embodiment of the present invention, a method for deploying robots in collaboration with unmanned aerial vehicles is provided. To verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.

[0087] Based on the above, the following tests were conducted. The threshold in this invention can be adjusted according to actual circumstances:

[0088] The robot captures the drone's motion trajectory using a camera; a servo control framework based on trajectory planning and tracking controls the robot arm to complete the line-grabbing and loading actions; the drone moves according to a visual SLAM algorithm, cooperating with the robot to complete the task; key parameters during the experiment are recorded, such as drone speed, altitude, and robot line-grabbing time; the experiment is repeated multiple times, and the average value is taken to improve the reliability of the experimental data. The experimental results are shown in the table below:

[0089] Table 1

[0090] Experiment 1 10 50 8.5 95 Experiment 2 15 60 9.5 90 Experiment 3 20 70 11.0 85 Experiment 4 25 80 12.5 80 Experiment 5 30 90 14.0 75

[0091] The table above shows that as the speed of the drone increases, the difficulty of drone-robot collaborative operation increases. However, experimental data shows that within a reasonable speed range, the method of the present invention still has a high success rate in grasping the line. The drone's altitude has little impact on the success rate of grasping the line, indicating that the method of the present invention has strong adaptability within a certain altitude range. The robot's line grasping time increases slightly with the increase of the drone's speed, but remains within an acceptable range overall.

[0092] The UAV and robot collaborative operation method based on the visual servo control framework of the present invention has shown high line grabbing success rate and strong adaptability in experiments. It can be applied to UAV and robot collaborative operation in the power grid field to improve operation efficiency and accuracy.

[0093] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0094] Example 3

[0095] The third embodiment of the present invention differs from the first two embodiments in that:

[0096] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0097] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0098] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0099] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0100] Example 4

[0101] Reference Figure 2 This is the fourth embodiment of the present invention. This embodiment provides a robot deployment system for drone collaboration, including a construction module, a determination module, a release module and a line-grabbing module.

[0102] The building blocks are used to construct a visual servo control framework based on two-layer predictive control to achieve motion control of UAVs.

[0103] The determination module is used to determine the release control parameters of the drone-launched robot and the gripping control parameters of the robot gripping the wire, based on the visual servo control framework.

[0104] The release module is used to control the drone to perform the operation of releasing the robot according to the release control parameters.

[0105] The wire-grabbing module is used to control the robot to perform wire-grabbing operations according to the wire-grabbing control parameters.

[0106] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for deploying robots in collaboration with unmanned aerial vehicles (UAVs), characterized in that: include, A visual servo control framework based on two-layer predictive control is constructed to realize the motion control of UAVs. The upper layer control is used to realize trajectory planning of UAVs, and the lower layer control is used to realize trajectory tracking of UAVs. The release control parameters for the drone-launched robot and the gripping control parameters for the robot's gripping line are determined based on the visual servo control framework. The drone executes the release robot operation according to the release control parameters, and the robot executes the wire-grabbing operation according to the wire-grabbing control parameters; The robot's perception of the drone's motion trajectory includes the following two cases: the robot carrying a motion sensing element obtains the drone's motion trajectory, and the drone carrying a motion sensing element obtains the drone's motion trajectory. The robot, equipped with motion sensing elements, obtains the drone's motion trajectory by sensing changes in the robot's own motion state. The drone's motion trajectory is then calculated using a preprocessing formula, expressed as follows: The location of the drone is The robot's location is l is the distance between the drone and the robot. For robot angle parameters, The angle parameters between the drone and the robot; The acquisition of the drone's motion trajectory by the motion sensing element carried by the drone includes data interaction between the drone and the robot. The robot does not carry a motion sensing element, but it knows the data output by the motion sensing element carried by the drone through interaction with the drone. The robot then calculates the motion trajectory according to the above preprocessing formula to obtain the drone's motion trajectory. The servo control framework includes a visual servo framework based on a two-layer predictive control structure. The motion control process consists of two layers: trajectory planning and trajectory tracking. The upper layer performs optimization processing for vibration suppression, disturbance rejection, and constraint handling in trajectory planning, while the lower layer only performs tracking control of the target trajectory. This allows the lower layer to complete the control of the drone releasing the robot and the robot grasping the line without making significant changes. Among them, the constraint processing during the motion process includes mechanical constraints, visibility constraints, and control quantity constraints; The trajectory planning includes visually identifying targets corresponding to power transmission lines, and performing vibration suppression, disturbance rejection, and constraint processing based on the drone's motion trajectory. This involves using visual sensors onboard the drone to identify the power transmission lines, acquiring their position and shape information, and then comparing the identified lines with the drone's current position to calculate the distance *d* and angle parameters between the drone and the power transmission lines. Based on the relative position of the drone and the power transmission line, as well as the drone's movement trajectory, trajectory planning is performed. After obtaining the desired trajectory, the planned trajectory data is output to the lower trajectory tracking layer. The motion trajectory includes motion parameters, current environmental parameters, and preset job movement path parameters; The trajectory planning data includes data on the control process required for releasing the robot and controlling the robot to grasp the line, through parameters of motion and specified constraints, to achieve special operations. The trajectory tracking includes determining the parameters for the drone to release the robot and the robot to grasp the line based on the trajectory planning data and under motion constraints. After receiving the trajectory planning data from the trajectory planning layer, the trajectory tracking layer calculates the control input through a predictive control algorithm and sends it to the control systems of the drone and the robot to achieve tracking of the desired trajectory. The control model of the drone is represented as follows: Based on the current state and the system model, the predicted state value for a future period is calculated as follows: The predictive control algorithm is as follows: in, Let f represent the throttle, servo, and pitch control of the UAV, and let f be the dynamic model of the system. This is a predicted state value for a future time point. The current state. Current control input, , Here are the parameters for the controller, and n is the size of the control window; The motion constraints include: angle restrictions on robot joints, ensuring that the target corresponding to the power transmission line is always located in the image space, and speed restrictions on the actuators.

2. The method for deploying robots in collaboration with unmanned aerial vehicles (UAVs) as described in claim 1, characterized in that: The drone performs the operation of releasing the robot according to the release control parameters, including solving the problem of robot localization and map building when moving in an unknown environment through the following specific steps: When the robot starts, it begins the sensor data fusion process. After the data from multiple sensors—LiDAR and IMU—is fused, feature extraction is performed. The SIFT algorithm is used to extract feature points, resulting in feature points A1, A2, A3 and B1, B2, B3. The extracted feature data is then matched with feature points in the map to establish data associations, solving the robot's localization problem in unknown environments. Taking A1 and B1 as an example, the distance d and angle difference between them are calculated. : = arctan in( , )and( , The coordinates of A1 and B1 are given, respectively. The calculated distance and angle difference are compared with a set threshold: when... pixels and If the matching relationship between A1 and B1 is considered reliable, then the output data is associated with the data. pixels and If the feature point relationship is mismatched, the feature distance is adjusted. If the error is within an acceptable range, the matching relationship is output as the corrected parameter, and the matching ends. Once the data association is established, motion estimation is performed to estimate the robot's trajectory in the environment, as well as its relative position and posture. The feature points and associated information extracted from the trajectory are integrated into the map to construct a map of the robot's environment for loop closure detection. When the robot loses its localization, it is relocated to re-determine its position on the map.

3. The method for deploying robots in collaboration with unmanned aerial vehicles (UAVs) as described in claim 2, characterized in that: The parameters for the robot's line-grabbing include the control parameters output by the UAV after the visual servo control framework has completed data analysis and control. Execute the operation to release the robot and output the corresponding control parameter P. S To ensure the robot safely detaches from the drone, after the robot is released by the drone, the robot will operate according to the control parameters P output by the visual servo control framework. S After adjusting its posture and position, the robotic arm grasps the power transmission line at a specified time, position, and angle. Once successfully grasped, the robotic arm adjusts its gripping force and angle based on control parameters output by the visual servo control framework to ensure stable operation on the power line. Following successful deployment, the visual servo control framework continues to generate corresponding control parameters P. S1 P S2 P S3 P Sn This ensures that the robot can move stably on the power transmission line, and that the operation of adjusting its position and posture enables the robot to successfully complete its task.

4. A system employing the drone-assisted robot deployment method as described in any one of claims 1 to 3, characterized in that: Includes a build module, a determine module, a release module, and a capture module; The construction module is used to build a visual servo control framework based on two-layer predictive control to achieve motion control of the UAV. The determining module is used to determine the release control parameters of the drone-released robot and the gripping control parameters of the robot gripping the line based on the visual servo control framework. The release module is used to control the drone to perform the operation of releasing the robot according to the release control parameters; The wire-grabbing module is used to control the robot to perform wire-grabbing operations according to the wire-grabbing control parameters.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3.

Images