A vision-based positioning-based unmanned aerial vehicle (UAV) hoisting control system and method for photovoltaic panels.

Abstract

This application provides a vision-based positioning-based unmanned aerial vehicle (UAV) hoisting control system and method for photovoltaic (PV) panels. The system controls the UAV to fly above the target PV panel and acquires real-time images of the panel's installation frame. Based on the PV panel's dynamic response identifier in the installation frame image, pose drift is estimated to obtain pose drift estimation features. Visual servo control is then performed based on these features to ensure the servo positioning drift is less than a drift threshold. Dynamic motion data of the UAV's robotic arm end effector is acquired. An expected impedance controller is constructed using the dynamic motion data and a particle swarm optimization algorithm. This controller is then used for compliant control, generating dynamic compliant commands for the UAV. Motion control of the UAV is then applied according to these commands, enabling it to complete the hoisting and hooking of the target PV panel. The technical solution provided in this application can achieve precise positioning and hooking of the target PV panel using visual servo control.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) control technology, and more specifically, to a vision-based positioning-based UAV hoisting control system and method for photovoltaic panels. Background Technology

[0002] Visual positioning-based drone hoisting control for photovoltaic panels is a complex technology integrating computer vision, drone flight control, robotic arm operation, path planning, and dynamics modeling. It aims to achieve high-precision positioning, transportation, and installation of photovoltaic panels with little or no human intervention. During execution, visual servo control, as a key technology, dynamically adjusts the drone's flight attitude and the robotic arm's end effector position through real-time image feature and pose information feedback, ensuring that the photovoltaic panel hoisting point remains highly aligned with the target installation point.

[0003] However, in existing technologies, high-precision positioning in complex environments still faces challenges, especially under conditions such as strong light, reflection, dust, rain, and snow, where visual features are easily lost, leading to pose drift or recognition errors. This places extremely high demands on the attitude stability and positioning control accuracy of hoisting drones. Furthermore, the parameter adaptation of compliant control and the force control safety strategy during the contact process still need to ensure efficiency and stability while taking into account the differences in installation postures and load characteristics. Therefore, how to achieve precise positioning of target photovoltaic panels and hoisting hooks in visual servo control is a difficult problem facing the industry. Summary of the Invention

[0004] This application provides a vision-based photovoltaic panel drone hoisting control system and method, which can achieve precise positioning and hoisting hook connection of target photovoltaic panels through vision servo control.

[0005] In a first aspect, this application provides a vision-based positioning-based unmanned aerial vehicle (UAV) hoisting control method for photovoltaic panels, comprising the following steps:

[0006] Control the hoisting drone to fly from the current position to above the target photovoltaic panel, and collect images of the installation frame of the target photovoltaic panel in real time after the hoisting drone hovers;

[0007] Based on the photovoltaic panel dynamic response identification code in the installation frame image, pose drift estimation is performed to obtain the pose drift estimation features of the hoisting drone. Based on the pose drift estimation features, visual servo control is performed on the hoisting drone, thereby making the servo positioning drift of the hoisting drone in visual servo control less than the drift threshold.

[0008] The dynamic motion data of the end effector of the hoisting drone is acquired. The desired impedance controller of the end effector of the hoisting drone is constructed by the dynamic motion data and the particle swarm algorithm. The desired impedance controller is used to perform compliant control, thereby generating dynamic compliant commands for the hoisting drone.

[0009] The hoisting drone is motion controlled according to the dynamic compliant command, thereby enabling the hoisting drone to complete the hoisting and hooking of the target photovoltaic panel.

[0010] In some embodiments, controlling the hoisting drone to fly from its current position to above the target photovoltaic panel specifically includes:

[0011] Obtain the current position of the hoisting drone and the current position of the target photovoltaic panel in the world coordinate system;

[0012] The hovering point of the hoisting drone is set based on the current position of the target photovoltaic panel;

[0013] The path planner of the hoisting drone generates collision-free flight information for the hoisting drone based on the hovering point;

[0014] Based on the collision-free flight information, the hoisting drone is controlled to fly from its current position to above the target photovoltaic panel.

[0015] In some embodiments, the pose drift estimation based on the photovoltaic panel dynamic response identifier code in the installation frame image to obtain the pose drift estimation features of the hoisting drone specifically includes:

[0016] Distortion correction is performed on the installation frame image, and then the photovoltaic panel dynamic response identification code of the target photovoltaic panel is extracted from the corrected installation frame image;

[0017] Corner detection is performed on the dynamic response identifier code of the photovoltaic panel to obtain the detected corner sequence of the dynamic response identifier code of the photovoltaic panel;

[0018] Visual drift estimation is performed based on the detected corner point sequence, thereby obtaining the visual drift vector and visual rotation vector between the hoisting drone and the dynamic response identification code of the photovoltaic panel;

[0019] The pose drift estimation features of the hoisting UAV are constructed based on the visual drift vector and the visual rotation vector.

[0020] In some embodiments, visual servo control of the hoisting drone based on the pose drift estimation features specifically includes:

[0021] The pose drift estimation features are subjected to drift transformation to obtain the servo positioning drift amount of the hoisting UAV;

[0022] A servo positioning controller for the hoisting drone is constructed using the servo positioning drift amount and particle swarm algorithm.

[0023] The servo positioning control quantity for the hoisting drone is generated based on the servo positioning controller.

[0024] The servo positioning control quantity is sent to the underlying flight controller of the hoisting drone, thereby completing the visual servo control of the hoisting drone.

[0025] In some embodiments, the dynamic motion data of the end effector of the hoisting drone includes the real-time position, speed, and acceleration of the end effector.

[0026] In some embodiments, constructing the desired impedance controller at the end of the hoisting drone's robotic arm using the dynamic motion data and particle swarm optimization algorithm specifically includes:

[0027] The particle swarm optimization algorithm is used to optimize the compliant coefficient sequence, and the optimization result of the compliant coefficient sequence is obtained.

[0028] Based on the optimization results of the compliance coefficient sequence and the dynamic motion data, a desired impedance controller for the end effector of the hoisting drone's robotic arm is constructed.

[0029] In some embodiments, motion control of the hoisting drone according to the dynamic compliance command involves sending the dynamic compliance command to the motion controller of the hoisting drone, and then the motion controller controls the hoisting drone to adjust its posture according to the dynamic compliance command.

[0030] Secondly, this application provides a vision-based positioning-based photovoltaic panel drone hoisting control system for executing a vision-based positioning-based photovoltaic panel drone hoisting control method, including:

[0031] The image acquisition module is used to control the hoisting drone to fly from the current position to above the target photovoltaic panel, and to acquire images of the installation frame of the target photovoltaic panel in real time after the hoisting drone hovers.

[0032] The visual positioning module is used to estimate the pose drift based on the photovoltaic panel dynamic response identification code in the installation frame image, obtain the pose drift estimation features of the hoisting drone, and perform visual servo control on the hoisting drone based on the pose drift estimation features, so that the servo positioning drift of the hoisting drone in visual servo control is less than the drift threshold.

[0033] The compliant control module is used to acquire dynamic motion data of the end effector of the hoisting drone's robotic arm, construct a desired impedance controller for the end effector of the hoisting drone's robotic arm using the dynamic motion data and particle swarm optimization algorithm, and use the desired impedance controller to perform compliant control, thereby generating dynamic compliant commands for the hoisting drone.

[0034] The motion control module is used to control the motion of the hoisting drone according to the dynamic compliant command, thereby enabling the hoisting drone to complete the hoisting hook of the target photovoltaic panel.

[0035] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described vision-based positioning-based photovoltaic panel drone hoisting control method.

[0036] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described vision-based positioning-based UAV hoisting control method for photovoltaic panels.

[0037] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0038] This application provides a vision-based positioning-based photovoltaic panel drone hoisting control system and method. The system controls the drone to fly from its current position to above the target photovoltaic panel, and after hovering, it acquires real-time images of the photovoltaic panel's installation frame. Based on the photovoltaic panel's dynamic response identifier in the installation frame image, pose drift estimation is performed to obtain the drone's pose drift estimation features. Based on these features, visual servo control is applied to the drone, ensuring that the servo positioning drift is less than a drift threshold. Dynamic motion data of the drone's robotic arm end effector is acquired. A desired impedance controller for the drone's robotic arm end effector is constructed using this dynamic motion data and a particle swarm optimization algorithm. The desired impedance controller is used for compliant control, generating dynamic compliant commands for the drone. Motion control is applied to the drone according to these commands, enabling it to complete the hoisting and hooking of the target photovoltaic panel.

[0039] Therefore, in this application, firstly, by estimating the pose drift based on the dynamic response identifier of the photovoltaic panel in the installation frame image, and using the obtained pose drift estimation features for visual servo control, the small displacement and attitude deviation of the UAV relative to the target photovoltaic panel can be perceived in real time and accurately, thereby enabling feedforward compensation and closed-loop fine-tuning in the control loop. Secondly, by acquiring the dynamic motion data of the end effector of the hoisting UAV and optimizing the desired impedance controller using the particle swarm optimization algorithm, the end effector can be made to have adaptive compliance characteristics. This not only improves the safety and dynamic adaptability of the end effector in complex working environments, but also complements the visual servo control, thereby effectively improving the positioning accuracy of the target photovoltaic panel. Finally, the hoisting UAV is motion controlled according to the dynamic compliance command, enabling it to achieve high-precision pose adjustment and adaptive compliance response when approaching the target photovoltaic panel and performing hooking operations. Visual servo provides high-precision spatial positioning and drift correction, while dynamic compliance motion ensures safe and stable end action, achieving compliant approach and reliable hooking to the target hooking point, significantly improving positioning accuracy and the success rate of hoisting hooking operations.

[0040] In summary, the technical solution adopted in this application can achieve precise positioning and hoisting hooks for target photovoltaic panels in visual servo control. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this embodiment of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is an exemplary flowchart of a vision-based positioning-based unmanned aerial vehicle (UAV) hoisting control method for photovoltaic panels, as shown in some embodiments of this application.

[0043] Figure 2 This is an exemplary flowchart illustrating visual servo control of a hoisting drone according to some embodiments of this application;

[0044] Figure 3 This is a schematic diagram of the structure of a vision-based positioning-based photovoltaic panel drone hoisting control system according to some embodiments of this application;

[0045] Figure 4 This is a schematic diagram of the structure of a computer device for implementing a vision-based positioning-based unmanned aerial vehicle (UAV) hoisting control method for photovoltaic panels, according to some embodiments of this application. Detailed Implementation

[0046] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0047] This application provides a vision-based positioning-based photovoltaic panel drone hoisting control system and method. The core of the system is to control the drone to fly from its current position to above the target photovoltaic panel, and to acquire real-time images of the installation frame of the target photovoltaic panel after the drone hovers. Based on the dynamic response identifier code of the photovoltaic panel in the installation frame image, pose drift estimation is performed to obtain the pose drift estimation features of the drone. Based on these features, visual servo control is applied to the drone, ensuring that the servo positioning drift is less than a drift threshold. Dynamic motion data of the drone's robotic arm end effector is acquired. A desired impedance controller for the end effector is constructed using this dynamic motion data and a particle swarm optimization algorithm. The desired impedance controller is used for compliant control, generating dynamic compliant commands for the drone. Motion control is applied to the drone according to these commands, enabling it to complete the hoisting and hooking of the target photovoltaic panel. This method enables precise positioning and hooking of the target photovoltaic panel using visual servo control.

[0048] To better understand the above technical solutions, a detailed description of the technical solutions will be provided below in conjunction with the accompanying drawings and specific embodiments. (Refer to...) Figure 1 The figure is an exemplary flowchart of a vision-based positioning-based UAV hoisting control method for photovoltaic panels, according to some embodiments of this application. The figure mainly includes the following steps:

[0049] In step S101, the hoisting drone is controlled to fly from its current position to above the target photovoltaic panel, and the installation frame image of the target photovoltaic panel is acquired in real time after the hoisting drone hovers.

[0050] In some embodiments, controlling the hoisting drone to fly from its current position to above the target photovoltaic panel can be achieved in the following ways:

[0051] Obtain the current position of the hoisting drone and the current position of the target photovoltaic panel in the world coordinate system;

[0052] The hovering point of the hoisting drone is set based on the current position of the target photovoltaic panel;

[0053] The path planner of the hoisting drone generates collision-free flight information for the hoisting drone based on the hovering point;

[0054] Based on the collision-free flight information, the hoisting drone is controlled to fly from its current position to above the target photovoltaic panel.

[0055] In practical implementation, firstly, the onboard IMU, RTK / GNSS, visual odometry, and other sensors of the hoisting drone can be tightly coupled to a state estimator (such as an extended Kalman filter or factor graph optimizer) in the world coordinate system through time synchronization. This outputs the current position of the hoisting drone, while simultaneously using an onboard camera to detect the target area, thus obtaining the current position of the target photovoltaic panel in the world coordinate system. Then, the hovering point of the hoisting drone can be set based on the current position of the target photovoltaic panel. Specifically, 2m in the normal direction of the current position of the target photovoltaic panel is defined as the hovering point of the hoisting drone. In actual implementation, the safety distance in the normal direction can be adaptively adjusted according to the slope and clamp size, and is not limited here. Furthermore, a hoisting... The path planner for the hoisting drone generates collision-free flight information based on the hovering point. This involves inputting the hovering point's location information into the hoisting drone's path planner as the flight endpoint. Within the feasible space, using drone dynamics, attitude tilt angle, maximum speed / acceleration, and load-introduced swing constraints as boundary conditions, it solves for the collision-free trajectory from the current position to the flight endpoint. These boundary conditions and the collision-free trajectory then constitute the hoisting drone's collision-free flight information. Finally, based on this collision-free flight information, the hoisting drone can be controlled to fly from its current position to above the target photovoltaic panel. This involves inputting the collision-free flight information into the hoisting drone's motion controller for trajectory tracking, thereby controlling the hoisting drone to fly from its current position to above the target photovoltaic panel, i.e., to reach the hovering point.

[0056] In some embodiments, the installation frame image of the target photovoltaic panel is acquired in real time after the hoisting drone hovers. Specifically, after controlling the hoisting drone to hover at the hovering point, the high-definition camera on the hoisting drone can be used to take real-time pictures of the installation frame of the target photovoltaic panel, thereby obtaining the installation frame image of the target photovoltaic panel.

[0057] In step S102, pose drift estimation is performed based on the photovoltaic panel dynamic response identification code in the installation frame image to obtain the pose drift estimation features of the hoisting drone. Based on the pose drift estimation features, visual servo control is performed on the hoisting drone, thereby making the servo positioning drift of the hoisting drone in visual servo control less than the drift threshold.

[0058] It should be noted that in this application, the installation frame image of the target photovoltaic panel includes a photovoltaic panel dynamic response identification code. The outer structure of the photovoltaic panel dynamic response identification code is a large solid black square border, which can help to quickly find the outline in the image and initially lock the approximate area of ​​the mark. It can be detected even at a distance. The inner structure of the photovoltaic panel dynamic response identification code adopts the AprilTag3 encoding format to uniquely identify each photovoltaic panel and can store simple information. It has stronger anti-blurring and anti-occlusion capabilities, and high-contrast circular feature points are embedded in the four corners and the center of the internal encoding area.

[0059] In some embodiments, the pose drift estimation of the hoisting drone can be performed based on the photovoltaic panel dynamic response identifier code in the installation frame image to obtain the pose drift estimation features. Specifically, this can be achieved in the following manner:

[0060] Distortion correction is performed on the installation frame image, and then the photovoltaic panel dynamic response identification code of the target photovoltaic panel is extracted from the corrected installation frame image;

[0061] Corner detection is performed on the dynamic response identifier code of the photovoltaic panel to obtain the detected corner sequence of the dynamic response identifier code of the photovoltaic panel;

[0062] Visual drift estimation is performed based on the detected corner point sequence, thereby obtaining the visual drift vector and visual rotation vector between the hoisting drone and the dynamic response identification code of the photovoltaic panel;

[0063] The pose drift estimation features of the hoisting UAV are constructed based on the visual drift vector and the visual rotation vector.

[0064] In practical implementation, firstly, distortion correction can be performed on the installation frame image. This involves using calibrated camera-in-situ distortion coefficients to perform distortion removal and pixel correction within the installation frame image. This allows the extraction of the target photovoltaic panel's dynamic response identifier from the corrected image. Specifically, after using the Canny edge detection algorithm, the `findContours` function is called to find all closed contours in the corrected installation frame image. These contours are then approximated as polygons to filter out those approximating quadrilaterals. Further filtering is then performed based on area, convexity, etc., initially selecting potential marker bounding boxes. Each candidate quadrilateral region undergoes perspective transformation to correct it into a standard-sized square image. Finally, a pre-trained `AprilTag` function is used... The decoder decodes the corrected square image and identifies its ID to obtain the photovoltaic panel dynamic response identification code of the target photovoltaic panel. Then, corner detection is performed on the photovoltaic panel dynamic response identification code. After successful identification, the cornerSubPix function can be used within the original correction area to accurately locate the center coordinates of the circular feature points embedded in the previous design at the sub-pixel level, thereby obtaining the pixel coordinates of each detected corner point in the photovoltaic panel dynamic response identification code. The sequence of all detected corner points can be used as the detected corner point sequence of the photovoltaic panel dynamic response identification code.

[0065] In addition, in specific implementation, visual drift estimation can be performed based on the detected corner point sequence. This involves obtaining the world coordinates of each detected corner point in the sequence and using a pinhole camera model to obtain the camera intrinsic matrix and distortion coefficients through camera calibration. The distortion coefficients are vectors used to describe the radial and tangential distortion of the lens. The camera intrinsic matrix, distortion coefficients, world coordinates, and pixel coordinates of each detected corner point are then input into the solvePnP function for calculation, resulting in the visual drift vector and visual rotation vector between the hoisting UAV and the photovoltaic panel dynamic response identification code. The visual drift vector includes the positional drift between the hoisting UAV and the photovoltaic panel dynamic response identification code in the horizontal, vertical, and longitudinal axes, while the visual rotation vector includes the rotation angles between the hoisting UAV and the photovoltaic panel dynamic response identification code in the roll, pitch, and yaw directions. Finally, the pose drift estimation features of the hoisting UAV can be constructed based on the visual drift vector and the visual rotation vector, i.e., the feature vector composed of the visual drift vector and the visual rotation vector is used as the pose drift estimation features of the hoisting UAV.

[0066] Preferably, in some embodiments, reference is made to Figure 2 As shown, this figure is an exemplary flowchart of visual servo control of a hoisting drone according to some embodiments of this application. In this embodiment, visual servo control of the hoisting drone based on the pose drift estimation features can be achieved by the following steps:

[0067] In step S1021, the pose drift estimation features are subjected to drift transformation to obtain the servo positioning drift amount of the hoisting UAV;

[0068] In step S1022, the servo positioning drift amount and particle swarm algorithm are used to construct a servo positioning controller for the hoisting drone;

[0069] In step S1023, a servo positioning control quantity for the hoisting drone is generated based on the servo positioning controller.

[0070] In step S1024, the servo positioning control quantity is sent to the underlying flight controller of the hoisting drone, thereby completing the visual servo control of the hoisting drone.

[0071] In practical implementation, firstly, the pose drift estimation features can be transformed to obtain the servo positioning drift of the hoisting UAV. The servo positioning drift is mapped to the pose drift estimation features in the hoisting UAV's own reference frame; that is, the position drift in the pose drift estimation features is not processed, but the rotation angles in the pose drift estimation features are converted to Euler angles to obtain the servo positioning drift. Then, the servo positioning drift and particle swarm optimization algorithm can be used to construct a servo positioning controller for the hoisting UAV. This servo positioning controller can be expressed by the following formula:

[0072] U = Kp * Error + Kd * d(Error) / dt

[0073] Where U represents the servo positioning control quantity, Kp represents the proportional gain matrix, Kd represents the differential gain matrix, Error represents the servo positioning drift, and d(Error) / dt represents the derivative of the servo positioning drift. For example, the servo positioning control quantity can be [Vx, Vy, Vz, ωx, ωy, ωz], which are the forward speed, translation speed, ascent speed, roll angle, pitch angle, and yaw angle of the hoisting UAV in the body coordinate system, respectively. The proportional gain matrix is ​​usually a diagonal matrix containing multiple gain values, where each gain value independently adjusts the response intensity of the corresponding channel. The differential gain matrix is ​​used to introduce damping, suppress system oscillation, and improve stability. The derivative of the servo positioning drift can be expressed as the current servo positioning drift minus the previous servo positioning drift divided by time. In actual implementation, the optimal proportional gain matrix and differential gain matrix can be searched online using the particle swarm optimization algorithm to construct a servo positioning controller adapted to the current drift state. The multi-peak error surface is processed and converged quickly by utilizing the swarm search characteristics of the particle swarm optimization algorithm.

[0074] In addition, in specific implementation, the servo positioning control quantity for the hoisting UAV can be generated based on the servo positioning controller. That is, the servo positioning controller optimized by particle swarm optimization can be run to output the servo positioning control quantity for the hoisting UAV in real time. This servo positioning control quantity includes the hoisting UAV's velocity command and attitude correction angle in three-dimensional space, which is used to fine-tune the position and attitude of the hoisting UAV to eliminate drift. At the same time, the drift rate feedforward term is combined to enhance the response speed and suppress continuous drift. Finally, the servo positioning control quantity can be sent to the hoisting UAV's underlying flight controller. That is, the calculated servo positioning control quantity is sent to the UAV's underlying flight controller through a secure interface. The underlying flight controller converts it into motor thrust and attitude control surface commands and executes high-frequency closed-loop adjustment, thereby correcting the UAV's position and attitude in real time, realizing drift compensation based on visual servoing, and thus completing the visual servo control of the hoisting UAV.

[0075] In some embodiments, the servo positioning drift of the hoisting UAV in visual servo control is made less than a drift threshold. Specifically, firstly, a drift response model of the hoisting UAV under different operating environments can be established based on historical experimental data. This drift response model is then optimized using particle swarm optimization to obtain an optimal set of drift thresholds, which includes position component thresholds and angle component thresholds. Then, in actual implementation, the visual servo control in the above steps is executed iteratively until each component of the obtained servo positioning drift is less than the set drift threshold. For example, the forward speed, translation speed, and ascent speed in the servo positioning drift are all less than the position component threshold, and the roll angle, pitch angle, and yaw angle are all less than the angle component threshold. Specifically, when the forward speed, translation speed, and ascent speed are all lower than the position component threshold, and the roll angle, pitch angle, and yaw angle are all lower than the angle component threshold, the system determines that the hoisting UAV has met the drift constraint requirements, thereby achieving high-precision hovering and stable positioning, and thus realizing precise hovering and fine-tuning of the UAV above the target photovoltaic panel.

[0076] It should be noted that by estimating the pose drift based on the dynamic response identification code of the photovoltaic panel in the installation frame image, and using the obtained pose drift estimation features for visual servo control, the small displacement and attitude deviation of the UAV relative to the target photovoltaic panel can be perceived in real time and accurately. This allows for feedforward compensation and closed-loop fine-tuning in the control loop, ensuring that the servo positioning drift of the UAV remains below the set drift threshold during hovering and movement. This effectively eliminates the small drift caused by wind disturbance, load swing, visual measurement noise, and the UAV's own dynamic characteristics, ensuring accurate position and attitude control of the UAV in three-dimensional space and improving the stability and safety of the hoisting operation.

[0077] In step S103, dynamic motion data of the end effector of the hoisting drone is acquired. A desired impedance controller for the end effector of the hoisting drone is constructed using the dynamic motion data and particle swarm optimization algorithm. The desired impedance controller is used for compliant control, thereby generating dynamic compliant commands for the hoisting drone.

[0078] It should be noted that the dynamic motion data of the end effector of the hoisting drone includes the real-time position, velocity, and acceleration of the end effector. Specifically, high-precision sensors, including joint encoders (to acquire joint angles) and IMUs (to measure linear acceleration and angular velocity), can be installed on the end effector and various joints of the hoisting drone, ensuring time synchronization of all sensors. Secondly, through forward kinematics calculations, the joint angle data is mapped to the real-time position of the end effector in three-dimensional space. Then, the velocity and acceleration are calculated using the change in end effector position over time, and fused together with the linear acceleration and angular velocity measured by the IMU to improve the accuracy and response frequency of velocity and acceleration. Finally, the processed end effector position, velocity, and acceleration data are output with a unified timestamp, forming complete dynamic motion data of the end effector, achieving high-precision real-time perception of the motion state of the hoisting drone's end effector.

[0079] In some embodiments, the construction of the desired impedance controller at the end effector of the hoisting drone's robotic arm using the dynamic motion data and particle swarm optimization algorithm can be achieved in the following manner:

[0080] The particle swarm optimization algorithm is used to optimize the compliant coefficient sequence, and the optimization result of the compliant coefficient sequence is obtained.

[0081] Based on the optimization results of the compliance coefficient sequence and the dynamic motion data, a desired impedance controller for the end effector of the hoisting drone's robotic arm is constructed.

[0082] In practical implementation, firstly, a particle swarm optimization (PSO) algorithm can be used to optimize the compliance coefficient sequence, which includes the desired mass coefficient, damping coefficient, and stiffness coefficient. When optimizing using PSO, a particle swarm is initialized, with each particle carrying a candidate parameter vector and a velocity vector. In each iteration, for each particle: its candidate parameter vector is loaded into the impedance control simulation environment, the control actions generated by the impedance controller are run, and the corresponding performance costs are calculated. Based on the costs, the individual optimality and global optimality of the particles are updated, and the update rules are followed according to the standard PSO algorithm. This iteration continues until convergence or the stopping condition is met, thus obtaining the optimization result of the compliance coefficient sequence. Then, based on the optimization result of the compliance coefficient sequence and dynamic motion data, a desired impedance controller for the end effector of the hoisting drone's robotic arm can be constructed. In actual implementation, this desired impedance controller can be expressed by the following formula:

[0083] F_desired = M * (d²X / dt²) + B * (dX / dt) + K * (X - X0)

[0084] Where F_desired represents the desired interaction force output by the desired impedance controller, M, B and K are the mass coefficient, damping coefficient and stiffness coefficient in the optimization result of the compliance coefficient sequence, respectively, and X, dX / dt and d²X / dt² represent the real-time position, velocity and acceleration of the robotic arm end in the dynamic motion data, respectively.

[0085] In some embodiments, using the desired impedance controller for compliant control to generate dynamic compliant commands for the hoisting drone can be achieved in the following ways:

[0086] Obtain the actual force at the end of the lifting drone's robotic arm;

[0087] The desired interaction force on the end effector of the lifting drone's robotic arm is determined based on the desired impedance controller.

[0088] Based on the actual force and the desired interaction force, a compliant inverse solution is performed on the desired impedance controller to generate dynamic compliant commands for the hoisting drone.

[0089] In practice, firstly, the actual force at the end of the hoisting drone's robotic arm can be obtained, which can be measured by a six-axis torque sensor installed at the end of the robotic arm. Then, the desired interaction force on the end of the hoisting drone's robotic arm can be output using a desired impedance controller. Finally, a compliant inverse solution can be performed on the desired impedance controller based on the actual force and the desired interaction force, that is, the deviation between the actual force and the desired interaction force can be calculated. Based on this deviation, the motion command that needs to be adjusted can be obtained through the desired impedance controller, so that the deviation between the actual force and the desired interaction force is zero. This motion command can be used as a dynamic compliant command for the hoisting drone, which includes position correction, speed adjustment, and attitude fine-tuning information.

[0090] It should be noted that by acquiring dynamic motion data from the end effector of the hoisting drone's robotic arm and optimizing the desired impedance controller using a particle swarm optimization algorithm, the robotic arm can acquire adaptive compliant characteristics. This allows it to automatically adjust its motion stiffness and damping when subjected to external interference or contact with the target photovoltaic panel, absorbing impacts and reducing the impact of minor positional deviations on system stability. This not only improves the safety and dynamic adaptability of the end effector in complex operating environments but also complements visual servo control: visual servo provides precise posture adjustment, while compliant control compensates for transient impacts and minor deviations caused by wind disturbances, attitude errors, or hook contact. The combination of the two effectively improves the positioning accuracy of the target photovoltaic panel, ensures a smooth and reliable hoisting and hooking process, avoids slippage, collisions, or misalignment caused by rigid control, and achieves high-precision and safe automated drone hoisting operations.

[0091] In step S104, the hoisting drone is motion controlled according to the dynamic compliant command, thereby enabling the hoisting drone to complete the hoisting and hooking of the target photovoltaic panel.

[0092] In some embodiments, motion control of the hoisting drone according to the dynamic compliance command involves sending the dynamic compliance command to the motion controller of the hoisting drone, and then the motion controller controls the hoisting drone to adjust its posture according to the dynamic compliance command.

[0093] In practice, firstly, the dynamic compliance commands generated by the desired impedance controller can be encapsulated into standardized control signals and transmitted to the motion controller of the hoisting UAV via a high-speed communication link. After receiving the commands, the motion controller calculates the required thrust distribution and attitude adjustment parameters by combining the current flight status data of the UAV (position, attitude, velocity, acceleration, etc.) and the desired trajectory in the mission space coordinate system. Subsequently, the motion controller adjusts the UAV's main rotor, yaw control, and robotic arm joint drive in real time through the flight control unit (FCU), enabling the UAV to perform subtle attitude adjustments and compliant motion compensation while maintaining overall stability. During execution, the system continuously monitors the end effector force, attitude error, and dynamic feedback. If contact impact or attitude drift is detected, the controller will quickly correct based on the dynamic compliance commands to achieve small and smooth adjustments. Finally, the UAV end effector can achieve precise alignment and smooth hooking with the target photovoltaic panel under safe and stable conditions, ensuring efficient completion of the hoisting action and avoiding structural damage or positioning failure caused by attitude errors.

[0094] It should be noted that motion control of the hoisting drone based on dynamic compliant commands enables it to achieve high-precision posture adjustment and adaptive compliant response when approaching the target photovoltaic panel and performing hooking operations. This avoids collisions or deviations caused by wind disturbances, load swaying, or minor visual errors. Compliant control automatically absorbs impact and adjusts the direction of force at the moment of contact, reducing slippage or jamming caused by rigid control, thereby protecting the photovoltaic panel and the drone's robotic arm structure. Simultaneously, compliant control and visual servo control complement each other: visual servo provides high-precision spatial positioning and drift correction, while dynamic compliant motion ensures safe and smooth end-effector movements, achieving compliant approach and reliable hooking at the target hooking point. This not only significantly improves positioning accuracy and operational success rate but also reduces the risk of damage to the photovoltaic panel and drone during assembly.

[0095] Therefore, in this application, firstly, by estimating the pose drift based on the dynamic response identifier of the photovoltaic panel in the installation frame image, and using the obtained pose drift estimation features for visual servo control, the small displacement and attitude deviation of the UAV relative to the target photovoltaic panel can be perceived in real time and accurately, thereby enabling feedforward compensation and closed-loop fine-tuning in the control loop. Secondly, by acquiring the dynamic motion data of the end effector of the hoisting UAV and optimizing the desired impedance controller using the particle swarm optimization algorithm, the end effector can be made to have adaptive compliance characteristics. This not only improves the safety and dynamic adaptability of the end effector in complex working environments, but also complements the visual servo control, thereby effectively improving the positioning accuracy of the target photovoltaic panel. Finally, the hoisting UAV is motion controlled according to the dynamic compliance command, enabling it to achieve high-precision pose adjustment and adaptive compliance response when approaching the target photovoltaic panel and performing hooking operations. Visual servo provides high-precision spatial positioning and drift correction, while dynamic compliance motion ensures safe and stable end action, achieving compliant approach and reliable hooking to the target hooking point, significantly improving positioning accuracy and the success rate of hoisting hooking operations.

[0096] In summary, the technical solution adopted in this application can achieve precise positioning and hoisting hooks for target photovoltaic panels in visual servo control.

[0097] In another aspect, in some embodiments, this application provides a vision-based positioning-based photovoltaic panel drone hoisting control system, with reference to... Figure 3 The figure is a schematic diagram of a vision-based positioning photovoltaic panel drone hoisting control system according to some embodiments of this application. The vision-based positioning photovoltaic panel drone hoisting control system includes:

[0098] The image acquisition module 201 is used to control the hoisting drone to fly from its current position to above the target photovoltaic panel, and to acquire images of the installation frame of the target photovoltaic panel in real time after the hoisting drone hovers.

[0099] The visual positioning module 202 is used to estimate the pose drift based on the photovoltaic panel dynamic response identification code in the installation frame image, obtain the pose drift estimation features of the hoisting drone, and perform visual servo control on the hoisting drone based on the pose drift estimation features, so that the servo positioning drift of the hoisting drone in visual servo control is less than the drift threshold.

[0100] The compliant control module 203 is used to acquire dynamic motion data of the end effector of the hoisting drone's robotic arm, construct a desired impedance controller for the end effector of the hoisting drone's robotic arm using the dynamic motion data and particle swarm optimization algorithm, and use the desired impedance controller to perform compliant control, thereby generating dynamic compliant commands for the hoisting drone.

[0101] The motion control module 204 is used to control the motion of the hoisting drone according to the dynamic compliant command, thereby enabling the hoisting drone to complete the hoisting hook of the target photovoltaic panel.

[0102] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, the processor being configured to acquire the code and execute the above-described vision-based positioning-based photovoltaic panel drone hoisting control method.

[0103] In some embodiments, reference Figure 4 The figure is a schematic diagram of the structure of a computer device implementing a vision-based positioning-based photovoltaic panel drone hoisting control method according to some embodiments of this application. The vision-based positioning-based photovoltaic panel drone hoisting control method in the above embodiments can be... Figure 4 The computer device shown is used to implement this, and the computer device includes at least one processor 301, a communication bus 302, a memory 303, and at least one communication interface 304.

[0104] The processor 301 can be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more devices used to control the execution of the vision-based positioning-based photovoltaic panel drone hoisting control method described in this application.

[0105] The communication bus 302 can be used to transmit information between the aforementioned components.

[0106] The memory 303 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 303 may exist independently and be connected to the processor 301 via the communication bus 302. The memory 303 may also be integrated with the processor 301.

[0107] The memory 303 stores program code for executing the scheme of this application, and its execution is controlled by the processor 301. The processor 301 executes the program code stored in the memory 303. The program code may include one or more software modules. In the above embodiments, the determination of the vision-based positioning-based photovoltaic panel UAV hoisting control method can be implemented by the processor 301 and one or more software modules in the program code in the memory 303.

[0108] Communication interface 304 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0109] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0110] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0111] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described vision-based positioning-based UAV hoisting control method for photovoltaic panels.

[0112] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0113] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for controlling the hoisting of photovoltaic panels using a drone based on vision positioning, characterized in that, Includes the following steps: Control the hoisting drone to fly from the current position to above the target photovoltaic panel, and collect images of the installation frame of the target photovoltaic panel in real time after the hoisting drone hovers; Based on the dynamic response identification code of the photovoltaic panel in the installation frame image, pose drift is estimated to obtain the pose drift estimation features of the hoisting drone. Based on the pose drift estimation features, visual servo control is performed on the hoisting drone to establish a drift response model of the hoisting drone under different operating environments. Particle swarm optimization is used to optimize and search the drift response model to obtain a drift threshold set, which includes position component thresholds and angle component thresholds. The above visual servo control is executed cyclically until each component of the obtained servo positioning drift is less than the corresponding threshold in the drift threshold set, at which point it is determined that the hoisting drone has met the drift constraint requirements. The dynamic motion data of the end effector of the hoisting drone is acquired, and the desired impedance controller of the end effector of the hoisting drone is constructed using the dynamic motion data and particle swarm optimization algorithm to obtain the actual force at the end effector of the hoisting drone. The desired interaction force on the end effector of the lifting drone's robotic arm is determined based on the desired impedance controller. Based on the actual force and the desired interaction force, a compliant inverse solution is performed on the desired impedance controller to generate dynamic compliant commands for the hoisting drone. The hoisting drone is motion controlled according to the dynamic compliant command, thereby enabling the hoisting drone to complete the hoisting and hooking of the target photovoltaic panel.

2. The method for controlling the hoisting of photovoltaic panels by a drone based on visual positioning as described in claim 1, characterized in that, Controlling the hoisting drone to fly from its current position to above the target photovoltaic panel specifically includes: Obtain the current position of the hoisting drone and the current position of the target photovoltaic panel in the world coordinate system; The hovering point of the hoisting drone is set based on the current position of the target photovoltaic panel; The path planner of the hoisting drone generates collision-free flight information for the hoisting drone based on the hovering point; Based on the collision-free flight information, the hoisting drone is controlled to fly from its current position to above the target photovoltaic panel.

3. The method for controlling the hoisting of photovoltaic panels by a UAV based on visual positioning as described in claim 1, characterized in that, Based on the dynamic response identifier of the photovoltaic panel in the installation frame image, pose drift estimation is performed to obtain the pose drift estimation features of the hoisting drone, which specifically include: Distortion correction is performed on the installation frame image, and then the photovoltaic panel dynamic response identification code of the target photovoltaic panel is extracted from the corrected installation frame image; Corner detection is performed on the dynamic response identifier code of the photovoltaic panel to obtain the detected corner sequence of the dynamic response identifier code of the photovoltaic panel; Visual drift estimation is performed based on the detected corner point sequence, thereby obtaining the visual drift vector and visual rotation vector between the hoisting drone and the dynamic response identification code of the photovoltaic panel; The pose drift estimation features of the hoisting UAV are constructed based on the visual drift vector and the visual rotation vector.

4. The method for controlling the hoisting of photovoltaic panels by a drone based on visual positioning as described in claim 1, characterized in that, Visual servo control of the hoisting UAV based on the aforementioned pose drift estimation features specifically includes: The pose drift estimation features are subjected to drift transformation to obtain the servo positioning drift amount of the hoisting UAV; A servo positioning controller for the hoisting drone is constructed using the servo positioning drift amount and particle swarm algorithm. The servo positioning control quantity for the hoisting drone is generated based on the servo positioning controller. The servo positioning control quantity is sent to the underlying flight controller of the hoisting drone, thereby completing the visual servo control of the hoisting drone.

5. The method for controlling the hoisting of photovoltaic panels by a drone based on visual positioning as described in claim 1, characterized in that, The dynamic motion data of the end effector of the hoisting drone includes the real-time position, speed, and acceleration of the end effector.

6. The method for controlling the hoisting of photovoltaic panels by a drone based on visual positioning as described in claim 1, characterized in that, The construction of the desired impedance controller at the end of the hoisting drone's robotic arm using the dynamic motion data and particle swarm optimization algorithm specifically includes: The particle swarm optimization algorithm is used to optimize the compliant coefficient sequence, and the optimization result of the compliant coefficient sequence is obtained. Based on the optimization results of the compliance coefficient sequence and the dynamic motion data, a desired impedance controller for the end effector of the hoisting drone's robotic arm is constructed.

7. The method for controlling the hoisting of photovoltaic panels by a drone based on visual positioning as described in claim 1, characterized in that, Motion control of the hoisting drone according to the dynamic compliant command involves sending the dynamic compliant command to the motion controller of the hoisting drone, and then the motion controller controls the hoisting drone to adjust its posture according to the dynamic compliant command.

8. A vision-based positioning-based photovoltaic panel drone hoisting control system, used to execute the vision-based positioning-based photovoltaic panel drone hoisting control method as described in any one of claims 1 to 7, characterized in that, include: The image acquisition module is used to control the hoisting drone to fly from the current position to above the target photovoltaic panel, and to acquire images of the installation frame of the target photovoltaic panel in real time after the hoisting drone hovers. The visual positioning module is used to estimate the pose drift based on the photovoltaic panel dynamic response identification code in the installation frame image, obtain the pose drift estimation features of the hoisting drone, and perform visual servo control on the hoisting drone based on the pose drift estimation features, so that the servo positioning drift of the hoisting drone in visual servo control is less than the drift threshold. The compliant control module is used to acquire dynamic motion data of the end effector of the hoisting drone's robotic arm, construct a desired impedance controller for the end effector of the hoisting drone's robotic arm using the dynamic motion data and particle swarm optimization algorithm, and use the desired impedance controller to perform compliant control, thereby generating dynamic compliant commands for the hoisting drone. The motion control module is used to control the motion of the hoisting drone according to the dynamic compliant command, thereby enabling the hoisting drone to complete the hoisting hook of the target photovoltaic panel.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing code, and the processor being configured to retrieve the code and execute the vision-based positioning-based photovoltaic panel drone hoisting control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the vision-based positioning-based unmanned aerial vehicle (UAV) hoisting control method for photovoltaic panels as described in any one of claims 1 to 7.

Images