Unmanned aerial vehicle multi-load synchronous control system

Abstract

The invention relates to the technical field of unmanned aerial vehicle control, and discloses an unmanned aerial vehicle multi-load synchronous control system, and the system comprises a linkage control unit which is used for detecting an image shot by an unmanned aerial vehicle, obtaining a camera imaging plane deviation according to a detection result, and combining with a kinematic model to solve an expected instruction of the unmanned aerial vehicle and a holder; the multi-load configuration unit is used for determining control logic according to the load type and the expected instruction and issuing an operation instruction to the load according to the control logic so as to realize linkage control of the load and the unmanned aerial vehicle controller; the control instruction adjusting unit is used for acquiring the flight state of the unmanned aerial vehicle when the expected instruction is executed, and calculating a disturbance value of the unmanned aerial vehicle to determine a final control instruction of the unmanned aerial vehicle by taking the flight state as input; the equipment is an airborne computing power unit. The active-disturbance-rejection pose correction technology is introduced, real-time compensation of disturbance such as wind disturbance, load sudden change and hanging point deviation is achieved, and the working efficiency and the complex scene adaptability are improved.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) control technology, and more specifically, to a multi-payload synchronous control system for UAVs. Background Technology

[0002] With the widespread application of drones in power line inspection, emergency rescue, and operational tasks, their mission complexity is constantly increasing, and single payload configurations are no longer sufficient to meet the requirements. Existing technologies mainly have the following shortcomings: Lack of unified interface and coordinated control for multiple payloads: When performing missions, UAVs often carry multiple payloads such as gimbal cameras, infrared thermal imagers, drop devices, and robotic arms. However, the existing flight control system lacks a universal interface standard, which makes it impossible for the payloads to form effective coordination.

[0003] Insufficient coupling between flight control and gimbal: Existing UAVs only support independent stabilization control of gimbal cameras, lacking flight control and gimbal linkage strategies based on visual target recognition, making it difficult to maintain target tracking and stable imaging in situations with sudden changes in target position and dynamic scenes.

[0004] Attitude stability is severely affected by disturbances: Multiple loads cause center of gravity shift and aerodynamic characteristics changes. Existing control strategies based on accurate dynamic models suffer from model mismatch in real-world complex scenarios, making it difficult to achieve real-time compensation, which leads to flight attitude oscillations or instability.

[0005] Fragmented mission flow: UAV image acquisition, target recognition, payload actions and ground station interaction are usually implemented separately, lacking a closed-loop system for the entire process, which is not conducive to efficient mission execution and anomaly early warning.

[0006] Therefore, there is an urgent need for a UAV control method and system that can achieve gimbal camera and flight control coordination, unified adaptation of multiple payloads, and real-time anti-disturbance compensation to overcome the shortcomings of existing technologies.

[0007] There are currently no effective solutions to the problems in the relevant technologies. Summary of the Invention

[0008] In view of the problems in related technologies, this invention proposes a multi-payload synchronous control system for unmanned aerial vehicles (UAVs) to overcome the aforementioned technical problems existing in the prior art.

[0009] Therefore, the specific technical solution adopted by the present invention is as follows: A multi-payload synchronous control system for unmanned aerial vehicles (UAVs), the system comprising: The linkage control unit is used to detect images captured by the drone, obtain the camera imaging plane deviation based on the detection results, and combine it with the kinematic model to calculate the expected commands of the drone and the gimbal. The multi-load configuration unit is used to determine the control logic based on the load type and expected instructions, and to issue operation instructions to the load according to the control logic, so as to realize the linkage control between the load and the UAV controller. The control command adjustment unit is used to acquire the flight state of the UAV when executing the desired command, take the flight state as input, calculate the disturbance value of the UAV, and determine the final control command of the UAV. The onboard computing unit is used to drive the gimbal and drone according to the expected instructions and final control instructions, collect power equipment information obtained by the camera and sensor probes on the drone, judge the performance of the power equipment based on the power equipment information, and output the drone's operation decision suggestions based on the performance.

[0010] Preferably, the linkage control unit includes: The image detection model building module is used to collect historical aerial images of target electronic devices taken by drones under different conditions, and to build a target detection model using historical aerial images and deep learning technology. The detection and recognition module is used to perform detection processing on real-time images captured by the drone using a target detection model, extract real-time image features, and obtain the location of the electronic device based on the real-time image features. The deviation analysis module is used to obtain the image coordinates of the electronic device on the real-time image based on the position of the electronic device, and to build a projection model based on the image coordinates to analyze the deviation of the camera imaging plane. The gimbal expectation command generation module is used to output control commands for the UAV and gimbal based on the camera imaging plane deviation, and to use fuzzy control to control the operation status of the UAV and gimbal based on the control commands.

[0011] Preferably, the method of obtaining the image coordinates of the electronic device on the real-time image based on the position of the electronic device, and establishing a projection model based on the image coordinates to analyze the camera imaging plane deviation includes: The image coordinates of the electronic device on the real-time image are obtained based on the position of the electronic device, and the image coordinates are combined with the set of feature points of the electronic device in three-dimensional space to establish a projection model. The rotation matrix and translation vector of the camera on the UAV are solved based on the projection model, and the feature points in the three-dimensional space are back-projected onto the real-time image captured by the UAV using the rotation matrix and translation vector. The back projection error is calculated based on the back projection results, and the average error of the back projection error is obtained. When the average error is less than the pixel error threshold, the transformation process from the image coordinate system to the camera coordinate system is completed. A UAV attitude rotation matrix is ​​established based on the yaw angle when the UAV captures real-time images. The camera coordinate system is then transformed into a global coordinate system using the UAV attitude rotation matrix. An inverse transformation is performed on the global coordinate system, and the Euclidean distance of the global coordinate system before and after the inverse transformation is compared to output the camera imaging plane deviation.

[0012] Preferably, the control commands for the UAV and gimbal are output based on the camera imaging plane deviation, and the operating status of the UAV and gimbal is controlled by fuzzy control based on the control commands, including: Based on the camera imaging plane deviation, the offset of the electronic device in the global coordinate system is output as the UAV control command, and the back projection deviation generated during the camera coordinate system transformation is sent as the gimbal's desired rotation angle. The deviation and rate of change of the current position of the gimbal and the target position of the power equipment are obtained based on the expected rotation angle of the gimbal, and are used as fuzzy control inputs for the deviation and rate of change of the deviation to reflect the degree and trend of the target deviating from the field of view of the gimbal. The deviation, the rate of change of deviation, and the fuzzy control output are divided into several fuzzy levels, and corresponding quantization factors are established. The fuzzy levels, quantization factors, and gimbal control characteristics are combined to establish a fuzzy control rule base. The output fuzzy value of each rule is calculated using the inference method based on the fuzzy rule base, and the output fuzzy values ​​of all rules are synthesized using the centroid method to obtain the fuzzy control output value. The control output value is then calculated based on the deviation. The fuzzy control output value and the control output value are combined according to weights to obtain the control quantity used to drive the gimbal servo motor, which drives the gimbal to complete the corresponding rotation action, and adjusts the UAV's heading and speed according to the UAV control command.

[0013] Preferably, the multi-load configuration unit includes: The flight control measurement update module is used to install the payload to the UAV mounting point, send self-test commands, and determine the control logic of the adapted payload based on the payload type and payload parameters. The control logic includes the payload working mode and the expected commands of the UAV and gimbal. The communication link establishment module is used to establish a communication link based on the payload's load parameters and optimize the communication quality of the communication link based on communication feasibility to ensure stable communication between the payload and the UAV. The payload flight control linkage module is used by the UAV controller to send operation instructions to the payload according to the control logic. The payload receives the operation instructions through the communication link, executes the corresponding actions, and provides feedback on the execution status. The collaborative scheduling optimization module is used to aggregate the execution status of payloads using the UAV controller, and optimize the flight path and payload operation sequence of the UAV based on the execution status and task priority.

[0014] Preferably, establishing a communication link based on the payload's load parameters and optimizing the communication quality of the communication link based on communication feasibility to ensure stable communication between the payload and the UAV includes: The UAV controller automatically configures the UAV's communication parameters based on the payload parameters reported by the payload, and negotiates a communication link with the payload. After the negotiation is completed, the UAV controller sends a link establishment confirmation command to the payload, and the payload sends back a link ready signal, thus completing the establishment of a two-way communication link. Using communication feasibility and quality dynamic assessment technology, the comprehensive link quality index of the two-way communication link is calculated based on signal strength, round-trip time delay and interference intensity, and the comprehensive link quality index is compared with the minimum quality index. If the overall link quality index is greater than or equal to the minimum quality index, it means that the bidirectional communication link quality meets the requirements. If the overall link quality index is less than the minimum quality index, load parameter optimization is performed until the bidirectional communication link quality meets the requirements. An optimization function is constructed by combining the quality of the two-way communication link with the priority of the payload tasks using the UAV controller. The optimization function is then solved to prioritize the payload tasks, thereby ensuring stable communication between the payload and the UAV.

[0015] Preferably, the control command adjustment unit includes: The dynamic model and observer design module is used to establish a dynamic model with the attitude loop and velocity loop of the UAV as objects, and to design an extended state observer based on the total perturbation of the attitude loop and the total perturbation of the velocity loop. The estimation output module is used to obtain the angular velocity and linear velocity of the UAV when executing the desired command, and use them as input to the extended state observer to output attitude loop and velocity loop perturbation estimates. The UAV adjustment module is used to inject the attitude loop disturbance estimate and the velocity loop disturbance estimate as negative feedforward into the attitude loop and velocity loop control inputs, and adjust the control torque command and thrust command of the UAV in combination with the dynamic model.

[0016] Preferably, the attitude loop disturbance estimate and the velocity loop disturbance estimate are used as negative feedforward injections into the attitude loop and velocity loop as control inputs, and the control torque command and thrust command of the UAV are adjusted in conjunction with the dynamic model, including: Based on the dynamic model, the desired angular velocity and desired linear velocity of the UAV are output, and the attitude loop error and velocity loop error are calculated by combining the angular velocity and linear velocity. Based on the rate of change of the output error, the nonlinear feedback value of the attitude loop and the nonlinear value of the velocity loop are designed. The attitude loop disturbance estimates and velocity loop disturbance estimates are used as negative feedforward injections into the attitude loop and velocity loop control inputs. They are combined with the attitude loop nonlinear feedback values ​​and velocity loop nonlinear values ​​for feedforward compensation, and the compensated UAV control torque and thrust commands are output as the final control commands.

[0017] Preferably, the onboard computing unit of the equipment includes: The drive module is used to drive the gimbal and drone according to the expected instructions and final control instructions; The insulation performance calculation module is used to segment and identify water droplet characteristics after water spraying on insulators captured by a gimbal camera using an adaptive threshold algorithm, and calculate the contact angle by combining the ellipse equation to output the hydrophobicity level and determine the insulation performance of the insulator. The line energization detection module is used to collect electric field signals within the target range of the line using an induction probe mounted on a drone, and output the energization status of the line after performing feature analysis on the electric field signals.

[0018] Preferably, an adaptive threshold algorithm is used to segment and identify water droplet characteristics in insulator images captured by a gimbal camera after water spraying, and the contact angle is calculated using the ellipse equation to output the hydrophobicity level. The insulation performance of the insulator is then judged, including: The system receives color images transmitted from the gimbal camera, converts these images into grayscale images, uses adaptive thresholding technology to determine segmentation thresholds, and then... Figure 2 Binarization is used to separate the water droplet region from the insulator background, resulting in a binarized image; Erosion, dilation and connected region marking are performed on the binarized image to mark the independent water droplet regions in the binarized image. Valid water droplets are selected according to preset conditions. The coordinates of all pixels on the edge contour of the valid water droplets are input into the ellipse fitting model to calculate the tangent slope of the valid water droplets. The contact angle of the effective water droplets is output based on the tangent slope. All effective water droplets are summarized, and the summarized results are compared with the hydrophobicity level relationship table to judge the insulation performance of the insulator.

[0019] The beneficial effects of this invention are as follows: 1. This invention enables autonomous linkage control between flight control and gimbal, ensuring the continuity and high precision of target recognition and tracking. It also proposes a universal interface standard for multiple payloads of UAVs, supporting plug-and-play payloads. The flight control automatically identifies the payload type and updates the control strategy. Furthermore, it introduces self-disturbance rejection attitude correction technology to achieve real-time compensation for disturbances such as wind disturbance, sudden load changes, and attachment point offset. This constructs an intelligent control system for the entire mission process, forming a closed loop from image acquisition, target recognition, payload action, data reporting to ground station early warning, thereby improving operational efficiency and adaptability to complex scenarios.

[0020] 2. This invention enables autonomous linkage control between the UAV and the gimbal, improving target tracking accuracy and shooting stability. The plug-and-play functionality of multiple payloads significantly enhances the UAV's multi-task adaptability and operational efficiency. Self-disturbance compensation improves stability, maintaining stable flight even under wind disturbances and sudden load changes. Meanwhile, the closed-loop system throughout the entire process, from image acquisition and mission actions to ground station early warning, enables intelligent operation. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic block diagram of a multi-payload synchronous control system for an unmanned aerial vehicle (UAV) according to an embodiment of the present invention. Figure 2 This is a flowchart of the autonomous linkage control between the gimbal camera and the UAV body in a multi-payload synchronous control system for a UAV according to an embodiment of the present invention. Figure 3 This is a flowchart illustrating the configuration of a multi-payload UAV in a multi-payload synchronous control system according to an embodiment of the present invention. Figure 4 This is an application roadmap for a multi-payload synchronous control system for unmanned aerial vehicles (UAVs) according to an embodiment of the present invention.

[0023] In the picture: 1. Linkage control unit; 2. Multi-load configuration unit; 3. Control command adjustment unit; 4. Equipment onboard computing power unit. Detailed Implementation

[0024] To further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention. These drawings are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention.

[0025] According to an embodiment of the present invention, a multi-payload synchronous control system for unmanned aerial vehicles (UAVs) is provided.

[0026] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figure 1 and Figure 4 As shown, according to an embodiment of the present invention, a multi-payload synchronous control system for unmanned aerial vehicles includes: The linkage control unit 1 is used to detect images captured by the UAV, obtain the camera imaging plane deviation based on the detection results, and combine it with the kinematic model to calculate the expected commands of the UAV and the gimbal.

[0027] In one embodiment, the linkage control unit 1 includes: The image detection model building module is used to collect historical aerial images of target electronic devices taken by drones under different conditions, and to build a target detection model using historical aerial images and deep learning technology. The detection and recognition module is used to perform detection processing on real-time images captured by the drone using a target detection model, extract real-time image features, and obtain the location of the electronic device based on the real-time image features. The deviation analysis module is used to obtain the image coordinates of the electronic device on the real-time image based on the position of the electronic device, and to build a projection model based on the image coordinates to analyze the deviation of the camera imaging plane. The gimbal expectation command generation module is used to output control commands for the UAV and gimbal based on the camera imaging plane deviation, and to use fuzzy control to control the operation status of the UAV and gimbal based on the control commands.

[0028] In one embodiment, obtaining the image coordinates of the electronic device on a real-time image based on its position, and establishing a projection model based on these image coordinates to analyze camera imaging plane deviation includes: The image coordinates of the electronic device on the real-time image are obtained based on the position of the electronic device, and the image coordinates are combined with the set of feature points of the electronic device in three-dimensional space to establish a projection model. The rotation matrix and translation vector of the camera on the UAV are solved based on the projection model, and the feature points in the three-dimensional space are back-projected onto the real-time image captured by the UAV using the rotation matrix and translation vector. The back projection error is calculated based on the back projection results, and the average error of the back projection error is obtained. When the average error is less than the pixel error threshold, the transformation process from the image coordinate system to the camera coordinate system is completed. A UAV attitude rotation matrix is ​​established based on the yaw angle when the UAV captures real-time images. The camera coordinate system is then transformed into a global coordinate system using the UAV attitude rotation matrix. An inverse transformation is performed on the global coordinate system, and the Euclidean distance of the global coordinate system before and after the inverse transformation is compared to output the camera imaging plane deviation.

[0029] In one embodiment, outputting control commands for the UAV and gimbal based on camera imaging plane deviation, and using fuzzy control to control the operating state of the UAV and gimbal based on the control commands includes: Based on the camera imaging plane deviation, the offset of the electronic device in the global coordinate system is output as the UAV control command, and the back projection deviation generated during the camera coordinate system transformation is sent as the gimbal's desired rotation angle. The deviation and rate of change of the current position of the gimbal and the target position of the power equipment are obtained based on the expected rotation angle of the gimbal, and are used as fuzzy control inputs for the deviation and rate of change of the deviation to reflect the degree and trend of the target deviating from the field of view of the gimbal. The deviation, the rate of change of deviation, and the fuzzy control output are divided into several fuzzy levels, and corresponding quantization factors are established. The fuzzy levels, quantization factors, and gimbal control characteristics are combined to establish a fuzzy control rule base. The output fuzzy value of each rule is calculated using the inference method based on the fuzzy rule base, and the output fuzzy values ​​of all rules are synthesized using the centroid method to obtain the fuzzy control output value. The control output value is then calculated based on the deviation. The fuzzy control output value and the control output value are combined according to weights to obtain the control quantity used to drive the gimbal servo motor, which drives the gimbal to complete the corresponding rotation action, and adjusts the UAV's heading and speed according to the UAV control command.

[0030] The multi-load configuration unit 2 is used to determine the control logic according to the load type and the expected command, and to issue operation commands to the load according to the control logic, so as to realize the linkage control between the load and the UAV controller.

[0031] In one embodiment, the multi-load configuration unit 2 includes: The flight control measurement update module is used to install the payload to the UAV mounting point, send self-test commands, and determine the control logic of the adapted payload based on the payload type and payload parameters. The control logic includes the payload working mode and the expected commands of the UAV and gimbal. The communication link establishment module is used to establish a communication link based on the payload's load parameters and optimize the communication quality of the communication link based on communication feasibility to ensure stable communication between the payload and the UAV. The payload flight control linkage module is used by the UAV controller to send operation instructions to the payload according to the control logic. The payload receives the operation instructions through the communication link, executes the corresponding actions, and provides feedback on the execution status. The collaborative scheduling optimization module is used to aggregate the execution status of payloads using the UAV controller, and optimize the flight path and payload operation sequence of the UAV based on the execution status and task priority.

[0032] In one embodiment, establishing a communication link based on the payload's load parameters and optimizing the communication quality of the communication link based on communication feasibility to ensure stable communication between the payload and the UAV includes: The UAV controller automatically configures the UAV's communication parameters based on the payload parameters reported by the payload, and negotiates a communication link with the payload. After the negotiation is completed, the UAV controller sends a link establishment confirmation command to the payload, and the payload sends back a link ready signal, thus completing the establishment of a two-way communication link. Using communication feasibility and quality dynamic assessment technology, the comprehensive link quality index of the two-way communication link is calculated based on signal strength, round-trip time delay and interference intensity, and the comprehensive link quality index is compared with the minimum quality index. If the overall link quality index is greater than or equal to the minimum quality index, it means that the bidirectional communication link quality meets the requirements. If the overall link quality index is less than the minimum quality index, load parameter optimization is performed until the bidirectional communication link quality meets the requirements. An optimization function is constructed by combining the quality of the two-way communication link with the priority of the payload tasks using the UAV controller. The optimization function is then solved to prioritize the payload tasks, thereby ensuring stable communication between the payload and the UAV.

[0033] The control command adjustment unit 3 is used to acquire the flight state of the UAV when executing the desired command, use the flight state as input, calculate the disturbance value of the UAV, and determine the final control command of the UAV.

[0034] In one embodiment, the control command adjustment unit 3 includes: The dynamic model and observer design module is used to establish a dynamic model with the attitude loop and velocity loop of the UAV as objects, and to design an extended state observer based on the total perturbation of the attitude loop and the total perturbation of the velocity loop. The estimation output module is used to obtain the angular velocity and linear velocity of the UAV when executing the desired command, and use them as input to the extended state observer to output attitude loop and velocity loop perturbation estimates. The UAV adjustment module is used to inject the attitude loop disturbance estimate and the velocity loop disturbance estimate as negative feedforward into the attitude loop and velocity loop control inputs, and adjust the control torque command and thrust command of the UAV in combination with the dynamic model.

[0035] In one embodiment, the attitude loop perturbation estimate and the velocity loop perturbation estimate are injected as negative feedforward control inputs into the attitude loop and velocity loop, and the control torque command and thrust command of the UAV are adjusted in conjunction with the dynamic model, including: Based on the dynamic model, the desired angular velocity and desired linear velocity of the UAV are output, and the attitude loop error and velocity loop error are calculated by combining the angular velocity and linear velocity. Based on the rate of change of the output error, the nonlinear feedback value of the attitude loop and the nonlinear value of the velocity loop are designed. The attitude loop disturbance estimates and velocity loop disturbance estimates are used as negative feedforward injections into the attitude loop and velocity loop control inputs. They are combined with the attitude loop nonlinear feedback values ​​and velocity loop nonlinear values ​​for feedforward compensation, and the compensated UAV control torque and thrust commands are output as the final control commands.

[0036] The onboard computing unit 4 is used to drive the gimbal and drone according to the expected instructions and final control instructions, collect power equipment information obtained by the camera and sensor probes on the drone, judge the performance of the power equipment based on the power equipment information, and output the drone's operation decision suggestions based on the performance.

[0037] In one embodiment, the onboard computing unit 4 includes: The drive module is used to drive the gimbal and drone according to the expected instructions and final control instructions; The insulation performance calculation module is used to segment and identify water droplet characteristics after water spraying on insulators captured by a gimbal camera using an adaptive threshold algorithm, and calculate the contact angle by combining the ellipse equation to output the hydrophobicity level and determine the insulation performance of the insulator. The line energization detection module is used to collect electric field signals within the target range of the line using an induction probe mounted on a drone, and output the energization status of the line after performing feature analysis on the electric field signals.

[0038] In one embodiment, an adaptive threshold algorithm is used to segment and identify water droplet characteristics in insulator images captured by a gimbal camera after water spraying. The contact angle is then calculated using the ellipse equation to output the hydrophobicity level, and the insulator's insulation performance is determined by: The system receives color images transmitted from the gimbal camera, converts these images into grayscale images, uses adaptive thresholding technology to determine segmentation thresholds, and then... Figure 2 Binarization is used to separate the water droplet region from the insulator background, resulting in a binarized image; Erosion, dilation and connected region marking are performed on the binarized image to mark the independent water droplet regions in the binarized image. Valid water droplets are selected according to preset conditions. The coordinates of all pixels on the edge contour of the valid water droplets are input into the ellipse fitting model to calculate the tangent slope of the valid water droplets. The contact angle of the effective water droplets is output based on the tangent slope. All effective water droplets are summarized, and the summarized results are compared with the hydrophobicity level relationship table to judge the insulation performance of the insulator.

[0039] To facilitate understanding of the above technical solutions of the present invention, the working principle or operation method of the present invention in actual process will be described in detail below.

[0040] I. Vision-based autonomous linkage control between gimbal camera and UAV body; like Figure 2As shown, a target detection algorithm is used to identify multiple targets, such as power equipment, and the desired commands for the UAV and gimbal are calculated by combining camera imaging plane deviation and UAV / gimbal kinematic model. The specific steps are as follows: (1) Dataset preparation and model training: Collecting image data containing various types of power equipment is the foundation for target detection. This image data includes aerial images of target power equipment taken by UAVs at different times and under different weather conditions, as well as close-up images of target power equipment taken by ground inspection personnel. At the same time, these image data need to be labeled to clearly indicate the category, location, and other information of each power device in the image. The labeling work is usually completed with the help of professional image labeling tools. The labeling results will serve as supervision signals for training the model, guiding the model to learn the correct feature representation.

[0041] (2) Model Training: Using deep learning technology, a suitable object detection model, such as the YOLO series of single-stage detection algorithms, is selected for training. During the training process, the labeled dataset is divided into a training set, a validation set, and a test set. The training set is used to learn the model parameters, the validation set is used to adjust the model hyperparameters to prevent overfitting, and the test set is used to evaluate the final performance of the model. By inputting the images in the training set into the model, the model calculates the error between the predicted result and the true result based on the labeled information. The backpropagation algorithm is used to continuously adjust the model parameters, so that the error gradually decreases, thereby enabling the model to accurately extract the features of power equipment and have the ability to identify power equipment.

[0042] (3) Detection and recognition: The trained model is applied to the actual scene to detect the real-time images captured by the UAV. When the UAV flies over the area of ​​power equipment, the images captured by the camera are transmitted to the onboard computing device in real time. The model processes these images and extracts image features through operations such as convolutional layers and pooling layers. Based on the learned power equipment feature patterns, it determines whether there is power equipment in the image and determines its location and type. For example, the model can accurately identify whether there are abnormalities such as broken insulators or broken lines in the image, as well as the specific location and model of the tower, providing accurate data support for the subsequent calculation of the expected instructions by combining the kinematic model.

[0043] (4) Camera imaging plane deviation analysis: The deviation of the target in the camera image coordinate system can be obtained by the image captured by the gimbal camera. The desired position of the UAV and the desired gimbal command are obtained by the process of image coordinate system, camera coordinate system, rotation adjustment, global coordinate system and control command. The PnP (Perspective-n-Point) algorithm is used to transform from image coordinate system to camera coordinate system. The rotation and translation of the camera are solved by the known 3D points (the four corner points marked) and the corresponding 2D image points. The transformation from camera coordinate system to global coordinate system is done by matrix multiplication between camera coordinate system and UAV attitude rotation matrix.

[0044] For example, before rotation, the rotatedtvec (camera coordinate system) is: [0.2, 0.1, 2.5], 0.2m in front, 0.1m to the left, and 2.5m in height, while the poseGlb (UAV attitude rotation matrix) is: ; The rotated tvec (global coordinate system): rotatedtvec = poseGlb@rotatedtvec, where the global coordinate system's X-axis is North, Y-axis is East, and Z-axis is Down. Based on the transformation theory of the PnP model, in coordinate system transformation, the set of known 3D spatial feature points... ; and its pixel coordinates p on the image plane i =[u i ,v i ] T A projection model can be established using the PnP algorithm: ; Where K is the camera intrinsic parameter matrix, R is the rotation matrix, t is the translation vector, and s is the scaling factor. In this embodiment, after completing the PnP solution, the obtained R and t are used to backproject the 3D points onto the image plane to calculate the reprojection error. ; in, , Let be the pixel coordinates obtained by backprojection from the PnP estimation results, when the average error satisfies: ; Where, ε thA pixel error threshold of 1 to 3 pixels is used to determine if the conversion accuracy is reliable; otherwise, the following self-calibration process is automatically triggered: re-extracting image corner points and optimizing sub-pixel accuracy; combining historical frame feature points for temporal consistency weighting; re-executing the EPnP (Efficient PnP) algorithm or LM optimization to solve PnP. This closed-loop mechanism enhances the dynamic stability of PnP based on error feedback. Unlike the traditional static single-to-global coordinate system rotation matrix directionality, this embodiment adds inverse transformation consistency detection after attitude mapping. Let the UAV attitude rotation matrix be: ; Where θ represents the yaw angle of the UAV about the vertical axis, and the camera coordinates are converted to global coordinates as follows: Its inverse transformation is: ; By comparing the Euclidean distances of the original and inverse transform results: ; If E>δ th If the value is 1e-3m, it is assumed that the transformation matrix has an incorrect sign direction or rotation order, and matrix orthogonality correction and sign normalization are automatically performed.

[0045] (5) Send desired gimbal commands: The final output of the camera imaging plane deviation analysis is the offset (in meters) marked in the X, Y, and Z directions of the global coordinate system, which will be directly used for the control commands of the UAV. At the same time, the camera coordinate system transformation deviation (back projection error) will be sent as the desired rotation angle of the gimbal.

[0046] In servo control, a composite algorithm of PID and fuzzy control is adopted, enabling the gimbal to rotate rapidly when the target deviation is large and to enter fine-tuning when the deviation is small, thereby improving tracking stability. The specific steps are as follows: Construct control parameters and deviation quantification rules: These are based on the deviation e between the current position of the gimbal and the target position of the power equipment, and the rate of change e of the deviation. c As inputs to fuzzy control, both directly reflect the degree and trend of the target deviating from the gimbal's field of view, and are expressed as deviation e and deviation rate of change e. c The fuzzy control output u is divided into 7 fuzzy levels (such as negative large, negative medium, negative small, zero, positive small, positive medium, positive large) and corresponding quantization factors are established. According to the requirements of power equipment inspection scenarios, such as the field of view of the PTZ, a deviation threshold E0 (such as 5°) is set. When |e|>E0, it is judged as a large deviation scenario, and when |e|≤E0, it is judged as a small deviation scenario.

[0047] Establish a fuzzy control rule base: Based on manual inspection experience and gimbal control characteristics, formulate "if-then" fuzzy rules. The core logic is that the larger the deviation, the larger the control output; the larger the deviation change rate, the smoother the control output.

[0048] Example rule: If the deviation e is positive, the rate of change of deviation e c If e is zero, then the fuzzy output u is positive (large deviation, full speed rotation); if e is positive small, e c If the value is positive and small, then u is positive and small (smooth adjustment under small deviation). Through multiple simulations and field tests, contradictory rules are eliminated and boundary rules are added, i.e., control strategies when the deviation approaches zero, forming a complete fuzzy rule library containing 25 to 49 rules.

[0049] Fuzzy reasoning and sharpening calculation: This involves combining the real-time collected deviation e and deviation change rate e... c The fuzzy quantity is converted into a fuzzy quantity within the fuzzy universe by a membership function, and its membership degree to each fuzzy level is determined. Based on the fuzzy rule base, the output fuzzy quantity of each rule is calculated using the maximum and minimum inference method, and the output fuzzy quantities of all rules are synthesized using the centroid method to obtain the precise output value U of the fuzzy control. f .

[0050] Real-time calculation of PID control parameters: Calculate the PID control output value U based on the deviation e. p The formula is: (where K) p For proportionality coefficient, K i For the integral coefficient, K d (where K is the differential coefficient), the initial values ​​of the PID parameters are determined through engineering tuning methods (such as the critical proportional gain method). p0 K i0 K d0 This ensures the steady-state accuracy of the system under small deviations.

[0051] Adaptive allocation and output of control weights: Define fuzzy control weight λ and PID control weight 1-λ (λ∈[0,1]), and the weights change dynamically with the deviation e: When |e|>E0 (large deviation): λ=0.7 to 0.9, fuzzy control is the main method, and large control quantities are output quickly to drive the gimbal to rotate quickly and align with the target; When |e|≤E0 (small deviation): λ=0.1 to 0.3, PID control is the main method, and the steady-state error is offset by the synergistic action of proportional, integral and derivative control to achieve fine adjustment; Transition interval (e.g.) ): λ changes linearly to avoid gimbal jitter caused by sudden changes in control quantity.

[0052] The fuzzy control output U f and PID control output Up The final control quantity of the gimbal servo motor is obtained by weighted synthesis. The gimbal is driven to complete the corresponding rotation action. The current position and attitude information are collected in real time by the encoder and gyroscope built into the gimbal, and the new deviation e and the rate of change of deviation e are calculated. c .

[0053] The new e and e c The composite algorithm is re-inputted to achieve closed-loop iteration of the control process until the deviation e approaches zero. The gimbal and the UAV adopt a fast-slow cooperative control mechanism: when the target position changes suddenly, the gimbal first rotates quickly to align with the target, and then guides the UAV to gradually adjust its course and speed through data feedback, ultimately realizing the autonomous linkage control between the UAV and the gimbal.

[0054] II. Multi-payload configuration method and general interface design for UAVs; Standardized mechanical, electrical, and communication interfaces are designed to enable plug-and-play functionality between the payload and the flight control system. When the flight control system detects a new payload, it automatically identifies its type and parameters, updates the control strategy, and establishes a two-way communication link to achieve autonomous, coordinated control between the payload and the flight control system. The specific steps are as follows: (1) Load physical connection and electrical self-test; The operator installs the payload onto the UAV mounting point using the standardized mechanical interface. Once the mechanism is in place, a mechanical connection confirmation signal is triggered. The flight controller detects the power supply circuit continuity through the electrical interface to confirm that the payload is stably connected to the power grid, and simultaneously sends a power-on self-test command to the payload. After receiving the command, the payload completes its own hardware self-test, such as the status of the lens, sensors, and communication module, and sends feedback information on self-test readiness or fault status to the flight controller through the electrical interface.

[0055] (2) The flight control system automatically identifies the load type and parameters; The flight controller sends identity and parameter query commands to payloads that have completed self-testing via a standardized communication interface. The command format is predefined according to the communication protocol. After receiving the query command, the payload actively reports its core information, including payload type (e.g., visible light inspection camera or infrared thermal imager), operating parameters (e.g., field of view, resolution, operating voltage range), control command set, and status feedback field definitions. After receiving the information, the flight controller matches and verifies it against the built-in payload parameter database to complete payload type identification and parameter registration.

[0056] (3) Update the flight control strategy; Based on the identified payload type and parameters, the flight controller calls or generates control logic adapted to the payload from the control strategy library. Its core is to clarify the linkage rules of how the flight controller commands the payload and how the payload cooperates with the flight controller. The rules include the matching rules of the payload's working mode and flight status, the coordination rules of the payload's actions and the gimbal / UAV's attitude, such as when the payload adjusts the shooting angle, the flight controller synchronously compensates for the UAV's attitude deviation, and the priority scheduling rules of multiple payloads, such as when an emergency inspection is carried out, the infrared camera has a higher priority than the visible light camera.

[0057] If it is a new type of payload, the flight controller will automatically generate a basic control strategy according to the general control strategy template, combined with the instruction set and parameters reported by the payload, to ensure basic linkage capability.

[0058] (4) Establish a two-way communication link (data interaction channel); The flight controller automatically configures its own communication module parameters based on the communication parameters reported by the payload, and completes communication link negotiation with the payload. After the negotiation is completed, the flight controller sends a link establishment confirmation command to the payload, and the payload feeds back a link ready signal. The two-way communication link is officially established, that is, the flight controller can send control commands to the payload, and the payload can feed back status and operation data to the flight controller in real time. When multiple payloads are connected, the flight controller assigns an independent communication address to each payload and manages the communication link using time-division multiplexing or priority preemption to avoid data conflicts.

[0059] After the link is established, a dynamic assessment mechanism for communication feasibility and quality is introduced to monitor link stability, data transmission reliability, and potential anomalies in real time. The flight control system uses an embedded link quality assessment model to comprehensively consider indicators such as signal strength RSSI, packet reception rate PRR, round-trip time RTT, packet loss rate, and interference intensity to calculate a comprehensive link quality index. ; In the formula, Q i The link quality score for the i-th payload is given, with α1 to α5 being weighting coefficients that can be adaptively adjusted based on payload type, task priority, and environmental interference. When Q... i min At that time, the communication parameter optimization module is automatically triggered to dynamically adjust the time slot allocation t. s Channel selection f c Or transmission power P t Optimizing link performance can be expressed as: ; In multi-payload parallel communication scenarios, the flight control system simultaneously introduces data conflict detection and semantic consistency verification mechanisms to verify the measurement data z transmitted by the payloads. i,j Perform Mahalanobis distance verification on the i-th load and j-th data: ;​ Where, Σ i,j With Σ k,l These are the load data covariance matrices, If a semantic conflict is detected, data retransmission, fusion, or filtering is triggered, and the timestamp sequence {t} is processed. i,j The order consistency of messages with the payload identifier is checked to ensure that there are no duplicates or out-of-order messages, thereby preventing communication anomalies from affecting task execution.

[0060] To optimize the scheduling of multi-payload communication, the flight control system establishes an adaptive resource allocation model, assigning the link quality Q of each link as the threshold. i With load mission priority w i Combine these steps to construct an optimization function: ; Where 'a' represents communication resource allocation actions (such as time slots, channels, power, etc.), and D i (a) is the delay function, and β is the adjustment coefficient. By solving the problem using reinforcement learning or dynamic programming methods, priority management, dynamic time slot adjustment, and link optimization among multiple payload tasks can be achieved, enabling high-priority payloads to maintain reliable connections in complex communication environments.

[0061] The above integrated mechanism realizes closed-loop control from link establishment, communication feasibility assessment, real-time quality monitoring, data conflict detection to resource scheduling optimization. It has self-adaptive, self-correcting and self-repairing capabilities, which can significantly improve the stability, reliability and real-time performance of UAV multi-payload collaborative operations, ensure accurate execution of payload commands and high integrity of mission data, and provide stable and reliable communication support for UAV mission execution in complex environments.

[0062] (5) Autonomous linkage operation between payload and flight control; Based on mission requirements and updated control strategies, the flight controller issues targeted control commands to the payload, such as "turn on the visible light camera, adjust the field of view to 30°, and aim at the power tower 100 meters ahead." After receiving the commands, the payload executes the corresponding actions and provides real-time feedback on the operation execution status and information according to a unified protocol.

[0063] After receiving feedback information, the flight controller makes real-time decision adjustments: if the payload reports "target not aligned", it issues angle correction commands according to the control strategy; if multiple payloads are operating, it adjusts the flight attitude according to the feedback of the payload's working status, such as controlling the UAV to hover stably when the payload is taking pictures; if the payload reports a fault, it automatically triggers fault handling strategies, such as switching to a backup payload, reducing flight speed, or returning to home.

[0064] (6) Multi-load coordination and task scheduling optimization; The flight controller dynamically schedules all connected payloads in real time, combining their status information with task priorities. When the UAV flies near a power tower, it first controls the infrared camera to start thermal imaging detection, while the visible light camera prepares to capture details. After the detection results are fed back to the flight controller in real time, the flight controller adjusts the UAV's position so that the visible light camera can accurately capture abnormal areas. The payload execution status is synchronized to the flight control task scheduling module in real time to optimize subsequent flight paths and operation sequences.

[0065] The control strategy here refers to the full-process linkage rules covering payload identification, command issuance, status feedback, flight coordination, and multi-payload scheduling. This includes rules for matching payload types with flight control commands, rules for coordinating payload status with flight actions, rules for prioritizing multiple payloads, and rules for handling anomalies. The general interface design supports simultaneous operation of multiple payloads, with real-time feedback of payload execution status and information to the flight control system, such as... Figure 3 The diagram is used for mission scheduling and flight adjustments.

[0066] III. Self-disturbance rejection pose correction technology; Wind disturbances, center of gravity changes, and attachment point offsets are all considered as total disturbances. An Extended State Observer (ESO) is used to estimate and compensate for these disturbances online. The core advantage of ESO is that it does not require precise modeling of the specific form of the total disturbance; it only uses measurable outputs such as UAV attitude and velocity to estimate the system's own dynamic summation disturbance in real time. The disturbance estimate is then used for feedforward compensation. The specific steps are as follows: (1) Establish a simplified dynamic model of the UAV; A simplified dynamic model is established, using the UAV's attitude loop (roll, pitch, yaw) and velocity loop (X / Y / Z axis velocities) as the control objects. The attitude loop model is as follows: The velocity loop model is as follows: ; Where ω is the angular velocity vector, To control the torque, A / B is the model coefficient matrix, d a The attitude loop represents the total perturbation, v is the linear velocity vector (X / Z axis velocity), F is the thrust vector, C / D is the model coefficient matrix, and d v The total disturbance of the velocity loop (wind resistance, inertial force caused by changes in the center of gravity, etc.).

[0067] (2) Design and parameter tuning of the extended state observer; First, the total perturbation of the attitude loop is... Total disturbance of velocity loop They are respectively considered as "expansion state" (n is the state dimension of the original system), transforming the original system into an augmented system of states plus extended states. A nonlinear observer is designed for the augmented system, taking the attitude loop ESO as an example, in the following form: ; In the formula, These are the observed angular velocity values. These are the observed values ​​of angular acceleration. β1 represents the total perturbation estimate of the attitude loop; β1 to β3 represent the observer gain (tuned using the pole placement method). The function is a nonlinear saturation function (to improve observation accuracy with small errors), where α (0 < α < 1) and δ (small positive number) are nonlinear parameters. Parameter tuning principles: Adjust β1 to β3 according to the UAV's response speed requirements (the larger the gain, the faster the observation response, but it is more susceptible to noise interference); α is set to 0.5 to 0.8 (to balance linear and nonlinear characteristics); δ is set to 0.01 to 0.1 (to suppress measurement noise).

[0068] (3) Online estimation of total disturbance; ESO (Electronic State Observation) acquires the measurable state of the UAV in real time: angular velocity ω is obtained through a gyroscope, and linear velocity v is obtained through GPS / IMU, which serve as inputs to the observer; nonlinear observation calculations utilize the acquired ω / v, combined with the current output control torque, to perform ESO calculations. The thrust F is calculated in real time using the aforementioned nonlinear equations, and the estimated value of the attitude loop disturbance is output. Velocity loop disturbance estimate ;right , A first-order low-pass filter is used to remove high-frequency measurement noise, resulting in a smooth disturbance estimate. , .

[0069] (4) Total disturbance feedforward compensation; The smoothed disturbance estimate is used as the negative feedforward injection control input to the attitude loop and velocity loop to counteract the effect of the total disturbance on the system. Attitude loop compensation: Compensated control torque command , The base torque command under undisturbed conditions; velocity loop compensation: the compensated thrust command. F0 is the basic thrust command under undisturbed conditions; compensation is used to approximately transform the system into an undisturbed linear system to improve control robustness.

[0070] After injecting the disturbance estimate into the UAV's attitude and velocity loops, the angular velocity and thrust are dynamically adjusted through a nonlinear error feedback law to maintain flight stability. The core of the nonlinear error feedback law is to dynamically adjust the feedback strength according to the magnitude of the error. That is, when the error is large, the feedback is strengthened to quickly correct the deviation, and when the error is small, the feedback is weakened to avoid overshoot. Ultimately, smooth and precise adjustment of angular velocity and thrust is achieved. The specific steps are as follows: A. Define the control error; Calculate the control errors of the attitude loop and velocity loop, where the attitude loop error is eω=ωd -ω, where ω d ω is the desired angular velocity, generated by task instructions / path planning; ω is the measured angular velocity from the gyroscope; the velocity loop error is e. v =v d -v, where v d The desired linear velocity is generated by flight trajectory planning; v is the measured linear velocity from GPS / IMU, and the error rate of change is also calculated. ; B. Design a nonlinear feedback function (the core of the feedback law); The attitude loop nonlinear feedback law is designed using piecewise nonlinear functions: ; Among them, K pω K dω Let fal(-) be the proportional and differential feedback gain matrix, and fal(-) be a nonlinear function with the following characteristics: when (Large error): , By taking a value between 0.5 and 0.7, the feedback strength increases with the increase of error, thus achieving rapid correction. when (Large error): It features near-linear feedback to avoid high-frequency jitter.

[0071] The nonlinear feedback of the velocity loop is as follows: ; The design logic is consistent with the attitude loop, α v Take a value between 0.6 and 0.9.

[0072] C. Generate the final control command by combining disturbance compensation; The basic instruction for outputting the nonlinear feedback law , and the disturbance estimate of the ESO output ( , Perform feedforward compensation to obtain The final control command after limiting The command is sent to the motor controller to drive the motor and adjust its speed, thereby achieving dynamic adjustment of angular velocity and thrust. Simultaneously, error calculation, nonlinear feedback, disturbance compensation, and command issuance are repeated at a frequency of 100 to 200 Hz to achieve closed-loop iterative control until the error converges to an acceptable range (e.g., ...). ).

[0073] When the disturbance estimate exceeds the threshold, the mission replanning module is triggered to adjust the flight trajectory and operational actions to ensure operational continuity and safety.

[0074] IV. Simulation verification of flight control for multi-payload unmanned aerial vehicles; Complex operating conditions such as wind disturbance, hanging point offset, and sudden load changes are introduced into the simulation scenario to verify the robustness of the algorithm. An example of a power operation scenario is shown below: (1) Insulator hydrophobicity detection: The UAV gimbal camera takes pictures, starts the water spraying device, takes pictures of the insulator after water spraying, the computing module identifies the water droplet contact angle, and outputs the hydrophobicity level; the onboard computing unit 4 of the equipment is based on machine vision and digital image processing technology. Without manual intervention, it automatically extracts water droplet features and calculates the contact angle from the insulator after water spraying pictures taken by the gimbal camera. The core technologies include image segmentation, edge detection, curve fitting, etc. The specific steps are as follows: Image preprocessing: The onboard computing unit receives the RGB image transmitted from the gimbal camera and converts it into a grayscale image to reduce the color dimension and simplify calculations. Gaussian filtering is used to eliminate environmental noise in the image, while histogram equalization or adaptive contrast enhancement algorithms are used to improve the grayscale difference between the water droplets and the insulator surface, creating a clear boundary between the white / transparent water droplets and the dark / light-colored insulator surface, preparing for subsequent segmentation.

[0075] Water droplet region segmentation: Based on the preprocessed grayscale image, the Otsu adaptive thresholding algorithm is used to automatically determine the segmentation threshold, and the image is binarized (the water droplet region is set as the foreground "1" and the insulator background is set as "0") to achieve preliminary separation of the water droplets from the background. At the same time, the binarized image is subjected to erosion-dilation operation: first, the small foreground noise is removed by erosion, and then the true outline of the water droplet is restored by dilation to avoid gaps in the edges of the segmented water droplets. A connected component labeling algorithm is used to mark all independent water droplet regions in the image. Then, effective water droplets are selected according to preset conditions, and stable attached water droplets used to calculate the contact angle are retained.

[0076] Contact angle calculation: The onboard computing unit 4 of the equipment uses an ellipse fitting algorithm to calculate the contact angle, as follows: The coordinates (x, y) of all pixels on the edge contour of the water droplet are input into the ellipse fitting model, and the parameters of the ellipse equation are solved by the least squares method. Since the outline of the stably attached water droplet is approximately a part of an ellipse, the fitting result can accurately reflect the true shape of the water droplet.

[0077] Based on the equation of the ellipse and the already located contact points A and B, calculate the slope of the tangents to the water droplet at points A and B; then, using the vector angle formula, calculate the angles between the two tangents and the surface of the insulator (horizontal baseline), and take the average of the two angles as the final contact angle θ of the water droplet.

[0078] For multiple valid water droplets in the same insulator image, the contact angle is calculated separately and then the arithmetic mean is taken to reduce the error caused by the difference in the shape of individual water droplets and improve the measurement accuracy.

[0079] The hydrophobicity level of insulators is determined using the contact angle value as the core indicator. Referring to power industry standards (such as DL / T864-2018 "Room Temperature Curing Silicone Rubber Anti-Pollution Flashover Coating for High Voltage Insulators"), a correspondence between contact angle and hydrophobicity level is established, as shown in Table 1 below: Table 1. Correspondence between contact angle and hydrophobicity grade

[0080] The onboard computing unit 4 of the equipment packages the hydrophobicity level (such as HC2-strong hydrophobicity), the original contact angle value (such as 135.2°), and typical water droplet images (with contact angles marked), and transmits them to the UAV flight control system or ground control station, providing an intuitive basis for maintenance personnel to assess the insulation performance of insulators (the decrease in hydrophobicity may lead to the risk of flashover).

[0081] (2) Voltage detection and connection: The UAV carries a voltage detector and connection device. The voltage detection module detects the energized status of the line. If the power is off, the connection action is started. The device clamps the wire to complete the connection.

[0082] The core of the voltage detection module uses high-voltage electric field induction technology (capacitive coupling principle), which can detect the energized state without direct contact with the line. The specific implementation steps are as follows: The drone flies to the vicinity of the target line according to the mission instructions. It adjusts the attitude of the gimbal so that the induction probe of the voltage detection module is facing the line. The probe and the line maintain a preset safe distance, such as 10-30cm, to avoid the risk of discharge. The voltage detection module starts the initialization program, self-checks the status of the induction probe, signal processing circuit, and data transmission module, and calibrates the zero point to ensure that the detection system is in normal working condition.

[0083] Electric field signal acquisition and preprocessing: The induction probe acquires the electric field signal around the line. When the line is energized, it will generate an alternating electric field or electrostatic field. The probe converts the electric field signal into a weak voltage signal (mV level) through capacitive coupling effect. The weak voltage signal is amplified by the amplifier circuit inside the module, and then filtered by the filter circuit to remove 50Hz power frequency interference and high frequency noise to extract the effective signal, thus obtaining a stable and processable electrical signal.

[0084] Signal Feature Analysis and Liveness Detection: The pre-processed analog electrical signal is converted into a digital signal and transmitted to the built-in microprocessor (MCU) for feature analysis. The MCU calculates the core feature parameters of the signal: for AC lines, the focus is on calculating the signal amplitude and frequency; for DC lines, the focus is on calculating the DC component of the signal. The calculated feature parameters are compared with preset thresholds according to the line voltage level, such as 10kV, 110kV, 220kV, etc., in advance: if the feature parameters are greater than the preset thresholds and the signal frequency or DC component matches the characteristics of the line power supply type, the line is determined to be live; if the feature parameters are less than or equal to the preset thresholds and the signal has no stable power supply characteristics, the line is determined to be de-energized.

[0085] Transmission and Feedback of Detection Results: The voltage detection module transmits the detection results of energized or de-energized lines to the UAV flight control system through a standardized communication interface. Simultaneously, it provides feedback on the detection confidence level (e.g., a 99.8% confidence level indicating a power outage), providing a reliable basis for flight control decisions. Based on the safety requirements of voltage detection and connection operations, the energized state of the lines is mainly divided into three categories, clearly defining the operational risk level, as shown in Table 2. Table 2. Operational Risk Level Table

[0086] In summary, by leveraging the technical solution of this invention, target tracking and ultra-stable imaging are achieved through deep integration of vision and flight control, overcoming the limitations of traditional independent control of gimbal and flight control. It employs a PnP dynamic self-calibration closed-loop control architecture combined with fuzzy PID adaptive composite control. Furthermore, in the servo control stage, the weights of fuzzy control and PID control are dynamically allocated based on the target deviation. For large deviations, fuzzy control is prioritized for rapid alignment, while for small deviations, PID control is prioritized to compensate for steady-state errors. Simultaneously, combined with real-time data interaction between the gimbal and the UAV, a collaborative mechanism of fast gimbal adjustment and slow UAV compensation is achieved. This reduces image shake compared to traditional solutions, maintaining stable imaging of the target at the center of the field of view even with slight oscillations in the UAV's attitude. This meets the high-precision requirements for dynamic target capture in emergency rescue and clear acquisition of equipment details in power line inspection.

[0087] This invention addresses the pain points of cumbersome payload adaptation and coordination conflicts in existing systems by designing a complete solution with standardized interfaces, adaptive strategy generation, and dynamic priority scheduling. Through predefined identity recognition protocols and parameter interaction standards, the flight controller can complete payload type identification and parameter registration within 50ms and automatically call or generate adaptive control strategies (such as automatically adjusting the UAV's hovering stability parameters to match the camera's exposure requirements when an infrared camera is detected). When multiple payloads operate in parallel, resources are dynamically allocated based on task urgency and payload characteristics. For example, in emergency rescue, priority is given to ensuring the communication bandwidth and action response speed of the throwing device, while in power line inspection, priority is given to scheduling data interaction between the gimbal camera and the detection sensor. The efficiency of multi-payload collaborative operation is improved compared to traditional distributed control.

[0088] This invention differs from traditional disturbance rejection schemes that rely on precise dynamic models. It innovatively introduces an Extended State Observer (ESO) with coupled disturbance decoupling technology, treating wind disturbances, sudden load changes, attachment point offsets, and coupled disturbances from load actions as a unified total disturbance. The ESO estimates attitude and velocity loop disturbances online by collecting gyroscope angular velocity and GPS linear velocity data in real time, and injects them into the control loop through negative feedforward compensation. This can offset the effects of ±10m / s gusts, sudden loading / unloading of 5kg loads, and 10cm attachment point offsets. At the same time, for coupled disturbances generated by the extension and retraction of the robotic arm and the movement of the throwing device, a new decoupling algorithm based on particle swarm optimization (PSO) is added to separate coupled components and compensate them separately, preventing disturbances from being propagated and amplified within the system.

[0089] This invention constructs a closed-loop system encompassing image acquisition, target recognition, payload action, data feedback, early warning decision-making, and task optimization. Its onboard computing module can perform customized tasks such as target detection, hydrophobicity level determination, and non-contact voltage testing in real time, transmitting the results to the ground station via an encrypted communication link. The anomaly warning response time is ≤1 second, allowing maintenance personnel to remotely monitor the operational status in real time. It also possesses scene-adaptive capabilities, dynamically adjusting parameters according to the operation type: automatically increasing gimbal camera resolution and detection frequency during power line inspections; prioritizing delivery accuracy and communication stability during emergency rescues; and optimizing flight trajectory and hovering time during material delivery. The operational coverage is expanded compared to traditional systems.

[0090] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-payload synchronous control system for unmanned aerial vehicles (UAVs), characterized in that, The system includes: The linkage control unit is used to detect images captured by the drone, obtain the camera imaging plane deviation based on the detection results, and combine it with the kinematic model to calculate the expected commands of the drone and the gimbal. The multi-load configuration unit is used to determine the control logic based on the load type and expected instructions, and to issue operation instructions to the load according to the control logic, so as to realize the linkage control between the load and the UAV controller. The control command adjustment unit is used to acquire the flight state of the UAV when executing the desired command, take the flight state as input, calculate the disturbance value of the UAV, and determine the final control command of the UAV. The onboard computing unit is used to drive the gimbal and drone according to the expected instructions and final control instructions, collect power equipment information obtained by the cameras and sensors on the drone, judge the performance of the power equipment based on the power equipment information, and output the drone's operation decision suggestions based on the performance.

2. The UAV multi-payload synchronous control system according to claim 1, characterized in that, The linkage control unit includes: The image detection model building module is used to collect historical aerial images of target electronic devices taken by drones under different conditions, and to build a target detection model using historical aerial images and deep learning technology. The detection and recognition module is used to perform detection processing on real-time images captured by the drone using a target detection model, extract real-time image features, and obtain the location of the electronic device based on the real-time image features. The deviation analysis module is used to obtain the image coordinates of the electronic device on the real-time image based on the position of the electronic device, and to build a projection model based on the image coordinates to analyze the deviation of the camera imaging plane. The gimbal expectation command generation module is used to output control commands for the UAV and gimbal based on the camera imaging plane deviation, and to use fuzzy control to control the operation status of the UAV and gimbal based on the control commands.

3. The UAV multi-payload synchronous control system according to claim 2, characterized in that, The process of obtaining the image coordinates of the electronic device on the real-time image based on its position, and establishing a projection model based on the image coordinates to analyze the camera imaging plane deviation, includes: The image coordinates of the electronic device on the real-time image are obtained based on the position of the electronic device, and the image coordinates are combined with the set of feature points of the electronic device in three-dimensional space to establish a projection model. The rotation matrix and translation vector of the camera on the UAV are solved based on the projection model, and the feature points in the three-dimensional space are back-projected onto the real-time image captured by the UAV using the rotation matrix and translation vector. The back projection error is calculated based on the back projection results, and the average error of the back projection error is obtained. When the average error is less than the pixel error threshold, the transformation process from the image coordinate system to the camera coordinate system is completed. A UAV attitude rotation matrix is ​​established based on the yaw angle when the UAV captures real-time images. The camera coordinate system is then transformed into a global coordinate system using the UAV attitude rotation matrix. An inverse transformation is performed on the global coordinate system, and the Euclidean distance of the global coordinate system before and after the inverse transformation is compared to output the camera imaging plane deviation.

4. The UAV multi-payload synchronous control system according to claim 3, characterized in that, The step of outputting control commands for the UAV and gimbal based on the camera imaging plane deviation, and using fuzzy control to control the operating status of the UAV and gimbal based on the control commands, includes: Based on the camera imaging plane deviation, the offset of the electronic device in the global coordinate system is output as the UAV control command, and the back projection deviation generated during the camera coordinate system transformation is sent as the desired rotation angle of the gimbal. The deviation and rate of change of the current position of the gimbal and the target position of the power equipment are obtained based on the expected rotation angle of the gimbal, and are used as fuzzy control inputs for the deviation and rate of change of the deviation to reflect the degree and trend of the target deviating from the field of view of the gimbal. The deviation, the rate of change of deviation, and the fuzzy control output are divided into several fuzzy levels, and corresponding quantization factors are established. The fuzzy levels, quantization factors, and gimbal control characteristics are combined to establish a fuzzy control rule base. The output fuzzy value of each rule is calculated using the inference method based on the fuzzy rule base, and the output fuzzy values ​​of all rules are synthesized using the centroid method to obtain the fuzzy control output value. The control output value is then calculated based on the deviation. The fuzzy control output value and the control output value are combined according to weights to obtain the control quantity used to drive the gimbal servo motor, which drives the gimbal to complete the corresponding rotation action, and adjusts the UAV's heading and speed according to the UAV control command.

5. The UAV multi-payload synchronous control system according to claim 1, characterized in that, The multi-load configuration unit includes: The flight control measurement update module is used to install the payload to the UAV mounting point and send self-test commands, and determine the control logic of the adapted payload based on the payload type and payload parameters. The control logic includes the payload working mode and the expected commands of the UAV and gimbal. The communication link establishment module is used to establish a communication link based on the payload's load parameters and optimize the communication quality of the communication link based on communication feasibility to ensure stable communication between the payload and the UAV. The payload flight control linkage module is used by the UAV controller to send operation instructions to the payload according to the control logic. The payload receives the operation instructions through the communication link, executes the corresponding actions, and provides feedback on the execution status. The collaborative scheduling optimization module is used to aggregate the execution status of payloads using the UAV controller, and optimize the flight path and payload operation sequence of the UAV based on the execution status and task priority.

6. A multi-payload synchronous control system for unmanned aerial vehicles according to claim 5, characterized in that, The process of establishing a communication link based on the payload's load parameters and optimizing the communication quality of the link based on communication feasibility to ensure stable communication between the payload and the UAV includes: The UAV controller automatically configures the UAV's communication parameters based on the payload parameters reported by the payload, and negotiates a communication link with the payload. After the negotiation is completed, the UAV controller sends a link establishment confirmation command to the payload, and the payload sends back a link ready signal, thus completing the establishment of a two-way communication link. Using communication feasibility and quality dynamic assessment technology, the comprehensive link quality index of the two-way communication link is calculated based on signal strength, round-trip delay and interference intensity, and the comprehensive link quality index is compared with the minimum quality index. If the overall link quality index is greater than or equal to the minimum quality index, it means that the bidirectional communication link quality meets the requirements. If the overall link quality index is less than the minimum quality index, load parameter optimization is performed until the bidirectional communication link quality meets the requirements. An optimization function is constructed by combining the quality of the two-way communication link with the priority of the payload tasks using the UAV controller. The optimization function is then solved to prioritize the payload tasks, thereby ensuring stable communication between the payload and the UAV.

7. A multi-payload synchronous control system for unmanned aerial vehicles according to claim 1, characterized in that, The control command adjustment unit includes: The dynamic model and observer design module is used to establish a dynamic model with the attitude loop and velocity loop of the UAV as objects, and to design an extended state observer based on the total perturbation of the attitude loop and the total perturbation of the velocity loop. The estimation output module is used to obtain the angular velocity and linear velocity of the UAV when executing the desired command, and use them as input to the extended state observer to output attitude loop and velocity loop perturbation estimates. The UAV adjustment module is used to inject the attitude loop disturbance estimate and the velocity loop disturbance estimate as negative feedforward into the attitude loop and velocity loop control inputs, and adjust the control torque command and thrust command of the UAV in combination with the dynamic model.

8. A multi-payload synchronous control system for unmanned aerial vehicles according to claim 7, characterized in that, The step of injecting the attitude loop disturbance estimate and the velocity loop disturbance estimate as negative feedforward control inputs into the attitude loop and velocity loop, and adjusting the UAV's control torque command and thrust command in conjunction with the dynamic model, includes: Based on the dynamic model, the desired angular velocity and desired linear velocity of the UAV are output, and the attitude loop error and velocity loop error are calculated by combining the angular velocity and linear velocity. Based on the rate of change of the output error, the nonlinear feedback value of the attitude loop and the nonlinear value of the velocity loop are designed. The attitude loop disturbance estimates and velocity loop disturbance estimates are used as negative feedforward injections into the attitude loop and velocity loop control inputs. They are combined with the attitude loop nonlinear feedback values ​​and velocity loop nonlinear values ​​for feedforward compensation, and the compensated UAV control torque and thrust commands are output as the final control commands.

9. A multi-payload synchronous control system for unmanned aerial vehicles according to claim 1, characterized in that, The onboard computing unit of the device includes: The drive module is used to drive the gimbal and drone according to the expected instructions and final control instructions; The insulation performance calculation module is used to segment and identify water droplet characteristics after insulators are sprayed with water using an adaptive threshold algorithm captured by a gimbal camera, and calculate the contact angle by combining the ellipse equation to output the hydrophobicity level and determine the insulation performance of the insulator. The line energization detection module is used to collect electric field signals within the target range of the line using an induction probe mounted on a drone, and output the energization status of the line after performing feature analysis on the electric field signals.

10. A multi-payload synchronous control system for unmanned aerial vehicles according to claim 9, characterized in that, The process involves using an adaptive threshold algorithm to segment and identify water droplet characteristics in insulator images captured by a gimbal camera, and then combining this with the elliptic equation to calculate the contact angle and output the hydrophobicity level. The determination of the insulator's insulation performance includes: The system receives color images transmitted from the PTZ camera, converts the color images into grayscale images, uses adaptive thresholding technology to determine the segmentation threshold, and binarizes the grayscale images to separate the water droplet region from the insulator background, thus obtaining a binarized image. Erosion, dilation and connected region marking are performed on the binarized image to mark the independent water droplet regions in the binarized image. Valid water droplets are selected according to preset conditions. The coordinates of all pixels on the edge contour of the valid water droplets are input into the ellipse fitting model to calculate the tangent slope of the valid water droplets. The contact angle of the effective water droplets is output based on the tangent slope. All effective water droplets are summarized, and the summarized results are compared with the hydrophobicity level relationship table to judge the insulation performance of the insulator.

Images