Unmanned aerial vehicle hoisting control method and system based on industrial vision

Abstract

The invention relates to the technical field of unmanned aerial vehicle hoisting and control, in particular to an unmanned aerial vehicle hoisting control method based on industrial vision, which comprises the following steps: S1, an unmanned aerial vehicle acquires multi-angle visual image data of a load and real-time tension information of each hoisting rope during hoisting operation; and S2, performing three-dimensional point cloud reconstruction according to the data, extracting feature points to calculate a visual gravity center, determining the mechanical gravity center of the lifting rope according to the stress analysis of the lifting rope, and dynamically adjusting the visual gravity center based on the coordinate difference and the stress balance relationship between the two to obtain the comprehensive gravity center position and the speed state of the load. In the invention, through comprehensive gravity center modeling and self-adaptive control based on industrial vision and mechanics fusion, dynamic coordination and stable control of a load attitude and an unmanned aerial vehicle attitude are realized, so that the problems that a traditional unmanned aerial vehicle hoisting method mostly depends on a single vision identification mode, and a gravity center dynamic correction mechanism is lacked, so that the unmanned aerial vehicle cannot be hoisted easily are solved. And the problem that the load is easy to swing and lose stability is solved.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) hoisting and control technology, and in particular to a UAV hoisting control method and system based on industrial vision. Background Technology

[0002] In recent years, the application of drones in the industrial field has developed rapidly, playing a vital role, especially in scenarios such as high-altitude hoisting, power line maintenance, construction, and logistics transportation. As the complexity of hoisting tasks increases, single-drone or multi-drone collaborative hoisting of heavy components has become commonplace. However, complex environmental factors such as wind disturbances, nonlinear load swaying, multi-drone attitude coordination, and incomplete sensor information pose significant challenges to the stability, accuracy, and safety of drone hoisting systems. Traditional drone hoisting methods mostly rely on single-vision recognition, and the lack of a dynamic center of gravity correction mechanism makes the load prone to swaying and instability. Summary of the Invention

[0003] To overcome the above shortcomings, this invention provides a drone hoisting control method and system based on industrial vision, which aims to improve the problem that traditional drone hoisting methods mostly rely on single visual recognition and lack a dynamic center of gravity correction mechanism, resulting in the load being prone to swinging and instability.

[0004] In a first aspect, the present invention provides the following technical solution: a drone hoisting control method based on industrial vision, comprising the following steps: S1. During hoisting operations, the UAV collects multi-angle visual image data of the load and real-time tension information of each hoisting rope; S2. Based on the data, perform three-dimensional point cloud reconstruction and extract feature points to calculate the visual center of gravity. At the same time, determine the mechanical center of gravity of the suspension rope based on the force analysis of the suspension rope, and dynamically adjust the visual center of gravity based on the coordinate difference and force balance relationship between the two to obtain the comprehensive center of gravity position and velocity state of the load. S3. Based on the obtained comprehensive center of gravity state, establish a nonlinear dynamic model of the UAV, sling, and load. This model considers the elasticity of the sling, the inertial coupling of the load, and external disturbances, and is used to describe the dynamic response characteristics of the UAV hoisting system. S4. Based on the dynamic model, an adaptive control algorithm is used to generate UAV attitude and hoisting control commands. By prioritizing the adjustment of position error in the hovering state and prioritizing the adjustment of speed error in the moving state, dynamic correction and stable control of the load center of gravity are achieved. S5. While executing control commands, the UAV monitors changes in visual center of gravity and suspension cable tension in real time, and feeds the monitoring results back to the control module to form a closed-loop control in order to suppress load sway and maintain balance. S6. When the monitoring results indicate that the load attitude or the tension of the hoisting rope exceeds the threshold, the system triggers a safety mechanism to automatically decelerate, hover, or stop in an emergency until the load stabilizes or the hoisting task is completed.

[0005] By adopting the above technical solution, based on the integrated center of gravity modeling and adaptive control of industrial vision and mechanics, dynamic coordination and stable control of load attitude and UAV attitude are achieved, thereby improving the problem that traditional UAV hoisting methods mostly rely on single visual recognition and lack a dynamic center of gravity correction mechanism, which leads to load swinging instability.

[0006] Preferably, the image data and tensile information include: Before hoisting, the drone collects multi-angle images of the load using multiple industrial cameras. Each image is input into the 3D point cloud reconstruction module. A point cloud model of the load is generated using a stereo matching-based reconstruction algorithm. The point cloud is registered and features are extracted. The geometric center is calculated to obtain the visual centroid. The force values ​​of each suspension rope are obtained by the suspension rope tension sensor and the torque balance equation is established by combining the coordinates of the suspension point to calculate the mechanical center of gravity. Based on the coordinate difference between the visual center of gravity and the mechanical center of gravity and the mechanical constraints, the visual center of gravity is dynamically adjusted to obtain the comprehensive center of gravity position and velocity state of the load, and the comprehensive center of gravity state is input into the control module.

[0007] Preferably, the visual center of gravity and the mechanical center of gravity include: A nonlinear dynamic model of the UAV, sling, and load is established based on the comprehensive center of gravity state obtained by dynamic adjustment of the visual and mechanical center of gravity. The suspension rope is modeled as a flexible rod with elastic and damping properties; The relationship between the system's kinetic and potential energy was derived using the Lagrange equation, and a dynamic equation was established that includes the UAV thrust, cable tension, load inertia, and external disturbances. An adaptive update mechanism for mechanical model parameters is introduced to automatically correct model coefficients when the load mass or lifting point changes.

[0008] Preferably, the nonlinear dynamic model includes: The control module includes an upper-level load center of gravity adjustment control unit and a lower-level attitude control unit; The upper control unit generates load adjustment commands based on the overall center of gravity deviation, while the lower control unit generates attitude and thrust control commands. Increase the weight of position error while hovering, and increase the weight of speed error while moving. An adaptive algorithm is used to update the control gain in real time and output composite control commands.

[0009] Preferably, the UAV attitude and hoisting control commands include: During the execution of control commands by the drone, data on visual center of gravity and sling tension are continuously collected; The control module compares the target state with the current state, calculates the deviation, and automatically adjusts the feedback gain factor. When a change in pitch is detected, the thrust response rate is adjusted, and the control gain is reduced when the attitude stabilizes. The system parameters are periodically updated to maintain control accuracy.

[0010] Preferably, the changes in the visual center of gravity and the tension of the suspension rope include: The system sets multiple levels of safety thresholds, including the rope force threshold, the load attitude angle threshold, and the UAV attitude threshold; When the detected value reaches the warning threshold, the control module reduces the flight speed and adjusts the attitude; When the detected value exceeds the warning threshold, the control module switches to hover mode; When the detected value exceeds the limit threshold, an emergency stop command is executed and the suspension rope buffer or locking device is triggered. And record the exception information in the task log.

[0011] Preferably, the security mechanism includes: After the task is completed, the system extracts the attitude data, center of gravity change data, and rope force data during the hoisting process; Analyze the control deviation and update the control parameters; In subsequent tasks, load and update the parameters and perform parameter convergence calculations. Develop a control parameter library for task classification; The new parameters are written to the control module after the task is executed.

[0012] Preferably, the combined centroid of S2 and the nonlinear dynamic model of S3 include: Based on the comprehensive center of gravity state and nonlinear dynamics model, the vision system and lidar carried by the UAV synchronously collect environmental information around the hoisting path; The collected data is segmented and obstacle detection is performed to generate a 3D environmental model; Calculate wind direction, wind speed, and turbulence intensity by combining meteorological sensor data; The path planning module recalculates the flight path based on the environmental model and dynamic payload status; The control module adjusts the flight altitude, speed, and attitude parameters based on the updated path, and inputs the adjustment results into the closed-loop control module for execution.

[0013] Preferably, the control commands of S4 and the monitoring data of S5 include: When multiple drones participate in the hoisting operation, a synchronous communication mechanism is established through the master control node; The master control node allocates pull shares and spatial location targets to each drone; All UAVs share integrated center of gravity and attitude data; The control algorithm updates the thrust allocation matrix based on changes in force. When any drone deviates from its designated path, the remaining drones perform compensatory control to maintain load balance.

[0014] Secondly, the present invention provides the following technical solution: a drone hoisting control system based on industrial vision, comprising the following modules: The data acquisition module is used by the UAV to collect multi-angle visual image data of the load and real-time tension information of each hoisting rope during hoisting operations; The center of gravity fusion module is used to reconstruct a three-dimensional point cloud based on the data and extract feature points to calculate the visual center of gravity. At the same time, it determines the mechanical center of gravity of the suspension rope based on the force analysis of the suspension rope, and dynamically adjusts the visual center of gravity based on the coordinate difference and force balance relationship between the two to obtain the comprehensive center of gravity position and velocity state of the load. The dynamic modeling module is used to establish a nonlinear dynamic model of the UAV, sling, and load based on the obtained comprehensive center of gravity state. This model considers the elasticity of the sling, the inertial coupling of the load, and external disturbances, and is used to describe the dynamic response characteristics of the UAV hoisting system. The control command generation module is used to generate UAV attitude and hoisting control commands based on the dynamic model using an adaptive control algorithm. By prioritizing the adjustment of position error in the hovering state and prioritizing the adjustment of speed error in the moving state, dynamic correction and stable control of the load center of gravity are achieved. The feedback monitoring module is used to monitor changes in the visual center of gravity and the tension of the suspension rope in real time while the UAV executes control commands. The monitoring results are fed back to the control module to form a closed-loop control, so as to suppress load swing and maintain balance. The safety control module is used to trigger a safety mechanism to automatically decelerate, hover, or stop the machine when monitoring results indicate that the load attitude or rope tension exceeds a threshold, until the load stabilizes or the hoisting task is completed.

[0015] The present invention has the following beneficial effects: 1. In this invention, by integrating center of gravity modeling and adaptive control based on the fusion of industrial vision and mechanics, dynamic coordination and stable control of the load attitude and the UAV attitude are achieved, thereby improving the problem that traditional UAV hoisting methods mostly rely on single visual recognition and lack a dynamic center of gravity correction mechanism, which leads to the load being prone to swinging and instability.

[0016] 2. In this invention, a comprehensive center of gravity state is established based on multi-angle visual images and rope tension information, thereby realizing real-time identification of load position and speed. This improves the problem that traditional center of gravity calculation methods are mostly based on static geometric models, which cannot reflect the force changes during the hoisting process, resulting in insufficient control accuracy.

[0017] 3. In this invention, by constructing a nonlinear dynamic model that considers the elasticity of the hoisting rope and external disturbances, an accurate description of the dynamic response of the UAV hoisting system can be achieved. This improves the problem of system control lag caused by the assumption of rigid connection in most traditional models, which do not consider the coupling of elasticity and inertia.

[0018] 4. In this invention, by establishing a closed-loop monitoring and safety control mechanism, the flight state is automatically adjusted when the load attitude or force is abnormal, thereby improving the problem that most traditional UAV hoisting systems lack a safety feedback mechanism and cannot identify the overload or attitude deviation of the hoisting rope in time, resulting in insufficient operational safety. Attached Figure Description

[0019] Figure 1 This is a flowchart of a UAV hoisting control method based on industrial vision proposed in this invention; Figure 2 This is a diagram illustrating the center-of-gravity fusion process of a UAV hoisting control method based on industrial vision proposed in this invention. Figure 3 This is a control strategy diagram for a drone hoisting control method based on industrial vision proposed in this invention; Figure 4 This is a multi-machine collaborative architecture diagram of a UAV hoisting control method based on industrial vision proposed in this invention; Figure 5 This invention presents a safety control process for a drone hoisting control method based on industrial vision. Figure 6 This is a module architecture diagram of an industrial vision-based drone hoisting control system proposed in this invention. Detailed Implementation

[0020] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1: In a first embodiment of the present invention, the present invention provides a drone hoisting control method based on industrial vision, such as... Figures 1-5 As shown, it includes the following steps: S1. During hoisting operations, the UAV collects multi-angle visual image data of the load and real-time tension information of each hoisting rope; Furthermore, the image data and tensile information include: Before hoisting, the drone uses multiple industrial cameras to capture multi-angle images of the load. Each image is input into the 3D point cloud reconstruction module. A point cloud model of the load is generated using a stereo matching-based reconstruction algorithm. The point cloud is registered and features are extracted. The geometric center is calculated to obtain the visual centroid. The force values ​​of each suspension rope are obtained by the suspension rope tension sensor and the torque balance equation is established by combining the coordinates of the suspension point to calculate the mechanical center of gravity. Based on the coordinate difference between the visual center of gravity and the mechanical center of gravity and the mechanical constraints, the visual center of gravity is dynamically adjusted to obtain the comprehensive center of gravity position and velocity state of the load, and the comprehensive center of gravity state is input into the control module.

[0022] Specifically, during hoisting operations, multi-angle visual image data of the load and real-time tension information of each hoisting rope are collected to establish a combined visual and mechanical model of the load's center of gravity.

[0023] Before the lifting operation, the UAV collects multi-angle image data of the load using industrial cameras installed at different locations on the aircraft. These images are input into a 3D point cloud reconstruction module, which uses a dense reconstruction algorithm based on stereo matching to generate a point cloud model of the load. This point cloud model is then spatially aligned using an ICP registration algorithm based on the different camera views. Principal component feature extraction is then performed to determine the geometric center coordinates of the load, denoted as . This coordinate is the visual center of gravity of the load.

[0024] At the same time, the cable tension sensor collects the force value of each cable. And record the spatial coordinates of its corresponding lifting point. According to the static equilibrium equations, the mechanical center of gravity of the load is... satisfy: ; in Let the lifting point position vector be... Let be the position vector of the center of gravity to be determined. This is obtained by solving the system of moment equilibrium equations. .

[0025] After obtaining the visual center of gravity and the mechanical center of gravity, the system calculates the coordinate difference between the two. This difference is then input into the mechanical constraint-based fusion adjustment module. This module uses the load-mass matrix... With force vector As a constraint, the visual center of gravity coordinates are corrected, and the overall center of gravity position is calculated. : ; in To integrate the adjustment coefficient matrix, the corrective weights for visual estimation errors under mechanical constraints are represented. The system synchronously calculates the velocity state of the load: The velocity vector of the load's center of gravity is obtained using the finite difference method. This reflects the dynamic attitude changes of the load.

[0026] Combined center of gravity position and velocity status data As an input signal, it is transmitted to the hoisting control module and outputs control commands for attitude control and path adjustment.

[0027] Data flow: Input data: multi-angle image sequences from the camera and signals from the suspension cable tension sensor; Processing: point cloud reconstruction → geometric registration → visual centroid calculation → torque balance calculation → coordinate difference calculation and fusion adjustment; Output data: composite centroid coordinates. With speed state The signal is transmitted to the attitude control module.

[0028] Through this process, the UAV system can acquire comprehensive center of gravity information of the load in real time, avoiding deviations caused by relying on a single visual or mechanical data, and providing basic input for subsequent hoisting stability control.

[0029] S2. Reconstruct the three-dimensional point cloud based on the data and extract feature points to calculate the visual center of gravity. At the same time, determine the mechanical center of gravity of the suspension rope based on the force analysis of the suspension rope, and dynamically adjust the visual center of gravity based on the coordinate difference and force balance relationship between the two to obtain the comprehensive center of gravity position and velocity state of the load. Furthermore, the visual center of gravity and the mechanical center of gravity include: A nonlinear dynamic model of the UAV, sling, and load is established based on the comprehensive center of gravity state obtained by dynamic adjustment of the visual and mechanical center of gravity. The suspension rope is modeled as a flexible rod with elastic and damping properties; The relationship between the system's kinetic and potential energy was derived using the Lagrange equation, and a dynamic equation was established that includes the UAV thrust, cable tension, load inertia, and external disturbances. An adaptive update mechanism for mechanical model parameters is introduced to automatically correct model coefficients when the load mass or lifting point changes.

[0030] Specifically, based on the multi-angle images and rope stress data acquired in stage S1, the point cloud reconstruction module first performs feature matching on the images and generates a dense depth map using disparity calculation. This depth map is then converted into a set of spatial coordinate points. A 3D point cloud of the load was constructed. Voxel filtering was used for point cloud denoising and downsampling, RANSAC plane segmentation was used to remove background points, and K-means clustering was used to separate the main load point cloud. Principal component analysis was used to calculate the geometric center of the load point cloud. ; in For the effective number of point clouds, The coordinates of the visual centroid of the load are given.

[0031] In the process of solving for the center of gravity of the mechanics, the tension of each suspension rope is obtained based on S1. and the location of the lifting points Establish the equilibrium equation: The center of gravity of the mechanical system is obtained by the least squares method. The difference in coordinates between the two This reflects visual estimation bias. The dynamic adjustment module corrects the visual center of gravity based on the force balance relationship, using a weighted correction model: ;in This is a dynamic weighting matrix determined based on the suspension rope stiffness and the degree of force balance, used to coordinate the reliability of visual and mechanical estimation results. (Integrated center of gravity position) The time rate of change is used to calculate the load velocity state: ;in The linear velocity characterizing the center of gravity of the load.

[0032] Based on the dynamically adjusted overall center of gravity state, a nonlinear dynamic model of the UAV, suspension rope, and load is established. The suspension rope is modeled as having an elastic coefficient. With damping coefficient For a flexible rod, the tension is expressed as: ;in This is the current length vector of the suspension rope. It is the stationary length.

[0033] The system uses the Lagrange equations to establish the dynamic relationship: ;in , As the system's kinetic energy, As potential energy, This is the generalized external force vector. Kinetic energy. This includes the translational and rotational energy of the drone, the elastic potential energy of the suspension cable, and the inertial energy of the load; potential energy This includes gravitational potential and the elastic potential of the sling. From this system of equations, we can derive the nonlinear dynamic equations that include the UAV thrust, sling tension, load inertia, and external disturbance terms: ;in For the system quality matrix, The Coriolis and centrifugal force matrix, For gravity, For the thrust input of the drone, It is an external disturbance force.

[0034] To ensure the model's adaptability to changes in load mass or lifting point position, an adaptive update mechanism for mechanical parameters is introduced. The system monitors the load attitude offset and force residual in real time, and automatically corrects the model parameters based on minimizing the following cost function: ;when When the threshold is exceeded, update using recursive least squares method. Equal model coefficients enable dynamic adjustment of the model.

[0035] Data input / output flow: Input data: multi-angle image point cloud, suspension rope tension, suspension point coordinates, load attitude information; Processing process: point cloud reconstruction → geometric center calculation → mechanical center of gravity solution → dynamic weighted correction → dynamic equation modeling and parameter update; Output data: comprehensive center of gravity position. Center of gravity speed Parameters of the nonlinear dynamic equations This is used for the next step of attitude control and path planning.

[0036] Through this process, the system can maintain the continuity of the center of gravity state estimation and the effectiveness of the dynamic model under load attitude and force changes, providing accurate model input for subsequent hoisting trajectory control.

[0037] S3. Based on the obtained comprehensive center of gravity state, establish a nonlinear dynamic model of the UAV, sling, and load. This model considers the elasticity of the sling, the inertial coupling of the load, and external disturbances, and is used to describe the dynamic response characteristics of the UAV hoisting system. Furthermore, the nonlinear dynamic model includes: The control module includes an upper-level load center of gravity adjustment control unit and a lower-level attitude control unit; The upper control unit generates load adjustment commands based on the overall center of gravity deviation, while the lower control unit generates attitude and thrust control commands. Increase the weight of position error while hovering, and increase the weight of speed error while moving. An adaptive algorithm is used to update the control gain in real time and output composite control commands.

[0038] Specifically, the overall center of gravity position obtained from S2 Center of gravity speed The system establishes a coupled dynamic model of the UAV, the sling, and the load. The UAV attitude is determined using Euler angles. It is stated that the suspension rope is modeled as having elasticity. and damping Flexible rods, the translational and rotational inertia of the load and The thrust vector of the UAV is denoted as... The external disturbance vector is denoted as The system's state vector is defined as: This includes the drone's spatial position, attitude, and the length of each suspension rope.

[0039] The system adopts Lagrangian dynamics: ;in Represents the mass and inertia matrix; Represents the Coriolis force and centrifugal force terms; The term represents gravity and the elastic restoring force of the suspension rope; the right end represents the thrust input and external disturbance force. Suspension rope tension. Determine using the following formula: ; in The vector of the suspension rope. This is the stationary length. The dynamic equations established based on this can characterize the feedback effect of load motion on the UAV's attitude and thrust.

[0040] The nonlinear dynamic model uses a control module divided into an upper-level load center of gravity adjustment control unit and a lower-level attitude control unit. The input to the upper-level control unit is the overall center of gravity position. With respect to the expected center of gravity The error is defined as: The upper-level control unit generates load adjustment commands through the PD control law. ;in and These are the position and velocity gain matrices, respectively. The weights in the gain matrices are dynamically adjusted according to different flight states: the position error weight is increased during hovering, and the velocity error weight is increased during path movement. The adjustments are based on flight mode parameters. Corresponding adjustment rules: ; in Indicates hover mode. Indicates the movement mode.

[0041] The lower-level attitude control unit receives the output from the upper-level unit. Combined with the current attitude state vector of the UAV The desired thrust and attitude command are calculated. Thrust distribution employs coordinate transformation: ;in This is the attitude rotation matrix, used to transfer the center of gravity control force to the unmanned aerial vehicle (UAV) system. Then, according to the thrust distribution equation: Command to obtain the motor angular velocity ,matrix Characterizes the propeller position and thrust constant.

[0042] The system uses an adaptive control algorithm to... , The algorithm updates in real time and is based on the principle of minimizing control error energy. Updated via gradient descent: ; in The learning rate is used to automatically adjust the control gain under different operating conditions, thereby achieving the adaptive characteristics of the control law.

[0043] Data input / output process, input data: overall center of gravity status Expected center of gravity trajectory UAV attitude information External disturbance estimation Processing steps: State error calculation → Solving the upper-level load adjustment control law → Dynamic gain adjustment → Attitude and thrust control command calculation → Parameter adaptive update; Output data: Control command set Attitude adjustment angle Motor speed command Used for actuator control.

[0044] This process establishes a coupled nonlinear dynamic model, incorporating the load's center of gravity, the rigging's mechanical properties, and the UAV's attitude into a single control framework. The hierarchical structure of the upper and lower control units enables the system to allocate control weights in real time according to the flight status, ensuring the continuity and adaptability of the control model as the load moves and environmental disturbances change.

[0045] S4. Based on the dynamic model, an adaptive control algorithm is used to generate UAV attitude and hoisting control commands. By prioritizing the adjustment of position error in hovering state and prioritizing the adjustment of speed error in moving state, dynamic correction and stable control of the load center of gravity are achieved. Furthermore, the UAV attitude and hoisting control commands include: During the execution of control commands by the drone, data on visual center of gravity and sling tension are continuously collected; The control module compares the target state with the current state, calculates the deviation, and automatically adjusts the feedback gain factor. When a change in pitch is detected, the thrust response rate is adjusted, and the control gain is reduced when the attitude stabilizes. The system parameters are periodically updated to maintain control accuracy.

[0046] Specifically, based on the nonlinear dynamic model established by S3, the control module inputs the comprehensive center of gravity state. , UAV attitude status and expected trajectory The control module uses an adaptive control law to generate UAV attitude and hoisting control commands, ensuring the load maintains a stable center of gravity during dynamic movement.

[0047] The attitude and center of gravity coupled control law is constructed, and the system error is defined as follows: ; The control law adopts a weighted adaptive form: ;in For integrated control commands, , For state mode parameters The relevant dynamic gain matrix. Parameters Used to characterize the current operating condition of the drone: while hovering. When moving The gain matrix is ​​adjusted according to the following rules: ;in , This is the initial gain. , This is the weighting adjustment coefficient. This setting ensures that position control takes precedence when hovering, and speed control takes precedence when moving. Then, the attitude control commands are transformed through the rotation matrix: ; in This is the attitude rotation matrix for the UAV, which converts the center of gravity control commands from the geographic coordinate system to the machine coordinate system. Ultimately, the thrust and attitude control variables... They were assigned to various implementing agencies.

[0048] The feedback gain factor automatic adjustment mechanism continuously acquires real-time visual center of gravity data while executing control commands. With the tension of the suspension rope Data. The control module calculates the real-time deviation: ; in The reference tension is used. The control module automatically adjusts the feedback gain based on the deviation amplitude. ; in For the feedback gain vector, This is an adjustment coefficient used to ensure the control gain responds in real time to changes in load attitude.

[0049] Amplitude detection and thrust response adjustment, amplitude angle The vision system calculates this based on the load's motion trajectory. ; in This refers to the height of the suspension point. When detected... When the yaw rate exceeds the threshold, the thrust response rate Temporarily increased: ;when When the control gain returns to its steady-state value: ;in It is a regulating factor used to limit system oscillations and suppress overshoot.

[0050] A parameter periodic update mechanism is used to prevent the model from drifting away from the real system. The system updates parameters at fixed time intervals. Internal key control parameter set Perform average update: ; in Represents a set of parameters. This represents the number of sampling points. The updated parameters are stored in the control module cache for use in the next control cycle.

[0051] Data input / output process, input data: overall center of gravity status Suspension rope tension data UAV attitude angle Expected trajectory Processing flow: Data acquisition → State error calculation → Adaptive control law solution → Feedback gain correction → Swing detection and thrust response adjustment → Periodic parameter update. Output data: Thrust and attitude control commands. Motor angular velocity command Adjusted parameter set .

[0052] This process enables synchronous closed-loop control of the payload's center of gravity and attitude during complex lifting operations by the UAV. The control module adaptively adjusts the feedback gain and thrust response rate, enabling the system to maintain a stable center of gravity and attitude balance under different flight conditions, and to maintain dynamic response capability to changes in the sling's mechanics.

[0053] S5. While executing control commands, the UAV monitors changes in visual center of gravity and suspension cable tension in real time, and feeds the monitoring results back to the control module to form a closed-loop control in order to suppress load sway and maintain balance. Furthermore, changes in visual center of gravity and rope tension include: The system sets multiple levels of safety thresholds, including the rope force threshold, the load attitude angle threshold, and the UAV attitude threshold; When the detected value reaches the warning threshold, the control module reduces the flight speed and adjusts the attitude; When the detected value exceeds the warning threshold, the control module switches to hover mode; When the detected value exceeds the limit threshold, an emergency stop command is executed and the suspension rope buffer or locking device is triggered. And record the exception information in the task log.

[0054] Specifically, during the execution of attitude and hoisting control commands by the UAV, the system collects visual center of gravity coordinate information and hoisting rope tension signals in real time, forming a continuous monitoring stream. The monitoring data serves as input to the control module, used to update dynamic model parameters and control commands.

[0055] Input data, the visual module outputs the load centroid position vector The tension of the suspension rope output by the force sensor. The attitude angle vector output by the UAV inertial navigation unit. The control module compares the above input with the target state and calculates the error term: ; in These are the system's set target center of gravity, desired tension, and attitude reference values. The error term is fed into the closed-loop controller, where the control quantity is calculated based on the overall deviation. ; in For the combined error vector, , , These are the proportional, integral, and derivative gain factors, respectively. The control module dynamically adjusts the gain parameters according to different operating conditions to maintain system stability.

[0056] The system presets three types of safety thresholds: rope stress threshold. Load attitude angle threshold UAV attitude threshold .

[0057] Based on the comparison between the real-time detection value and the threshold, the system executes a tiered response: when the detection value reaches the warning threshold... or At that time, the control module reduces the thrust response rate. And reduce flight speed; when the detected value exceeds the warning threshold If the attitude deviation exceeds the limit, the system switches to hover control mode; when the detected value exceeds the limit threshold... In the event of instability, an emergency stop command is triggered, activating the suspension rope buffer device or mechanical locking structure to prevent the load from falling.

[0058] Data processing and output flow: Input stage: Visual and tension sensors collect data, which is then filtered for noise suppression; Analysis stage: The control module extracts state deviations based on error calculation formulas; Decision stage: The system state level is determined based on safety thresholds; Execution stage: New attitude control commands are generated. The data is sent to the flight control system; feedback phase: after execution, the flight control system returns attitude change data, forming a new input closed loop; recording phase: each abnormality and threshold trigger event is written to the mission log for subsequent diagnosis.

[0059] Through the aforementioned closed-loop control structure, the system can detect changes in the tension of the hoisting rope and the position of the center of gravity at the initial stage of load oscillation, and adjust the thrust distribution and attitude angle within the control cycle. This mechanism can suppress the lateral sway of the load under inertia and maintain the overall balance of the UAV and the hoisting system.

[0060] The real-time threshold mechanism provides dynamic response capabilities for different risk levels, enabling the system to stabilize quickly at the flight level when sudden disturbances occur and to safely abort operations under extreme conditions.

[0061] This process does not rely on an external computing center. All signal acquisition, judgment and control are completed within the closed loop of the flight control module, ensuring that the control delay is within the millisecond range.

[0062] S6. When the monitoring results indicate that the load attitude or the tension of the hoisting rope exceeds the threshold, the system triggers a safety mechanism to automatically decelerate, hover, or stop in an emergency until the load returns to stability or the hoisting task is completed. Furthermore, the security mechanisms include: After the task is completed, the system extracts the attitude data, center of gravity change data, and rope force data during the hoisting process; Analyze the control deviation and update the control parameters; In subsequent tasks, load and update the parameters and perform parameter convergence calculations. Develop a control parameter library for task classification; The new parameters are written to the control module after the task is executed.

[0063] Specifically, when the system detects that the load attitude angle or the sling tension exceeds a preset threshold, the safety mechanism module immediately intervenes in the flight control loop. This mechanism uses a state determination function: ; Classify and determine the current state. For real-time suspension rope tension, For the load attitude angle, This represents the rate of change of attitude angle. (Function) Output three status instructions: : Perform deceleration control; : Perform hover control; : Perform an emergency shutdown.

[0064] The control module invokes different flight control strategies based on status commands. Deceleration control reduces the rate of thrust change. This reduces the dynamic response amplitude of the drone; hovering control is achieved by issuing speed commands. And set the attitude angular rate In an emergency shutdown, the thrust is directly reduced to zero. Simultaneously, the suspension rope locking mechanism is triggered. The switching logic for each mode is driven by a state machine to prevent control command conflicts.

[0065] After the task is completed, the system calls the data logging module to extract three types of data from this hoisting process: attitude angle time series. ; trajectory of center of gravity displacement ; Curve of change in suspension rope tension The control module performs deviation statistics on the above data and calculates the expected control error: ;in For ideal control commands, To actually execute the instructions, The duration of the task.

[0066] Based on the error results, the system adopts the following parameter update equation: ;in To control the gain vector, The learning rate is used. Control parameters are adjusted through gradient iteration.

[0067] To ensure parameter convergence across different tasks, the system performs parameter convergence calculations after each task: ; in This represents the parameter value from the previous task. This is the convergence coefficient. This calculation process forms a dynamic convergence mechanism for the control parameters, reducing control oscillations.

[0068] Data input / output process: Monitoring stage: acquire load attitude angle and suspension rope tension signals, and remove noise through a filtering module; Judgment stage: the state determination function calculates the current state. This triggers the corresponding safety control mode; Control phase: Output control commands. The process begins with the flight control module adjusting thrust and attitude; during the recording phase, control and sensor data are written to the mission log storage module; in the analysis phase, after the mission ends, the control module extracts the log data and performs error statistics and gain updates; during the learning phase, the updated control parameters are written to the control parameter library, forming a mission type index; and in the loading phase, the corresponding mission type control parameter set is called during subsequent mission initialization to achieve experience inheritance.

[0069] Through this safety mechanism, the system can execute multi-level responses when abnormal conditions are detected during hoisting tasks, forming a continuous safety control chain from deceleration to shutdown, to avoid the expansion of load swing or system instability.

[0070] The parameter analysis and update process after task completion enables the control module to have self-learning capabilities, automatically adjusting the gain configuration based on historical task errors. The parameter library forms control feature sets for different task categories, improving the control accuracy and convergence speed of subsequent hoisting processes.

[0071] This process requires no external intervention; all data extraction, error calculation, and parameter updates are automatically completed within the control module, forming a complete adaptive closed-loop learning system.

[0072] The combined centroid of S2 and the nonlinear dynamic model of S3 include: Based on the comprehensive center of gravity state and nonlinear dynamics model, the vision system and lidar on the UAV synchronously collect environmental information around the hoisting path; The collected data is segmented and obstacle detection is performed to generate a 3D environmental model; Calculate wind direction, wind speed, and turbulence intensity by combining meteorological sensor data; The path planning module recalculates the flight path based on the environmental model and dynamic payload status; The control module adjusts the flight altitude, speed, and attitude parameters based on the updated path, and inputs the adjustment results into the closed-loop control module for execution.

[0073] Specifically, based on the overall center of gravity state Based on the nonlinear dynamic model of the UAV-payload system, environmental modeling, meteorological calculations, and real-time path adjustment are performed on the UAV lifting path to form a dynamic closed-loop control process for complex environments.

[0074] Environmental perception and modeling: The UAV's onboard vision system and LiDAR module simultaneously collect environmental information around the hoisting path, acquiring depth images and point cloud data. Let the point cloud set be: ; in For spatial coordinates, The reflection intensity is used. The system performs a semantic segmentation function on the point cloud data: Output tags Indicates the obstacle type (ground, building, flyable area). The obstacle detection module constructs a set of obstacle boundaries in the environmental coordinate system. And generate a 3D model of the environment. .

[0075] Weather perception and wind field modeling: Weather sensors on drones collect wind speed data. ,wind direction and turbulence intensity The wind field model is expressed as: ;in air density, For the area of ​​force application, This is the drag coefficient. The wind force vector... As an input of external disturbances, a nonlinear dynamic model is used to correct thrust distribution.

[0076] The dynamic model is coupled for calculation, and the system is based on the combined load center of gravity state and the dynamic relationship: ; in Let T be the UAV's position vector, and T be the UAV's thrust. Let the tension vector of the suspension rope be... This is due to external wind disturbance forces. Let gravitational acceleration be the acceleration due to gravity. The control module solves the above equations in real time to obtain the system state vector: This state variable is used for path planning and control.

[0077] Path planning and control calculations; the path planning module is based on the environment model. and overall center of gravity status Perform cost function minimization operation: ; in For the target trajectory, As a cost for obstacle distance, , , These are the weighting coefficients. By minimizing... Obtain the new expected path .

[0078] The control module converts the path adjustment results into flight control variables: ; in For position control output, For attitude commands, This is the attitude rotation matrix. The control output is sent to the closed-loop execution unit for error comparison and correction with the real-time acquired feedback data.

[0079] Data input and output process, data acquisition layer: 3D point cloud data acquired by vision and LiDAR. Meteorological sensors collect wind field data Environment modeling layer: Performs point cloud segmentation and obstacle detection to generate an environment model. Dynamics calculation layer: Input load combined with center of gravity state Calculate the current system state based on the dynamic equations. Path planning layer: and Using the input as input, perform cost function optimization to obtain the target path. Control output layer: calculates control commands. The data is then output to the flight control module; feedback loop: the closed-loop control module collects flight attitude and force feedback, updates control gain and path weight, and completes dynamic adjustment.

[0080] By incorporating the integrated center of gravity state and nonlinear dynamics model into the path planning process, the system can adjust its flight trajectory and attitude commands in real time when load swings or environmental disturbances occur, enabling the UAV to maintain stable load balance in complex environments. The combination of wind field modeling and obstacle detection reduces the risk of collisions along the lifting path.

[0081] The control layer achieves dynamic coupling between flight control and environmental perception by solving the dynamic equations and optimizing the cost function in real time. The commands output to the closed-loop control module are immediate and updatable, ensuring the safety and continuity of UAV lifting missions under environmental fluctuations.

[0082] The control commands of S4 and the monitoring data of S5 include: When multiple drones participate in the hoisting operation, a synchronous communication mechanism is established through the master control node; The master control node allocates pull shares and spatial location targets to each drone; All UAVs share integrated center of gravity and attitude data; The control algorithm updates the thrust allocation matrix based on changes in force. When any drone deviates from its designated path, the remaining drones perform compensatory control to maintain load balance.

[0083] Specifically, in multi-UAV collaborative lifting missions, a distributed thrust distribution and real-time attitude feedback system is formed through the synchronous communication and force coordination mechanism of the master control node, ensuring that the multi-UAV system maintains overall mechanical balance and attitude stability under non-uniform load and environmental disturbance conditions.

[0084] Synchronous communication and task allocation among multiple UAVs: In a multi-UAV collaborative hoisting structure, a master control node is provided. With several subordinate drone nodes The master node periodically broadcasts system status data packets: ;in To take into account the overall center of gravity status, For the first The attitude state vector of the drone. The current thrust vector, This represents the positional deviation of the UAV relative to the payload's center of mass. All nodes achieve time synchronization through a low-latency communication link, ensuring phase consistency of the control signals.

[0085] The master control node is based on the load quality Distribution of lifting points Calculate the pull distribution share for each drone: ; in The number of drones involved in the hoisting. This is the component corrected based on load offset and attitude feedback. The master node generates the thrust allocation matrix: The matrix parameters are then sent to each UAV control module via the communication channel.

[0086] Force and attitude feedback are shared, and the sensor systems of each UAV collect the tension of the suspension cable in real time. Attitude angle and positional deviation Forming a feedback vector: The data is processed using a weighted fusion function: ;in The weighting coefficients, determined by the proportion of tension, are used to form the overall system state variables. And input the data into the master node for dynamic balance calculation.

[0087] Dynamic thrust adjustment and compensation control: The master control node executes the control law based on the difference between the current overall state and the target attitude. ;in Let be the target equilibrium state vector. , , These are the proportional, integral, and derivative gain matrices, respectively. The master control node uses the control quantity... Corrected thrust distribution matrix: When a drone is detected to have deviated from its target trajectory, the remaining drones will adjust their plans based on the updated... Automatically perform compensated thrust adjustment: ; in The compensation coefficient is calculated based on the geometric relationship of the lifting points. This represents the change in tension of the deviator.

[0088] Data input and output process, input layer: master control node receives the overall center of gravity of the payload. Attitude data of each UAV With tensile data Each UAV node synchronously receives the control matrix broadcast by the master control. .

[0089] Computation layer: The master control node calculates the thrust distribution for each UAV. According to the feedback vector Find the overall state ; Execution control law calculation And generate a new thrust matrix .

[0090] Output layer: Updated control commands are distributed to each UAV controller; each UAV adjusts its attitude and thrust according to the received commands; the new attitude and force status are collected and fed back again to form a closed loop.

[0091] Monitoring layer: The control system periodically detects tension and attitude fluctuations to determine whether they exceed the threshold; if the detection deviation is too large, the compensation control logic is triggered; if the deviation continues to increase, the master control node issues hovering or reallocation commands.

[0092] By introducing synchronous communication and a thrust distribution matrix for the master control node, each UAV can maintain overall mechanical balance when subjected to asymmetrical loads, attitude disturbances, or partial UAV deviations. A real-time force feedback sharing mechanism ensures the system can dynamically identify individual UAV anomalies and reduce load sway through compensatory control. The introduction of control law equations enables quantitative coordination of master-slave distributed control, giving the UAV swarm real-time coordination capabilities and self-stabilizing characteristics during lifting missions, meeting the safety and consistency requirements of multi-load-point lifting.

[0093] Example 2: In the maintenance of power transmission lines and the hoisting of high-altitude components in mountainous areas, multiple drones work together to lift heavy objects such as towers, insulator assemblies, or cable reels. During drone lifting, a comprehensive center of gravity and nonlinear dynamic model is established. Environmental and wind field data are acquired using a vision system, lidar, and meteorological sensors to generate a 3D model and dynamic path planning. In multi-drone collaboration, the master control node allocates the pulling force share and spatial target, while each drone shares attitude and force information. The control algorithm updates the thrust allocation matrix in real time. When the load attitude or hoisting cable tension exceeds a threshold, the system triggers deceleration, hovering, or emergency stop, and updates the control parameters to form a parameter library after the task. This scenario presents problems such as nonlinear instability caused by attitude coupling, path delay and control lag caused by environmental disturbances, multi-drone synchronization errors, and a lack of self-correction capabilities in the safety mechanism, affecting the stability and safety of the hoisting process. To solve these problems, this invention provides a drone hoisting control system based on industrial vision, the structure of which is as follows: Figure 6 As shown. The specific implementation process of this system is as follows: Specifically, the data acquisition module is used by the UAV to obtain multi-source information about the load during the lifting process. It acquires multi-view image data of the load through multi-angle vision cameras and uses a rope tension sensor to measure the force values ​​at each lifting point in real time. The images and force information output by the module are synchronized with the timeline for subsequent center of gravity calculation and dynamic modeling, ensuring accurate data support for the system during the lifting, transportation, and landing phases.

[0094] The center of gravity fusion module builds a 3D point cloud model based on the acquired images and force information, extracts key feature points to calculate the visual center of gravity, and simultaneously calculates the mechanical center of gravity based on the force distribution of the suspension rope. By analyzing the coordinate difference and force balance relationship between the visual and mechanical centers of gravity, the visual center of gravity is dynamically corrected to obtain the position and velocity state of the comprehensive center of gravity. The comprehensive center of gravity result serves as the core input for UAV attitude and lifting control, enabling an accurate description of the load's motion characteristics.

[0095] The dynamics modeling module establishes a nonlinear dynamic model of the UAV, sling, and load based on the overall center of gravity state. The model considers the elasticity and damping characteristics of the sling, as well as the coupling relationship between load inertia and external disturbances, to describe the dynamic response characteristics of the system at different flight phases. The model outputs the UAV's attitude, thrust, and load motion states, providing a structured dynamic equation foundation for the control module.

[0096] The control command generation module generates UAV attitude and hoisting control commands based on the output of the dynamic model through an adaptive control algorithm. In hovering mode, position error is the primary adjustment criterion, while in moving mode, speed error is the primary criterion. The control gain parameters are dynamically adjusted to enable the UAV to maintain stable load center of gravity and attitude control under different operating conditions.

[0097] The feedback monitoring module monitors changes in the visual center of gravity and sling tension in real time during the UAV's execution of control commands. By comparing the desired state with the actual state, it calculates the deviation and adjusts the control inputs to form a closed-loop control, thereby suppressing load sway and maintaining system balance. Monitoring data is continuously input into the control module, enabling control commands to be updated in real time according to the flight status.

[0098] The safety control module triggers a safety mechanism when the system detects that the load attitude or the force on the suspension rope exceeds a set threshold. It executes corresponding measures based on different levels of deviation, including deceleration, hovering, or emergency stop operations, to ensure operational safety. After the task is completed, the module analyzes the attitude, center of gravity, and force data throughout the process, updates the control parameters, and stores them in the parameter library for subsequent control corrections and accuracy improvements.

[0099] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A drone hoisting control method based on industrial vision, characterized in that, Includes the following steps: S1. During hoisting operations, the UAV collects multi-angle visual image data of the load and real-time tension information of each hoisting rope; S2. Based on the data, perform three-dimensional point cloud reconstruction and extract feature points to calculate the visual center of gravity. At the same time, determine the mechanical center of gravity of the suspension rope based on the force analysis of the suspension rope, and dynamically adjust the visual center of gravity based on the coordinate difference and force balance relationship between the two to obtain the comprehensive center of gravity position and velocity state of the load. S3. Based on the obtained comprehensive center of gravity state, establish a nonlinear dynamic model of the UAV, sling, and load. This model considers the elasticity of the sling, the inertial coupling of the load, and external disturbances, and is used to describe the dynamic response characteristics of the UAV hoisting system. S4. Based on the dynamic model, an adaptive control algorithm is used to generate UAV attitude and hoisting control commands. By prioritizing the adjustment of position error in the hovering state and prioritizing the adjustment of speed error in the moving state, dynamic correction and stable control of the load center of gravity are achieved. S5. While executing control commands, the UAV monitors changes in visual center of gravity and suspension cable tension in real time, and feeds the monitoring results back to the control module to form a closed-loop control in order to suppress load sway and maintain balance. S6. When the monitoring results indicate that the load attitude or the tension of the hoisting rope exceeds the threshold, the system triggers a safety mechanism to automatically decelerate, hover, or stop in an emergency until the load stabilizes or the hoisting task is completed.

2. The UAV hoisting control method based on industrial vision according to claim 1, characterized in that, The image data and tensile information include: Before hoisting, the drone collects multi-angle images of the load using multiple industrial cameras. Each image is input into the 3D point cloud reconstruction module. A point cloud model of the load is generated using a stereo matching-based reconstruction algorithm. The point cloud is registered and features are extracted. The geometric center is calculated to obtain the visual centroid. The force values ​​of each suspension rope are obtained by the suspension rope tension sensor and the torque balance equation is established by combining the coordinates of the suspension point to calculate the mechanical center of gravity. Based on the coordinate difference between the visual center of gravity and the mechanical center of gravity and the mechanical constraints, the visual center of gravity is dynamically adjusted to obtain the comprehensive center of gravity position and velocity state of the load, and the comprehensive center of gravity state is input into the control module.

3. The UAV hoisting control method based on industrial vision according to claim 1, characterized in that, The visual center of gravity and the mechanical center of gravity include: A nonlinear dynamic model of the UAV, sling, and load is established based on the comprehensive center of gravity state obtained by dynamic adjustment of the visual and mechanical center of gravity. The suspension rope is modeled as a flexible rod with elastic and damping properties; The relationship between the system's kinetic and potential energy was derived using the Lagrange equation, and a dynamic equation was established that includes the UAV thrust, cable tension, load inertia, and external disturbances. An adaptive update mechanism for mechanical model parameters is introduced to automatically correct model coefficients when the load mass or lifting point changes.

4. The UAV hoisting control method based on industrial vision according to claim 1, characterized in that, The nonlinear dynamic model includes: The control module includes an upper-level load center of gravity adjustment control unit and a lower-level attitude control unit; The upper control unit generates load adjustment commands based on the overall center of gravity deviation, while the lower control unit generates attitude and thrust control commands. Increase the weight of position error while hovering, and increase the weight of speed error while moving. An adaptive algorithm is used to update the control gain in real time and output composite control commands.

5. The UAV hoisting control method based on industrial vision according to claim 1, characterized in that, The UAV attitude and hoisting control commands include: During the execution of control commands by the drone, data on visual center of gravity and sling tension are continuously collected; The control module compares the target state with the current state, calculates the deviation, and automatically adjusts the feedback gain factor. When a change in pitch is detected, the thrust response rate is adjusted, and the control gain is reduced when the attitude stabilizes. The system parameters are periodically updated to maintain control accuracy.

6. The UAV hoisting control method based on industrial vision according to claim 1, characterized in that, The changes in the visual center of gravity and the tension of the suspension rope include: The system sets multiple levels of safety thresholds, including the rope force threshold, the load attitude angle threshold, and the UAV attitude threshold; When the detected value reaches the warning threshold, the control module reduces the flight speed and adjusts the attitude; When the detected value exceeds the warning threshold, the control module switches to hover mode; When the detected value exceeds the limit threshold, an emergency stop command is executed and the suspension rope buffer or locking device is triggered. And record the exception information in the task log.

7. The UAV hoisting control method based on industrial vision according to claim 1, characterized in that, The security mechanism includes: After the task is completed, the system extracts the attitude data, center of gravity change data, and rope force data during the hoisting process; Analyze the control deviation and update the control parameters; In subsequent tasks, load and update the parameters and perform parameter convergence calculations. Develop a control parameter library for task classification; The new parameters are written to the control module after the task is executed.

8. The UAV hoisting control method based on industrial vision according to claim 1, characterized in that, The combined centroid of S2 and the nonlinear dynamic model of S3 include: Based on the comprehensive center of gravity state and nonlinear dynamics model, the vision system and lidar carried by the UAV synchronously collect environmental information around the hoisting path; The collected data is segmented and obstacle detection is performed to generate a 3D environmental model; Calculate wind direction, wind speed, and turbulence intensity by combining meteorological sensor data; The path planning module recalculates the flight path based on the environmental model and dynamic payload status; The control module adjusts the flight altitude, speed, and attitude parameters based on the updated path, and inputs the adjustment results into the closed-loop control module for execution.

9. The UAV hoisting control method based on industrial vision according to claim 1, characterized in that, The control commands of S4 and the monitoring data of S5 include: When multiple drones participate in the hoisting operation, a synchronous communication mechanism is established through the master control node; The master control node allocates pull shares and spatial location targets to each drone; All UAVs share integrated center of gravity and attitude data; The control algorithm updates the thrust allocation matrix based on changes in force. When any drone deviates from its designated path, the remaining drones perform compensatory control to maintain load balance.

10. A drone hoisting control system based on industrial vision, characterized in that, The UAV hoisting control method based on industrial vision, as described in any one of claims 1-9, includes the following modules: The data acquisition module is used by the UAV to collect multi-angle visual image data of the load and real-time tension information of each hoisting rope during hoisting operations; The center of gravity fusion module is used to reconstruct a three-dimensional point cloud based on the data and extract feature points to calculate the visual center of gravity. At the same time, it determines the mechanical center of gravity of the suspension rope based on the force analysis of the suspension rope, and dynamically adjusts the visual center of gravity based on the coordinate difference and force balance relationship between the two to obtain the comprehensive center of gravity position and velocity state of the load. The dynamic modeling module is used to establish a nonlinear dynamic model of the UAV, sling, and load based on the obtained comprehensive center of gravity state. This model considers the elasticity of the sling, the inertial coupling of the load, and external disturbances, and is used to describe the dynamic response characteristics of the UAV hoisting system. The control command generation module is used to generate UAV attitude and hoisting control commands based on the dynamic model using an adaptive control algorithm. By prioritizing the adjustment of position error in the hovering state and prioritizing the adjustment of speed error in the moving state, dynamic correction and stable control of the load center of gravity are achieved. The feedback monitoring module is used to monitor changes in the visual center of gravity and the tension of the suspension rope in real time while the UAV executes control commands. The monitoring results are fed back to the control module to form a closed-loop control, so as to suppress load swing and maintain balance. The safety control module is used to trigger a safety mechanism to automatically decelerate, hover, or stop the machine when monitoring results indicate that the load attitude or rope tension exceeds a threshold, until the load stabilizes or the hoisting task is completed.

Images