Tilt-rotor unmanned aerial vehicle operation method and system based on visual servo and unmanned aerial vehicle

Abstract

The invention relates to the technical field of unmanned aerial vehicles, and discloses a tilt-rotor unmanned aerial vehicle operation method and system based on visual servo and an unmanned aerial vehicle, and the method comprises the steps: obtaining a flight airspace image of the tilt-rotor unmanned aerial vehicle, and determining surrounding obstacles according to the flight airspace image; the relative distance and the relative speed between the tilt-rotor unmanned aerial vehicle and surrounding obstacles are calculated, and the pose and the prediction trajectory of the tilt-rotor unmanned aerial vehicle are adjusted in combination with preset constraint conditions; and according to the relative distance and the relative speed between the tilt-rotor unmanned aerial vehicle and the surrounding obstacles, the current risk level is evaluated by using a risk scoring function, and early warning is performed according to the current risk level. An elastic protection structure is configured for a main rotor, collision and damage of the rotor are avoided, obstacles are accurately recognized in combination with airspace images, the relative distance and the relative speed are calculated in cooperation with multi-source data, the pose track is dynamically adjusted, complex scenes such as weak textures are adapted, graded early warning is conducted through a risk scoring function, and a control strategy is linked. And short-distance operation safety is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and more specifically to a method, system, and UAV for operating tiltrotor UAVs based on visual servoing. Background Technology

[0002] Tiltrotor drones achieve thrust vector adjustment by changing the tilt angle of their rotors / wings, combining the advantages of multi-rotor vertical takeoff and landing with fixed-wing cruise efficiency. They have been applied in scenarios such as power line inspection, building inspection, emergency search and rescue, and agricultural and forestry plant protection. In certain applications, such as agricultural spraying, power line inspection, or building structure inspection, drones need to operate close to the target object. In such cases, exposed wings may accidentally touch the work surface or other obstacles, leading to flight accidents. To reduce such risks, some products on the market have adopted protective nets or specially designed wing structures, but these solutions often affect the performance and efficiency of the drone to some extent. Furthermore, physical protection only solves the safety problem but cannot overcome the perception and positioning bottlenecks faced by drones in close-range operations, especially in complex scenarios such as power transmission lines.

[0003] Positioning inaccuracies caused by dynamic environmental interference. For some tasks requiring high-precision positioning, relying solely on manual remote control is insufficient. Even the most experienced operators may experience deviations due to visual errors or reaction delays. Furthermore, birds, moving vehicles, or other temporarily appearing dynamic objects in the operating environment can severely interfere with the UAV's automatic visual positioning system, introducing erroneous visual information and causing positioning drift or even failure.

[0004] Secondly, the lack of scene texture leads to feature loss. In weakly textured environments such as power transmission lines and large building surfaces, there is a lack of rich and unique visual feature points (e.g., a uniformly colored sky, smooth cables and towers). This makes it difficult for visual localization algorithms that rely on traditional feature point matching to extract a sufficiently stable and numerous number of feature points, thus failing to provide continuous and reliable pose estimation for UAVs. In complex or hazardous environments, such as operating in confined spaces or under severe weather conditions, the combined limitations of human operators and the aforementioned technologies further increase the risk of mission failure. Summary of the Invention

[0005] This invention provides a method, system, and drone for operating tiltrotor drones based on visual servoing, in order to solve the problem of collisions and rotor damage caused by inaccurate visual detection during close-range operations.

[0006] In a first aspect, the present invention provides a visual servoing-based operation method for a tiltrotor unmanned aerial vehicle (UAV). The tiltrotor UAV includes a main rotor and a tail rotor. Each main rotor is equipped with a corresponding elastic protection structure, which is connected to the main body of the tiltrotor UAV via an adjustable retaining ring. The method includes: Acquire airspace images of the tiltrotor UAV and determine surrounding obstacles based on the airspace images; Calculate the relative distance and relative velocity between the tiltrotor UAV and surrounding obstacles, and adjust the attitude and predicted trajectory of the tiltrotor UAV in combination with preset constraints; Based on the relative distance and relative speed between the tiltrotor drone and surrounding obstacles, a risk scoring function is used to assess the current risk level, and an early warning is issued based on the current risk level.

[0007] The present invention provides a visual servo-based tiltrotor UAV operation method. By configuring an elastic protection structure on the main rotor, it avoids rotor collision damage while maintaining flight flexibility. It accurately identifies obstacles by combining airspace images, calculates relative distance and relative speed with multi-source data, and dynamically adjusts the pose and trajectory to adapt to complex operation scenarios such as weak textures. Through risk scoring function-based graded early warning and linkage control strategy, it ensures the safety of close-range operations, while improving positioning and trajectory control accuracy, enabling tiltrotor UAVs to operate safely and efficiently in complex scenarios.

[0008] In one optional implementation, the elastic protection structure includes: two elastic supports, the projection of the elastic supports on the horizontal plane does not coincide with the projection of the main rotor, and the projection length of the elastic supports in the direction perpendicular to the central axis of the tilt-rotor UAV is greater than the projection length of the main rotor. The adjustable retaining ring has a spring locking mechanism.

[0009] The visual servo-based tilt-rotor UAV operation method provided by this invention, through the dual elastic brackets and non-overlapping projection layout, not only avoids the rotor operation space, but also forms more sufficient collision protection with a longer projection length, effectively reducing the risk of rotor hitting obstacles; coupled with the spring-locked adjustable retaining ring, it can achieve tool-free quick disassembly and assembly, and adapt to different size arms, improving the versatility and maintenance convenience of the protection structure.

[0010] In one alternative implementation, determining surrounding obstacles based on flight airspace imagery includes: Target segmentation is performed on the flight airspace image to obtain instance masks for each independent target; Analyze the motion information of adjacent frames in the flight airspace image to filter out dynamic regions in the flight airspace image; The dynamic region is compared with the instance mask of each individual target to generate a dynamic mask for each moving target in the current frame, which serves as the surrounding dynamic obstacle.

[0011] The visual servo-based tiltrotor UAV operation method provided by this invention accurately identifies independent targets in the airspace through instance segmentation and motion analysis. At the same time, it distinguishes dynamic objects and generates dynamic masks to achieve accurate positioning of dynamic obstacles such as birds and moving vehicles. This effectively avoids the impact of dynamic interference on positioning, enabling the UAV to perceive risks more reliably in complex operating environments and providing accurate obstacle information support for subsequent obstacle avoidance and trajectory adjustment.

[0012] In one alternative implementation, determining surrounding obstacles based on flight airspace images further includes: Calculate the effective corner density and feature point count of a single frame of the flight airspace image. If the effective corner density is lower than a preset density threshold or the feature point count is consistently lower than a preset count threshold, then the current image is a weak texture scene or a sparse feature scene; otherwise, it is a normal scene. For weak texture or sparse feature scenes, corner features and line segment features are extracted, the weight of line segment features is increased, the threshold for determining points within line segments is relaxed, and the dependence on the number of feature points is reduced to identify obstacles in weak texture or sparse feature scenes.

[0013] The visual servo-based tiltrotor UAV operation method provided by this invention accurately identifies weak texture / sparse feature scenes by determining the corner density and the number of feature points, avoiding recognition failures caused by insufficient features. For such scenes, the feature extraction strategy is adjusted to increase the feature weight of line segments and relax the threshold of points within line segments, solving the problem of insufficient features under weak texture, ensuring the effectiveness of obstacle recognition, and enabling the UAV to stably perceive obstacles in weak texture operation environments such as power lines and smooth buildings. This improves the adaptability and positioning accuracy of operations in complex scenes and further enhances the reliability of close-range operations.

[0014] In one optional implementation, the relative distance and relative velocity between the tiltrotor UAV and surrounding obstacles are calculated, and the pose and predicted trajectory of the tiltrotor UAV are adjusted in conjunction with preset constraints, including: Acquire the current pose, current speed, and distance data between the tiltrotor UAV and surrounding obstacles, and calculate the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles based on the current pose, current speed, and distance data between the tiltrotor UAV and surrounding obstacles. Based on the initial trajectory and preset constraints, the desired pose and predicted trajectory of the tiltrotor UAV are generated. Based on the desired pose and predicted trajectory, a lookup table or solution is performed to determine the motion control commands for the tiltrotor UAV, and the motion of the tiltrotor UAV is adjusted based on the motion control commands.

[0015] The visual servo-based tiltrotor UAV operation method provided by this invention can accurately calculate relative distance and speed by fusing the UAV's current pose, speed, and obstacle distance data, providing a reliable basis for trajectory adjustment. It generates the desired pose by combining the initial trajectory with preset constraints, and then outputs control commands through table lookup / solution, ensuring the rationality of the trajectory and improving the timeliness of control response. This allows the UAV to dynamically adapt to obstacle changes during close-range operations, avoiding collision risks while ensuring trajectory accuracy. This effectively improves flight stability and safety during operations, while also enhancing the efficiency and controllability of task execution.

[0016] In one alternative implementation, the current risk level is assessed using a risk scoring function based on the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles, including: A continuous risk scoring function is constructed, and the smaller the relative distance and the greater the relative speed between the tiltrotor UAV and surrounding obstacles, the larger the risk scoring function. Based on the relative distance and relative speed between the tiltrotor drone and surrounding obstacles, the current risk score is calculated using a risk scoring function. The current risk score is matched with the preset risk level range to determine the current risk level corresponding to the current risk score.

[0017] The visual servo-based tiltrotor UAV operation method provided by this invention accurately quantifies the risk level between the UAV and obstacles by constructing a continuous risk scoring function that increases with decreasing relative distance and increasing relative speed. By matching the risk score with a preset level range, the risk level can be clearly determined, making the risk assessment more accurate and objective, avoiding the bias of subjective judgment, and providing a reliable basis for subsequent graded early warning and control strategy adjustment. This effectively improves the risk perception capability of UAVs during close-range operations.

[0018] In one optional implementation, the risk levels include: Normal, Attention, Alert, and Danger. Early warning is issued based on the current risk level, including: If the current risk level is normal, there is no need to report it. If the current risk level is "concerned", then reduce the upper limit of the drone's speed and tighten the weight of the attitude angle constraint. If the current risk level is alert, then reduce the drone's speed limit and increase the weight of obstacle avoidance cost; If the current risk level is dangerous, the drone will be forced to enter hovering mode or obstacle avoidance mode.

[0019] The visual servo-based tiltrotor UAV operation method provided by this invention achieves refined risk management through a gradient response strategy of normal, attention, alert, and danger levels. It maintains operational efficiency at low risk, gradually tightens flight constraints and increases obstacle avoidance priority at the attention and alert levels, and forcibly triggers a safety mode at the danger level, avoiding overreaction that affects the operation rhythm. It also enables rapid hazard avoidance at high risk, effectively balancing operational efficiency and safety protection, and allowing the UAV to dynamically adapt to risk changes when operating in close-range complex environments.

[0020] Secondly, the present invention provides a visual servo-based tiltrotor unmanned aerial vehicle (UAV) operation system. The tiltrotor UAV includes a main rotor and a tail rotor. Each main rotor is equipped with a corresponding elastic protection structure. The elastic protection structure is connected to the main body of the tiltrotor UAV via an adjustable retaining ring. The system includes: The visual servo positioning module is used to acquire the flight airspace image of the tiltrotor UAV and determine the surrounding obstacles based on the flight airspace image; The distance detection and motion adjustment module is used to calculate the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles, and adjust the pose and predicted trajectory of the tiltrotor UAV in combination with preset constraints. The risk detection and early warning module is used to assess the current risk level based on the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles using a risk scoring function, and to issue an early warning based on the current risk level.

[0021] Thirdly, the present invention provides a tiltrotor unmanned aerial vehicle (UAV) comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the method described in the first aspect or any corresponding embodiment thereof.

[0022] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0023] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2This is a schematic diagram of the first type of operation method for a tiltrotor UAV based on visual servoing according to an embodiment of the present invention; Figure 3 This is a three-dimensional structural diagram of the UAV in the visual servo-based tilt-rotor UAV operation method according to an embodiment of the present invention. Figure 4 This is a schematic diagram of a second process for a tiltrotor unmanned aerial vehicle (UAV) operation method based on visual servoing according to an embodiment of the present invention. Figure 5 This is a structural block diagram of a tiltrotor unmanned aerial vehicle (UAV) operation system based on visual servoing according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the hardware structure of a tiltrotor drone according to an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0027] In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0028] As an optional application scenario of this invention, such as Figure 1 As shown, the drone operation system may include at least one terminal device and at least one server. Figure 1 The system is illustrated in the example, which includes a computer 101, a mobile terminal 102, and a server 103, and the terminal devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 through a network 110.

[0029] Specifically, the terminal device can be a smartphone, tablet, laptop, PDA, desktop computer, game console, smart TV, smart wearable device, in-vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, etc. Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server providing cloud services. Network 110 can be a wired or wireless network, examples of which include, but are not limited to, the Internet, corporate intranet, local area network, wide area network, mobile communication network, and combinations thereof.

[0030] This invention provides a visual servoing-based tiltrotor UAV operation method. By configuring an elastic protection structure on the main rotor and optimizing the processing of dynamic targets and weak textures, the method can improve the accuracy of visual detection and avoid collisions during close-range operations.

[0031] According to an embodiment of the present invention, a method for operating a tiltrotor unmanned aerial vehicle based on visual servoing is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0032] This embodiment provides a visual servoing-based tiltrotor UAV operation method, which can be used in the aforementioned computer system. Figure 2 This is a flowchart of a visual servo-based tiltrotor UAV operation method according to an embodiment of the present invention. The tiltrotor UAV includes a main rotor and a tail rotor. Each main rotor is equipped with a corresponding elastic protection structure, which is connected to the main body of the tiltrotor UAV via an adjustable retaining ring. Figure 2 As shown, the process includes the following steps: Step S201: Obtain the flight airspace image of the tiltrotor UAV and determine the surrounding obstacles based on the flight airspace image.

[0033] Specifically, image data of the flight airspace is collected by visual sensors (such as cameras) carried by the drone, and then image analysis technology is used to identify and determine key information such as the type and location of surrounding obstacles from the images, providing environmental input for subsequent risk assessment and trajectory adjustment.

[0034] Step S202: Calculate the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles, and adjust the attitude and predicted trajectory of the tiltrotor UAV in combination with preset constraints.

[0035] Specifically, based on information about surrounding obstacles and combined with the UAV's own state data collected by sensors, the relative distance and relative speed to the obstacles are calculated. Then, based on the initial mission trajectory, and incorporating preset constraints such as safety and performance, the desired pose and predicted trajectory are generated through trajectory planning and control algorithms, thereby realizing the dynamic adjustment of the UAV's flight state.

[0036] Step S203: Based on the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles, the current risk level is assessed using a risk scoring function, and an early warning is issued based on the current risk level.

[0037] Specifically, relative distance and relative speed are used to calculate real-time risk scores through a preset risk scoring function and match the corresponding risk levels. Differentiated early warning and control strategies (such as speed limits, attitude constraints, and forced risk avoidance) are then triggered based on different risk levels to ensure the safety and controllability of the operation process.

[0038] The visual servo-based tiltrotor UAV operation method provided in this embodiment avoids rotor collision damage by configuring an elastic protection structure on the main rotor, while also ensuring flight flexibility. It accurately identifies obstacles by combining airspace images, calculates relative distance and relative speed using multi-source data, and dynamically adjusts the pose and trajectory to adapt to complex operation scenarios such as weak textures. Through risk scoring function-based graded early warning and linkage control strategies, it ensures safety in close-range operations, while improving positioning and trajectory control accuracy, enabling tiltrotor UAVs to operate safely and efficiently in complex scenarios.

[0039] In some optional implementations, the elastic protection structure includes: two elastic supports, the projections of the elastic supports and the main rotor on the horizontal plane not coinciding, and the projected length of the elastic supports in the direction perpendicular to the central axis of the tilt-rotor UAV being greater than the projected length of the main rotor; and an adjustable retaining ring that is a spring locking mechanism.

[0040] Specifically, such as Figure 3The diagram shows a three-dimensional structural representation of a tiltrotor drone. Each main rotor is protected by two elastic supports with adjustable retaining rings. Made of high-strength, lightweight, and highly elastic composite materials (such as carbon fiber or glass fiber reinforced plastic), the supports ensure rapid recovery from impact without damage. The elastic supports can be L-shaped or ring-shaped, U-shaped or semi-enclosed frames, or lightweight mesh or honeycomb skeletons to achieve similar cushioning and rebound effects. The vertical portion surrounds and protects the rotor, while the horizontal portion is securely fixed to the drone body using specially designed adjustable retaining rings, forming a stable support structure. To reduce weight and increase elasticity, the support surface is designed with multiple regularly arranged weight-reducing holes. The adjustable retaining rings can be wedge-locked, threaded clamped, or a combination of sliding grooves and locating pins. Replaceable bushings adapt to different arm cross-sectional shapes and the size differences of different drone models, ensuring good compatibility and stability. This allows users to easily install or remove the protective supports without tools, facilitating maintenance and replacement.

[0041] The elastic protection structure consists of an elastic support and an adjustable retaining ring. The material is a reinforced composite material (e.g., carbon fiber or glass fiber reinforced polymer), or a thin-walled metal or composite sandwich, maintaining a balance between elasticity and impact resistance. The support wall thickness is 1–3 mm (preferably 2 mm). Regular weight-reducing holes are formed in the vertical and horizontal sections, with a hole diameter of 2–5 mm and a hole spacing of 6–12 mm, to reduce the bending stiffness per unit length and achieve impact rebound. The minimum static clearance between the support and the blade is 5–20 mm (preferably 10 mm). The retaining ring uses a spring locking mechanism with a locking tongue travel of 1–3 mm, adapting to boom tube diameters of 18–28 mm (achieved through replaceable bushings), with an assembly tolerance of ±0.5 mm, enabling tool-free installation / removal.

[0042] The visual servo-based tilt-rotor UAV operation method provided in this embodiment avoids the rotor operation space and forms more sufficient collision protection with a longer projection length, effectively reducing the risk of rotor hitting obstacles. With the spring-locked adjustable retaining ring, tool-free quick disassembly and assembly can be achieved, and it can be adapted to different size arms, improving the versatility and maintenance convenience of the protection structure.

[0043] This embodiment provides a visual servoing-based tiltrotor UAV operation method, which can be used in the aforementioned computer system. Figure 4 This is a flowchart of a visual servo-based tiltrotor UAV operation method according to an embodiment of the present invention. The tiltrotor UAV includes a main rotor and a tail rotor. Each main rotor is equipped with a corresponding elastic protection structure, which is connected to the main body of the tiltrotor UAV via an adjustable retaining ring. Figure 4 As shown, the process includes the following steps: Step S301: Obtain the flight airspace image of the tiltrotor UAV and determine the surrounding obstacles based on the flight airspace image.

[0044] Specifically, step S301 includes: Step S3011: Target segmentation is performed on the flight airspace image to obtain instance masks for each independent target.

[0045] Specifically, to achieve high-precision positioning and control, especially in confined spaces or harsh environments, cameras with high resolution, wide dynamic range, and low latency were selected to ensure image clarity and real-time performance. These cameras provide high-quality image data under various lighting conditions and support multispectral sensors to enhance perception capabilities in specific environments. Equipped with a high-performance image processing chip, images can be analyzed and processed in real time, accurately capturing targets even in low light or complex backgrounds. By applying advanced computer vision algorithms such as feature point detection, target tracking, and stereo matching, real-time environmental perception is achieved, assisting the UAV in precise positioning. Furthermore, Simultaneous Localization and Mapping (SLAM) technology is integrated, enabling the UAV to build maps in unknown environments and simultaneously determine its own location.

[0046] In terms of visual servoing and localization, the front end first uses a deep learning-based instance segmentation network to perform pixel-level target segmentation on the camera image, obtaining instance masks for each type of independent target. The specific segmentation method is a mature existing technology, which will not be elaborated here.

[0047] Step S3012: Analyze the motion information of adjacent frames in the flight airspace image and filter out the dynamic regions in the flight airspace image.

[0048] Specifically, optical flow or tracking matching is performed on feature points and line segments between adjacent frames. By comparing the consistency between their pixel motion direction and the predicted self-motion of the aircraft, regions that significantly deviate from the overall disparity field are identified as dynamic regions. The result is then combined with the instance segmentation results to generate a dynamic mask for the current frame, thus identifying dynamic targets in the flight airspace image. The disparity field refers to the degree of positional offset of different objects in an image from the UAV's perspective. The disparity of stationary objects conforms to the overall scene pattern, and objects on the same plane have similar disparities. However, the disparity of dynamic objects (such as a bird suddenly flying by) is significantly inconsistent with the disparity of surrounding stationary objects.

[0049] Step S3013: Compare the dynamic region with the instance mask of each independent target to generate a dynamic mask for each moving target in the current frame, which serves as the surrounding dynamic obstacle.

[0050] Specifically, in the feature extraction stage, point features and line segment features are extracted and retained only outside the dynamic mask; features located near the mask boundary that may be affected by dynamic interference are given lower weights or directly removed in the back-end optimization, thereby effectively reducing the reprojection error introduced by dynamic targets.

[0051] The visual servo-based tiltrotor UAV operation method provided in this embodiment accurately identifies independent targets in the airspace through instance segmentation and motion analysis. At the same time, it distinguishes dynamic objects and generates dynamic masks to achieve accurate positioning of dynamic obstacles such as birds and moving vehicles. This effectively avoids the impact of dynamic interference on positioning, enabling the UAV to perceive risks more reliably in complex operating environments and providing accurate obstacle information support for subsequent obstacle avoidance and trajectory adjustment.

[0052] In some optional implementations, step S301 above further includes: Step S3014: Calculate the effective corner density and feature point count of a single frame of the flight airspace image. If the effective corner density is lower than a preset density threshold or the feature point count is consistently lower than a preset count threshold, then the current image is a weak texture scene or a sparse feature scene; otherwise, it is a normal scene.

[0053] Specifically, to address weak texture scenes such as power transmission lines and metal components (i.e., areas with slow image grayscale changes, significantly insufficient number of edges and corners, and small local gradient variance), the system counts the number of effective corners, line segments, and image gradient energy per unit pixel area for each frame. When the corner density is lower than a set threshold or the number of points within a feature is consistently lower than a certain threshold, the system automatically determines the current scene to be a weak texture scene or a sparse feature scene.

[0054] Step S3015: For weak texture scenes or sparse feature scenes, extract corner features and line segment features, increase the weight of line segment features, relax the threshold for determining points within line segments, reduce the dependence on the number of feature points, and identify obstacles in weak texture scenes or sparse feature scenes.

[0055] Specifically, in the back-end optimization, the weight of line feature residuals in the overall objective function is increased, the threshold for determining points within line segments is appropriately relaxed, and the dependence on the number of point features is reduced, so that the system still has sufficient observability even when features are sparse. Back-end pose optimization uses joint point-to-surface or point-to-line geometric residuals and introduces robust kernel functions such as Huber or Cauchy to suppress anomalous matching, outputting state estimates of the UAV's pose, velocity, and covariance, providing continuous, confidence-based input for subsequent control modules.

[0056] Dynamic region processing can employ instance segmentation to obtain pixel-level masks, or semantic segmentation combined with optical flow consistency detection to achieve region filtering. Feature extraction and localization, besides point-line joint feature methods, can utilize direct or semi-direct methods to construct photometric errors, or primarily use sparse features supplemented by edge / contour constraints to improve the observability of weak textures. Target recognition and wire detection can introduce a learning-based detection network as a priori on top of edge detection and Hough transform; the order of their use can be switched between the mapping and tracking stages depending on the scenario. Multi-source fusion, in addition to extended Kalman filtering, can employ unscented Kalman filtering or graph optimization backends to achieve unified state estimation. If GNSS is limited or unavailable, UWB, visual markers, or artificial feature boards can be introduced as auxiliary localization sources; the system can be integrated after extrinsic parameter calibration and time synchronization without altering the main workflow. Early warning risk assessment, besides explicit distance and relative velocity rules, can also use data-trained scoring models or learned cost mappings, but a deterministic mapping table with control parameters must still be retained to meet engineering controllability requirements.

[0057] The visual servo-based tiltrotor UAV operation method provided in this embodiment accurately identifies weak texture / sparse feature scenes by determining the corner density and the number of feature points, avoiding recognition failures caused by insufficient features. For such scenes, the feature extraction strategy is adjusted to increase the feature weight of line segments and relax the threshold of points within line segments, solving the problem of insufficient features under weak texture and ensuring the effectiveness of obstacle recognition. This enables the UAV to stably perceive obstacles in weak texture operation environments such as power lines and smooth buildings, improving the adaptability and positioning accuracy of operations in complex scenes and further enhancing the reliability of close-range operations.

[0058] Step S302: Calculate the relative distance and relative speed between the tilt-rotor UAV and surrounding obstacles, and adjust the attitude and predicted trajectory of the tilt-rotor UAV in combination with preset constraints.

[0059] Specifically, step S302 includes: Step S3021: Obtain the current pose, current speed, and distance data between the tilt-rotor UAV and surrounding obstacles, and calculate the relative distance and relative speed between the tilt-rotor UAV and surrounding obstacles based on the current pose, current speed, and distance data between the tilt-rotor UAV and surrounding obstacles.

[0060] Specifically, the drone obtains its current position / attitude (pose), flight speed, and distance data to surrounding obstacles through its sensors (such as visual cameras, ranging modules, IMUs, etc.); then, by combining its own pose, speed, and distance to obstacles, it obtains the relative distance between the drone and the obstacle (the actual distance between the drone and the obstacle) through spatial geometric operations (such as coordinate difference calculation), and obtains the relative speed (the speed at which the drone approaches / moves away from the obstacle) through velocity vector operations.

[0061] Distance warning and judgment are achieved through simultaneous distance data acquisition using multiple types of distance sensors (such as ultrasonic sensors, infrared sensors, or laser rangefinders). These sensors are integrated and installed at pre-defined key locations on the UAV body, forming a fixed arrangement with a defined geometric relationship to the UAV body, rotors, and field of view, creating a multi-point sensing network. The ultrasonic sensors (2-4m), infrared sensors (0.05-2m), and laser rangefinders (0.1-12m) have adjustable pitch angles of 0-20° and sampling frequencies of 50-100Hz. Each distance sensor monitors the surrounding environment in real time and detects approaching obstacles, reducing blind spots and improving assembly and maintenance operability. To minimize sensor obstruction, the distance sensors can be integrated into the protective components for a unified layout or distributed on the fuselage underside or landing gear. A unified installation standard and calibration process ensures consistent field of view coverage and geometry.

[0062] The integrated intelligent early warning algorithm receives data from various distance sensors, distinguishes between static and dynamic obstacles, and predicts potential collision risks based on the drone's current speed and direction. It automatically triggers different levels of warning signals based on the obstacle's distance and relative speed, ranging from visual or auditory alerts to automatically activating obstacle avoidance procedures to adjust the flight path to avoid hazards. Furthermore, the early warning system is adaptive, dynamically adjusting warning parameters according to different flight missions and environmental conditions to ensure the most appropriate protection in all situations.

[0063] Step S3022: Based on the initial trajectory and preset constraints, generate the desired pose and predicted trajectory of the tilt-rotor UAV.

[0064] Specifically, based on the initial flight trajectory preset by the mission, and combined with safety constraints (such as minimum obstacle avoidance distance, maximum speed limit, etc.), the target position / attitude (desired pose) that the UAV needs to reach next is generated through trajectory planning algorithms (such as path interpolation), and the flight path (predicted trajectory) from the current state to the desired pose is predicted.

[0065] The system employs a visual-inertial SLAM approach based on feature point and line segment fusion: point features (threshold 0.6–0.8 pixels), line segment point threshold 1–3 pixels, and residuals using point-to-line distance with a robust kernel (Huber / Cauchy). When the dynamic region accounts for >40%, the line segment threshold is automatically relaxed to 2–4 pixels, and point features are weighted less. The system fuses with IMU / RTK-GNSS / ranging data at 100–200Hz, with a map resolution of 0.03–0.08m / pixel. The system provides operating modes: fixed-point hovering, trajectory following, multi-point inspection, and automatic obstacle avoidance; each mode specifies a speed limit, minimum turning radius, and safe distance from the target (0.5–1.0m for power line scenarios).

[0066] The drone rotor protection method not only effectively prevents rotor collisions with obstacles but also enhances the drone's intelligence level. It is particularly suitable for tasks requiring close-range operations in confined spaces or complex environments, such as building inspections and power line patrols. Furthermore, in outdoor operations with high wind speeds or other uncertainties, the protection device significantly reduces the risk of accidental collisions. Compared to traditional solutions, this protection device is simple in structure, easy to manufacture, and reduces production costs. It also minimizes repair expenses due to accidents, while improving user experience and ease of operation.

[0067] In addition to model predictive control, closed-loop control can employ hierarchical PID, LQR, or adaptive / gain scheduling structures to achieve coordinated control of attitude, position, and tilt angle. When the warning level changes, parameter switching can be accomplished either through table lookup, interpolation transition, or state machine-triggered mode switching to avoid abrupt changes. Obstacle avoidance path generation, besides cost-based local replanning, can be replaced with vector field histograms, artificial potential fields, or local trajectory generators using sampling methods, with constraint boundaries still provided by the risk assessment module. In addition to fixed-point hovering, trajectory following, and multi-point inspection, mission modes can include trajectory photography, fixed-distance and fixed-amplitude scanning, or semi-automatic micro-management modes. Regarding the workflow sequence, mapping and recognition can either be performed sequentially or in parallel online. Data recording and report generation can be preprocessed on the airborne end or completed at the ground station or edge server to adapt to different computing power and link conditions.

[0068] Step S3023: Based on the desired pose and predicted trajectory, perform a lookup table or solve the problem to determine the motion control command for the tiltrotor UAV, and adjust the motion of the tiltrotor UAV based on the motion control command.

[0069] Specifically, in terms of perception fusion, unified timestamp alignment and extrinsic parameter calibration are performed on sensors such as cameras, inertial measurement units, ranging, and real-time kinematic (RTK) / Global Navigation Satellite System (GNSS) sensors. Rigid body transformation relationships from each sensor to the body coordinate system are established using known installation poses. State equations and noise models for each observation are constructed within a unified filtering / graph optimization framework. The fusion module monitors the residual statistics and data update rate of each sensor in real time. When a sensor experiences an anomaly such as a sudden increase in residuals, prolonged data loss, or self-test failure, the observations from that source are automatically downweighted or temporarily removed. Based on the current mission mode and risk status, corresponding degradation and fallback strategies are triggered (e.g., speed limitation, hovering, or returning to the home line according to a preset route), ensuring continuous availability of the system even in complex environments such as GNSS limitations, sudden changes in illumination, or partial obstruction.

[0070] Visual servo data is fused with data from other sensors (such as IMU, GPS, and barometer) to form a comprehensive sensing system. Kalman filtering or other data fusion algorithms improve positioning accuracy and stability. Advanced control strategies such as PID controllers, adaptive control, or model predictive control (MPC) are employed to precisely adjust the UAV's attitude and position based on real-time feedback, ensuring stable flight along a predetermined trajectory, even under varying wind speeds or electromagnetic interference. For tasks requiring centimeter-level positioning accuracy, integrated RTK (Real-Time Dynamic Differential) GPS technology further enhances positioning accuracy. Multiple preset operating modes, such as hovering, trajectory following, multi-point inspection, and automatic obstacle avoidance, allow users to select the appropriate mode based on specific task requirements. The system automatically plans the optimal path and dynamically adjusts flight parameters according to actual conditions, ensuring efficient completion of each mission. The built-in task scheduler automatically adjusts task priorities to ensure optimal resource utilization and supports remote monitoring, allowing operators to view the UAV's operational status in real time and intervene manually when necessary.

[0071] A closed-loop control structure is adopted to address the characteristics of the tiltrotor mechanism (which can be implemented as model predictive control (MPC), adaptive control, or PID / LQR, etc.). Its inputs include the current pose / velocity estimate from the fusion of visual servoing and multi-source perception, the desired trajectory given by the mission mode, and safety constraints output by the obstacle avoidance and warning module (e.g., minimum safe distance, available flight corridor, maximum permissible attitude angle and velocity, etc.). The control logic first generates a desired pose and trajectory reference based on the current mode (fixed-point hovering, trajectory following, multi-point inspection, or automatic obstacle avoidance). Then, obstacle avoidance constraints and warning constraints are introduced based on this reference. Control commands for attitude, thrust, and tilt angle are obtained through online solving or table lookup, and safety constraints are prioritized over performance objectives.

[0072] The visual servo-based tiltrotor UAV operation method provided in this embodiment can accurately calculate relative distance and speed by fusing the UAV's current pose, speed, and obstacle distance data, providing a reliable basis for trajectory adjustment. It generates the desired pose by combining the initial trajectory and preset constraints, and then outputs control commands through table lookup / solution, ensuring the rationality of the trajectory and improving the timeliness of control response. This allows the UAV to dynamically adapt to changes in obstacles during close-range operations, avoiding collision risks and ensuring trajectory accuracy. This effectively improves the stability and safety of flight during operations, while also enhancing the efficiency and controllability of task execution.

[0073] Step S303: Based on the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles, the current risk level is assessed using a risk scoring function, and an early warning is issued based on the current risk level.

[0074] Specifically, step S303 includes: Step S3031: Construct a continuous risk scoring function. The smaller the relative distance between the tiltrotor UAV and surrounding obstacles and the greater the relative speed, the larger the risk scoring function.

[0075] Specifically, the early warning and risk assessment module calculates the distance and relative speed between the UAV and surrounding obstacles or targets based on distance sensors and visual / fusion pose estimation. It constructs a continuous risk scoring function (e.g., a function that monotonically increases as distance decreases and relative speed increases). Using the relative distance (denoted as d) and relative speed (denoted as v) between the UAV and the obstacle as input, a continuous risk scoring function is designed, for example, risk score = k × (v / d), where k is a coefficient. The function value is made to monotonically increase as the relative distance d decreases or the relative speed v increases. That is, the closer the UAV is to the obstacle and the faster it approaches, the higher the risk score, thereby accurately quantifying the severity of real-time risks.

[0076] Step S3032: Calculate the current risk score using a risk scoring function based on the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles.

[0077] Specifically, by substituting the relative distance and relative speed between the drone and the obstacle collected in real time, the specific risk score value corresponding to the current moment can be directly calculated, transforming the abstract distance / speed relationship into a quantifiable risk indicator.

[0078] Step S3033: Match the current risk score with the preset risk level range to determine the current risk level corresponding to the current risk score.

[0079] Specifically, several tiered thresholds are set, and score ranges corresponding to four risk levels—"Normal," "Attention," "Alert," and "Danger"—are pre-defined. For example, 0-20 points is "Normal," 20-40 points is "Attention," etc., discretizing the risk score into four risk levels. When the risk score crosses a threshold, causing a change in the warning level, tiered adjustments to control parameters and constraints are automatically triggered: for example, at the "Attention" level, the speed limit is moderately reduced and the attitude angle constraint weight is tightened; at the "Alert / Danger" level, the speed limit is significantly reduced, the obstacle avoidance cost weight is increased, or hovering / obstacle avoidance mode is forcibly entered. Changes in warning levels are managed through a state machine, supporting interpolation / gradual transitions to avoid abrupt changes in control variables. The safety distance, speed limit, and control weight templates corresponding to different mission modes and different warning levels are pre-configured by the ground station. During flight, command-based switching enables the structure-perception-control-mission system to work collaboratively under a unified strategy, forming a reusable and engineering-implementable complete operational closed loop.

[0080] Visual servo control-based automated systems are not only suitable for high-precision tasks in confined spaces or complex environments, such as building inspections, power line inspections, and agricultural monitoring, but also maintain stable flight performance and high-precision positioning capabilities under adverse weather conditions such as strong winds, rain, and snow. The introduction of automated systems greatly simplifies operational procedures, reduces the skill requirements for operators, minimizes operational errors caused by human factors, and improves task success rates and safety.

[0081] The visual servo-based tiltrotor UAV operation method provided in this embodiment accurately quantifies the risk level between the UAV and obstacles by constructing a continuous risk scoring function that increases with decreasing relative distance and increasing relative speed. Combined with the risk score matching within a preset level range, it achieves a clear determination of the risk level, making risk assessment more accurate and objective, avoiding the bias of subjective judgment, and providing a reliable basis for subsequent graded early warning and control strategy adjustment, effectively improving the risk perception capability of UAVs during close-range operations.

[0082] In some optional implementations, risk levels include: Normal, Attention, Alert, and Danger. Warnings are issued based on the current risk level, including: If the current risk level is normal, there is no need to report it. If the current risk level is "concerned", then reduce the upper limit of the drone's speed and tighten the weight of the attitude angle constraint. If the current risk level is alert, then reduce the drone's speed limit and increase the weight of obstacle avoidance cost; If the current risk level is dangerous, the drone will be forced to enter hovering mode or obstacle avoidance mode.

[0083] The visual servo-based tiltrotor UAV operation method provided in this embodiment achieves refined risk management through a gradient response strategy of normal, attention, alert, and danger levels. It maintains operational efficiency at low risk, gradually tightens flight constraints and increases obstacle avoidance priority at the attention and alert levels, and forcibly triggers a safety mode at the danger level, avoiding overreaction that affects the operation rhythm. It also enables rapid hazard avoidance at high risk, effectively balancing operational efficiency and safety protection, and allowing the UAV to dynamically adapt to risk changes when operating in close-range complex environments.

[0084] This embodiment also provides a visual servoing-based tiltrotor UAV operation system, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the systems described in the following embodiments are preferably implemented in software, hardware implementations, or a combination of software and hardware, are also possible and contemplated.

[0085] This embodiment provides a visual servo-based tiltrotor UAV operation system. The tiltrotor UAV includes a main rotor and a tail rotor. Each main rotor is equipped with a corresponding elastic protection structure, which is connected to the main body of the tiltrotor UAV via an adjustable retaining ring. Figure 5 As shown, the system includes: The visual servo positioning module 501 is used to acquire the flight airspace image of the tilt-rotor UAV and determine the surrounding obstacles based on the flight airspace image.

[0086] The distance detection and motion adjustment module 502 is used to calculate the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles, and adjust the pose and predicted trajectory of the tiltrotor UAV in combination with preset constraints.

[0087] The risk detection and early warning module 503 is used to assess the current risk level based on the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles using a risk scoring function, and to issue an early warning based on the current risk level.

[0088] In some alternative implementations, the visual servo positioning module 501 includes: The instance segmentation unit is used to segment targets in the flight airspace image to obtain an instance mask for each independent target.

[0089] The dynamic analysis unit is used to analyze the motion information of adjacent frames in the flight airspace image and filter out the dynamic regions in the flight airspace image.

[0090] The dynamic target determination unit is used to compare the dynamic region with the instance mask of each individual target to generate a dynamic mask for each moving target in the current frame, which serves as the surrounding dynamic obstacle.

[0091] The scene determination unit is used to calculate the effective corner density and feature point number of a single frame of the flight airspace image. If the effective corner density is lower than the preset density threshold or the feature point number is continuously lower than the preset number threshold, the current image is a weak texture scene or a sparse feature scene; otherwise, it is a normal scene.

[0092] The special scene obstacle judgment unit is used to extract corner features and line segment features for weak texture scenes or feature sparse scenes, increase the weight of line segment features, relax the threshold for judging points inside line segments, reduce the dependence on the number of feature points, and identify obstacles in weak texture scenes or feature sparse scenes.

[0093] In some alternative implementations, the distance detection and motion adjustment module 502 includes: The motion parameter calculation unit is used to acquire the current pose, current speed, and distance data between the tilt-rotor UAV and surrounding obstacles, and to calculate the relative distance and relative speed between the tilt-rotor UAV and surrounding obstacles based on the current pose, current speed, and distance data between the tilt-rotor UAV and surrounding obstacles.

[0094] The pose and trajectory generation unit is used to generate the desired pose and predicted trajectory of a tiltrotor UAV based on the initial trajectory and preset constraints.

[0095] The motion adjustment unit is used to look up tables or solve problems based on the desired pose and predicted trajectory to determine the motion control commands of the tiltrotor UAV, and adjust the motion of the tiltrotor UAV based on the motion control commands.

[0096] In some optional implementations, the risk detection and early warning module 503 includes: The risk scoring function construction unit is used to construct a continuous risk scoring function. The smaller the relative distance and the greater the relative speed between the tiltrotor UAV and surrounding obstacles, the larger the risk scoring function.

[0097] The risk scoring unit is used to calculate the current risk score based on the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles using a risk scoring function.

[0098] The risk level determination unit is used to match the current risk score with a preset risk level range to determine the current risk level corresponding to the current risk score.

[0099] The visual servo-based tiltrotor UAV operation system provided in this embodiment of the invention can execute the visual servo-based tiltrotor UAV operation method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0100] Figure 6 This is a structural schematic diagram of a tiltrotor unmanned aerial vehicle (UAV) provided in an embodiment of the present invention.

[0101] The following is a detailed reference. Figure 6 The diagram illustrates a suitable structural design for implementing a tiltrotor drone according to an embodiment of the present invention. The tiltrotor drone may include a processor (e.g., a central processing unit, graphics processor, etc.) 601, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 602 or a program loaded from memory 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of the tiltrotor drone. The processor 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to bus 604.

[0102] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; memory devices 608 including, for example, magnetic tape, hard disk, etc.; and communication devices 609. Communication device 609 allows the tiltrotor drone to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 The tiltrotor drone with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown, and may alternatively implement or have more or fewer devices.

[0103] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a memory 608, or installed from a ROM 602. When the computer program is executed by the processor 601, it performs the functions defined in the visual servo-based tiltrotor UAV operation method of the embodiments of the present invention.

[0104] Figure 6 The tiltrotor drone shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0105] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the visual servo-based tiltrotor UAV operation method shown in the above embodiments is implemented.

[0106] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for operating a tiltrotor unmanned aerial vehicle based on visual servoing, characterized in that, The tiltrotor UAV includes a main rotor and a tail rotor. Each main rotor is equipped with a corresponding elastic protection structure. The elastic protection structure is connected to the main body of the tiltrotor UAV via an adjustable retaining ring. The method includes: Acquire an airspace image of the tiltrotor UAV and determine surrounding obstacles based on the airspace image; Calculate the relative distance and relative velocity between the tiltrotor UAV and surrounding obstacles, and adjust the attitude and predicted trajectory of the tiltrotor UAV in combination with preset constraints; Based on the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles, the current risk level is assessed using a risk scoring function, and an early warning is issued based on the current risk level.

2. The method according to claim 1, characterized in that, The elastic protection structure includes: two elastic supports, the projection of the elastic supports on the horizontal plane does not coincide with the projection of the main rotor, and the projection length of the elastic supports perpendicular to the central axis of the tilt-rotor UAV is greater than the projection length of the main rotor. The adjustable retaining ring is a spring locking mechanism.

3. The method according to claim 1, characterized in that, Based on the flight airspace image, surrounding obstacles are determined, including: The flight airspace image is segmented to obtain an instance mask for each independent target; Analyze the motion information of adjacent frames in the flight airspace image to filter out dynamic regions in the flight airspace image; The dynamic region is compared with the instance mask of each individual target to generate a dynamic mask for each moving target in the current frame, which serves as the surrounding dynamic obstacle.

4. The method according to claim 1, characterized in that, Determining surrounding obstacles based on the flight airspace image also includes: Calculate the effective corner density and feature point count of a single frame of the flight airspace image. If the effective corner density is lower than a preset density threshold or the feature point count is consistently lower than a preset count threshold, then the current image is a weak texture scene or a sparse feature scene; otherwise, it is a normal scene. For weak texture or sparse feature scenes, corner features and line segment features are extracted, the weight of line segment features is increased, the threshold for determining points within line segments is relaxed, and the dependence on the number of feature points is reduced to identify obstacles in weak texture or sparse feature scenes.

5. The method according to claim 1, characterized in that, Calculate the relative distance and relative velocity between the tiltrotor UAV and surrounding obstacles, and adjust the UAV's pose and predicted trajectory based on preset constraints, including: Acquire the current pose, current speed, and distance data between the tiltrotor UAV and surrounding obstacles, and calculate the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles based on the current pose, current speed, and distance data between the tiltrotor UAV and surrounding obstacles. Based on the initial trajectory and preset constraints, the desired pose and predicted trajectory of the tiltrotor UAV are generated. Based on the desired pose and predicted trajectory, a lookup table or solution is performed to determine the motion control command for the tiltrotor UAV, and the motion of the tiltrotor UAV is adjusted based on the motion control command.

6. The method according to claim 1, characterized in that, Based on the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles, the current risk level is assessed using a risk scoring function, including: A continuous risk scoring function is constructed, whereby the smaller the relative distance between the tilt-rotor UAV and surrounding obstacles and the greater the relative speed, the larger the risk scoring function becomes. Based on the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles, the current risk score is calculated using a risk scoring function. The current risk score is matched with the preset risk level range to determine the current risk level corresponding to the current risk score.

7. The method according to claim 6, characterized in that, The risk levels include: Normal, Attention, Alert, and Danger. Warnings are issued based on the current risk level, including: If the current risk level is normal, there is no need to report it. If the current risk level is "concerned", then reduce the upper limit of the drone's speed and tighten the weight of the attitude angle constraint. If the current risk level is alert, then reduce the drone's speed limit and increase the weight of obstacle avoidance cost; If the current risk level is dangerous, the drone will be forced to enter hovering mode or obstacle avoidance mode.

8. A tilt-rotor unmanned aerial vehicle (UAV) operation system based on visual servoing, characterized in that, The tiltrotor UAV includes a main rotor and a tail rotor. Each main rotor is equipped with a corresponding elastic protection structure. The elastic protection structure is connected to the main body of the tiltrotor UAV via an adjustable retaining ring. The system includes: The visual servo positioning module is used to acquire the flight airspace image of the tiltrotor UAV and determine the surrounding obstacles based on the flight airspace image; The distance detection and motion adjustment module is used to calculate the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles, and adjust the pose and predicted trajectory of the tiltrotor UAV in combination with preset constraints. The risk detection and early warning module is used to assess the current risk level based on the relative distance and relative speed between the tiltrotor UAV and surrounding obstacles using a risk scoring function, and to issue an early warning based on the current risk level.

9. A tilt-rotor unmanned aerial vehicle, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the method of any one of claims 1 to 7.

Images