Unmanned aerial vehicle autonomous approaching and netting control system based on visual servo

Abstract

The invention discloses an unmanned aerial vehicle autonomous approaching and netting control system based on visual servo, and belongs to the technical field of unmanned aerial vehicle control. The high-precision two-axis pan-tilt mechanism, the binocular vision module, the netting launching device and the electromechanical system main control unit of a hierarchical control algorithm are integrated, automatic recognition, accurate distance measurement, stable tracking and intelligent striking of a dynamic target are achieved, and the two-axis pan-tilt mechanism enables the netting launching device to have the motion freedom degree independent of an unmanned aerial vehicle platform. The effective striking range of the system is effectively expanded by combining a high-precision target positioning and servo tracking algorithm based on binocular vision, the capturing precision and the adaptive capacity of a dynamic target are remarkably improved, and through fusion of deep learning target detection, binocular vision pose calculation and a hierarchical control strategy, the dynamic target capturing precision and the adaptive capacity of the system are improved. The full-process independence from target identification, tracking approximation to attack decision making is realized, and a multi-subsystem efficient and collaborative working architecture is formed through unified scheduling of the main control unit.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle control, and particularly relates to a visual servo-based unmanned aerial vehicle autonomous approaching and net capture control system. BACKGROUND

[0002] Illegal intrusion of unmanned aerial vehicles poses a real threat to critical infrastructure, public safety and even national security. Existing net capture interception schemes mainly fall into two categories: one is to rely on the operator to manually control the unmanned aerial vehicle for aiming and launching. This method highly depends on the experience and reaction speed of the operator, and when dealing with high-speed and highly maneuverable targets, there are problems of reaction lag, high operation difficulty and low success rate, especially in complex lighting or background environment, the accuracy of manual distance and timing judgment is difficult to guarantee. The second is to use a net capture unmanned aerial vehicle with a certain degree of autonomy.

[0003] Existing autonomous schemes usually have obvious limitations: for example, some systems use simple pre-programmed trajectories or rough positioning based on monocular vision, lack accurate and real-time three-dimensional pose perception ability of the target, resulting in a very small effective capture window, often requiring the host and target unmanned aerial vehicles to maintain a nearly relative static state, poor adaptability in actual combat; some systems try to introduce visual servoing, but most of them use pan-tilt mechanisms with insufficient degrees of freedom or low control accuracy, making it difficult to stably track dynamic targets, and the visual system is usually a monocular camera, which cannot provide accurate distance information, resulting in inaccurate launch timing judgment and low capture success rate. SUMMARY

[0004] The purpose of the present application is to provide a visual servo-based unmanned aerial vehicle autonomous approaching and net capture control system to solve the problems raised in the background.

[0005] To achieve the above purpose, the present application provides the following technical scheme: a visual servo-based unmanned aerial vehicle autonomous approaching and net capture control system, comprising a two-axis pan-tilt mechanism, a net capture launching device, a binocular vision module and a master control unit.

[0006] The two-axis pan-tilt mechanism is configured to drive the entire pan-tilt system to rotate 360 degrees around the vertical axis, and is also used to drive the net capture launching device to pitch around the horizontal axis.

[0007] The net capture launching device is configured as the execution terminal of the net capture control system, and when the master control unit issues a launch instruction, the net will be shot at high speed towards the target together with the internal capture net, completing the capture task.

[0008] The binocular vision module is configured as two parameters consistent, calibrated gray level cameras, two gray level cameras are installed in a horizontal parallel manner, the line between the optical centers of the gray level cameras is parallel to the yaw axis of the holder, and the binocular images of the target region are collected in real time through the gray level cameras.

[0009] The main control unit is configured to send control instructions to two holder motors of the two-axis holder mechanism through a motor drive interface, to realize closed-loop servo control of the holder, to control the net catching launching device to perform a net catching triggering action through a general input and output interface, and to continuously interact with the flight controller of the unmanned aerial vehicle through a serial communication interface, and to send a flight path instruction generated based on visual sensing to guide the unmanned aerial vehicle to complete autonomous approaching maneuvers.

[0010] Further, the two-axis holder mechanism is a top-side mechanical layout, the top of the two-axis holder mechanism is provided with a yaw axis motor, the yaw axis motor is directly fixed to the bottom of the unmanned aerial vehicle body through a mounting plate, and the side of the two-axis holder mechanism is provided with a pitch axis motor, the pitch axis motor is rigidly connected to the output end of the yaw axis through a U-shaped support arm.

[0011] Further, the net catching launching device is arranged at the geometric center of the two-axis holder mechanism, and the net catching launching device is fixedly connected to the output end of the pitch axis of the holder through a support; a front end interface of the net catching launching device is used for quickly installing a detachable net elastic load, a folding state of a catching net is encapsulated in the net elastic load, a striker driven by a pre-compressed spring and a trigger structure are arranged inside the net catching launching device, and a rear part of the net catching launching device is a high-pressure gas chamber, and a carbon dioxide cylinder is arranged in the high-pressure gas chamber.

[0012] Further, the binocular vision module is installed directly below the two-axis holder mechanism and the net catching launching device combination, the binocular vision module is fixedly connected to the holder main body structure through a rigid connecting piece, and the viewing angle of the binocular vision module moves synchronously with the two-axis holder mechanism.

[0013] Further, the main control unit carries a target detection and recognition model, whether the target in the binocular image collected by the binocular vision module is an unmanned aerial vehicle is determined based on the target detection and recognition model, and the three-dimensional relative position of the target relative to the center of the unmanned aerial vehicle body is determined.

[0014] After obtaining the accurate three-dimensional position information of the target unmanned aerial vehicle, the main control unit enters an autonomous tracking and striking control stage, guides the host unmanned aerial vehicle to maneuver to a safe distance behind the target unmanned aerial vehicle through a hierarchical control strategy, and executes a net catching strike when a specific condition is met.

[0015] Further, the target detection and recognition model carried by the main control unit is specifically:

[0016] Based on the open-source unmanned aerial vehicle detection dataset and manually annotated self-collected data, the manually annotated annotation information includes the boundary box and class label of the target;

[0017] The open-source data and self-collected data are combined to construct a special dataset;

[0018] YOLOv5 target detection algorithm is used as the basic network of the model, and the special dataset is used for transfer learning and end-to-end training;

[0019] The trained model is pruned and quantized and optimized, and is converted and mounted on the main control unit;

[0020] The real-time video stream collected by the left camera is input into the deployed target detection and recognition model to identify whether the target is an unmanned aerial vehicle.

[0021] Further, the main control unit determines the three-dimensional relative position of the target relative to the center of the unmanned aerial vehicle body, specifically including:

[0022] The epipolar corrected binocular image is subjected to dense stereo matching to calculate the disparity value of each pixel;

[0023] For the detected target region, a matching cost calculation method based on Census transformation or mutual information is used, combined with a multi-path optimization strategy to generate a disparity map ;

[0024] In the target boundary box, the representative disparity of the target as a whole is extracted by weighted average value statistics ;

[0025] After obtaining the representative disparity of the target , the straight-line distance depth value of the target to the optical center of the left camera is calculated according to the triangulation geometric model of the binocular camera , the depth value is calculated by the following formula:

[0026]

[0027] wherein, the depth value represents the axis coordinate of the target in the left camera coordinate system, represents the baseline length of the binocular camera, is a constant value, represents the focal length of the camera, represents the calculated representative disparity value;

[0028] After obtaining the depth value and the image coordinate After that, through the camera pinhole model, the target is converted from two-dimensional image coordinates to three-dimensional coordinates in the left camera coordinate system :

[0029]

[0030] wherein, represents the principal point coordinates of the image, represents the focal length of the camera in the and directions;

[0031] The target point is converted from the camera coordinate system to the UAV body coordinate system through the pre-calibrated extrinsic matrix between the camera and the UAV body:

[0032]

[0033] The three-dimensional relative position vector of the target relative to the center of the UAV body is obtained .

[0034] Further, the master control unit enters the autonomous tracking and attack control phase, specifically including:

[0035] Desired relative pose generation, the desired relative pose is generated according to the tactical requirements, and the desired relative pose is defined as:

[0036]

[0037] wherein, represents that the desired relative distance is 10 meters, and the negative sign indicates that the own machine is located behind the target; represents that the desired lateral deviation is zero; represents that the desired height deviation is zero;

[0038] Relative pose error calculation, the system calculates the relative pose error in the body coordinate system in real time:

[0039]

[0040] wherein, is the actual relative pose calculated.

[0041] Further, after obtaining the relative pose error, a speed instruction is generated according to the relative pose error, and the speed instruction is represented as:

[0042]

[0043] wherein, represents the speed instruction in the body coordinate system, proportional, integral, derivative gain matrix.

[0044] convert the speed command into a UAV attitude command, which is expressed as:

[0045]

[0046]

[0047]

[0048] wherein, represent the desired roll, pitch, yaw angles, represent the estimated heading angle of the target UAV, represent the control gain.

[0049] Further, the execution of the net capture attack when certain conditions are met, specifically includes:

[0050] Continuous monitoring of certain conditions to determine the best attack opportunity, the certain conditions include:

[0051]

[0052] wherein, represent the position error threshold, represent the relative speed threshold, represent the pose stability duration, represent the stability time threshold;

[0053] When all certain conditions are met at the same time, the system determines that it has entered the optimal attack position, immediately generates a trigger signal, and the main control unit sends a high-level pulse signal to the TT motor, triggering the firing of the carbon dioxide cylinder, and executing the net capture attack.

[0054] Compared with the prior art, the beneficial effects of the present application are:

[0055] 1. The two-axis gimbal mechanism of the present application enables the net capture launching device to have the freedom of movement independent of the UAV platform, and in combination with the high-precision target positioning based on binocular vision and the servo tracking algorithm, effectively expands the effective attack range of the system, and significantly improves the capture accuracy and adaptability of dynamic targets.

[0056] 2. The present application realizes the full-process automation from target recognition, tracking and approaching to attack decision by fusing deep learning target detection, binocular vision pose solving and hierarchical control strategy, greatly reduces the dependence on manual operation, and improves the response speed and intelligence level of the system in complex environments.

[0057] 3.The application forms a working architecture of efficient cooperation of multiple subsystems by the master control unit uniformly scheduling vision, gimbal, flight control and striking unit, and significantly improves the stability and task completion reliability of the system in a real disturbance environment by combining a hierarchical control algorithm combining inner and outer loops and a multiple fault-tolerant mechanism. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 It is a schematic diagram of the net capture load of the unmanned aerial vehicle of the application;

[0059] Figure 2 It is a schematic diagram of the system target detection and identification model construction process of the application;

[0060] Figure 3 It is a schematic diagram of the target three-dimensional relative position calculation process of the application;

[0061] Figure 4 It is a schematic diagram of the unmanned aerial vehicle autonomous tracking and pose control process of the application;

[0062] Figure 5 It is a schematic diagram of the net capture striking trigger decision process of the application.

[0063] In the figure: 1, yaw axis motor; 2, pitch axis motor; 3, mounting plate; 4, U-shaped support arm; 5, yaw axis; 6, net launching device; 7, bracket; 8, net bomb load; 9, binocular vision module; 10, master control unit. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0065] Please refer to Figures 1-5 The application provides the following technical solutions:

[0066] An unmanned aerial vehicle autonomous approach and net capture control system based on visual servoing, comprising a two-axis gimbal mechanism, a net launching device, a binocular vision module and a master control unit;

[0067] The two-axis gimbal mechanism is configured to drive the entire gimbal system to rotate 360 degrees around the vertical axis, and is also used to drive the net launching device to pitch around the horizontal axis;

[0068] The net shooting device is configured as an execution terminal of the net shooting control system, and after the master control unit issues a shooting instruction, the net shooting device shoots the net and the internal catching net at a high speed towards a target to complete a capturing task.

[0069] The binocular vision module is configured as two gray level cameras with consistent parameters and calibrated, and the two gray level cameras are installed in a horizontal parallel manner, a line between optical centers of the gray level cameras is parallel to a yaw axis of the holder, and the binocular vision module collects binocular images of a target area in real time through the gray level cameras.

[0070] The master control unit is configured to send control instructions to two holder motors of the two-axis holder mechanism through a motor driving interface to realize closed-loop servo control of the holder, control the net shooting device to execute a net shooting triggering action through a general input and output interface, and continuously interact with a flight controller of the unmanned aerial vehicle through a serial communication interface to send a flight path instruction generated based on visual sensing to guide the unmanned aerial vehicle to complete autonomous approaching maneuvers.

[0071] In the above embodiment, by integrating the two-axis holder mechanism, the net shooting device, the binocular vision module and the master control unit, a complete and highly efficient unmanned aerial vehicle autonomous approaching and net shooting control system is constructed, the master control unit as a core scheduling center establishes stable data interaction and control links with other components through diversified interfaces to realize precise control of the whole system, the two-axis holder mechanism has a unique top- side mechanical layout, which gives the net shooting device flexible movement ability independent of the unmanned aerial vehicle platform, the yaw axis rotating by 360 degrees in combination with the horizontal axis that can be tilted greatly expands the working range of the net shooting device, enabling it to quickly respond to targets in different directions and angles, the binocular vision module provides high-quality image data support for target identification and positioning, and the net shooting device as an execution terminal breaks the limitations of traditional unmanned aerial vehicle capturing systems in which different functions are fragmented and poor in collaboration, greatly improves the integration level of the system, and can adapt to complex working environments to provide reliable hardware support and collaborative foundation for subsequent precise target capturing.

[0072] The two-axis holder mechanism has a top-side mechanical layout, the top of the two-axis holder mechanism is provided with a yaw axis motor, the yaw axis motor is directly fixed to the bottom of the unmanned aerial vehicle body through a mounting plate, and the side of the two-axis holder mechanism is provided with a tilt axis motor, the tilt axis motor is rigidly connected to the output end of the yaw axis through a U-shaped support arm.

[0073] The two-axis holder mechanism enables the net shooting device to have movement freedom independent of the unmanned aerial vehicle platform, in combination with high-precision target positioning based on binocular vision and a servo tracking algorithm, effectively expands the effective striking range of the system, and significantly improves the capturing precision and adaptability to dynamic targets.

[0074] The net shooting device is arranged at the geometric center of the two-axis holder mechanism, and is fixedly connected with the pitch axis output end of the holder through a support; the front end interface of the net shooting device is used for quickly installing a detachable net load, the net load is encapsulated with a folded net, the net shooting device is internally provided with a striker driven by a pre-compressed spring and a trigger structure, and the rear part of the net shooting device is a high-pressure gas chamber, and the high-pressure gas chamber accommodates a carbon dioxide cylinder;

[0075] The working process of the net shooting device is as follows: when the main control unit issues a shooting instruction, the TT motor is controlled to act, the trigger is released through a transmission mechanism, the pre-compressed spring drives the striker to impact and pierce the sealing film of the carbon dioxide cylinder at a high speed. The high-pressure carbon dioxide gas is released instantaneously, generating a strong thrust, and the net load and the net inside it are shot at a high speed towards the target, completing the capture task.

[0076] The binocular vision module is installed below the two-axis holder mechanism and the net shooting device combination, and is fixedly connected with the holder main body structure through a rigid connecting piece, and the viewing angle of the binocular vision module moves synchronously with the two-axis holder mechanism;

[0077] The main control unit is a high-performance embedded processing mainboard, which is physically installed below the entire holder and the shooting mechanism, and is directly fixed with the unmanned aerial vehicle body.

[0078] In the above embodiment, the direct fixation of the yaw axis motor and the unmanned aerial vehicle body and the rigid connection of the pitch axis motor through the U-shaped support arm ensure the stability and reliability of the mechanism movement, reduce the error in the movement process, the net shooting device is arranged at the geometric center of the two-axis holder mechanism and is stably connected through a support, ensuring the accuracy during shooting, the combination of the internal mechanical structure and the high-pressure gas chamber provides power support for the high-speed shooting of the net load, the rigid connection of the binocular vision module and the holder main body enables the viewing angle to move synchronously with the holder, ensuring the continuity and accuracy of image acquisition, the installation position of the main control unit not only ensures the stability of itself, but also facilitates data interaction and signal transmission with the unmanned aerial vehicle body and other components, effectively reducing the mechanical loss and signal interference of the system during operation, improving the operation efficiency and stability of the system, and significantly improving the aiming accuracy of the net shooting device, the image acquisition quality of the binocular vision module and the instruction transmission efficiency of the main control unit.

[0079] The main control unit is loaded with a target detection and recognition model, which determines whether the target in the binocular image collected by the binocular vision module is an unmanned aerial vehicle based on the target detection and recognition model, and determines the three-dimensional relative position of the target relative to the center of the unmanned aerial vehicle body.

[0080] After obtaining the accurate three-dimensional position information of the target UAV, the master control unit enters the autonomous tracking and attack control phase, guides the host UAV to maneuver to a safe distance behind the target UAV through a hierarchical control strategy, and executes the net capture attack when certain conditions are met.

[0081] The system realizes the full-process autonomy from target recognition, tracking and approaching to attack decision through the fusion of deep learning target detection, binocular vision pose solving and hierarchical control strategy, greatly reduces the dependence on manual operation, and improves the response speed and intelligence level of the system in complex environments.

[0082] The target detection and recognition model carried by the master control unit is specifically:

[0083] Based on the open-source UAV detection dataset and manually annotated self-collected data, the self-collected data includes flight videos of different types of UAVs in different real backgrounds, and the real backgrounds include but are not limited to urban sky, forest land and above buildings. The manually annotated annotation information includes the boundary box and class label of the target.

[0084] Merge open source data and self-collected data to build a special dataset;

[0085] YOLOv5 target detection algorithm is used as the basic network of the model, and the special dataset is used for transfer learning and end-to-end training;

[0086] The trained model is pruned and quantized for optimization, and is converted and carried on the master control unit;

[0087] The real-time video stream collected by the left eye camera is input into the deployed target detection and recognition model to identify whether the target is a UAV.

[0088] In the above embodiment, the target detection and recognition model carried by the master control unit is constructed by integrating public open source datasets and diversified self-collected datasets, and a special dataset covering feature information of different types of unmanned aerial vehicles in various real backgrounds is constructed, which greatly enriches the diversity and comprehensiveness of the data, provides sufficient and high-quality data basis for model training, and effectively improves the target recognition ability of the model in different scenes; In the model construction process, the YOLOv5 target detection algorithm is used as the basic network, combined with the transfer learning and end-to-end training method, the advantages of the algorithm in the field of target detection are fully utilized, the training period of the model is shortened, and the detection accuracy and generalization ability of the model are improved; The subsequent pruning and quantization optimization operation reduces the volume of the model and reduces the occupation of the hardware resources of the master control unit, ensures the efficient operation of the model on the embedded platform, improves the response speed of the system; The target detection and recognition model can quickly and accurately identify the unmanned aerial vehicle target in the image collected by the binocular vision module, provide timely and reliable target information for subsequent target positioning, tracking and attack decision, greatly reduce the dependence of the system on manual intervention, enable the system to complete the target recognition task independently in complex environment, and significantly improve the intelligent level and practical value of the system.

[0089] The master control unit determines the three-dimensional relative position of the target relative to the center of the unmanned aerial vehicle body, specifically including:

[0090] The epipolar corrected binocular image is subjected to dense stereo matching to calculate the disparity value of each pixel;

[0091] For the detected target region, a matching cost calculation method based on Census transformation or mutual information is used, combined with a multi-path optimization strategy to generate a disparity map ;

[0092] In the target bounding box, the representative disparity representing the overall distance of the target is extracted by weighted average value statistics ;

[0093] After obtaining the representative disparity of the target , the straight line distance depth value of the target to the optical center of the left camera is calculated according to the triangulation geometric model of the binocular camera , the depth value is calculated by the following formula:

[0094]

[0095] wherein, the depth value represents the axis coordinate of the target in the left camera coordinate system, represents the baseline length of the binocular camera, is a constant value, represents the focal length of the camera, represents the calculated representative disparity value;

[0096] After obtaining the depth value and image coordinates , the target is converted from two-dimensional image coordinates to three-dimensional coordinates in the left camera coordinate system through the camera pinhole model :

[0097]

[0098] wherein, represents the principal point coordinates of the image, represents the focal length of the camera in the and directions, which together constitute the intrinsic matrix of the camera, obtained through pre-calibration;

[0099] Through the extrinsic matrix (rotation matrix and translation vector ) between the pre-calibrated camera and the UAV body, the target point is converted from the camera coordinate system to the UAV body coordinate system:

[0100]

[0101] The three-dimensional relative position vector of the target relative to the center of the UAV body is obtained .

[0102] In the above embodiment, the representative disparity is extracted by weighted average statistics, and the depth value is calculated based on the binocular camera triangulation geometric model, which ensures the accuracy of the depth value and provides accurate distance parameters for three-dimensional calculation of the target position. In the coordinate conversion process, the camera pinhole model and the pre-calibrated intrinsic and extrinsic matrices are used to realize accurate conversion from two-dimensional image coordinates to the camera coordinate system, and then to the UAV body coordinate system. The final target three-dimensional relative position vector can accurately reflect the position relationship of the target relative to the machine, solving the problems of insufficient accuracy and poor stability of traditional positioning methods in complex environments. It provides reliable position data for the subsequent main control unit to develop reasonable tracking strategy and attack decision, enabling the UAV to accurately grasp the position change of the target and timely adjust its own motion state, significantly improving the tracking accuracy and adaptability of the system to dynamic targets.

[0103] The main control unit enters the autonomous tracking and attack control phase, which specifically includes:

[0104] Desired relative pose generation: generate the desired relative pose according to the tactical requirements. In order to achieve effective attack, the machine needs to occupy a favorable position behind the target. The desired relative pose is defined as:

[0105]

[0106] where, represents the desired relative distance of 10 meters, and the negative sign indicates that the UAV is behind the target; represents the desired lateral deviation of zero, i.e., maintaining the same axis as the target; represents the desired height deviation of zero, i.e., maintaining the same height as the target;

[0107] Relative pose error calculation, the system real-time computer body coordinate system under the relative pose error:

[0108]

[0109] where, is the actual relative pose calculated.

[0110] After obtaining the relative pose error, the speed command is generated according to the relative pose error. The controller adopts the classic PID control structure, which can handle the current value, cumulative value and trend of position error at the same time, ensuring that the system has good dynamic performance and steady-state accuracy. The speed command is represented as:

[0111]

[0112] where, represents the speed command in the body coordinate system, represents the proportional, integral, and derivative gain matrix.

[0113] Convert the speed command to the UAV attitude command. Considering the dynamics of the UAV, ensure that the generated control command can achieve the desired motion and will not exceed the physical limits of the UAV. The attitude command is represented as:

[0114]

[0115]

[0116]

[0117] where, represents the desired roll, pitch, and yaw angles, represents the estimated heading angle of the target UAV, represents the control gain.

[0118] Continuous monitoring of specific conditions to determine the best attack opportunity, specific conditions include:

[0119]

[0120] wherein, represents a position error threshold, represents a relative speed threshold, represents a pose stability duration, represents a stability time threshold;

[0121] When all the specific conditions are met at the same time, the system determines that the optimal striking position has been entered, generates a trigger signal immediately, and the main control unit sends a high-level pulse signal to the TT motor to trigger the firing of the carbon dioxide cylinder and execute the net capture strike.

[0122] Through the unified scheduling of the main control unit, the vision, the gimbal, the flight control and the striking unit form a high-efficiency collaborative working architecture of multiple subsystems, combined with the hierarchical control algorithm of the inner and outer loops and the multiple fault-tolerant mechanism, the stability and task completion reliability of the system in the real disturbance environment are significantly improved.

[0123] In the above embodiment, the expected relative pose generated according to the tactical demand clearly defines the optimal tracking position of the host, providing a clear target direction for the tracking process, and the real-time calculation of the relative pose error can timely reflect the deviation of the host from the expected pose, and the speed instruction generated based on the PID control structure fully considers the current value, cumulative value and change trend of the position error, ensuring that the system has good dynamic performance and steady-state accuracy, so that the host can quickly and smoothly adjust the motion state to approach the expected pose. When converting the speed instruction into an attitude instruction, the dynamics of the unmanned aerial vehicle are fully combined to avoid the situation that the instruction exceeds the physical limit of the unmanned aerial vehicle, ensuring the safety and stability of the unmanned aerial vehicle motion. In the striking opportunity judgment link, through the monitoring of multiple key parameters such as position error, relative speed and pose stability duration, strict striking conditions are formulated to ensure that the system can only execute the net capture strike when it enters the optimal striking position, greatly improving the success rate of the strike and avoiding capture failure due to improper timing. The system can autonomously complete the entire process from tracking to striking, reducing the difficulty and intensity of manual operation, and improving the response speed and task completion efficiency of the system in complex environments.

[0124] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any skilled person in the art can make equivalent replacement or change according to the technical solution and inventive concept of the present application within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A visual servoing-based autonomous approach and net-capture control system for unmanned aerial vehicles (UAVs), characterized in that, Includes a two-axis gimbal mechanism, a net capture and launch device, a binocular vision module, and a main control unit; The two-axis gimbal mechanism is configured to drive the entire gimbal system to rotate 360 ​​degrees around the vertical axis, and at the same time drive the net capture and launch device to pitch around the horizontal axis. The net-capturing launching device is configured as the execution terminal of the net-capturing control system. When the main control unit issues a launch command, it launches the net projectile and the net inside at high speed toward the target to complete the capture mission. The binocular vision module is configured with two grayscale cameras with identical parameters and calibrated. The two grayscale cameras are installed horizontally side by side, and the line connecting the optical centers of the grayscale cameras is parallel to the yaw axis of the gimbal. The grayscale cameras are used to acquire binocular images of the target area in real time. The main control unit is configured to send control commands to the two gimbal motors of the two-axis gimbal mechanism through the motor drive interface to realize closed-loop servo control of the gimbal, control the net capture and launch device to execute net capture triggering actions through the general input / output interface, continuously interact with the UAV's own flight controller through the serial communication interface, and send flight path commands based on visual perception to guide the UAV to complete autonomous approach maneuvers.

2. The visual servoing-based UAV autonomous approach and net-capture control system as described in claim 1, characterized in that, The two-axis gimbal mechanism has a top-side mechanical layout. A yaw axis motor is installed at the top of the two-axis gimbal mechanism. The yaw axis motor is directly fixed to the bottom of the UAV fuselage through a mounting plate. A pitch axis motor is installed on the side of the two-axis gimbal mechanism. The pitch axis motor is rigidly connected to the output end of the yaw axis through a U-shaped support arm.

3. The visual servoing-based UAV autonomous approach and net-capture control system as described in claim 1, characterized in that, The net-capturing launcher is located at the geometric center of the two-axis gimbal mechanism. The net-capturing launcher is fixedly connected to the pitch axis output end of the gimbal via a bracket. The front interface of the net-capturing launcher is used for quick installation of a detachable net projectile load. The net projectile load contains a folded net. The net-capturing launcher is equipped with a firing pin driven by a pre-compression spring and a trigger structure. The rear of the net-capturing launcher is a high-pressure gas chamber, which contains a carbon dioxide gas cylinder.

4. The visual servoing-based UAV autonomous approach and net-capture control system as described in claim 1, characterized in that, The binocular vision module is installed directly below the combination of the two-axis gimbal mechanism and the net capture and launch device. The binocular vision module is fixedly connected to the main structure of the gimbal through a rigid connector, and the viewing angle of the binocular vision module moves synchronously with the two-axis gimbal mechanism.

5. A visual servoing-based UAV autonomous approach and net-capture control system as described in claim 1, characterized in that, The main control unit is equipped with a target detection and recognition model. Based on the target detection and recognition model, it identifies whether the target in the binocular image collected by the binocular vision module is a drone and determines the three-dimensional relative position of the target with respect to the center of the drone body. After obtaining the precise three-dimensional position information of the target UAV, the main control unit enters the autonomous tracking and strike control phase. It guides the UAV to maneuver to a safe distance behind the target UAV through a hierarchical control strategy, and performs net capture and strike when specific conditions are met.

6. The visual servoing-based UAV autonomous approach and net-capture control system as described in claim 5, characterized in that, The target detection and recognition model carried by the main control unit is specifically as follows: Manual annotation is performed based on publicly available open-source drone detection datasets and self-collected data. The self-collected data includes flight videos of different drone models under different real-world backgrounds. The annotation information for manual annotation includes the bounding box and category label of the target. Merge open-source data with self-collected data to build a dedicated dataset; The YOLOv5 object detection algorithm is used as the base network of the model, and transfer learning and end-to-end training are performed using the dedicated dataset. After training, the model undergoes pruning and quantization optimization before being converted and integrated into the main control unit. The real-time video stream captured by the left eye camera is input into the deployed target detection and recognition model to identify whether the target is a drone.

7. A visual servoing-based UAV autonomous approach and net-capture control system as described in claim 5, characterized in that, The main control unit determines the three-dimensional relative position of the target with respect to the center of the UAV body, specifically including: Dense stereo matching is performed on the epipolar-corrected binocular images to calculate the disparity value of each pixel; For the detected target region, a disparity map is generated by using a matching cost calculation method based on Census transform or mutual information, combined with a multi-path optimization strategy. ; Within the target bounding box, a representative disparity representing the overall distance to the target is extracted by statistically calculating a weighted average. ; In obtaining the representative parallax of the target Then, based on the triangulation geometry model of the binocular camera, the straight-line distance depth value from the target to the optical center of the left camera was calculated. The depth value Calculated using the following formula: ； Among them, depth value Indicates the target in the left camera coordinate system Axis coordinates Indicates the baseline length of the stereo camera. For constant values, Indicates the camera's focal length. This represents the calculated representative disparity value; Obtaining depth value and image coordinates Then, using the camera pinhole model, the target is transformed from two-dimensional image coordinates to three-dimensional coordinates in the left camera coordinate system. : ； in, Represents the coordinates of the principal point of the image. Indicates the camera is in and Focal length in direction; By using the pre-calibrated extrinsic parameter matrix between the camera and the drone body, the target point is transformed from the camera coordinate system to the drone body coordinate system: ； Obtain the three-dimensional relative position vector of the target with respect to the center of the UAV body. .

8. The visual servoing-based UAV autonomous approach and net-capture control system as described in claim 5, characterized in that, The main control unit enters the autonomous tracking and strike control phase, specifically including: Desired relative pose generation: Based on tactical requirements, the desired relative pose is generated. The desired relative pose is defined as follows: ； in, This indicates a desired relative distance of 10 meters, and the negative sign indicates that the machine is located behind the target. This indicates that the expected lateral deviation is zero; This indicates that the expected height deviation is zero; Relative pose error calculation: Real-time relative pose error in the computer's body coordinate system. ； in, This represents the calculated actual relative pose.

9. A visual servoing-based UAV autonomous approach and net-capture control system as described in claim 6, characterized in that, After obtaining the relative pose error, a velocity command is generated based on the relative pose error. The velocity command is expressed as follows: ； in, This indicates the velocity command in the body coordinate system. Represents the proportional, integral, and differential gain matrices; The speed command is converted into a UAV attitude command, which is represented as follows: ； ； ； in, Indicates the desired roll, pitch, and yaw angles. This represents the estimated heading angle of the target UAV. This indicates the control gain.

10. A visual servoing-based UAV autonomous approach and net-capture control system as described in claim 5, characterized in that, The execution of the net capture and strike when specific conditions are met specifically includes: Continuously monitor specific conditions to determine the optimal timing for strikes; these specific conditions include: ； in, Indicates the position error threshold. Indicates the relative velocity threshold. Indicates the duration of pose stability. Indicates the stable time threshold; When all specific conditions are met simultaneously, the system determines that it has entered the optimal strike position, immediately generates a trigger signal, and the main control unit sends a high-level pulse signal to the TT motor to trigger the firing pin to fire the carbon dioxide cylinder and execute the net capture strike.

Images