A multi-joint self-balancing adjustment platform based on an inspection unmanned vehicle and its usage method

Abstract

The present invention relates to an adaptive landing platform and a usage method based on a six-wheel independent drive inspection unmanned vehicle, including devices such as a six-wheel independent drive unmanned vehicle, an adjustable landing platform, three double-joint support arms, a multi-sensor fusion system, etc., and also includes a method for multi-source data fusion analysis and adaptive adjustment during the landing process of an unmanned aerial vehicle. The landing platform is equipped with three double-joint support arms, and the horizontal maintenance of the platform is achieved through the attitude adjustment of the support arms. The system fuses the six-axis sensor of the platform, the remote sensing terrain data of the unmanned aerial vehicle, the camera, and the attitude feedback data of the unmanned aerial vehicle, and uses deep learning algorithms for analysis and processing, and realizes the real-time adaptive adjustment of the landing platform through the control system. It completes providing a stable and safe landing place for the unmanned aerial vehicle in complex environments such as grasslands, and realizes multi-level compliant reception, providing strong technical support for the application of the unmanned aerial vehicle in complex terrains.

Description

Technical Field

[0001] The present invention relates to the technical fields of multi-joint self-balancing regulation, multi-sensor data fusion, deep learning neural network algorithms, and precise drone landing technology, and particularly relates to a multi-joint self-balancing regulation platform based on an inspection unmanned vehicle and a using method thereof. Background Art

[0002] With the rapid development of drone technology, its applications in fields such as industrial inspection, emergency rescue, and scientific exploration are becoming increasingly widespread. Especially in complex terrains and harsh environments, drones have become a highly potential tool due to their flexibility and efficiency. However, traditional drone landing systems often struggle to achieve precise and safe landings when faced with uneven terrains and changing environments, which severely limits the application effect of drones in complex environments. At the same time, the combination of drone technology and artificial intelligence technology has become a key force driving the progress of multiple industries. The high-definition cameras, laser rangefinders, and high-performance computing platforms carried by drones enable real-time data acquisition and analysis, providing unprecedented possibilities for precise landing and environmental perception.

[0003] During the traditional drone landing process, it usually relies on preset landing points and simple visual navigation systems. However, this method often suffers from positioning errors or unstable postures when faced with complex terrains. Uneven ground may cause the drone to overturn or be damaged, while adverse weather conditions may affect the accuracy of the visual system. These problems not only increase the usage risks of drones but also limit their application scope in special environments.

[0004] In addition, most existing drone landing platforms adopt fixed structures and lack the ability to adapt to complex terrains. Although some studies have attempted to use simple adjustable mechanisms, it is still difficult to achieve real-time and precise attitude adjustment when faced with large-angle inclinations or rapidly changing terrains. At the same time, how to effectively fuse multi-source sensor data and use deep learning neural network technology for efficient and accurate analysis to improve the landing accuracy and stability of drones in complex environments remains an urgent problem to be solved in the current technological development.

[0005] Therefore, developing a landing platform that can adapt to complex terrains, has real-time self-balancing ability, and can cooperate intelligently with drones is of great significance for improving the working efficiency and safety of drones in various complex environments. This not only requires solving the challenges in hardware design but also requires innovation in software algorithms, data fusion, and real-time control to achieve precise and safe landings of drones in various complex environments. Summary of the Invention

[0006] The purpose of the present invention is to provide a multi-joint self-balancing adjustment platform based on an inspection unmanned vehicle and its usage method to solve various problems faced in the above operation scenarios.

[0007] To achieve the above object, the present application provides the following technical solutions:

[0008] A multi-joint self-balancing adjustment platform based on an inspection unmanned vehicle, characterized by comprising: an unmanned vehicle chassis, an adjustable landing platform, a double-joint support arm, and a multi-sensor system. An adjustable landing platform is installed above the unmanned vehicle chassis. The adjustable landing platform is inlaid and connected with the unmanned vehicle chassis through a fixed base plate. There are three limit sliding grooves above the fixed base plate, and three double-joint support arms arranged in an equilateral triangle are respectively installed above the sliding grooves. The upper ends of the double-joint support arms are adsorbed and fixed to a transparent platform through powerful suction cups. The unmanned vehicle chassis is equipped with a lithium battery pack, a high-performance industrial computer, and a main controller; an inertial measurement unit, a high-definition camera, and a laser rangefinder are installed on the adjustable landing platform.

[0009] The lithium battery pack and the high-performance industrial computer are fixed above the fixed base plate on the side and are connected and powered through a power cord. The high-performance industrial computer is also connected to the main controller at the edge of the large fixed base plate through a high-speed USB cable for power supply and data transmission.

[0010] The top of the unmanned vehicle chassis is horizontally and vertically symmetrically connected and fixed by 2 lower side long bars and 4 support long bars. Two front frame connecting arms are respectively connected and fixed to 2 side long bars. The front support arm and the middle support arm are both connected and fixed to the front-middle arm connecting bar. Two pillow blocks are connected and fixed above the front-middle arm connecting bar. A steel shaft passes through the two pillow blocks and steel sheet fasteners, and the four steel sheet fasteners are respectively fixed to the two front frame connecting arms. Four vertical short bars, four side long bars, and four support long bars are vertically and crosswise connected and fixed to form the overall frame of the back of the unmanned vehicle. The connection and fixation of the above structures are all achieved by tightening and locking through trapezoidal nuts and the side groove structures of profiles.

[0011] The upper part of the rear support arm of the unmanned vehicle is connected to the side long bar, and the lower part is fixedly connected through four screw holes on the motor support. The motor support is fixedly connected to the M3508 motor through four screw holes on the other side of the motor support and is connected to the obstacle-crossing tire through a bearing sleeve.

[0012] The double-joint support arm consists of a high-precision servo motor, an active joint, a driven joint, a universal ball shaft, a force feedback sensor, and a powerful suction cup. The high-precision servo motor is connected to the main controller and the lithium battery pack respectively through internal data lines to achieve power supply and control connection. The high-precision servo motor controls the rotation angles of the active joint and the driven joint. There is a high-precision servo connected and fixed at the connection of the active joint and the driven joint. The force feedback sensor monitors the supporting force in real time and sends the feedback data to the main controller through the Bluetooth module inside it. Both the high-precision servo motor and the high-precision servo have a self-locking function. In the case of no signal transmission, there is a large torque in both the high-precision servo motor and the high-precision servo, and they cannot be rotated randomly. They can only be rotationally controlled through the pulse signal sent by the main controller. The universal ball shaft and the powerful suction cup are fixed by the inlay of a rubber ball and a groove, and there is a certain pre-tightening force at the connection. This pre-tightening force is sufficient to make the drone land smoothly without shaking. In the default state of the movement of the double-joint support arm, the universal ball shaft follows the movement of the double-joint support arm with the vertical upward direction. The rotation of the universal ball shaft is passively triggered by the pulling and pushing forces generated by the interaction of the double-joint support arms with the transparent landing platform when they move coordinately.

[0013] The fixed bottom plate of the adjustable landing platform is a support plate modified from a PCB circuit board. The pins of three high-precision servo motors, three high-definition cameras, and three force feedback sensors, electronic components, are all fixed on the fixed bottom plate by welding. The main controller, also welded on the fixed bottom plate, realizes the power supply and data transmission to the above-mentioned component devices.

[0014] The six-axis attitude sensor is installed at the center position below the adjustable landing platform to monitor the spatial attitude of the platform in real time. The high-definition cameras are installed in the spatial gaps of the three double-joint support arms of the platform, fixed in an equilateral triangle position, and are used to capture visual information during the landing process of the drone. The laser rangefinder is installed at the center position of the platform and is used to accurately measure the distance between the drone and the platform. The chute provides multiple preset installation positions for the double-joint support arm to facilitate dealing with special application requirements. When the unmanned vehicle moves, the relative position between the limit chute and the double-joint support arm will not change.

[0015] The high-performance industrial computer incorporates a multi-source data fusion algorithm, integrating deep learning and adaptive control algorithms, and processes the data from the six-axis attitude sensor, high-definition cameras, laser rangefinder, and the feedback from the drone in real time to calculate the optimal platform attitude adjustment strategy.

[0016] During the landing process of the drone, the control system first compares the surrounding terrain information with the pre-loaded remote sensing data of the drone to determine the terrain at the current position. The high-performance industrial control computer combines the body attitude data read and recognized by the six-axis attitude sensor, performs attitude calculation through dynamic weighted fusion, outputs command information to the main controller, and the main controller converts it into PWM pulse information and transmits it to the high-precision servo motor and high-precision steering gear for preliminary attitude adjustment. Subsequently, the high-definition camera and laser rangefinder are used to track the landing trajectory of the drone in real time and fine-tune the platform attitude. When the drone touches the platform and gradually reduces the rotor speed, the unmanned vehicle control system calculates the spatial motion trajectory of the double-joint support arm based on the force feedback sensor data and the horizontal offset azimuth information of the drone landing, and performs compliant reception of the drone to minimize the landing impact of the drone to the greatest extent.

[0017] The high-performance industrial control computer is configured with a multi-source data real-time fusion analysis algorithm, which combines a convolutional neural network and a recurrent neural network to form an end-to-end adaptive control model, processes the data streams from various sensors in real time, predicts the motion trajectory of the drone, and optimizes the attitude adjustment strategy of the landing platform to ensure the accurate and safe landing of the drone under complex terrain conditions.

[0018] The end-to-end adaptive control model is implemented using the PyTorch deep learning framework. When extracting features, a multi-modal fusion network is used to process different sensor data: the IMU data of the six-axis attitude sensor is processed through a D convolutional neural network; the high-definition camera images are extracted with features through the ResNet50 backbone network; the laser rangefinder data is extracted with point cloud features through PointNet++; the telemetry data feedback from the drone is processed through the LSTM network. All features are fused through the attention mechanism (SelfAttention) to form a unified feature representation.

[0019] A method for using a multi-joint self-balancing adjustment platform based on an inspection unmanned vehicle includes the following steps:

[0020] S1, Use the OpenCV library to preprocess the images taken by the three high-definition cameras, including denoising, histogram equalization, and normalization; use the PCL library to filter and downsample the laser rangefinder point cloud data; use the Pandas library to perform time synchronization and interpolation processing on the IMU and telemetry data;

[0021] S2, Object detection and tracking: Use the YOLOv5 algorithm to perform real-time detection of the drone landing using the video streams of the three high-definition cameras, and use the DeepSORT algorithm for multi-object tracking. Model optimization is performed through TensorRT, and a real-time processing speed of 30FPS is achieved on the NVIDIA Jetson AGX Xavier;

[0022] S3, Trajectory Prediction: Integrate the image parsing data of YOLOv5 and the depth distance data of the laser rangefinder, and input them into the sequence-to-sequence model (Seq2Seq with Attention) of the Transformer architecture to predict the trajectory of the drone in the next 5 seconds. The input of the model is the position and attitude data of the past 30 frames, and the output is the predicted three-dimensional coordinate sequence;

[0023] S4, Platform Attitude Optimization: Construct a reinforcement learning environment based on PyTorch, and use the PPO (Proximal Policy Optimization) algorithm to train the attitude adjustment strategy. The reward function considers factors such as the transparency of the landing platform, the energy consumption of high-precision servo motors and high-precision steering gears, and the adjustment speed as considerations of the reward function;

[0024] S5, Adaptive Fusion Strategy: Design a dynamic weight allocation mechanism based on Softmax, and adaptively adjust the weights of object detection, trajectory prediction, and reinforcement learning strategies according to the detection confidence, trajectory prediction error, and current system state;

[0025] S6, Safety Constraints: Use the model predictive control method in control theory, use the MATLAB Simscape Multibody tool to establish a high-precision dynamic model of the platform, adopt the extended Kalman filter EKF to fuse multi-source sensor data, and estimate the complete state of the transparent landing platform in real time, including position, attitude, speed, and acceleration. And use the ACADO Toolkit to achieve real-time NMPC solution, including the weighted sum of the horizontal error of the transparent landing platform, energy consumption, and motion smoothness index, and set the motion constraint conditions of high-precision servo motors and high-precision steering gears to ensure that the generated control instructions are within the capabilities of high-precision servo motors and high-precision steering gears, making the movement of the transparent landing platform safe and reliable;

[0026] S7, Real-time Optimization: Convert the PyTorch model into the TensorRT format and deploy it on the Jetson AGX Xavier main control platform; adopt the half-precision (F16P) quantization technology to improve the inference speed while ensuring accuracy, use CUDA programming to achieve GPU acceleration of key point cloud processing and matrix operations, adopt a multi-threaded parallel processing strategy to parallelize data preprocessing, object detection, trajectory prediction, and control instruction generation tasks, and use ROS2 (Robot Operating System 2) as the software framework to achieve efficient communication and data distribution synchronization between the high-performance industrial computer of the unmanned vehicle and the main control module of the drone.

[0027] When the drone enters the final landing stage, the control system switches to fine control mode. The high-definition camera and laser rangefinder continue to provide high-frequency position feedback, based on which the platform posture is continuously fine-tuned. The force feedback sensor monitors the contact force in real time, and the control system accurately adjusts the damping coefficient of the double-joint support arm to achieve smooth acceptance.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1. High adaptability: The multi-joint self-balancing adjustment mechanism can adapt to various complex terrains, significantly improving the landing success rate of the drone in different environments. The mechanism can quickly adjust the height and angle of the double-joint support arm according to the real-time perceived terrain conditions, providing a stable landing platform for the drone, even in areas with large slopes or irregular terrain.

[0030] 2. Precise control: Through multi-sensor data fusion and deep learning algorithms, precise control of the drone landing process is achieved, greatly improving the safety and stability of landing. The control system can calculate the optimal landing path and posture in real time, and ensure that the drone can land accurately on the platform through visual guidance and wireless communication.

[0031] 3. High degree of intelligence: Using advanced artificial intelligence technology, the system can learn and optimize autonomously, and continuously improve its performance in complex environments. By analyzing historical data through machine learning algorithms, the control system can predict potential landing risks and formulate more reasonable landing strategies.

[0032] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0034] Figure 1 It is a schematic diagram of the structure of an unmanned inspection vehicle and a multi-joint self-balancing adjustment platform in one embodiment of the present invention;

[0035] Figure 2 A schematic diagram of the structure of a control system and related components in one embodiment of the present invention;

[0036] Figure 3 This is a schematic structural diagram of a multi-joint self-balancing adjustment platform in one embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of the structure of the unmanned vehicle chassis in one embodiment of the present invention;

[0038] Figure 5 Schematic diagram of the system operation technical process in an embodiment of the present invention;

[0039] Meanings of each label in the attached drawings:

[0040] 1 - Unmanned vehicle chassis. 2 - Adjustable landing platform, 3 - Double-joint support arm, 4 - Multi-sensor system;

[0041] 1-1 Lithium battery pack, 1-2 Industrial control computer, 1-3 Main controller, 1-4 Side long bar, 1-5 Rear support arm, 1-6 Obstacle-crossing tire, 1-7 Front frame connecting arm, 1-8 Bearing with seat, 1-9 Steel shaft, 1-10 Front-middle arm connecting bar, 1-11 Steel sheet fastener, 1-12 Motor support, 1-13 Bearing sleeve, 1-14 Middle support arm, 1-15 M3508 motor, 1-16 Front support arm, 1-17 Support long bar, 1-18 Vertical short bar.

[0042] 2-1 Fixed bottom plate, 2-2 Limit sliding groove, 2-3 Strong suction cup, 2-4 Transparent landing platform;

[0043] 3-1 High-precision servo motor, 3-2 Active joint, 3-3 Driven joint, 3-4 Universal ball shaft, 3-5 Force feedback sensor, 3-5-1 Bluetooth module, 3-6 High-precision steering gear;

[0044] 4-1 - Six-axis attitude sensor, 4-2 - High-definition camera, 4-3 - Laser rangefinder. Detailed implementation manners

[0045] The present invention will be further described below in conjunction with the attached drawings and embodiments.

[0046] It should be noted that the following detailed descriptions are all exemplary and are intended to provide further explanations of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0047] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0048] Without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0049] The technical solution of the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0050] A multi-joint self-balancing adjustment platform based on an inspection unmanned vehicle, characterized in that it includes: an unmanned vehicle chassis 1, an adjustable landing platform 2, a double-joint support arm 3, a multi-sensor system 4, and, an adjustable landing platform 2 is installed above the unmanned vehicle chassis 1, the adjustable landing platform 2 is inlaid and connected with the unmanned vehicle chassis 1 through a fixed bottom plate 2-1, there are three limit chutes 2-2 above the fixed bottom plate 2-1, and three double-joint support arms 3 arranged in an equilateral triangle are respectively installed above the chutes 2-2, and the upper end of the double-joint support arm 3 is adsorbed and fixed to the transparent platform 2-4 through a strong suction cup 2-3. The unmanned vehicle chassis 1 is equipped with a lithium battery pack 1-1, a high-performance industrial computer 1-2, and a main controller 1-3; a six-axis attitude sensor 4-1, a high-definition camera 4-2, and a laser rangefinder 4-3 are installed on the adjustable landing platform 2.

[0051] The lithium battery pack 1-1 and the high-performance industrial computer 1-2 are fixed above the fixed bottom plate 2-1 on the side, and are powered by a power cord. The high-performance industrial computer 1-2 is also connected to the main controller 1-3 at the edge of the large fixed bottom plate 2-1 through a high-speed USB cable for power supply and data transmission.

[0052] The top of the unmanned vehicle chassis 1 is horizontally and vertically symmetrically connected and fixed by 2 side length bars 1-4 and 4 support long bars 1-17 on the lower side. Two front frame connecting arms 1-7 are respectively connected and fixed to 2 side length bars 1-4. The front support arm 1-16 and the middle support arm 1-14 are both connected and fixed to the front-middle arm connecting bar 1-10. Two pedestal bearings 1-8 are connected and fixed above the front-middle arm connecting bar 1-10. A steel shaft 1-9 passes through the two pedestal bearings 1-8 and the steel sheet fasteners 1-11. The four steel sheet fasteners 1-11 are respectively fixed to the two front frame connecting arms 1-7. The four vertical short bars 1-18, the four side length bars 1-4, and the four support long bars 1-17 are vertically and crosswise connected and fixed to form the overall frame of the back of the unmanned vehicle. The connection and fixation of the above structures are all realized by tightening and locking through the trapezoidal nut and the side groove structure of the profile.

[0053] The upper part of the rear support arm 1-5 of the unmanned vehicle is connected to the side length bar 1-4, and the lower part is fixedly connected through four screw holes on the motor support 1-12. The motor support 1-12 is fixedly connected to the M3508 motor 1-15 through four screw holes on the other side of the motor support 1-12, and is connected to the obstacle-crossing tire 1-6 through a bearing sleeve 1-13.

[0054] The double-joint support arm 3 consists of a high-precision servo motor 3-1, an active joint 3-2, a driven joint 3-3, a universal ball shaft 3-4, a force feedback sensor 3-5, and a powerful suction cup 3-6. Among them, the high-precision servo motor 3-1 is respectively connected to the main controller 1-3 and the lithium battery pack 1-1 through internal data lines to achieve power supply and control connection; the high-precision servo motor 3-1 controls the rotation angles of the active joint 3-2 and the driven joint 3-3. There is a high-precision servo 3-6 at the connection of the active joint 3-2 and the driven joint 3-3 for connection and fixation. The force feedback sensor 3-5 monitors the supporting force in real time and sends the feedback data to the main controller 1-3 through its internal Bluetooth module 3-5-1. Both the high-precision servo motor 3-1 and the high-precision servo 3-6 have a self-locking function. In the absence of signal transmission, there is a large torque in both the high-precision servo motor 3-1 and the high-precision servo 3-6, and they cannot be rotated arbitrarily. They can only be rotationally controlled through the pulse signal sent by the main controller 1-3. The universal ball shaft 3-4 and the powerful suction cup 2-3 are fixed by the inlay of a rubber ball and a groove, and there is a certain pre-tightening force at the connection. This pre-tightening force is sufficient to make the drone land stably without shaking. In the default state of the movement of the double-joint support arm 3, the universal ball shaft 3-4 keeps moving in the vertically upward direction following the double-joint support arm 3. The rotation of the universal ball shaft 3-4 is passively triggered by the pulling and pushing forces generated by the interaction of three double-joint support arms 3 on the transparent landing platform 2-4 when they move coordinately.

[0055] The fixed base plate 2-1 of the adjustable landing platform 2 is a support plate modified from a PCB circuit board. The pins of three high-precision servo motors 3-1, three high-definition cameras 4-2, and three force feedback sensors 3-5, electronic components, are all fixed on the fixed base plate 2-1 by welding. The main controller 1-3, also welded on the fixed base plate 2-1, realizes the power supply and data transmission to the above-mentioned component devices.

[0056] The six-axis attitude sensor 4-1 is installed at the center position below the adjustable landing platform 2 to monitor the spatial attitude of the platform in real time; the high-definition camera 4-2 is installed in the spatial gaps of the three double-joint support arms 3 of the platform and fixed at an equilateral triangle position to capture visual information during the landing process of the drone; the laser rangefinder 4-3 is installed at the center position of the platform to accurately measure the distance between the drone and the platform; the chute 2-2 provides multiple preset installation positions for the double-joint support arm 3 to facilitate dealing with special application requirements. When the unmanned vehicle moves, the relative position between the limit chute 2-2 and the double-joint support arm 3 will not change.

[0057] The high-performance industrial computer 1-2 has a built-in multi-source data fusion algorithm, which integrates deep learning and adaptive control algorithms, and processes data from the six-axis attitude sensor 4-1, the high-definition camera 4-2, the laser rangefinder 4-3 and the drone feedback in real time to calculate the optimal platform attitude adjustment strategy.

[0058] During the landing of the drone, the control system first compares the surrounding terrain information with the pre-loaded drone remote sensing data to determine the current terrain. The performance industrial computer 1-2 reads and identifies the vehicle posture data in combination with the six-axis attitude sensor 4-1, performs attitude solution through dynamic weighted fusion, and outputs command information to the main controller 1-3, which is converted into PWM pulse information and transmitted to the high-precision servo motor 3-1 and high-precision steering gear 3-6 for preliminary attitude adjustment. Subsequently, the high-definition camera 4-2 and the laser rangefinder 4-3 are used to track the landing trajectory of the drone in real time, and the platform attitude is fine-tuned. When the drone contacts the platform and begins to gradually reduce the rotor speed, the unmanned vehicle control system calculates the spatial motion trajectory of the double-joint support arm 3 based on the force feedback sensor 3-5 data and the horizontal offset azimuth information of the drone landing, and performs a smooth acceptance of the drone to minimize the landing impact of the drone.

[0059] The high-performance industrial computer 1-2 is equipped with a real-time fusion analysis algorithm for multi-source data, which integrates convolutional neural networks and recursive neural networks to form an end-to-end adaptive control model. It processes data streams from various sensors in real time, predicts the motion trajectory of the UAV, and optimizes the landing platform attitude adjustment strategy to ensure accurate and safe landing of the UAV under complex terrain conditions.

[0060] The end-to-end adaptive control model is implemented using the PyTorch deep learning framework. When extracting features, a multimodal fusion network is used to process different sensor data: the IMU data of the six-axis attitude sensor is processed through a 1D convolutional neural network; the high-definition camera image is extracted through the ResNet50 backbone network; the laser rangefinder data is extracted through PointNet++ for point cloud features; and the telemetry data fed back by the drone is processed through the LSTM network. All features are fused through the attention mechanism (Self-Attention) to form a unified feature representation.

[0061] A method for using a multi-joint self-balancing adjustment platform based on an inspection unmanned vehicle comprises the following steps:

[0062] S1, use the OpenCV library to pre-process the images taken by the three high-definition cameras 4-2, including denoising, histogram equalization and normalization; use the PCL library to filter and downsample the laser rangefinder point cloud data; use the Pandas library to perform time synchronization and interpolation processing on the IMU and telemetry data;

[0063] S2, Object Detection and Tracking: The YOLOv5 algorithm is adopted to perform real-time detection on the drone landing using the video streams of three high-definition cameras 4-2. The DeepSORT algorithm is used for multi-object tracking. The model is optimized through TensorRT to achieve a real-time processing speed of 30 FPS on NVIDIA Jetson AGX Xavier;

[0064] S3, Trajectory Prediction: The image parsing data of YOLOv5 and the depth distance data of the laser rangefinder 4-3 are fused and input into the sequence-to-sequence model (Seq2Seq with Attention) of the Transformer architecture to predict the trajectory of the drone in the next 5 seconds. The input of the model is the position and attitude data of the past 30 frames, and the output is the predicted three-dimensional coordinate sequence;

[0065] S4, Platform Attitude Optimization: A reinforcement learning environment based on PyTorch is constructed, and the PPO (Proximal Policy Optimization) algorithm is used to train the attitude adjustment strategy. The reward function considers factors such as the transparency of the landing platform 2-4 degrees, the energy consumption and adjustment speed of the high-precision servo motor 3-1 and the high-precision steering gear 3-6 as the considerations of the reward function;

[0066] S5, Adaptive Fusion Strategy: Design a dynamic weight allocation mechanism based on Softmax, and adaptively adjust the weights of object detection, trajectory prediction, and reinforcement learning strategies according to the detection confidence, trajectory prediction error, and current system state;

[0067] S6, Safety Constraints: Use the model predictive control method in control theory, use the MATLAB Simscape Multibody tool to establish a high-precision dynamic model of the platform, adopt the extended Kalman filter (EKF) to fuse multi-source sensor data, and estimate the complete state of the transparent landing platform 2-4 in real time, including position, attitude, speed, and acceleration. And use the ACADO Toolkit to achieve real-time NMPC solution, including the weighted sum of the level error, energy consumption, and motion smoothness index of the transparent landing platform 2-4, and set the motion constraint conditions of the high-precision servo motor 3-1 and the high-precision steering gear 3-6 to ensure that the generated control instructions are within the capabilities of the high-precision servo motor 3-1 and the high-precision steering gear 3-6, so as to ensure the safety and reliability of the movement of the transparent landing platform 2-4;

[0068] S7, real-time optimization: convert the PyTorch model into TensorRT format and deploy it on the Jetson AGX Xavier main control platform; use half-precision (FP16) quantization technology to improve the inference speed while ensuring accuracy, use CUDA programming to achieve GPU acceleration of key point cloud processing and matrix operations, adopt multi-threaded parallel processing strategy to parallelize data preprocessing, target detection, trajectory prediction and control instruction generation tasks, and use ROS2 (Robot Operating System 2) as the software framework to achieve efficient communication and data distribution synchronization between the unmanned vehicle's high-performance industrial computer 1-2 and the drone's main control module.

[0069] When the drone enters the final landing stage, the control system switches to fine control mode. The high-definition camera and laser rangefinder continue to provide high-frequency position feedback, based on which the platform posture is continuously fine-tuned. The force feedback sensor monitors the contact force in real time, and the control system accurately adjusts the damping coefficient of the double-joint support arm to achieve smooth acceptance.

Claims

1. A multi-joint self-balancing adjustment platform based on an inspection unmanned vehicle, characterized in that include: Driverless vehicle chassis (1), adjustable landing platform (2), double-joint support arm (3), multi-sensor system (4). An adjustable landing platform (2) is installed above the driverless vehicle chassis (1). The adjustable landing platform (2) is inlaid and connected with the driverless vehicle chassis (1) through a fixed bottom plate (2-1). There are three limit sliding grooves (2-2) above the fixed bottom plate (2-1). Three double-joint support arms (3) arranged in an equilateral triangle are respectively installed above the sliding grooves (2-2). The upper ends of the double-joint support arms (3) are adsorbed and fixed to the transparent platform (2-4) through strong suction cups (2-3). The driverless vehicle chassis (1) is equipped with a lithium battery pack (1-1), an industrial computer (1-2), and a main controller (1-3). An six-axis attitude sensor (4-1), a high-definition camera (4-2), and a laser rangefinder (4-3) are installed on the adjustable landing platform (2). The top of the driverless vehicle chassis (1) is horizontally and vertically symmetrically connected and fixed by 2 side-length bars (1-4) on the lower side and 4 support bars (1-17). Two front frame connecting arms (1-7) are respectively connected and fixed to 2 side-length bars (1-4). The front support arm (1-16) and the middle support arm (1-14) are both connected and fixed to the front-middle arm connecting bar (1-10). Two pedestal bearings (1-8) are connected and fixed above the front-middle arm connecting bar (1-10). A steel shaft (1-9) passes through the two pedestal bearings (1-8) and the steel sheet fasteners (1-11). The four steel sheet fasteners (1-11) are respectively fixed to the two front frame connecting arms (1-7). The four vertical short bars (1-18), the four side-length bars (1-4), and the four support bars (1-17) are vertically and crosswise connected and fixed to form the overall frame of the back of the driverless vehicle. The upper part of the rear support arm (1-5) is connected to the side-length bar (1-4), and the lower part is fixedly connected through four screw holes on the motor support (1-12). The motor support (1-12) and the M3508 motor (1-15) are fixed through four screw holes on the other side of the motor support (1-12), and are connected to the obstacle-crossing tire (1-6) through a bearing sleeve (1-13). The double-joint support arm (3) consists of a high-precision servo motor (3-1), a driving joint (3-2), a driven joint (3-3), a universal ball shaft (3-4), a force feedback sensor (3-5), and a strong suction cup (2-3). Among them, the high-precision servo motor (3-1) is respectively connected to the main controller (1-3) and the lithium battery pack (1-1) through internal data lines to achieve power supply and control connection;The high-precision servo motor (3-1) controls the rotation angles of the active joint (3-2) and the driven joint (3-3). The connection between the active joint (3-2) and the driven joint (3-3) is fixedly connected by a high-precision steering gear (3-6). The force feedback sensor (3-5) monitors the supporting force in real time and transmits the feedback data to the main controller (1-3) through the Bluetooth module (3-5-1) inside it. Both the high-precision servo motor (3-1) and the high-precision steering gear (3-6) have a self-locking function. In the absence of signal transmission, there is a large torque at both the high-precision servo motor (3-1) and the high-precision steering gear (3-6). The universal ball shaft (3-4) and the powerful suction cup (2-3) are fixedly embedded through a rubber ball and a groove, and there is a certain pre-tightening force at the connection. This pre-tightening force is sufficient to make the drone land smoothly without shaking. In the default state of the movement of the double-joint support arm (3), the universal ball shaft (3-4) follows the movement of the double-joint support arm (3) with the vertical upward direction. The rotation of the universal ball shaft (3-4) is passively triggered by the pulling and pushing forces exerted on the transparent landing platform (2-4) when the three double-joint support arms (3) move coordinately.

2. The multi-joint self-balancing adjustment platform based on the inspection unmanned vehicle according to claim 1, characterized in that The fixed base plate (2-1) of the adjustable landing platform (2) is a support plate modified from a PCB circuit board. The pins of the electronic components of three high-precision servo motors (3-1), three high-definition cameras (4-2) and three force feedback sensors (3-5) are fixed to the fixed base plate (2-1) by welding. The main controller (1-3) which is also welded to the fixed base plate (2-1) realizes power supply and data transmission for the components.

3. The multi-joint self-balancing adjustment platform based on the inspection unmanned vehicle according to claim 2, wherein The six-axis attitude sensor (4-1) is installed at the center position below the adjustable landing platform (2) to monitor the spatial attitude of the platform in real time; A high-definition camera (4-2) is installed in the space gaps between the three double-joint support arms (3) of the platform, and is fixed in an equilateral triangle position, and is used to capture visual information during the landing of the UAV; a laser rangefinder (4-3) is installed at the exact center of the platform, and is used to accurately measure the distance between the UAV and the platform; the slide groove (2-2) provides a plurality of preset installation positions for the double-joint support arm (3), so as to facilitate the response to special application requirements; when the unmanned vehicle is moving, the relative position of the limit slide groove (2-2) and the double-joint support arm (3) will not change.

4. The multi-joint self-balancing adjustment platform based on the inspection unmanned vehicle according to claim 3, wherein, The industrial computer (1-2) has a built-in multi-source data fusion algorithm, which integrates deep learning and adaptive control algorithms, processes data from a six-axis attitude sensor (4-1), a high-definition camera (4-2), a laser rangefinder (4-3) and feedback from a drone in real time, and calculates an optimal platform attitude adjustment strategy.

5. The multi-joint self-balancing adjustment platform based on the inspection unmanned vehicle according to claim 4, wherein, During the landing process of the UAV, the control system first compares the surrounding terrain information with the pre-loaded UAV remote sensing data to determine the terrain at the current location. The industrial control computer (1-2) reads and recognizes the vehicle body posture data in combination with the six-axis attitude sensor (4-1), performs attitude calculation through dynamic weighted fusion, outputs command information to the main controller (1-3), and the main controller (1-3) converts it into PWM pulse information and transmits it to the high-precision servo motor (3-1) and the high-precision steering gear (3-6) for preliminary attitude adjustment. Subsequently, the high-definition camera (4-2) and the laser rangefinder (4-3) are used to track the landing trajectory of the UAV in real time and fine-tune the platform attitude. When the UAV contacts the platform and begins to gradually reduce the rotor speed, the unmanned vehicle control system calculates the spatial motion trajectory of the double-joint support arm (3) based on the force feedback sensor (3-5) data and the horizontal offset azimuth information of the landing of the UAV, and performs a smooth acceptance of the UAV to minimize the landing impact of the UAV.

6. A method for using a multi-joint self-balancing adjustment platform based on the inspection unmanned vehicle according to claim 4, characterized in that, The industrial control computer (1-2) is configured with a multi-source data real-time fusion analysis algorithm, which integrates a convolutional neural network and a recursive neural network to form an end-to-end adaptive control model, processes data streams from various sensors in real time, predicts the motion trajectory of the drone, optimizes the landing platform attitude adjustment strategy, and ensures accurate and safe landing of the drone under complex terrain conditions; The described end-to-end adaptive control model is implemented using the PyTorch deep learning framework. When extracting features, a multi-modal fusion network is used to process different sensor data: the IMU data of the six-axis attitude sensor is processed through an ID convolutional neural network; the features of the high-definition camera images are extracted by the ResNet50 backbone network; the lidar data is used to extract point cloud features through PointNet++; the telemetry data fed back by the drone is processed by the LSTM network, and all features are fused through the attention mechanism to form a unified feature representation.

7. The method according to claim 6, characterized in that, It includes the following steps: S1, Data preprocessing: Use the OpenCV library to preprocess the images, including denoising, histogram equalization, and normalization; use the PCL library to filter and downsample the lidar point cloud data; use the Pandas library to perform time synchronization and interpolation processing on the IMU and telemetry data; S2, Object detection and tracking: Adopt the YOLOv5 algorithm to use three high-definition cameras (4-2) to perform real-time detection of the drone landing, and use the DeepSORT algorithm for multi-object tracking; The model is optimized through TensorRT to achieve a real-time processing speed of 30 FPS on the NVIDIA Jetson AGX Xavier; S3, Trajectory prediction: Fuse the image parsing data of YOLOv5 and the depth distance data of the lidar (4-3), and input them into the sequence-to-sequence model of the Transformer architecture to predict the trajectory of the drone in the next 5 seconds. The model input is the position and attitude data of the past 30 frames, and the output is the predicted three-dimensional coordinate sequence; S4, Platform attitude optimization: Build a reinforcement learning environment based on PyTorch, and use the PPO algorithm to train the attitude adjustment strategy; the reward function considers the transparency of the landing platform (2-4), the energy consumption and adjustment speed of the high-precision servo motor (3-1) and the high-precision servo actuator (3-6) as the considerations of the reward function; S5, Adaptive fusion strategy: Design a dynamic weight allocation mechanism based on Softmax, and adaptively adjust the weights of object detection, trajectory prediction, and reinforcement learning strategies according to the detection confidence, trajectory prediction error, and current system state; S6, Safety constraints: Use the model predictive control method in control theory, use the MATLAB Simscape Multibody tool to establish a high-precision dynamic model of the platform, adopt the extended Kalman filter to fuse multi-source sensor data, and estimate the complete state of the transparent landing platform (2-4) in real time, including position, attitude, speed, and acceleration, and use the ACADO Toolkit to achieve real-time NMPC solution, including the weighted sum of the horizontal error, energy consumption, and motion smoothness index of the transparent landing platform (2-4), and set the motion constraint conditions of the high-precision servo motor (3-1) and the high-precision servo actuator (3-6) to ensure that the generated control instructions are within the capabilities of the high-precision servo motor (3-1) and the high-precision servo actuator (3-6), so as to ensure the safety and reliability of the movement of the transparent landing platform (2-4); S7, Real-time Optimization: Convert the PyTorch model to the TensorRT format and deploy it on the Jetson AGX Xavier main control platform; adopt the half-precision quantization technology to improve the inference speed while ensuring the accuracy, use CUDA programming to achieve GPU acceleration for key point cloud processing and matrix operations, adopt the multi-thread parallel processing strategy to parallelize the data preprocessing, target detection, trajectory prediction, and control instruction generation tasks, and use ROS2 as the software framework to achieve efficient communication and data distribution synchronization between the industrial control computer (1-2) of the unmanned vehicle and the main control module of the unmanned aerial vehicle.

Images