Intelligent manipulator control method of unmanned aerial vehicle

Abstract

The invention belongs to the technical field of unmanned aerial vehicles, and particularly relates to an intelligent manipulator control method of an unmanned aerial vehicle, which comprises the following steps that an intelligent manipulator is used for operating a flight remote controller, and the intelligent manipulator receives flight data from the unmanned aerial vehicle and calculates the flight data through an algorithm model; specific numerical values such as a course angle, a pitch angle and an azimuth angle needing to be adjusted of the unmanned aerial vehicle are obtained, and the manipulator is driven to control and adjust a remote controller of the unmanned aerial vehicle through refined mechanical operation of the electric motor, so that final control of the unmanned aerial vehicle is realized. According to the invention, the stability and safety problems possibly caused by manual operation are avoided, the dependence on an unmanned aerial vehicle operator is reduced, large-scale cluster unmanned aerial vehicle control can be realized, and accurate cooperative work between unmanned aerial vehicles can be realized.

Description

Technical Field

[0001] The invention belongs to the technical field of unmanned aerial vehicles, and in particular relates to an intelligent manipulator control method for an unmanned aerial vehicle. Background Art

[0002] At present, most intelligent unmanned aerial vehicles are controlled by people on the ground. Micro, light and small intelligent unmanned aerial vehicles use the flight remote control and ground station system that are equipped with the aircraft to upload and download aircraft data. Medium and large intelligent unmanned aerial vehicles use special control cabins, and there are dedicated people responsible for the take-off and landing control and flight control of the aircraft.

[0003] Traditional micro, light, and small intelligent unmanned aerial vehicles are all controlled by human hands using aircraft remote controls. Human hands controlling aircraft through remote controls have the accumulation of experience, the growth of proficiency, the inevitable problem of fatigue, and the problem of fluctuations in physical and mental conditions, which lead to safety hazards in the operation of unmanned aerial vehicles. Summary of the invention

[0004] The purpose of the present invention is to provide an intelligent manipulator control method for an unmanned aerial vehicle, which utilizes a manipulator to simulate and replace human hands to operate an aircraft remote control, thereby avoiding the drawbacks of manual operation and solving the problems raised in the above-mentioned background technology.

[0005] To achieve the above-mentioned purpose, the present invention adopts the following technical scheme: a method for controlling an unmanned aerial vehicle with an intelligent manipulator, comprising the following steps: using an intelligent manipulator to operate a flight remote controller, wherein the intelligent manipulator receives flight data from the unmanned aerial vehicle and adjusts the joystick and buttons on the flight remote controller according to the flight data, thereby controlling the flight attitude, heading angle and pitch angle of the unmanned aerial vehicle.

[0006] Preferably, the intelligent manipulator is driven by a servo motor, and the operating force and angle of the intelligent manipulator are adjusted in real time according to the flight status information of the unmanned aerial vehicle.

[0007] Preferably, the flight data includes but is not limited to video stream information transmitted from the camera of the unmanned aerial vehicle and flight sensor data information, and the information is obtained through a monocular or binocular camera and a GPS, RTK sensor receiver.

[0008] Preferably, the processing of the flight data includes acquiring point cloud data through a camera for environmental perception, using a SI FT algorithm for feature extraction to identify environmental obstacles or target objects, and adjusting the manipulator operation through a PID controller.

[0009] Preferably, the intelligent manipulator receives instructions from the ground station computer, parses the operation steps to be executed according to the instructions, and operates the dual joysticks of the remote control through the grabbing and pushing and pulling actions of the manipulator to control the forward, backward, left and right turns, ascent and descent flight actions of the unmanned aerial vehicle.

[0010] Preferably, during the flight of the unmanned aerial vehicle, the flight status information of the unmanned aerial vehicle is fed back to the intelligent manipulator through the visual system, and the intelligent manipulator adjusts the operating force and angle of the joystick or switch in real time according to the flight status information.

[0011] Preferably, after the unmanned aerial vehicle completes its mission or returns to land, the intelligent manipulator executes a landing procedure according to instructions, including but not limited to shutting down power or disconnecting control.

[0012] Preferably, the control instructions and data are transmitted between the unmanned aerial vehicle and the remote controller via the 2.4G / 5.8G frequency band. The data is transmitted between the unmanned aerial vehicle and the ground station computer via the 5.8G frequency band. The ground station computer and the intelligent manipulator send control signals to the manipulator via cable communication, and the intelligent manipulator adjusts the angle and speed of the joint and the action of the end effector according to the instructions.

[0013] Technical effects and advantages of the present invention: Compared with the prior art, the intelligent manipulator control method of an unmanned aerial vehicle proposed by the present invention has the following advantages:

[0014] The present invention uses an intelligent manipulator to replace manual operation of the flight remote control. The precise manipulator, under the control of software, timely and accurately adjusts the aircraft remote control through the manipulator according to the data sent by various sensors of the aircraft, controls the flight attitude, adjusts the heading angle and the pitch angle, thereby achieving the purpose of controlling the aircraft and avoiding the stability and safety problems that may be caused by manual operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of the intelligent manipulator of the present invention operating the flight remote controller;

[0016] Figure 2 This is a system block diagram of the intelligent manipulator of the present invention operating the flight remote controller;

[0017] Figure 3 Flow chart of SIFT target detection of the present invention;

[0018] Figure 4 It is a structural schematic diagram of the unmanned aerial vehicle, ground station computer and intelligent manipulator of the present invention.

[0019] In the picture: 1. Unmanned aerial vehicle; 2. Ground station computer; 3. Intelligent manipulator; 4. Flight remote control. DETAILED DESCRIPTION

[0020] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0021] The present invention provides an intelligent manipulator control method for an unmanned aerial vehicle, comprising the following steps: using an intelligent manipulator 3 to operate a flight remote controller 4, the intelligent manipulator 3 receives flight data from an unmanned aerial vehicle 1, and adjusts the joystick and buttons on the flight remote controller according to the flight data, thereby controlling the flight attitude, heading angle and pitch angle of the unmanned aerial vehicle.

[0022] The present invention uses an intelligent manipulator to replace manual operation of the flight remote controller. The precise manipulator, under the control of software, timely and accurately adjusts the aircraft remote controller through the manipulator according to the data sent by the aircraft sensors, controls the flight attitude, adjusts the heading angle and pitch angle, and achieves the purpose of controlling the aircraft, avoiding the stability and safety problems that may be caused by manual operation. The purpose of controlling the aircraft is achieved, the stability and safety problems that may be caused by manual operation are avoided, and the purpose of controlling the aircraft in all weather and all terrains can be achieved.

[0023] Specifically, flight data includes but is not limited to video stream information transmitted by the unmanned aerial vehicle camera and flight sensor data information, and the information is obtained through a monocular or binocular camera and a GPS, RTK sensor receiver.

[0024] like Figure 2 As shown, the data acquisition module has the video stream information transmitted by the aircraft camera and the aircraft's own flight sensor data information. The implementation steps are as follows:

[0025] (1) Accessing camera video stream information: The image transmission system of the intelligent unmanned aerial vehicle transmits downlink images to the aircraft remote control. The manipulator is equipped with a monocular or binocular camera, which is similar to the human eye and perceives the image transmission image to assist in analyzing the flight position and collect data according to the flight mission.

[0026] (2) Flight sensor data information: collects data from the aircraft flight control platform and various flight sensors.

[0027] Data processing and transmission module

[0028] like Figure 3As shown in the figure, point cloud data is obtained through the camera for environmental perception, while monitoring the flight status of the drone, including data such as altitude, speed, and tilt angle. Different data are aligned in time and space, and a timestamp synchronization mechanism is used to ensure that multi-source data is consistent within the same time range. The SIFT algorithm is used to extract image transmission features to achieve recognition of environmental obstacles or target objects. The complete process of SIFT target detection is as follows:

[0029] Key point detection: In the target detection task, we first need to detect the key points in the target image and the scene image. The SIFT algorithm detects stable feature points in the image by generating Gaussian pyramids and Gaussian difference pyramids.

[0030] Feature descriptors: For each detected keypoint, SIFT calculates its orientation, scale, and local feature descriptors. The SIFT feature descriptor is a 128-dimensional vector generated based on the gradient orientation information near the keypoint.

[0031] Feature matching: Match the key points in the target image with the key points in the scene image. SIFT uses the Euclidean distance (or other distance metric) between descriptors to perform the matching.

[0032] RANSAC geometry verification: A set of potential matching point pairs is obtained through feature matching, but there may be false matches. In order to eliminate false matches, the RANSAC (random sampling consensus) algorithm is often used for geometry verification to further improve the accuracy of target detection.

[0033] Target positioning: Through the geometric transformation matrix, the outline or boundary of the target image is mapped to the scene image to determine the position and posture of the target in the scene.

[0034] By analyzing the optical flow between image frames, the moving direction and speed of the drone are estimated to assist flight control. Based on the data processing results, operation instructions are generated and the PID controller is used to adjust the manipulator operation. For example, if the drone is detected to deviate from the predetermined trajectory, the operation instructions for adjusting the remote control joystick are generated by modifying the PID parameters and mapped to the specific remote control operation to return the drone to the correct trajectory.

[0035] In addition, the processing of flight data includes acquiring point cloud data through the camera for environmental perception, using the SI FT algorithm for feature extraction to identify environmental obstacles or target objects, and adjusting the manipulator operation through the PID controller.

[0036] Furthermore, the intelligent manipulator is driven by a servo motor, and the operating force and angle of the intelligent manipulator are adjusted in real time according to the flight status information of the unmanned aerial vehicle.

[0037] Furthermore, the control principle of the robot is mainly composed of the following three parts:

[0038] (1) Feedback control: During the flight of the UAV, the visual system feeds back the current flight status (such as speed, direction, attitude, etc.) to the computer host of the ground station. The manipulator adjusts the operating force and angle of the joystick or switch in real time based on the flight status information to ensure that the UAV executes according to the predetermined route or flight strategy.

[0039] (2) Motor drive: The joints of the robot are driven by servo motors. After receiving instructions from the controller, the motors will rotate or move according to the preset path and speed, driving the joints of the robot to perform corresponding actions.

[0040] (3) Control algorithm: A complex PID control algorithm is run inside the ground host of the ground station to adjust the motion trajectory and speed of the manipulator, enabling it to flexibly respond to various operational requirements.

[0041] Specifically, the intelligent manipulator receives instructions from the ground station computer 2, parses the operation steps to be executed according to the instructions, and operates the dual joysticks of the remote controller through the grabbing and pushing and pulling actions of the manipulator to control the forward, backward, left and right turns, ascent and descent flight actions of the unmanned aerial vehicle. Furthermore, after the unmanned aerial vehicle completes the mission or returns to land, the intelligent manipulator executes the landing procedure according to the instructions, including but not limited to turning off the power or disconnecting the control. The ground station computer collects the downlink data sent by the intelligent unmanned aerial vehicle to grasp the flight status of the intelligent unmanned aerial vehicle, changes the position of the manipulator on the remote controller through software, and changes the position of the manipulator to change the state of the remote controller stick and button position, and achieves the purpose of changing the flight attitude of the intelligent unmanned aerial vehicle through the data uplink of the remote controller.

[0042] The aircraft sensor data is linked down to the ground control system, which uses control software to analyze and sense data such as the aircraft's position and attitude. The intelligent unmanned aerial vehicle is controlled by adjusting the remote control of the intelligent unmanned aerial vehicle through a manipulator to accomplish the flight mission.

[0043] During the flight of the UAV, the flight status information of the UAV is fed back to the intelligent manipulator through the visual system, and the intelligent manipulator adjusts the operating force and angle of the joystick or switch in real time according to the flight status information.

[0044] The above-mentioned unmanned aerial vehicle and the remote controller transmit control commands and data interactively through the 2.4G / 5.8G frequency band; the unmanned aerial vehicle and the ground station computer transmit data through the 5.8G frequency band; the ground station computer and the intelligent manipulator send control signals to the manipulator through cable communication, and the intelligent manipulator adjusts the angle and speed of the joints and the action of the end effector according to the commands.

[0045] Specifically, according to the preset flight mission, the manipulator receives instructions and parses the operating steps that need to be performed (such as pushing rods, toggling switches, etc.). The manipulator operates the dual joysticks of the remote control through grabbing and pushing and pulling actions to adjust the drone's forward / backward, left and right turns, ascent / descent and other basic flight actions; during the flight of the drone, the current flight status (such as speed, direction, attitude, etc.) is fed back to the manipulator system through the visual system. The manipulator adjusts the operating force and angle of the joystick or switch in real time according to the flight status information to ensure that the drone executes according to the predetermined route or flight strategy. After the mission is completed or the drone returns to land, the manipulator executes the landing procedure according to the instructions, turns off the power or disconnects the control. Figure 1 The detailed process is as follows:

[0046] Start: Start the entire control system.

[0047] Set target position: Set the target flight position of the UAV.

[0048] The robot grabs the remote control and controls the drone to take off: The intelligent robot is driven by a servo motor to simulate human hand movements to operate the remote control and make the unmanned aerial vehicle take off.

[0049] Visual system detection: The UAV collects image information through the camera and inputs the point cloud data into the computer vision system for analysis.

[0050] Calculate position deviation: Calculate the deviation between the current position of the UAV and the target position through the data fed back by the visual system.

[0051] PID control algorithm: Use the PID (proportional-integral-differential) control algorithm to calculate the corresponding control quantity based on the position deviation.

[0052] Adjust the remote sensing control signal: According to the results of the PID algorithm, adjust the robot's operating force and angle of the remote control to achieve precise control of the unmanned aerial vehicle.

[0053] Send control signal: Send the adjusted control signal to the UAV to make it fly along the preset path.

[0054] Reaching a stable state: The UAV reaches the target position and maintains a stable state.

[0055] End: After completing the mission, the UAV lands and the manipulator releases the remote control, ending the entire control process.

[0056] The present invention uses a manipulator to operate the remote controller that comes with the intelligent unmanned aerial vehicle to achieve a method of controlling the intelligent unmanned aerial vehicle. In this method, as long as the aircraft remote controller is used to control the flight, there is no need to change the hardware and software of the intelligent unmanned aerial vehicle, and it can be adapted to operate various brands and models of intelligent unmanned aerial vehicles. There are many advantages of controlling the intelligent unmanned aerial vehicle by a manipulator. It will not cause fatigue. As long as the performance of the aircraft supports it, it can be operated all the time. It has strong stability and avoids the uncertainty of manual operation. The intelligent unmanned aerial vehicle can be operated in unsafe areas, such as underground pipes and toxic and harmful areas, without the pilot being in danger. It can respond 24 hours a day, and even control the aircraft in a dark night without vision environment to fly accurately.

[0057] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for controlling an intelligent manipulator of an unmanned aerial vehicle, characterized in that: The following steps are involved: An intelligent manipulator is used to operate the flight remote controller. The intelligent manipulator receives flight data from the unmanned aerial vehicle and adjusts the joystick and buttons on the flight remote controller according to the flight data, thereby controlling the flight attitude, heading angle and pitch angle of the unmanned aerial vehicle.

2. The intelligent manipulator control method of an unmanned aerial vehicle according to claim 1, characterized in that: The intelligent manipulator is driven by a servo motor, and the operating force and angle of the intelligent manipulator are adjusted in real time according to the flight status information of the unmanned aerial vehicle.

3. The intelligent manipulator control method of an unmanned aerial vehicle according to claim 1, characterized in that: The flight data includes but is not limited to video stream information transmitted by the camera of the unmanned aerial vehicle and flight sensor data information, and the information is obtained through a monocular or binocular camera and a GPS, RTK sensor receiver.

4. The intelligent manipulator control method of an unmanned aerial vehicle according to claim 1, characterized in that: The processing of the flight data includes acquiring point cloud data through a camera for environmental perception, using a SIFT algorithm for feature extraction to identify environmental obstacles or target objects, and adjusting the manipulator operation through a PID controller.

5. The intelligent manipulator control method of an unmanned aerial vehicle according to claim 1, characterized in that: The intelligent manipulator receives instructions from the ground station computer, parses the operation steps to be executed according to the instructions, and operates the dual joysticks of the remote control through the grabbing and pushing and pulling actions of the manipulator to control the forward, backward, left and right turns, ascent and descent flight actions of the unmanned aerial vehicle.

6. The intelligent manipulator control method of an unmanned aerial vehicle according to claim 1, characterized in that: During the flight of the UAV, the flight status information of the UAV is fed back to the intelligent manipulator through the visual system, and the intelligent manipulator adjusts the operating force and angle of the joystick or switch in real time according to the flight status information.

7. The intelligent manipulator control method of an unmanned aerial vehicle according to claim 1, characterized in that: After the unmanned aerial vehicle completes its mission or returns to land, the intelligent manipulator executes a landing procedure according to instructions, including but not limited to shutting down power or disconnecting control.

8. The intelligent manipulator control method for an unmanned aerial vehicle according to claim 1, characterized in that: The control commands and data are transmitted between the UAV and the remote controller via the 2.4G / 5.8G frequency band.

9. The intelligent manipulator control method of an unmanned aerial vehicle according to claim 8, characterized in that: Data is transmitted between the UAV and the ground station computer via the 5.8G frequency band.

10. The intelligent manipulator control method of an unmanned aerial vehicle according to claim 9, characterized in that: The ground station computer sends control signals to the intelligent manipulator via cable communication, and the intelligent manipulator adjusts the angle and speed of the joints and the movement of the end effector according to the instructions.

Images