Unmanned aerial vehicle control method based on multi-sensor fusion

Abstract

The invention discloses an unmanned aerial vehicle control method based on multi-sensor fusion, and relates to the technical field of unmanned aerial vehicle control, and the method comprises the steps: collecting a target image through a high-speed vision camera carried on an unmanned aerial vehicle; processing the target image based on an improved adaptive threshold algorithm, and determining a landing platform of the unmanned aerial vehicle; calculating horizontal deviation data, vertical height data, yaw angle data, pitch angle data and roll angle data of the unmanned aerial vehicle relative to the landing platform; acquiring the speed and acceleration of the unmanned aerial vehicle in real time; on the basis of the relative position prediction model, relative pose information of the unmanned aerial vehicle and the landing platform is updated in real time through an interpolation algorithm; and performing flight path planning according to the relative pose information, and controlling the unmanned aerial vehicle to land on a landing platform in stages. According to the method, the real-time performance of unmanned aerial vehicle control is improved, the control delay of the unmanned aerial vehicle is effectively reduced, and the control precision of landing of the unmanned aerial vehicle based on the landing platform is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle control, and particularly relates to a method for controlling unmanned aerial vehicles based on multi-sensor fusion. BACKGROUND

[0002] Unmanned aerial vehicles, also known as drones, are devices that are controlled by radio remote control equipment and self-provided program control devices, or are operated completely or intermittently by on-board computers.

[0003] Traditional autonomous following and landing control of unmanned aerial vehicles mainly relies on positioning information provided by the global positioning system (GPS) or the Beidou satellite navigation system (BDS). However, satellite navigation technology has inherent limitations in practical applications, and its positioning accuracy is usually within 1 to 5 meters, which cannot meet the needs of close-range high-precision following and landing.

[0004] In order to meet the needs of close-range high-precision following and landing of unmanned aerial vehicles, a visual measurement system is introduced in the prior art, which calculates the relative pose of the unmanned aerial vehicle and the landing platform by recognizing the landing platform through an on-board camera. However, the traditional visual measurement system generally uses a fixed threshold algorithm for image recognition processing. For example, target features of the landing platform image are extracted by color filtering (such as RGB separation) or fixed threshold binary method, but such methods have low image processing efficiency, high target false detection rate, and cannot adapt to real-time detection in dynamic environments under uneven lighting, shadow blocking or complex background environments.

[0005] Therefore, a method for controlling unmanned aerial vehicles based on multi-sensor fusion is proposed. SUMMARY

[0006] To solve some or all of the technical problems existing in the prior art, the present application provides a method for controlling unmanned aerial vehicles based on multi-sensor fusion.

[0007] The technical solution of the present application is as follows:

[0008] A method for controlling unmanned aerial vehicles based on multi-sensor fusion is provided, comprising:

[0009] A high-speed vision camera mounted on the unmanned aerial vehicle acquires a target image;

[0010] An improved adaptive threshold algorithm is used to process the target image to determine the landing platform of the unmanned aerial vehicle;

[0011] The horizontal offset data, vertical height data, yaw angle data, pitch angle data and roll angle data of the unmanned aerial vehicle relative to the landing platform are calculated;

[0012] The speed and acceleration of the unmanned aerial vehicle are collected in real time through an inertial measurement unit arranged on the unmanned aerial vehicle;

[0013] Based on the relative position prediction model, the relative position and posture information of the unmanned aerial vehicle and the landing platform is updated in real time according to the horizontal offset data, vertical height data, yaw angle data, pitch angle data and roll angle data of the unmanned aerial vehicle relative to the landing platform, and the speed and acceleration of the unmanned aerial vehicle through an interpolation algorithm.

[0014] After the unmanned aerial vehicle obtains the landing instruction, the relative position and posture information is used for path planning, and the unmanned aerial vehicle is controlled to land on the landing platform in stages.

[0015] In some optional embodiments, the relative position prediction model is represented as:

[0016]

[0017] P t+1 = P t + V t * Δt + 0.5 * a t * (Δt) 2 t+1 P t+1 represents the relative position and posture of the unmanned aerial vehicle and the landing platform at t+1, P t represents the relative position and posture of the unmanned aerial vehicle and the landing platform at t, V t represents the speed of the unmanned aerial vehicle at t, a t represents the acceleration of the unmanned aerial vehicle at t, and Δt represents the time difference between t+1 and t. t t t

[0018] In some optional embodiments, the unmanned aerial vehicle is controlled to land on the landing platform in stages, including: controlling the unmanned aerial vehicle to fly directly above the landing platform, controlling the unmanned aerial vehicle to track the landing platform, and controlling the unmanned aerial vehicle to reduce the flight height and land on the landing platform.

[0019] In some optional embodiments, after the unmanned aerial vehicle obtains the landing instruction, the relative position and posture information is used for path planning, and the unmanned aerial vehicle is controlled to land on the landing platform in stages, including:

[0020] After the unmanned aerial vehicle obtains the landing instruction, the relative position and posture information is used for path planning, and the unmanned aerial vehicle is controlled to land on the landing platform in stages, including:

[0021] When it is determined that the unmanned aerial vehicle meets the instruction execution condition, the relative position and posture information of the unmanned aerial vehicle and the landing platform is used for path planning, the unmanned aerial vehicle is controlled to fly directly above the landing platform, and the unmanned aerial vehicle is controlled to track the landing platform in real time.

[0022] When it is detected that the height data of the unmanned aerial vehicle and the landing platform is less than a preset height threshold, the unmanned aerial vehicle is controlled to land on the landing platform.

[0023] ​​​In some optional embodiments, after the UAV obtains the landing instruction, the relative pose information is used to plan a flight path, and the UAV is controlled to land on the landing platform in stages, and the method further comprises:

[0024] When the tracking target is lost during the flight of the UAV tracking the landing platform, the UAV is controlled to adjust the flight height and search for the landing platform again.

[0025] In some optional embodiments, after the UAV obtains the landing instruction, the relative pose information is used to plan a flight path, and the UAV is controlled to land on the landing platform in stages, and the method further comprises:

[0026] The flight path planning instruction is obtained, analyzed and processed by the outer loop controller, the flight path control instruction is obtained, and the UAV is controlled to fly through the inner loop controller;

[0027] The first attitude flight path information of the UAV is collected in real time through the inner loop sensor, the deviation between the first attitude flight path information and the flight path control instruction is analyzed, the first compensation instruction is obtained, and the UAV is controlled to fly by the inner loop controller optimizing the flight path control instruction;

[0028] The second attitude flight path information of the UAV is collected in real time through the outer loop sensor, the deviation between the second attitude flight path information and the flight path planning instruction is analyzed, the second compensation instruction is obtained, and the UAV is controlled to fly by the outer loop controller optimizing the flight path planning instruction.

[0029] In some optional embodiments, the improved adaptive threshold algorithm is represented as:

[0030] T(x,y)=u(x,y)·(1+k·σ(x,y));

[0031] Wherein, T(x,y) represents the adaptive threshold value of the pixel point (x,y) in the image, u(x,y) represents the gray mean value in the set window with the pixel point (x,y) as the center, σ(x,y) represents the gray standard deviation in the set window with the pixel point (x,y) as the center, and k represents a dynamic adjustment coefficient.

[0032] The main advantages of the technical scheme of the present application are as follows:

[0033] The UAV control method based on multi-sensor fusion of the present application processes the target image based on the improved adaptive threshold algorithm, can eliminate the shadow interference in the target image, realizes accurate identification and positioning of the landing platform in a complex environment, collects the speed and acceleration of the UAV in real time through the inertial measurement unit, predicts and updates the relative pose information of the UAV and the landing platform based on the relative position prediction model and the interpolation algorithm, improves the real-time performance of the UAV control, effectively reduces the control delay of the UAV, and improves the control accuracy of the UAV landing based on the landing platform. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating the UAV control method based on multi-sensor fusion provided in an embodiment of the present invention.

[0036] Figure 2 This is a flowchart illustrating the landing platform of a UAV in a multi-sensor fusion-based UAV control method provided in an embodiment of the present invention.

[0037] Figure 3 A schematic diagram of the UAV control structure in the UAV control method based on multi-sensor fusion provided in an embodiment of the present invention;

[0038] Figure 4 A schematic diagram of a target image provided in an embodiment of the present invention;

[0039] Figure 5 This is a schematic diagram of an optimized binary target image provided in an embodiment of the present invention;

[0040] Figure 6 A schematic diagram of a feature contour image provided in an embodiment of the present invention;

[0041] Figure 7 A schematic diagram of target feature information provided in an embodiment of the present invention;

[0042] Figure 8 This is a schematic diagram of a preset visual identifier provided in an embodiment of the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0044] The technical solutions provided by the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0045] See Figure 1The embodiment of the present application provides a UAV control method based on multi-sensor fusion, which comprises the following steps:

[0046] Step 1, collecting a target image by a high-speed vision camera carried on the UAV;

[0047] Step 2, processing the target image based on an improved adaptive threshold algorithm to determine a landing platform of the UAV;

[0048] Step 3, calculating horizontal offset data, vertical height data, yaw angle data, pitch angle data and roll angle data of the UAV relative to the landing platform;

[0049] Step 4, collecting the speed and acceleration of the UAV in real time by an inertial measurement unit built on the UAV;

[0050] Step 5, based on a relative position prediction model, updating the relative pose information of the UAV and the landing platform in real time according to the horizontal offset data, vertical height data, yaw angle data, pitch angle data and roll angle data of the UAV relative to the landing platform, and the speed and acceleration of the UAV through an interpolation algorithm;

[0051] Step 6, after the UAV obtains a landing instruction, performing path planning according to the relative pose information to control the UAV to land on the landing platform in stages.

[0052] In the embodiment of the present application, the landing platform position of the UAV in the target image is determined by processing the target image based on the improved adaptive threshold algorithm, and the rotation matrix and translation vector of the high-speed vision camera relative to the landing platform are solved through a conventional pose solving algorithm according to the landing platform position of the UAV in the target image, so that the horizontal offset data, vertical height data, yaw angle data, pitch angle data and roll angle data of the UAV relative to the landing platform are calculated according to the rotation matrix and translation vector.

[0053] In the embodiment of the present application, the shadow interference in the target image can be eliminated by processing the target image based on the improved adaptive threshold algorithm, and the detection and recognition accuracy of the landing platform is improved.

[0054] In the embodiment of the present application, the relative pose information of the UAV and the landing platform comprises relative displacement information and relative attitude information; wherein the relative displacement information comprises horizontal offset data and vertical height data of the UAV relative to the landing platform; and the relative attitude information comprises yaw angle data, pitch angle data and roll angle data of the UAV relative to the landing platform.

[0055] When the unmanned aerial vehicle navigates to the vicinity of the landing platform, a high-speed vision camera is started to collect target images, and the images are processed to determine the position of the landing platform, during the image processing time, the flight state of the unmanned aerial vehicle is constantly changing, at this time, the relative pose information of the unmanned aerial vehicle and the landing platform calculated according to the image processing result is the past relative pose information, if the past relative pose information is directly transmitted to the unmanned aerial vehicle for flight control, the state of the unmanned aerial vehicle will always fall behind in time, resulting in a decrease in landing accuracy on the landing platform. Therefore, in the embodiment of the present application, considering the delay problem caused by the image processing process, the relative position prediction model and the interpolation algorithm are used to predict the relative pose information of the next frame, so as to update the relative pose information of the unmanned aerial vehicle and the landing platform in real time, thereby improving the landing accuracy.

[0056] The unmanned aerial vehicle control method based on multi-sensor fusion provided by the embodiment of the present application can eliminate shadow interference in the target image, realize accurate identification and positioning of the landing platform in a complex environment, by processing the target image based on the improved adaptive threshold algorithm; the speed and acceleration of the unmanned aerial vehicle are collected in real time by the inertial measurement unit, and the relative pose information of the unmanned aerial vehicle and the landing platform is predicted and updated based on the relative position prediction model and the interpolation algorithm, thereby improving the real-time performance of the unmanned aerial vehicle control, effectively reducing the control delay of the unmanned aerial vehicle, and improving the control accuracy of the unmanned aerial vehicle landing based on the landing platform.

[0057] Further, in the embodiment of the present application, the relative position prediction model is represented as:

[0058]

[0059] Wherein, P t+1 represents the relative pose of the unmanned aerial vehicle and the landing platform at t+1, P t represents the relative pose of the unmanned aerial vehicle and the landing platform at t, V t represents the speed of the unmanned aerial vehicle at t, and a t represents the acceleration of the unmanned aerial vehicle at t, and Δt represents the time difference between t+1 and t.

[0060] In the embodiment of the present application, the relative position prediction model is used to predict the relative pose of the unmanned aerial vehicle and the landing platform at the next time, which can effectively solve the delay problem caused by the image processing process, and improve the control accuracy of the unmanned aerial vehicle landing based on the landing platform.

[0061] Further, in the embodiment of the present application, the unmanned aerial vehicle is controlled to land on the landing platform in a segmented manner, including: controlling the unmanned aerial vehicle to fly above the landing platform, controlling the unmanned aerial vehicle to track the landing platform, and controlling the unmanned aerial vehicle to reduce the flight height and land on the landing platform.

[0062] In the embodiment of the present application, after the UAV obtains the landing instruction, the time consumed by the flight path planning according to the relative pose information is less than or equal to 2 seconds, and the control response time of the UAV is less than 50 ms.

[0063] In the embodiment of the present application, the UAV is controlled to land on the landing platform in a segmented manner, so that the control accuracy of the UAV landing on the landing platform is improved.

[0064] Further, in an optional embodiment of the present application, after the UAV obtains the landing instruction, the flight path is planned according to the relative pose information, and the UAV is controlled to land on the landing platform in stages, comprising:

[0065] After the UAV obtains the landing instruction, the task is evaluated to determine whether the UAV currently meets the instruction execution condition;

[0066] When it is determined that the UAV meets the instruction execution condition, the flight path is planned according to the horizontal offset data, the vertical height data, the yaw angle data, the pitch angle data and the roll angle data of the UAV relative to the landing platform, the UAV is controlled to fly to directly above the landing platform, and the UAV is controlled to fly in real time to track the landing platform;

[0067] When it is detected that the height data of the UAV relative to the landing platform is less than a preset height threshold, the UAV is controlled to land on the landing platform.

[0068] In the embodiment of the present application, after the UAV obtains the landing instruction, the task is evaluated to determine whether the execution of the landing instruction affects the safety of the UAV and whether the execution condition is met, when it is determined that the execution condition is met, the flight path is planned and optimized according to the horizontal offset data, the vertical height data, the yaw angle data, the pitch angle data and the roll angle data of the UAV relative to the landing platform, the UAV is controlled to fly to directly above the landing platform, and the UAV is controlled to fly in real time to track the landing platform, and when it is detected that the height data of the UAV relative to the landing platform is less than a preset height threshold, the UAV is controlled to land on the landing platform.

[0069] In the embodiment of the present application, after the UAV obtains the landing instruction, the task is evaluated to analyze the landing instruction, avoid the influence of the direct execution of the landing instruction on the flight safety of the UAV, and control the UAV to track the flight according to the horizontal offset data, the vertical height data, the yaw angle data, the pitch angle data and the roll angle data of the UAV relative to the landing platform, and the UAV is landed when the height data of the UAV is less than a preset height threshold, so that the landing safety of the UAV is effectively ensured through the segmented control.

[0070] Further, in an optional embodiment of the present application, after the UAV obtains the landing instruction, the flight path is planned according to the relative pose information, and the UAV is controlled to land on the landing platform in stages, further comprising:

[0071] When the tracking target is lost during the flight of the unmanned aerial vehicle tracking the landing platform, the unmanned aerial vehicle is controlled to adjust the flight height and re-search the landing platform.

[0072] Reference Figure 2 In the embodiment of the present application, after the landing platform is locked and the landing instruction is acquired, the unmanned aerial vehicle is controlled to fly to the directly above the landing platform according to the horizontal offset data, the vertical height data, the yaw angle data, the pitch angle data and the roll angle data of the unmanned aerial vehicle relative to the landing platform, to track the landing platform in real time, and when the height data of the unmanned aerial vehicle and the landing platform is detected to be less than a preset height threshold, the unmanned aerial vehicle is controlled to land on the landing platform.

[0073] In the embodiment of the present application, the unmanned aerial vehicle tracks the landing platform, and when the target of the landing platform tracked by the unmanned aerial vehicle is lost for more than 3 seconds, the flight height of the unmanned aerial vehicle is controlled to increase by 200 centimeters, and the landing platform is re-searched.

[0074] In the embodiment of the present application, the flight height of the unmanned aerial vehicle should not exceed the flight height threshold.

[0075] In the embodiment of the present application, when the target of the landing platform tracked by the unmanned aerial vehicle is lost, the flight height of the unmanned aerial vehicle is adjusted, and the landing platform is re-searched, so that the secondary search of the landing platform when the tracking target is lost is realized, and the automatic tracking detection of the landing platform is realized.

[0076] Reference Figure 3 Further, in an optional embodiment of the present application, after the unmanned aerial vehicle acquires the landing instruction, the relative pose information is used for path planning, and the unmanned aerial vehicle is controlled to land on the landing platform in stages, including:

[0077] The path planning instruction is acquired, analyzed and processed by the outer loop controller, the path control instruction is acquired, and the unmanned aerial vehicle is controlled to fly through the inner loop controller;

[0078] The first attitude path information of the unmanned aerial vehicle is collected in real time by the inner loop sensor, the deviation of the first attitude path information and the path control instruction is analyzed, the first compensation instruction is acquired, and the unmanned aerial vehicle is controlled to fly by the inner loop controller optimizing the path control instruction;

[0079] The second attitude path information of the unmanned aerial vehicle is collected in real time by the outer loop sensor, the deviation of the second attitude path information and the path planning instruction is analyzed, the second compensation instruction is acquired, and the unmanned aerial vehicle is controlled to fly by the outer loop controller optimizing the path planning instruction.

[0080] In the embodiment of the present application, the unmanned aerial vehicle is controlled to land on the landing platform by the double-loop control structure according to the acquired path planning instruction.

[0081] In the embodiment of the present application, the outer loop controller analyzes the flight path planning instruction, obtains the flight path control instruction, and controls the flight of the unmanned aerial vehicle through the inner loop controller.

[0082] In the embodiment of the present application, the inner loop sensor collects the first attitude flight path information of the unmanned aerial vehicle in real time, analyzes the deviation between the first attitude flight path information and the flight path control instruction, obtains the first compensation instruction, and controls the flight of the unmanned aerial vehicle by optimizing the flight path control instruction through the inner loop controller, so as to stabilize the flight attitude of the unmanned aerial vehicle.

[0083] In the embodiment of the present application, the outer loop sensor collects the second attitude flight path information of the unmanned aerial vehicle in real time, analyzes the deviation between the second attitude flight path information and the flight path planning instruction, obtains the second compensation instruction, and controls the flight of the unmanned aerial vehicle by optimizing the flight path planning instruction through the outer loop controller, so as to adjust the flight trajectory of the unmanned aerial vehicle in real time.

[0084] In the embodiment of the present application, the flight of the unmanned aerial vehicle according to the flight path planning is controlled through the double-loop structure of the outer loop controller, the outer loop sensor, the inner loop controller and the inner loop sensor, the flight attitude of the unmanned aerial vehicle is effectively stabilized, the flight trajectory is corrected in real time, and dynamic tracking of the landing platform is realized.

[0085] Further, in an optional embodiment of the present application, the target image is processed based on an improved adaptive threshold algorithm to determine the landing platform of the unmanned aerial vehicle, comprising:

[0086] The target image is subjected to grayscale and denoising processing to obtain a preprocessed target image.

[0087] The threshold value is dynamically adjusted based on the improved adaptive threshold algorithm to obtain an optimized binary target image.

[0088] The contour detection and polygon approximation are performed according to the optimized binary target image, the target feature information is extracted, and the matching degree between the target feature information and the preset visual identifier is detected to determine the landing platform position of the unmanned aerial vehicle in the image.

[0089] Reference Figure 4 and Figure 5 In the embodiment of the present application, the grayscale and denoising processing of the target image can reduce environmental interference, and the dynamic adjustment of the threshold value based on the improved adaptive threshold algorithm can eliminate the shadow interference in the preprocessed target image.

[0090] Reference Figure 6 and Figure 7 In the embodiment of the present application, the contour detection and polygon approximation are performed according to the optimized binary target image to obtain a feature contour image, and non-target contours are excluded to obtain target feature information.

[0091] Further, in the embodiment of the present application, the preset visual identifier is designed based on a double-square characteristic.

[0092] Reference Figure 8 In the embodiment of the present application, the preset visual identifier is designed based on a double-square structure and an arrow pattern, and specifically includes a first rectangular frame, a second rectangular frame located in the middle of the second rectangular frame, a first arrow pattern located in the first rectangular frame, and a second arrow pattern located in the middle of the second rectangular frame.

[0093] Based on the preset visual identifier set as above, the rectangular frame in the preset visual identifier can be used to calculate the position parameter of the unmanned aerial vehicle relative to the target, and the arrow in the preset visual identifier can provide angle information for the unmanned aerial vehicle. When the unmanned aerial vehicle is far away from the target, the position information of the unmanned aerial vehicle can be estimated by using the first rectangular frame and the arrow mark outside; when the unmanned aerial vehicle is close to the target, the first mark outside can be out of the field of view of the camera, and at this time, the position information of the unmanned aerial vehicle can be estimated according to the second rectangular frame and the arrow mark inside; in this way, the position information of the unmanned aerial vehicle relative to the target can be obtained more accurately.

[0094] In another optional embodiment, the preset visual identifier is designed based on a double-square structure and an H pattern, and specifically includes a first rectangular frame, a second rectangular frame located in the middle of the second rectangular frame, and an H pattern located in the middle of the second rectangular frame.

[0095] In another optional embodiment, the preset visual identifier is designed based on a concentric circle structure or a two-dimensional code pattern.

[0096] Further, in the embodiment of the present application, the improved adaptive threshold algorithm is represented as:

[0097] T(x, y) = u(x, y) · (1 + k · σ(x, y));

[0098] wherein T(x, y) represents the adaptive threshold value of the pixel point (x, y) in the image, u(x, y) represents the gray mean value in the set window with the pixel point (x, y) as the center, σ(x, y) represents the gray standard deviation in the set window with the pixel point (x, y) as the center, and k represents a dynamic adjustment coefficient.

[0099] wherein the value range of the dynamic adjustment coefficient is 0.1-0.3.

[0100] wherein the size of the set window is set according to actual requirements.

[0101] In the embodiment of the present application, the improved adaptive threshold algorithm dynamically calculates the threshold value of each pixel through local mean and standard deviation, which can effectively eliminate the shadow interference in the image and improve the robustness of image binarization.

[0102] In the embodiment of the present application, the shadow interference in the preprocessed target image is eliminated based on the improved adaptive threshold algorithm to obtain an optimized binary target image, and the contour detection and polygon approximation are performed according to the optimized binary target image, so that the extraction of the target feature information in the target image is realized, and the matching degree of the target feature information and the preset visual identifier is detected to determine the position of the unmanned aerial vehicle landing platform, thereby solving the problems of low image processing efficiency and high target false detection rate in the traditional technology by using the color filtering method or the fixed threshold binary method, and the target image can be effectively recognized and processed under the condition of uneven illumination, shadow shielding or complex background environment, thereby meeting the real-time detection demand in the dynamic environment.

[0103] It should be noted that, in this document, relational terms such as“first” and“second”, and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms“comprises”,“comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Furthermore, the terms“front”,“rear”,“left”,“right”,“upper”, and“lower” as used herein refer to the positions of the elements as shown in the figures.

[0104] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1.A method for controlling a UAV based on multi-sensor fusion, characterized in that, The method comprises the following steps: Collecting target images by a high-speed visual camera mounted on the unmanned aerial vehicle; Processing the target images based on an improved adaptive threshold algorithm to determine a landing platform of the unmanned aerial vehicle; Calculating horizontal offset data, vertical height data, yaw angle data, pitch angle data and roll angle data of the unmanned aerial vehicle relative to the landing platform; Real-time collecting speed and acceleration of the unmanned aerial vehicle by an inertial measurement unit mounted on the unmanned aerial vehicle; Based on a relative position prediction model, real-time updating relative pose information of the unmanned aerial vehicle and the landing platform according to the horizontal offset data, vertical height data, yaw angle data, pitch angle data and roll angle data of the unmanned aerial vehicle relative to the landing platform, and the speed and acceleration of the unmanned aerial vehicle through an interpolation algorithm; After the unmanned aerial vehicle obtains a landing instruction, performing path planning according to the relative pose information, and controlling the unmanned aerial vehicle to land on the landing platform in stages. 2.The multi-sensor fusion based UAV control method of claim 1, wherein, The relative position prediction model is expressed as: wherein P t+1 represents the relative pose of the UAV and the landing platform at time t+1, P t represents the relative pose of the UAV and the landing platform at time t, V t represents the velocity of the UAV at time t, a t represents the acceleration of the UAV at time t, and Δt represents the time difference between time t+1 and time t. 3.The multi-sensor fusion based UAV control method of claim 1, wherein, The controlling the unmanned aerial vehicle to land on the landing platform in stages comprises: controlling the unmanned aerial vehicle to fly above the landing platform, controlling the unmanned aerial vehicle to track the landing platform, and controlling the unmanned aerial vehicle to reduce flight height and land on the landing platform. 4.The multi-sensor fusion based UAV control method of claim 1, wherein, The controlling the unmanned aerial vehicle to land on the landing platform in stages after the unmanned aerial vehicle obtains the landing instruction according to the relative pose information comprises: After the unmanned aerial vehicle obtains the landing instruction, performing task evaluation to determine whether the unmanned aerial vehicle currently meets an instruction execution condition; When it is determined that the unmanned aerial vehicle meets the instruction execution condition, performing path planning according to the horizontal offset data, vertical height data, yaw angle data, pitch angle data and roll angle data of the unmanned aerial vehicle relative to the landing platform, controlling the unmanned aerial vehicle to fly above the landing platform, and controlling the unmanned aerial vehicle to track the landing platform in real time; When it is detected that height data of the unmanned aerial vehicle and the landing platform is less than a preset height threshold, controlling the unmanned aerial vehicle to land on the landing platform. 5.The multi-sensor fusion based UAV control method of claim 4, wherein, The controlling the unmanned aerial vehicle to land on the landing platform in stages after the unmanned aerial vehicle obtains the landing instruction according to the relative pose information further comprises: When tracking target loss occurs in the process that the unmanned aerial vehicle tracks the landing platform, controlling the unmanned aerial vehicle to adjust flight height and re-search the landing platform. 6.The multi-sensor fusion based UAV control method of claim 1, wherein, The controlling the unmanned aerial vehicle to land on the landing platform in stages after the unmanned aerial vehicle obtains the landing instruction according to the relative pose information comprises: Obtaining a path planning instruction, analyzing and processing the path planning instruction by an outer loop controller, obtaining a path control instruction, and controlling the unmanned aerial vehicle to fly through an inner loop controller; Real-time collecting first attitude path information of the unmanned aerial vehicle through an inner loop sensor, analyzing deviation of the first attitude path information from the path control instruction, obtaining a first compensation instruction, and optimizing the path control instruction to control the unmanned aerial vehicle to fly by the inner loop controller; Real-time collecting second attitude path information of the unmanned aerial vehicle through an outer loop sensor, analyzing deviation of the second attitude path information from the path planning instruction, obtaining a second compensation instruction, and optimizing the path planning instruction to control the unmanned aerial vehicle to fly by the outer loop controller. 7.The multi-sensor fusion based UAV control method of claim 1, wherein, The improved adaptive threshold algorithm is expressed as: T(x, y) = u(x, y) · (1 + k · σ(x, y)). Wherein, T(x, y) represents the adaptive threshold value of the pixel point (x, y) in the image, u(x, y) represents the gray mean value in the set window with the pixel point (x, y) as the center, sigma(x, y) represents the gray standard deviation in the set window with the pixel point (x, y) as the center, and k represents the dynamic adjustment coefficient.

Images