Method for generating flight trajectory, control device and unmanned aerial vehicle

Abstract

A method for generating a flight trajectory, a control device (40, 60) and an unmanned aerial vehicle (100), the method comprising: acquiring a specific image (20) and a specific curve (21-22) (S101); translating the back-projection points on the ground of each pixel point on the specific curve (21-22) (S101) to the flight altitude of the unmanned aerial vehicle to obtain a three-dimensional trajectory point set; the three-dimensional trajectory point set includes: the three-dimensional trajectory point set includes the three-dimensional trajectory points corresponding to the respective pixel points corresponding to the specific curve (21-22) (S101) on the specific image (20) in the ground coordinate system; generating a flight trajectory according to the three-dimensional trajectory point set, the flight trajectory being used to control the unmanned aerial vehicle to fly along the flight trajectory. Thereby, the unmanned aerial vehicle (100) can fly according to a specific curve (21-22) designed by the user, realizing the personalized design of the flight mode of the unmanned aerial vehicle (100) and improving the flexibility of the flight mode of the unmanned aerial vehicle (100).

Description

[0001] This application is a divisional application of the application with application number 201680012475.X, application date November 14, 2016, and invention title "Method for Generating Flight Trajectory, Control Device and Unmanned Aerial Vehicle". Technical Field

[0002] Embodiments of the present invention relate to the field of unmanned aerial vehicles, and in particular, to a method for generating a flight trajectory, a control device and an unmanned aerial vehicle. Background Art

[0003] In the prior art, an unmanned aerial vehicle can operate in different modes, which include but are not limited to, point-to-point flight, intelligent following, etc.

[0004] In the point-to-point flight mode, the user can select a flight target by clicking on a point or an area on the display device (such as a screen) of the unmanned aerial vehicle control terminal, and the unmanned aerial vehicle plans the shortest path to fly towards the flight target. In the intelligent following mode, the user can select a movable object (such as a person, an animal, etc.) on the display device (such as a screen) of the unmanned aerial vehicle control terminal to control the unmanned aerial vehicle to fly following the movable object.

[0005] However, the user may hope that the unmanned aerial vehicle flies along a specific trajectory, such as passing through specific points, round-trip flight, etc. In addition, when the user issues a task, there may not be an accurate target point temporarily, but hopes to send the position information of the final target point to the unmanned aerial vehicle after the unmanned aerial vehicle has flown a certain distance, and the existing flight modes cannot meet such requirements, resulting in a lack of personalized design for the flight mode of the unmanned aerial vehicle. Summary of the Invention

[0006] Embodiments of the present invention provide a method for generating a flight trajectory, a control device and an unmanned aerial vehicle to achieve flexible control of the flight mode of the unmanned aerial vehicle.

[0007] One aspect of embodiments of the present invention is to provide a method for generating a flight trajectory, including:

[0008] Obtain a specific image and a specific curve, where the specific curve is a curve drawn on the specific image;

[0009] Generate the specific curve into a flight trajectory according to the specific image and the specific curve, and the flight trajectory is used to control the unmanned aerial vehicle to fly along the flight trajectory.

[0010] Another aspect of embodiments of the present invention is to provide a control device, including one or more processors, working alone or in cooperation, and the one or more processors are used for:

[0011] Obtain a specific image and a specific curve, where the specific curve is a curve drawn on the specific image;

[0012] Generate the specific curve into a flight trajectory according to the specific image and the specific curve, and the flight trajectory is used to control the unmanned aerial vehicle to fly along the flight trajectory.

[0013] Another aspect of the embodiments of the present invention is to provide a control device, including:

[0014] An acquisition module, configured to acquire a specific image and a specific curve, where the specific curve is a curve drawn on the specific image;

[0015] A determination module, configured to generate the specific curve into a flight trajectory according to the specific image and the specific curve, and the flight trajectory is used to control the unmanned aerial vehicle to fly along the flight trajectory.

[0016] Another aspect of the embodiments of the present invention is to provide an unmanned aerial vehicle, including:

[0017] A fuselage;

[0018] A power system, installed on the fuselage, for providing flight power;

[0019] A flight controller, communicatively connected to the power system, for controlling the flight of the unmanned aerial vehicle;

[0020] The flight controller includes the above-mentioned control device.

[0021] The method for generating a flight trajectory, the control device and the unmanned aerial vehicle provided in this embodiment, through a specific curve drawn on a specific image, translate the back-projection points of each pixel point on the specific curve on the ground to the flight altitude of the unmanned aerial vehicle to obtain a three-dimensional trajectory point set; the three-dimensional trajectory point set includes: the three-dimensional trajectory point set includes the three-dimensional trajectory points corresponding to the respective pixel points corresponding to the specific curve on the specific image in the ground coordinate system, so as to generate a flight trajectory for controlling the unmanned aerial vehicle from the specific curve. The specific curve can be a specific curve set by the user on a static picture, or a specific curve set on one frame or multiple frames of images in a dynamic video. Correspondingly, the specific image can be a static picture, or one frame or multiple frames of images in a dynamic video. The specific curve drawn by the user on the specific image can be used to control the flight trajectory of the unmanned aerial vehicle, that is, the unmanned aerial vehicle can fly according to the specific curve designed by the user's personalization, realizing the personalized design of the flight mode of the unmanned aerial vehicle. Compared with the flight modes such as point-to-point flight and intelligent following in the prior art, the flexibility of the flight mode of the unmanned aerial vehicle is improved. Description of the Drawings

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0023] Figure 1 It is a flowchart of the method for generating a flight trajectory provided by an embodiment of the present invention;

[0024] Figure 1A It is a schematic diagram of the coordinate system provided by an embodiment of the present invention;

[0025] Figure 1B It is a schematic diagram of a specific curve set by a user on a planar image provided by an embodiment of the present invention;

[0026] Figure 2 It is a flowchart of the method for generating a flight trajectory provided by another embodiment of the present invention;

[0027] Figure 2A It is a schematic diagram of a projection ray provided by another embodiment of the present invention;

[0028] Figure 3 It is a flowchart of the method for generating a flight trajectory provided by another embodiment of the present invention;

[0029] Figure 3A It is a schematic diagram of three-dimensional trajectory points provided by an embodiment of the present invention;

[0030] Figure 3B It is a schematic diagram of three-dimensional trajectory points provided by an embodiment of the present invention;

[0031] Figure 3C It is a schematic diagram of three-dimensional trajectory points provided by an embodiment of the present invention;

[0032] Figure 3D It is a schematic diagram of three-dimensional trajectory points provided by an embodiment of the present invention;

[0033] Figure 4 It is a structural diagram of a control device provided by an embodiment of the present invention;

[0034] Figure 5 It is a structural diagram of a control device provided by another embodiment of the present invention;

[0035] Figure 6 It is a structural diagram of a control device provided by another embodiment of the present invention;

[0036] Figure 7 It is a structural diagram of an unmanned aerial vehicle provided by an embodiment of the present invention.

[0037] Reference Signs:

[0038] 10 - Image plane 02 - Upper left corner of the image plane

[0039] 01 - Projection point of the optical center 0 on the image plane 10 0 - Optical center of the imaging device

[0040] 03 - Projection point of the optical center 0 on the ground 20 - Specific image

[0041] 21 - Starting point of the specific curve 22 - End point of the specific curve 40 - Control device

[0042] 41 - One or more processors 42 - Sensor 43 - Display screen

[0043] 44 - Transmitter 45 - Receiver 50 - Receiver 51 - Transmitter

[0044] 60 - Control device 61 - Acquisition module 62 - Determination module 621 - Pre - processing unit

[0045] 622 - Determination unit 63 - Display module 64 - Receiving module 65 - Calculation module

[0046] 66 - Detection module 67 - Starting module 68 - Control module 69 - Sending module

[0047] 100 - Unmanned aerial vehicle 107 - Motor 106 - Propeller 117 - Electronic speed controller

[0048] 118 - Flight controller 108 - Sensing system 110 - Communication system

[0049] 102 - Support device 104 - Imaging device 112 - Ground station

[0050] 114 - Antenna 116 - Electromagnetic wave Detailed implementation manners

[0051] Next, the technical solutions in the embodiments of the present invention will be clearly described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0052] It should be noted that when a component is referred to as being "fixed to" another component, it can be directly on the other component or there may also be an intermediate component. When a component is considered to be "connected" to another component, it can be directly connected to the other component or there may be an intermediate component at the same time.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this invention belongs. The terms used in the description of the present invention herein are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0054] The following will describe in detail some embodiments of the present invention with reference to the accompanying drawings. Without conflict, the embodiments described below and the features in the embodiments may be combined with each other.

[0055] An embodiment of the present invention provides a method for generating a flight trajectory. Figure 1 It is a flowchart of the method for generating a flight trajectory provided by an embodiment of the present invention; Figure 1A It is a schematic diagram of a coordinate system provided by an embodiment of the present invention; Figure 1B It is a schematic diagram of a specific curve set by a user on a planar image in an embodiment of the present invention. The execution subject of this embodiment may be a ground station, i.e., a drone control terminal, or a flight controller. In this embodiment, the drone control terminal may include, but is not limited to, a head-mounted display (VR glasses, VR helmets, etc.), a mobile phone, a remote controller (such as a remote controller with a display screen), a smart bracelet, a tablet computer, etc. The unmanned aerial vehicle can work in different modes, and the modes include, but are not limited to, point-to-point flight, intelligent following, camera focusing, etc.

[0056] In the point-to-point flight mode, the user can select a flight target by clicking on a point or an area on the display device (such as a screen) of the drone control terminal, and the unmanned aerial vehicle can fly towards the flight target.

[0057] In the intelligent following mode, the user can control the unmanned aerial vehicle to follow a movable object (such as a person, an animal, etc.) by selecting a movable object on the display device (such as a screen) of the drone control terminal.

[0058] In the camera focusing mode, the user can control the imaging device (such as a camera) of the unmanned aerial vehicle to focus by clicking on a point or an area on the display device (such as a screen) of the drone control terminal.

[0059] The imaging device installed on the unmanned aerial vehicle can perform aerial photography. The image taken by the imaging device corresponds to an image coordinate system, the imaging device itself corresponds to a camera coordinate system, and the unmanned aerial vehicle has a ground coordinate system relative to the ground. The relationship between the image coordinate system, the camera coordinate system, and the ground coordinate system can be reflected by Figure 1A as Figure 1AAs shown in the figure, 10 represents the image plane where the image captured by the imaging device is located. If point 02 is the upper left corner of this image plane, taking point 02 as the coordinate origin, with the positive right direction of this image plane as the X-axis and the positive lower direction of this image plane as the Y-axis, a two-dimensional coordinate system can be established. The two-dimensional coordinate system composed of point 02, the X-axis, and the Y-axis is the image coordinate system.

[0060] If point 0 is the optical center of the imaging device, the X C axis is parallel to the X-axis, and the Y C axis is parallel to the Y-axis. The optical axis of the imaging device is the Z C axis. Then, taking point 0 as the origin, the X C axis, the Y C axis, and the Z C axis form a three-dimensional coordinate system, which is the camera coordinate system. In addition, the projection point of the optical center 0 on the image plane 10 is 01, and the coordinates of point 01 in the image coordinate system are (u0, v0). The distance from the optical center 0 to point 01 is the focal length f of the imaging device.

[0061] If the projection point of the optical center 0 on the ground is 03, taking the unmanned aerial vehicle as a reference object, with the positive right direction of the unmanned aerial vehicle as the X0-axis, the positive front direction of the unmanned aerial vehicle as the Y0-axis, and the direction perpendicular to the ground and upward as the Z0-axis, the three-dimensional coordinate system composed of point 03, the X0-axis, the Y0-axis, and the Z0-axis is the ground coordinate system. As Figure 1A shown, assume that point N is an arbitrary pixel point in the image plane, and the coordinates of pixel point N in the image coordinate system are (u, v). A ray can be formed from the optical center 0 of the imaging device through an arbitrary pixel point in the image plane, such as point N. This ray intersects the ground at a point. Assume the intersection point is P. Then, point P can be used as the back-projection point of pixel point N in the image plane on the ground.

[0062] As Figure 1 shown, the method in this embodiment may include:

[0063] Step S101, obtain a specific image and a specific curve, where the specific curve is a curve drawn on the specific image.

[0064] The execution subject of this embodiment can be a flight controller or a ground station, that is, the UAV control terminal. In this embodiment, the UAV control terminal may include, but is not limited to, head-mounted display glasses (VR glasses, VR helmets, etc.), mobile phones, remote controllers (such as remote controllers with displays), smart bracelets, tablet computers, etc. The unmanned aerial vehicle can work in different modes, and the modes include, but are not limited to, point-and-fly, intelligent following, camera focusing, etc. An imaging device is carried on the unmanned aerial vehicle, and this imaging device can be a camera or a video camera, and this imaging device can achieve aerial photography, and can capture both static pictures and dynamic videos.

[0065] When the execution entity of this embodiment is a ground station, there are various ways for the ground station to obtain a specific image and a specific curve. This embodiment provides at least the following three ways:

[0066] The first way:

[0067] The flight controller sends the real-time images captured by the imaging device, such as static pictures or dynamic videos, to the ground station. The ground station has a display screen. After receiving the static picture or dynamic video, the ground station displays the static picture or dynamic video on the display screen for the user to view. The display screen is a touch screen that can sense operations such as the user's swiping, clicking, touching, and tapping. The user can freely draw a specific curve on the static picture or dynamic video through this display screen, such as Figure 2 As shown, 20 represents a frame of the static picture or dynamic video captured by the imaging device carried by the unmanned aerial vehicle. A frame of the static picture or dynamic video can be a two-dimensional planar image or a three-dimensional image. In this embodiment, a two-dimensional planar image is taken as an example, and the specific picture of the planar image is not shown. The user draws a specific curve on the planar image displayed on this touch screen. For example, a specific curve from the starting point 21 to the ending point 22. The starting point 21 of this specific curve can represent the geographical location where the user is currently located, or it can be a point representing a certain specific location in the planar image. In addition, the ending point 22 of this specific curve can also be any point in the planar image, or it can be a point representing a certain specific location in the planar image. The specific curve drawn by the user from the starting point 21 to the ending point 22 can pass through specific points on the planar image or not pass through specific points on the planar image, and this specific curve is the motion trajectory that the user expects the unmanned aerial vehicle to follow during flight in the air.

[0068] If the user draws a specific curve on a dynamic video, since the dynamic video is composed of frame-by-frame images, the specific curve drawn by the user will be scattered on multiple frames of the dynamic video. Then the specific image can be multiple frames of the dynamic video that make up this specific curve, or it can be one frame of the multiple frames of the dynamic video that make up this specific curve. For example, the ground station can map the specific curve scattered on multiple frames of images to one frame of the multiple frames of images, such as the first frame of image. Then this first frame of image is the specific image including this specific curve. In the following steps, according to the height of the imaging device from the ground when the imaging device captures the first frame of image, the angle of the imaging device relative to the ground, and the coordinates of each pixel point on this specific curve in the image coordinate system where the first frame of image is located, the three-dimensional trajectory points of each pixel point on this specific curve in the ground coordinate system can be calculated. If the user draws a specific curve on a static picture or one frame of a dynamic video, then the specific image is the static picture or one frame of the dynamic video on which this specific curve is drawn.

[0069] The second type:

[0070] Based on the first method, after the ground station obtains a specific graphic and a specific curve, it uploads the specific graphic and the specific curve to the cloud platform. In this embodiment, the cloud platform can be a server, a server cluster, a distributed server, a virtual machine, a virtual machine cluster, etc. Other ground stations communicating with this cloud platform can download the specific graphic and the specific curve from this cloud platform at any time and anywhere. For example, ground station A and ground station B are respectively used to control two different unmanned aerial vehicles. Ground station A controls unmanned aerial vehicle A, and ground station B controls unmanned aerial vehicle B. Suppose ground station B has obtained a specific image and a specific curve through the above first method. Ground station B can upload the specific graphic and the specific curve to the cloud platform. Even if user A and user B have not added each other as friends through the same instant messaging software, as long as ground station A is connected to this cloud platform, user A can download the specific graphic and the specific curve from this cloud platform through ground station A, so that user A can control unmanned aerial vehicle A in the same way as user B controls unmanned aerial vehicle B.

[0071] The third type:

[0072] Ground station A and ground station B are respectively used to control two different unmanned aerial vehicles. For example, ground station A controls unmanned aerial vehicle A, and ground station B controls unmanned aerial vehicle B. Assuming that ground station B has obtained a specific image and a specific curve through the above first method, and there is real-time communication between ground station A and ground station B, then ground station B can share the specific image and the specific curve with ground station A, so that ground station A can control the flight trajectory of unmanned aerial vehicle A according to the specific image and the specific curve. For example, both ground station A and ground station B are tablet computers, and instant messaging software is installed on the two tablet computers respectively. User A operates ground station A, and user B operates ground station B. User A and user B log in to the same instant messaging software through their respective tablet computers, and user A and user B add each other as friends through the same instant messaging software. When user B obtains a specific image and a specific curve through ground station B using the above first method, and ground station B can control the flight trajectory of unmanned aerial vehicle B smoothly and power-savingly according to the specific image and the specific curve, then user B shares the specific image and the specific curve with user A through the instant messaging software on ground station B, so that user A can control unmanned aerial vehicle A in the same way as user B controls unmanned aerial vehicle B. In addition, ground station B can not only share the specific image and the specific curve with ground station A, but also share them with other ground stations, so that other ground stations can control their respective unmanned aerial vehicles to fly along the same trajectory. For example, in some celebration activities, this method can be used to control multiple unmanned aerial vehicles to fly along the same flight trajectory in chronological order. In addition, after ground station B shares the specific image and the specific curve with ground station A, the user corresponding to ground station A can also change the flight altitude of the unmanned aerial vehicle through ground station A, so as to control the unmanned aerial vehicle to fly along this flight trajectory at different altitudes. When multiple ground stations share the specific image and the specific curve sent by ground station B, these multiple ground stations can control their respective unmanned aerial vehicles to fly along the same flight trajectory at different altitudes, so as to achieve a shocking viewing effect.

[0073] When the execution subject of this embodiment is a flight controller, the flight controller obtains a specific image and a specific curve from the ground station through wireless transmission. The way for the ground station to obtain the specific image and the specific curve can be any one of the above three methods. Specifically, the ground station sends the specific image and the specific curve to the communication system of the unmanned aerial vehicle, and then the communication system transmits the specific image and the specific curve to the flight controller.

[0074] In addition, optionally, when the ground station or the flight controller acquires a specific image, it includes acquiring the height of the unmanned aerial vehicle relative to the ground when the imaging device captures the specific image, the angle of the imaging device relative to the ground, the position of the imaging device in the ground coordinate system, and the focal length of the imaging device. Among them, the angle of the imaging device relative to the ground includes at least one of the roll angle, pitch angle, and yaw angle of the imaging device. For example, when the flight controller sends the real-time image captured by the imaging device, such as a static picture or a dynamic video, to the ground station, the flight controller acquires the height of the unmanned aerial vehicle relative to the ground when the imaging device captures the real-time image, the angle of the imaging device relative to the ground, the position of the imaging device in the ground coordinate system, and the focal length of the imaging device, and stores the height of the unmanned aerial vehicle relative to the ground when the imaging device captures the real-time image, the angle of the imaging device relative to the ground, the position of the imaging device in the ground coordinate system, and the focal length of the imaging device in the memory of the unmanned aerial vehicle, or sends them to the ground station.

[0075] Step S102: Generate the specific curve into a flight trajectory according to the specific image and the specific curve, and the flight trajectory is used to control the unmanned aerial vehicle to fly along the flight trajectory.

[0076] In this embodiment, the flight controller can generate the specific curve into a flight trajectory according to the specific image and the specific curve, or the ground station can generate the specific curve into a flight trajectory according to the specific image and the specific curve. Specifically, since a planar image is composed of pixel points, each pixel point corresponds to a coordinate in the image coordinate system, and at the same time, the value of each pixel point represents the gray level or brightness of the pixel point. As Figure 1B shown, for the specific image 20, the specific curve from the starting point 21 to the ending point 22 is also composed of pixel points. If the Figure 1B shown specific image 20 is used as Figure 1A shown image plane 10, then for any pixel point on the specific curve 21-22, a ray can be formed from the optical center 0 of the imaging lens of the imaging device through this pixel point, and this ray intersects the ground at a point. The intersection point formed by this ray and the ground is the back-projection point of this pixel point on the ground. Thus, each pixel point on the specific curve 21-22 can be back-projected onto the ground to obtain the back-projection point of each pixel point on the ground. Since the unmanned aerial vehicle flies in the air at a certain height from the ground, therefore, by translating the back-projection point of each pixel point on the specific curve 21-22 on the ground to the flight height of the unmanned aerial vehicle when the imaging device captures the specific image, the three-dimensional coordinate points of each pixel point in the three-dimensional space, that is, the ground coordinate system, can be obtained. In this embodiment, the three-dimensional coordinate points are denoted as three-dimensional trajectory points.

[0077] According to the previous step, the user can draw a specific curve on a dynamic video, or draw a specific curve on a static picture or a frame of an image in a dynamic video. When the user draws a specific curve on a dynamic video, the specific curve will be scattered on multiple frames of the dynamic video, that is, the pixel points constituting the specific curve are distributed on multiple frames of the dynamic video. In this example, when determining the back-projection points of each pixel point on the ground, as Figure 1A shown, the specific image 20 of the image plane 10 can be the frame image where each pixel point is located, or any frame image of the multiple frames of the dynamic video where the specific curve is scattered. This any frame image can be the first frame image, the middle frame, or the last frame image of the multiple frame images.

[0078] The three-dimensional trajectory points corresponding to each pixel point respectively form a set of three-dimensional trajectory points. By using a trajectory generation algorithm for the set of three-dimensional trajectory points, a three-dimensional trajectory can be generated. The three-dimensional trajectory generated by using the trajectory generation algorithm satisfies the kinematic constraints of the unmanned aerial vehicle. This trajectory generation algorithm can be any algorithm in the prior art for generating a trajectory based on multiple trajectory points. Optionally, the trajectory generation algorithm selected in this embodiment is the minimum snap trajectory generation algorithm. The three-dimensional trajectory generated by using the minimum snap trajectory generation algorithm not only satisfies the kinematic constraints of the unmanned aerial vehicle, but also satisfies the smoothness constraint.

[0079] This three-dimensional trajectory can be used to control the flight of the unmanned aerial vehicle. Specifically, the unmanned aerial vehicle is controlled to fly along this three-dimensional trajectory. In this embodiment, the three-dimensional trajectory is the flight trajectory followed by the unmanned aerial vehicle when controlling its flight.

[0080] If the execution subject of this embodiment is a flight controller, after the flight controller generates the specific curve into a flight trajectory according to the specific image and the specific curve, it controls the unmanned aerial vehicle to fly in the air along the flight trajectory according to this flight trajectory. If the execution subject of this embodiment is a ground station, after the ground station generates the specific curve into a flight trajectory according to the specific image and the specific curve, it sends the flight trajectory to the flight controller so that the flight controller can control the unmanned aerial vehicle to fly in the air along the flight trajectory according to this flight trajectory.

[0081] In addition, in other embodiments, the flight controller or the ground station can also upload the flight trajectory to a specific server so that other flight controllers or other ground stations can directly download the flight trajectory from this specific server and control the flight of other unmanned aerial vehicles according to this flight trajectory. Or, when the execution subject of the flight trajectory generation method is a first ground station, the first ground station can also share the flight trajectory with a second ground station so that other ground stations can control the flight of other unmanned aerial vehicles according to this flight trajectory.

[0082] In this embodiment, a specific curve drawn on a specific image is used to generate a flight trajectory for controlling an unmanned aerial vehicle. The specific curve can be a specific curve set by a user on a static picture, or a specific curve set on one or more frames of an image in a dynamic video. Correspondingly, the specific image can be a static picture, or one or more frames of an image in a dynamic video. The specific curve drawn by the user on the specific image can be used to control the flight trajectory of the unmanned aerial vehicle, that is, the unmanned aerial vehicle can fly along the specific curve designed by the user in a personalized manner, realizing the personalized design of the flight mode of the unmanned aerial vehicle. Compared with the flight modes such as point-to-point flight and intelligent following in the prior art, the flexibility of the flight mode of the unmanned aerial vehicle is improved.

[0083] An embodiment of the present invention provides a method for generating a flight trajectory. Figure 2 It is a flowchart of a method for generating a flight trajectory provided by another embodiment of the present invention; Figure 2A It is a schematic diagram of a projection ray provided by another embodiment of the present invention. As Figure 2 shown, on the basis of the embodiment shown in Figure 1 The method for generating the specific curve into a flight trajectory according to the specific image and the specific curve may include:

[0084] Step S201, obtain the height of the imaging device from the ground when shooting the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device.

[0085] According to the above embodiment, when the specific image 20 is used as the Figure 1A shown image plane 10, point 0 is the optical center of the camera lens of the imaging device carried by the unmanned aerial vehicle. The projection point of the optical center 0 on the specific image 20 is 01, and the coordinates of point 01 in the image coordinate system where the specific image 20 is located are (u0, v0). The distance from the optical center 0 to point 01 is the focal length f of the imaging device. Point N is an arbitrary pixel point on the specific curve 21-22 in the specific image 20, and the coordinates of the pixel point N in the image coordinate system where the specific image 20 is located are (u, v). A ray can be formed from the optical center 0 of the camera lens of the imaging device through an arbitrary pixel point on the specific curve 21-22, such as point N, and this ray intersects the ground at a point. Assuming the intersection point is P, then point P can be used as the back-projection point of the pixel point N on the specific curve 21-22 on the ground.

[0086] As Figure 2AAs shown, point 0 is the optical center of the camera lens of the imaging device carried by the unmanned aerial vehicle, point P is the back-projection point on the ground of pixel point N on the specific curve 21-22, and the straight line where the optical center 0 and point P are located is the projection straight line denoted as OP. The height of the imaging device relative to the ground is the height of the optical center of the imaging device relative to the ground, that is, as Figure 2A shown, the height H, and the pitch angle of the imaging device relative to the ground is the angle θ as Figure 2A shown.

[0087] Step S202: Determine a three-dimensional trajectory point set according to the height of the imaging device from the ground when shooting the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device. The three-dimensional trajectory point set includes the three-dimensional trajectory points corresponding to each pixel point corresponding to the specific curve on the specific image in the ground coordinate system.

[0088] Specifically, the method for determining the three-dimensional trajectory point set according to the height of the imaging device from the ground when shooting the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device may include the following steps:

[0089] 1) Determine the back-projection point of the pixel point on the ground. The back-projection point is the intersection point of the projection ray passing through the optical center of the camera lens of the imaging device and the pixel point and the ground.

[0090] 2) Determine the coordinate position of the back-projection point in the camera coordinate system according to the coordinates of the pixel point in the image coordinate system where the specific image is located and the focal length of the imaging device.

[0091] Specifically, according to the coordinates (u, v) of pixel point N in the image coordinate system where the specific image 20 is located, the coordinates (u0, v0) of point 01 in the image coordinate system where the specific image 20 is located, the focal length f of the imaging device, and the height H of the imaging device relative to the ground, the coordinate position x of the back-projection point P of pixel point N on the specific curve 21-22 on the ground in the camera coordinate system can be determined by using formula (1):

[0092] x = k(u - u0, v - v0, f) T (1)

[0093] where k is a parameter characterizing the depth of field of the plane image, k is related to the height H of the imaging device relative to the ground, and the greater the height H of the imaging device relative to the ground, the greater k is.

[0094] 3) Determine the coordinate position of the back-projected point in the ground coordinate system based on the coordinate position of the back-projected point in the camera coordinate system;

[0095] Specifically, one feasible way to determine the coordinate position of the back-projected point in the ground coordinate system based on the coordinate position of the back-projected point in the camera coordinate system is as follows: Determine the extrinsic parameters of the camera coordinate system relative to the ground coordinate system based on the height of the imaging device from the ground when taking the specific image and the angle of the imaging device relative to the ground; Determine the coordinate position of the back-projected point in the ground coordinate system based on the coordinate position of the back-projected point in the camera coordinate system and the extrinsic parameters of the camera coordinate system relative to the ground coordinate system.

[0096] Since there is a conversion relationship between the camera coordinate system and the ground coordinate system, specifically, the relationship between the camera coordinate system and the ground coordinate system can be represented by the rotation matrix R and the translation vector t. The rotation matrix R and the translation vector t are the extrinsic parameters of the camera coordinate system relative to the ground coordinate system. Determine the rotation matrix R and the translation vector t respectively according to formulas (2) and (3):

[0097]

[0098]

[0099] Where, H represents the height of the imaging device relative to the ground. In this embodiment, the height of the imaging device relative to the ground is approximately the height of the optical center 0 of the camera lens of the imaging device relative to the ground, and θ represents the pitch angle of the imaging device relative to the ground.

[0100] According to formulas (1), (2), and (3), the coordinates of the back-projected point in the camera coordinate system can be converted into the coordinates of the back-projected point in the ground coordinate system, and the coordinates of the back-projected point in the ground coordinate system can be expressed as formula (4)

[0101] x = kR(-θ)(u - u 0 , v - v 0 , f) T + t (4)

[0102] For formula (4), let the z-axis coordinate x z = 0, calculate k, and then substitute k into formula (4) to obtain the coordinates of the back-projected point P in the ground coordinate system.

[0103] Similarly to the back-projected point P, the coordinates of the back-projected point on the ground of any pixel point on the specific curve 21 - 22 in the specific image 20 in the ground coordinate system can be obtained. In addition, the specific shape of the specific curve 21 - 22 is not limited in this embodiment.

[0104] 4) Determine the three-dimensional trajectory point corresponding to the pixel point in the ground coordinate system based on the height from the ground when the imaging device captures the specific image and the coordinate position of the back-projection point in the ground coordinate system.

[0105] After determining the coordinates of the back-projection point of any pixel point on the specific curve 21-22 in the specific image 20 in the ground coordinate system according to the above steps, in the ground coordinate system, translate each back-projection point to the flight height of the unmanned aerial vehicle, and the three-dimensional coordinate points of each pixel point in the three-dimensional space, that is, the ground coordinate system, can be obtained. Since the three-dimensional coordinate points are the points that make up the flight trajectory of the unmanned aerial vehicle, in this embodiment, the three-dimensional coordinate points are denoted as three-dimensional trajectory points. The three-dimensional trajectory points corresponding to each pixel point respectively constitute a three-dimensional trajectory point set.

[0106] Step S203: Generate a flight trajectory according to the three-dimensional trajectory point set.

[0107] Using a trajectory generation algorithm for the three-dimensional trajectory point set, a three-dimensional trajectory can be generated. The three-dimensional trajectory generated by using the trajectory generation algorithm satisfies the kinematic constraints of the unmanned aerial vehicle. This trajectory generation algorithm can be any algorithm in the prior art for generating a trajectory based on multiple trajectory points. Optionally, the trajectory generation algorithm selected in this embodiment is the minimum snap trajectory generation algorithm. The three-dimensional trajectory generated by using the minimum snap trajectory generation algorithm not only satisfies the kinematic constraints of the unmanned aerial vehicle, but also satisfies the smoothness constraint.

[0108] This three-dimensional trajectory can be used to control the flight of the unmanned aerial vehicle. Specifically, control the unmanned aerial vehicle to fly along this three-dimensional trajectory. In this embodiment, the three-dimensional trajectory is the flight trajectory followed by the unmanned aerial vehicle when controlling its flight.

[0109] In this embodiment, based on any pixel point on the specific curve and the optical center of the camera lens of the imaging device, the back-projection points of each pixel point on the specific curve on the ground are determined. According to the height, angle of the imaging device relative to the ground, and the focal length of the imaging device, the coordinate position of the back-projection point in the camera coordinate system and the external parameters of the camera coordinate system relative to the ground coordinate system are determined. According to the coordinate position of the back-projection point in the camera coordinate system and the external parameters of the camera coordinate system relative to the ground coordinate system, the coordinate position of the back-projection point in the ground coordinate system is determined. According to the coordinate position of the back-projection point in the ground coordinate system, the coordinates of the three-dimensional trajectory points can be accurately calculated, realizing the accurate calculation of the three-dimensional trajectory, that is, the flight trajectory, and thus realizing the accurate control of the unmanned aerial vehicle.

[0110] An embodiment of the present invention provides a method for generating a flight trajectory. Figure 3Flowchart of the method for generating a flight trajectory provided in another embodiment of the present invention; Figure 3A Schematic diagram of three-dimensional trajectory points provided in an embodiment of the present invention; Figure 3B Schematic diagram of three-dimensional trajectory points provided in an embodiment of the present invention; Figure 3C Schematic diagram of three-dimensional trajectory points provided in an embodiment of the present invention; Figure 3D Schematic diagram of three-dimensional trajectory points provided in an embodiment of the present invention. As Figure 3 shown, based on the embodiment shown in Figure 2 the method for generating a flight trajectory according to the three-dimensional trajectory point set may include:

[0111] Step S301: Preprocess the three-dimensional trajectory point set to obtain a preprocessed three-dimensional trajectory point set.

[0112] Due to the randomness of the user's drawing of a specific curve, the specific curve may not necessarily meet the motion performance constraints of the unmanned aerial vehicle. Therefore, it is necessary to preprocess each three-dimensional trajectory point, that is, the three-dimensional trajectory point set, determined in the above embodiment. The purpose of the preprocessing is to ensure that the flight trajectory formed by the preprocessed three-dimensional trajectory point set meets the kinematic constraints of the unmanned aerial vehicle. In this embodiment, the method for preprocessing each three-dimensional trajectory point may include at least one of the following:

[0113] 1) Obtain the maximum flight distance of the unmanned aerial vehicle, and preprocess the three-dimensional trajectory point set according to the maximum flight distance.

[0114] Specifically, calculate the length of the three-dimensional trajectory formed by the three-dimensional trajectory point set; if the length of the three-dimensional trajectory formed by the three-dimensional trajectory point set is greater than the maximum flight distance, delete some three-dimensional trajectory points in the three-dimensional trajectory point set so that the length of the three-dimensional trajectory formed by the remaining three-dimensional trajectory points in the three-dimensional trajectory point set is less than the maximum flight distance of the unmanned aerial vehicle.

[0115] According to the above embodiments, each three-dimensional trajectory point corresponds to a three-dimensional coordinate in the ground coordinate system. Based on the three-dimensional coordinates of each three-dimensional trajectory point, the distance between every two adjacent three-dimensional trajectory points can be calculated, and the sum of the distances between every two adjacent three-dimensional trajectory points is the total length of the three-dimensional trajectory formed by the three-dimensional trajectory point set. Since the maximum flight distance of the unmanned aerial vehicle is limited, if the total length of the three-dimensional trajectory is greater than the maximum flight distance of the unmanned aerial vehicle, it is necessary to limit the flight distance of the unmanned aerial vehicle. The specific way of limitation can be to delete some three-dimensional trajectory points in the three-dimensional trajectory point set. For example, delete the three-dimensional trajectory points at the beginning part or the end part of the three-dimensional trajectory point set. It can also be to delete one or two three-dimensional trajectory points every other one within the preset range of the three-dimensional trajectory point set, so that the total length of the three-dimensional trajectory formed by the remaining three-dimensional trajectory points in the three-dimensional trajectory point set is less than or equal to the maximum flight distance of the unmanned aerial vehicle. In this embodiment, the maximum flight distance of the unmanned aerial vehicle can be the curved distance that the unmanned aerial vehicle flies along the three-dimensional trajectory of the curve, or the straight-line distance between the starting three-dimensional trajectory point and the ending three-dimensional trajectory point.

[0116] 2) Obtain the density of at least some consecutive three-dimensional trajectory points in the three-dimensional trajectory point set, and preprocess the at least some consecutive three-dimensional trajectory points according to the density.

[0117] Specifically, determine the number of three-dimensional trajectory points in the three-dimensional trajectory point set that are located within the preset range; if the number of three-dimensional trajectory points within the preset range is greater than the threshold, reduce the number of three-dimensional trajectory points within the preset range, or obtain substitute points within the preset range and replace all the three-dimensional trajectory points within the preset range with the substitute points within the preset range. If the number of three-dimensional trajectory points within the preset range is less than or equal to the threshold, increase the number of three-dimensional trajectory points within the preset range, that is, increase the number of three-dimensional trajectory points in the locally low-density range of the three-dimensional trajectory point set.

[0118] For example, when a user depicts a specific curve, the pixel points at the starting part of the specific curve may be relatively dense, that is, there are many pixel points within a short distance, resulting in the three-dimensional trajectory points corresponding to the pixel points at the starting part of the specific curve being relatively dense in the ground coordinate system. To determine the density of the three-dimensional trajectory points in the ground coordinate system, in this embodiment, the number of three-dimensional trajectory points located within a preset range is determined in the ground coordinate system; if the number of the three-dimensional trajectory points within the preset range is greater than a threshold, the number of the three-dimensional trajectory points within the preset range is reduced, or alternatively, replacement points within the preset range are obtained, and all the three-dimensional trajectory points within the preset range are replaced with the replacement points within the preset range. The replacement point can be one or more three-dimensional trajectory points within the preset range, or the center point or centroid of the geometric figure formed by all the three-dimensional trajectory points within the preset range, or the center point or centroid of the geometric plane formed by some of the three-dimensional trajectory points within the preset range.

[0119] 3) Obtain the jitter degree of a specific three-dimensional trajectory point in the three-dimensional trajectory point set, and preprocess the specific three-dimensional trajectory point according to the jitter degree.

[0120] Specifically, if the jitter degree of the specific three-dimensional trajectory point is less than the threshold, the specific three-dimensional trajectory point is removed; and / or, if the jitter degree of the specific three-dimensional trajectory point is not less than the threshold, the specific three-dimensional trajectory point is retained.

[0121] The jitter degree of the specific three-dimensional trajectory point is determined according to the distance from the next three-dimensional trajectory point of the specific three-dimensional trajectory point to the straight line where the specific three-dimensional trajectory point and the previous three-dimensional trajectory point of the specific three-dimensional trajectory point are located.

[0122] For example, when a user depicts a specific curve, jitter may occur, resulting in multiple locally curved segments in the specific curve depicted. To reduce the jitter degree of the specific curve, in this embodiment, the three-dimensional trajectory points with a smaller jitter degree can be removed.

[0123] Such as Figure 3AAs shown, points A, B, C, and D are the three-dimensional trajectory points of four adjacent pixel points on a specific curve in the ground coordinate system. Point A is the previous three-dimensional trajectory point of point B, and point C is the next three-dimensional trajectory point of point B. Similarly, point B is the previous three-dimensional trajectory point of point C, and point D is the next three-dimensional trajectory point of point C. Draw a perpendicular line from point C to the line where points A and B are located, and the perpendicular line intersects the extension line of AB at point C1. The distance between point C and point C1 can be used to characterize the jitter degree of point B. If the distance between point C and point C1 is less than the threshold, it means that the jitter degree of the three-dimensional trajectory point B is less than the threshold, then remove point B. If the distance between point C and point C1 is greater than the threshold, then retain the three-dimensional trajectory point B. In this embodiment, assume that the distance between point C and point C1 is less than the threshold, then as Figure 3B shown, remove the three-dimensional trajectory point B.

[0124] As Figure 3B shown, after removing the three-dimensional trajectory point B, draw a perpendicular line from point D to the line where points A and C are located, and the perpendicular line intersects the extension line of AC at point D1. The distance between point D and point D1 can be used to characterize the jitter degree of point C. If the distance between point D and point D1 is less than the threshold, it means that the jitter degree of the three-dimensional trajectory point C is less than the threshold, then remove point C. If the distance between point D and point D1 is greater than the threshold, then retain the three-dimensional trajectory point C. In this embodiment, assume that the distance between point D and point D1 is greater than the threshold, then retain the three-dimensional trajectory point C, and starting from the three-dimensional trajectory point C, continue the method of judging the jitter degree of each subsequent three-dimensional trajectory point starting from point A, and judge the jitter degree of each subsequent three-dimensional trajectory point after the three-dimensional trajectory point C until all the three-dimensional trajectory points are traversed.

[0125] 4) Generate a three-dimensional trajectory based on at least some consecutive three-dimensional trajectory points in the three-dimensional trajectory point set, and preprocess the at least some consecutive three-dimensional trajectory points according to the curvature of the three-dimensional trajectory.

[0126] Specifically, if the curvature of the three-dimensional trajectory at the first three-dimensional trajectory point is greater than the threshold, then obtain a replacement point, where the first three-dimensional trajectory point is one of the at least some consecutive three-dimensional trajectory points, and the curvature of the curve formed by the replacement point and the two adjacent three-dimensional trajectory points before and after the first three-dimensional trajectory point at the replacement point is less than the curvature of the three-dimensional trajectory at the first three-dimensional trajectory point; replace the first three-dimensional trajectory point with the replacement point.

[0127] The obtaining of the substitute point includes: obtaining a first intermediate point between the first three-dimensional trajectory point and the previous three-dimensional trajectory point of the first three-dimensional trajectory point, and obtaining a second intermediate point between the first three-dimensional trajectory point and the next three-dimensional trajectory point of the first three-dimensional trajectory point, where the first intermediate point and the second intermediate point are the substitute points; or, obtaining the center or centroid of a triangle formed by the first three-dimensional trajectory point, the previous three-dimensional trajectory point of the first three-dimensional trajectory point, and the next three-dimensional trajectory point of the first three-dimensional trajectory point, where the center or centroid of the triangle is the substitute point.

[0128] For example, when an unmanned aerial vehicle turns, the adjustment of its angle is limited. When the curvature of the curve formed by the three-dimensional trajectory points is large, the unmanned aerial vehicle will not be able to fly strictly along the flight trajectory. Therefore, when preprocessing each three-dimensional trajectory point, it is necessary to remove the points with large curvature in order to obtain a smooth flight trajectory, so that the unmanned aerial vehicle flies along the smooth flight trajectory.

[0129] As Figure 3C shown, points A, B, and C are three adjacent three-dimensional trajectory points. Point A is the previous three-dimensional trajectory point of point B, and point C is the next three-dimensional trajectory point of point B. Connect points A, B, and C with a smooth curve. According to the mathematical formula, the curvature of curve ABC at point B can be calculated. If the curvature of curve ABC at point B is greater than the threshold, point B needs to be removed. If the curvature of curve ABC at point B is less than the threshold, then point B is retained. According to Figure 3C shown, the curvature of curve ABC at point B is large, and curve ABC is relatively steep at point B, making curve ABC not smooth. Therefore, in order to make the unmanned aerial vehicle fly along a smooth trajectory, a substitute point can be obtained, and the substitute point replaces point B, so that the curvature of the curve formed by point A, point C, and the substitute point at the substitute point is less than the curvature of curve ABC at point B. In this embodiment, the substitute point can be one point or multiple points.

[0130] Optionally, take the midpoint D of line segment AB and the midpoint E of line segment BC, and use midpoint D and midpoint E to replace point B, that is, remove point B and supplement midpoint E and midpoint D. The curve ADEC formed by point A, point D, point E, and point C is much smoother than curve ABC.

[0131] In addition, as Figure 3D shown, the center or centroid G of the triangle formed by points A, B, and C can also be used to replace point B, because the curvature of the curve formed by point A, point C, and the center or centroid G of triangle ABC at the center or centroid G is less than the curvature of curve ABC at point B.

[0132] In addition, for each three-dimensional trajectory point other than points A, B, and C, the curvature judgment and preprocessing are carried out in the same way.

[0133] Step S302: According to the preprocessed three-dimensional trajectory point set, use a trajectory generation algorithm to determine the flight trajectory, and the flight trajectory satisfies the kinematic constraints of the unmanned aerial vehicle.

[0134] After the above preprocessing, a preprocessed three-dimensional trajectory point set can be obtained. For the preprocessed three-dimensional trajectory point set, by using a trajectory generation algorithm, a flight trajectory that satisfies the kinematic constraints of the unmanned aerial vehicle can be obtained. In this embodiment, the trajectory generation algorithm can be a minimum-jerk trajectory generation algorithm. The flight trajectory generated by using the minimum-jerk trajectory generation algorithm not only satisfies the kinematic constraints of the unmanned aerial vehicle, but also satisfies the smoothness constraints of the unmanned aerial vehicle.

[0135] In addition, when the unmanned aerial vehicle flies along the flight trajectory, detect whether there are obstacles in the part of the flight trajectory in front of the unmanned aerial vehicle; if there are obstacles in the part of the flight trajectory in front of the unmanned aerial vehicle, activate the obstacle avoidance function of the unmanned aerial vehicle; after the unmanned aerial vehicle bypasses the obstacle, control the unmanned aerial vehicle to return to the flight trajectory.

[0136] After obtaining the flight trajectory that satisfies the kinematic constraints and smoothness constraints according to the above steps, the flight controller controls the unmanned aerial vehicle to fly along the flight trajectory. When the unmanned aerial vehicle flies along the flight trajectory, the radar device installed on the unmanned aerial vehicle can be used to detect whether there are obstacles in the part of the flight trajectory in front of the unmanned aerial vehicle. If there are obstacles, activate the obstacle avoidance function of the unmanned aerial vehicle. After the unmanned aerial vehicle successfully avoids the obstacle, the flight controller controls the unmanned aerial vehicle to fly back to the flight trajectory again.

[0137] In this embodiment, before determining the flight trajectory according to the three-dimensional trajectory point set, each three-dimensional trajectory point in the three-dimensional trajectory point set is preprocessed. The purpose of the preprocessing is to ensure that the flight trajectory formed by the preprocessed three-dimensional trajectory point set satisfies the motion performance constraints of the unmanned aerial vehicle, and solves the problem that the specific curve set by the user on the specific image does not satisfy the motion performance constraints of the unmanned aerial vehicle due to the randomness of the user's description of the specific curve; in addition, when the unmanned aerial vehicle flies along the flight trajectory, the radar installed on the unmanned aerial vehicle is used to detect whether there are obstacles in the part of the flight trajectory in front of the unmanned aerial vehicle. If there are obstacles, activate the obstacle avoidance function of the unmanned aerial vehicle, so that the unmanned aerial vehicle successfully bypasses the obstacle. After the unmanned aerial vehicle successfully bypasses the obstacle, the flight controller controls the unmanned aerial vehicle to continue flying along the flight trajectory, ensuring the safety of the unmanned aerial vehicle.

[0138] An embodiment of the present invention provides a control device. Figure 4The structural diagram of the control device provided by the embodiment of the present invention is as follows Figure 4 As shown, the control device 40 includes one or more processors 41, working alone or in cooperation, and a sensor 42; wherein, the one or more processors 41 are configured to: obtain a specific image and a specific curve, wherein the specific curve is a curve drawn on the specific image; generate the specific curve into a flight trajectory according to the specific image and the specific curve, and the flight trajectory is used to control the unmanned aerial vehicle to fly along the flight trajectory.

[0139] Specifically, the control device 40 is a ground station or a flight controller.

[0140] When the control device 40 is a ground station, or when the ground station includes the control device 40, optionally, the control device 40 further includes: a transmitter 44 communicatively connected to the one or more processors 41, and the transmitter 44 is configured to send the flight trajectory to the flight controller of the unmanned aerial vehicle.

[0141] When the control device is a flight controller, or when the flight controller includes the control device, optionally, the control device further includes: a receiver communicatively connected to the one or more processors, and the receiver is configured to receive the flight trajectory sent by the ground station, and the one or more processors are further configured to control the unmanned aerial vehicle to fly along the flight trajectory.

[0142] In an embodiment of the present invention, when the control device 40 is a ground station, or when the ground station includes the control device 40, the one or more processors 41 are configured to obtain a real-time image captured by an imaging device carried on the unmanned aerial vehicle; the control device 40 further includes: a display screen 43, and the display screen 43 is configured to display the real-time image; and sense a specific curve drawn on the real-time image displayed on the display screen; the one or more processors 41 are configured to obtain the specific curve and the specific image, and the specific image includes at least a part of the real-time image where the specific curve is located.

[0143] There are two ways to implement the one or more processors 41 to obtain the specific curve and the specific image as follows:

[0144] 1) The one or more processors 41 download the specific image and the specific curve from the cloud platform;

[0145] 2) The control device 40 is a first ground station, or the first ground station includes the control device 40, and the control device 40 further includes: a receiver 45 communicatively connected to the one or more processors 41, and the receiver 45 is configured to receive the specific image and the specific curve sent by a second ground station.

[0146] The specific principle and implementation manner of the flight controller provided by the embodiment of the present invention are both Figure 1Similar to the illustrated embodiments, details will not be described herein again.

[0147] In this embodiment, a specific curve drawn on a specific image is used to generate a flight trajectory for controlling the unmanned aerial vehicle. The specific curve can be a specific curve set by the user on a static picture, or a specific curve set on one or more frames of an image in a dynamic video. Correspondingly, the specific image can be a static picture, or one or more frames of an image in a dynamic video. The specific curve drawn by the user on the specific image can be used to control the flight trajectory of the unmanned aerial vehicle, that is, the unmanned aerial vehicle can fly along the specific curve designed by the user's personalization, realizing the personalized design of the flight mode of the unmanned aerial vehicle. Compared with the flight modes such as point-to-point flight and intelligent following in the prior art, the flexibility of the flight mode of the unmanned aerial vehicle is improved.

[0148] An embodiment of the present invention provides a control device. Figure 5 It is a structural diagram of the control device provided by another embodiment of the present invention. In this embodiment, the control device 40 is a flight controller, or the flight controller includes the control device 40. Based on including one or more processors 41, working alone or in cooperation, and a sensor 42, the control device 40 further includes: a receiver 50 communicatively connected to one or more processors 41, and the receiver 50 is used to receive the specific image and the specific curve sent by the ground station. One or more processors 41 are also used to control the unmanned aerial vehicle to fly along the flight trajectory. In this embodiment, the way for one or more processors 41 to obtain the specific image and the specific curve can be to obtain the specific image and the specific curve from the ground station, or to download the specific image and the specific curve from the cloud platform.

[0149] In addition, the control device 40 further includes: a transmitter 51 communicatively connected to one or more processors 41, and the transmitter 51 is used to send the real-time image captured by the imaging device carried on the unmanned aerial vehicle to the ground station.

[0150] One or more processors 41 are specifically configured to obtain a specific image and a specific curve in the following ways: obtaining the height of the imaging device from the ground when taking the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device; one or more processors 41 are specifically configured to generate the specific curve into a flight trajectory based on the specific image and the specific curve in the following ways: determining a three-dimensional trajectory point set according to the height of the imaging device from the ground when taking the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device, where the three-dimensional trajectory point set includes the three-dimensional trajectory points corresponding to each pixel point corresponding to the specific curve on the specific image in the ground coordinate system; generating a flight trajectory based on the three-dimensional trajectory point set.

[0151] An achievable way for one or more processors 41 to generate a flight trajectory based on the three-dimensional trajectory point set is: preprocessing the three-dimensional trajectory point set to obtain a preprocessed three-dimensional trajectory point set; determining the flight trajectory according to the preprocessed three-dimensional trajectory point set by using a trajectory generation algorithm, where the flight trajectory satisfies the kinematic constraints of the unmanned aerial vehicle.

[0152] The ways for one or more processors 41 to preprocess the three-dimensional trajectory point set include at least one of the following:

[0153] 1) Obtaining the maximum flight distance of the unmanned aerial vehicle and preprocessing the three-dimensional trajectory point set according to the maximum flight distance;

[0154] Specifically, when one or more processors 41 preprocess the three-dimensional trajectory point set according to the maximum flight distance, they are specifically configured to calculate the length of the three-dimensional trajectory formed by the three-dimensional trajectory point set; if the length of the three-dimensional trajectory formed by the three-dimensional trajectory point set is greater than the maximum flight distance, then deleting some three-dimensional trajectory points in the three-dimensional trajectory point set so that the length of the three-dimensional trajectory formed by the remaining three-dimensional trajectory points in the three-dimensional trajectory point set is less than the maximum flight distance of the unmanned aerial vehicle.

[0155] 2) Obtaining the density of at least some consecutive three-dimensional trajectory points in the three-dimensional trajectory point set and preprocessing the at least some consecutive three-dimensional trajectory points according to the density;

[0156] Specifically, when one or more processors 41 preprocess the at least partially continuous three-dimensional trajectory point set according to the density, they are specifically configured to: determine the number of three-dimensional trajectory points within a preset range in the three-dimensional trajectory point set; if the number of three-dimensional trajectory points within the preset range is greater than a threshold, reduce the number of three-dimensional trajectory points within the preset range, or obtain replacement points within the preset range, and replace all the three-dimensional trajectory points within the preset range with the replacement points within the preset range.

[0157] 3) Obtain the jitter degree of a specific three-dimensional trajectory point in the three-dimensional trajectory point set, and preprocess the specific three-dimensional trajectory point according to the jitter degree;

[0158] Specifically, when one or more processors 41 preprocess the specific three-dimensional trajectory point according to the jitter degree, they are specifically configured to: when the jitter degree of the specific three-dimensional trajectory point is less than a threshold, remove the specific three-dimensional trajectory point; and / or when the jitter degree of the specific three-dimensional trajectory point is not less than the threshold, retain the specific three-dimensional trajectory point.

[0159] The jitter degree of the specific three-dimensional trajectory point is determined according to the distance from the next three-dimensional trajectory point of the specific three-dimensional trajectory point to the straight line where the specific three-dimensional trajectory point and the previous three-dimensional trajectory point of the specific three-dimensional trajectory point are located.

[0160] 4) Generate a three-dimensional trajectory according to at least partially continuous three-dimensional trajectory points in the three-dimensional trajectory point set, and preprocess the at least partially continuous three-dimensional trajectory points according to the curvature of the three-dimensional trajectory.

[0161] Specifically, when one or more processors 41 preprocess the at least partially continuous three-dimensional trajectory points according to the curvature of the three-dimensional trajectory, they are specifically configured to: when the curvature of the three-dimensional trajectory at a first three-dimensional trajectory point is greater than a threshold, obtain a replacement point, where the first three-dimensional trajectory point is a three-dimensional trajectory point in the at least partially continuous three-dimensional trajectory points, and the curvature of the curve formed by the replacement point and the two adjacent three-dimensional trajectory points of the first three-dimensional trajectory point at the replacement point is less than the curvature of the three-dimensional trajectory at the first three-dimensional trajectory point; replace the first three-dimensional trajectory point with the replacement point.

[0162] Optionally, when one or more processors 41 obtain a replacement point, specifically for: obtaining a first intermediate point between the first three-dimensional trajectory point and the previous three-dimensional trajectory point of the first three-dimensional trajectory point, and obtaining a second intermediate point between the first three-dimensional trajectory point and the next three-dimensional trajectory point of the first three-dimensional trajectory point, where the first intermediate point and the second intermediate point are the replacement points; or, obtaining the center or centroid of a triangle formed by the first three-dimensional trajectory point, the previous three-dimensional trajectory point of the first three-dimensional trajectory point, and the next three-dimensional trajectory point of the first three-dimensional trajectory point, where the center or centroid of the triangle is the replacement point.

[0163] The specific principle and implementation manner of the flight controller provided in the embodiments of the present invention are all similar to Figure 3 the embodiments shown, and will not be described in detail here.

[0164] Before determining the flight trajectory according to the three-dimensional trajectory point set in this embodiment, preprocessing is performed on each three-dimensional trajectory point in the three-dimensional trajectory point set. The purpose of the preprocessing is to ensure that the flight trajectory formed by the preprocessed three-dimensional trajectory point set satisfies the motion performance constraints of the unmanned aerial vehicle, and solves the problem that the specific curve set by the user on a specific image does not satisfy the motion performance constraints of the unmanned aerial vehicle due to the randomness of the user's description of the specific curve; in addition, when the unmanned aerial vehicle flies along the flight trajectory, a radar provided on the unmanned aerial vehicle is used to detect whether there are obstacles in the part of the flight trajectory in front of the unmanned aerial vehicle. If there are obstacles, the obstacle avoidance function of the unmanned aerial vehicle is activated, so that the unmanned aerial vehicle successfully bypasses the obstacles. After the unmanned aerial vehicle successfully bypasses the obstacles, the flight controller controls the unmanned aerial vehicle to continue flying along the flight trajectory, ensuring the safety of the unmanned aerial vehicle.

[0165] The embodiments of the present invention provide a control device. In Figure 5Based on the technical solution provided by the illustrated embodiment, when determining the three-dimensional trajectory point set, one or more processors 41 specifically use the height of the imaging device from the ground when capturing the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device, and are specifically used for: determining the back-projection point of the pixel point on the ground, where the back-projection point is the intersection of the projection ray passing through the optical center of the camera lens of the imaging device and the pixel point and the ground; determining the coordinate position of the back-projection point in the camera coordinate system according to the coordinates of the pixel point in the image coordinate system where the specific image is located and the focal length of the imaging device; determining the coordinate position of the back-projection point in the ground coordinate system according to the coordinate position of the back-projection point in the camera coordinate system; and determining the three-dimensional trajectory point corresponding to the pixel point in the ground coordinate system according to the height of the imaging device from the ground when capturing the specific image and the coordinate position of the back-projection point in the ground coordinate system.

[0166] Specifically, to determine the coordinate position of the back-projection point in the ground coordinate system according to the coordinate position of the back-projection point in the camera coordinate system, it can be achieved through the following method: determining the external parameters of the camera coordinate system relative to the ground coordinate system according to the height of the imaging device from the ground when capturing the specific image and the angle of the imaging device relative to the ground; and determining the coordinate position of the back-projection point in the ground coordinate system according to the coordinate position of the back-projection point in the camera coordinate system and the external parameters of the camera coordinate system relative to the ground coordinate system.

[0167] In this embodiment, the trajectory generation algorithm includes: a minimum-sway trajectory generation algorithm.

[0168] In addition, as Figure 5 shown, the sensor 42 is communicatively connected to one or more processors 41. The sensor 42 is used to detect obstacles in the part of the flight trajectory in front of the unmanned aerial vehicle and send the detection result to one or more processors 41; one or more processors 41 determine whether there are obstacles in the part of the flight trajectory in front of the unmanned aerial vehicle according to the detection result; if there are obstacles in the part of the flight trajectory in front of the unmanned aerial vehicle, one or more processors 41 control the unmanned aerial vehicle to bypass the obstacles; and after the unmanned aerial vehicle bypasses the obstacles, one or more processors 41 control the unmanned aerial vehicle to return to the flight trajectory.

[0169] The specific principle and implementation manner of the flight controller provided by the embodiment of the present invention are similar to those of the Figure 2 illustrated embodiment, and will not be elaborated here.

[0170] In this embodiment, based on any pixel point on a specific curve and the optical center of the camera lens of the imaging device, the back-projection points of each pixel point on the specific curve on the ground are determined. According to the height and angle of the imaging device relative to the ground and the focal length of the imaging device, the coordinate positions of the back-projection points in the camera coordinate system and the external parameters of the camera coordinate system relative to the ground coordinate system are determined. Based on the coordinate positions of the back-projection points in the camera coordinate system and the external parameters of the camera coordinate system relative to the ground coordinate system, the coordinate positions of the back-projection points in the ground coordinate system are determined. Based on the coordinate positions of the back-projection points in the ground coordinate system, the coordinates of the three-dimensional trajectory points can be accurately calculated, realizing the accurate calculation of the three-dimensional trajectory, that is, the flight trajectory, thereby realizing the accurate control of the unmanned aerial vehicle.

[0171] An embodiment of the present invention provides a control device. Figure 6 For the structural diagram of the control device provided in another embodiment of the present invention, as Figure 6 shown, the control device 60 includes: an acquisition module 61 and a determination module 62. Among them, the acquisition module 61 is used to acquire a specific image and a specific curve, where the specific curve is a curve drawn on the specific image; the determination module 62 is used to generate the specific curve into a flight trajectory according to the specific image and the specific curve, and the flight trajectory is used to control the unmanned aerial vehicle to fly along the flight trajectory.

[0172] Optionally, the acquisition module 61 is specifically used to acquire a real-time image captured by an imaging device carried on the unmanned aerial vehicle; the control device 60 further includes: a display module 63 and a receiving module 64. The display module 63 is used to display the real-time image; the receiving module 64 is used to receive a specific curve drawn on the real-time image; the acquisition module 61 is specifically used to acquire a specific image, and the specific image includes at least a part of the real-time image where the specific curve is located.

[0173] In addition, the acquisition module 61 is used to download the specific image and the specific curve from the cloud platform, or the control device 60 can be a first ground station; the receiving module 64 is further used to receive the specific image and the specific curve sent by a second ground station.

[0174] In addition, when the acquisition module 61 acquires a specific image and a specific curve, the acquisition module 61 is specifically configured to acquire the height of the imaging device from the ground when taking the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device; when the determination module 62 generates the specific curve into a flight trajectory according to the specific image and the specific curve, the determination module 62 is specifically configured to determine a three-dimensional trajectory point set according to the height of the imaging device from the ground when taking the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device, and the three-dimensional trajectory point set includes the three-dimensional trajectory points corresponding to each pixel point corresponding to the specific curve on the specific image in the ground coordinate system respectively; and generate a flight trajectory according to the three-dimensional trajectory point set.

[0175] Optionally, the determination module 62 includes a preprocessing unit 621 and a determination unit 622. When the determination module 62 generates a flight trajectory according to the three-dimensional trajectory point set, the preprocessing unit 621 is configured to preprocess the three-dimensional trajectory point set to obtain a preprocessed three-dimensional trajectory point set; the determination unit 622 is configured to determine the flight trajectory according to the preprocessed three-dimensional trajectory point set by using a trajectory generation algorithm, and the flight trajectory satisfies the kinematic constraints of the unmanned aerial vehicle.

[0176] When the preprocessing unit 621 preprocesses the three-dimensional trajectory point set, the acquisition module 61 is further at least configured to: acquire the maximum flight distance of the unmanned aerial vehicle, acquire the density of at least some consecutive three-dimensional trajectory points in the three-dimensional trajectory point set, and acquire the jitter degree of a specific three-dimensional trajectory point in the three-dimensional trajectory point set; the preprocessing unit 621 is specifically configured to: preprocess the three-dimensional trajectory point set according to the maximum flight distance; preprocess the at least some consecutive three-dimensional trajectory points according to the density; preprocess the specific three-dimensional trajectory point according to the jitter degree; generate a three-dimensional trajectory according to at least some consecutive three-dimensional trajectory points in the three-dimensional trajectory point set, and preprocess the at least some consecutive three-dimensional trajectory points according to the curvature of the three-dimensional trajectory.

[0177] In addition, the control device 60 further includes: a calculation module 65. When the preprocessing unit 621 preprocesses the three-dimensional trajectory point set according to the maximum flight distance, the calculation module 65 is configured to calculate the length of the three-dimensional trajectory formed by the three-dimensional trajectory point set. If the length of the three-dimensional trajectory formed by the three-dimensional trajectory point set is greater than the maximum flight distance, the preprocessing unit 621 is configured to delete some three-dimensional trajectory points in the three-dimensional trajectory point set, so that the length of the three-dimensional trajectory formed by the remaining three-dimensional trajectory points in the three-dimensional trajectory point set is less than the maximum flight distance of the unmanned aerial vehicle.

[0178] When the preprocessing unit 621 preprocesses the at least partially continuous three-dimensional trajectory point set according to the density, the determination unit 622 is configured to determine the number of three-dimensional trajectory points in the three-dimensional trajectory point set that are located within a preset range. If the number of three-dimensional trajectory points within the preset range is greater than a threshold, the preprocessing unit 621 is configured to reduce the number of three-dimensional trajectory points within the preset range, or the acquisition module 61 is configured to acquire substitute points within the preset range, and the preprocessing unit 621 replaces all the three-dimensional trajectory points within the preset range with the substitute points within the preset range.

[0179] When the preprocessing unit 621 preprocesses the specific three-dimensional trajectory point according to the jitter degree, if the jitter degree of the specific three-dimensional trajectory point is less than a threshold, the preprocessing unit 621 is configured to remove the specific three-dimensional trajectory point; and / or, if the jitter degree of the specific three-dimensional trajectory point is not less than the threshold, the preprocessing unit 621 is configured to retain the specific three-dimensional trajectory point. The jitter degree of the specific three-dimensional trajectory point is determined according to the distance from the next three-dimensional trajectory point of the specific three-dimensional trajectory point to the straight line where the specific three-dimensional trajectory point and the previous three-dimensional trajectory point of the specific three-dimensional trajectory point are located.

[0180] When the preprocessing unit 621 preprocesses the at least partially continuous three-dimensional trajectory points according to the curvature of the three-dimensional trajectory, if the curvature of the three-dimensional trajectory at the first three-dimensional trajectory point is greater than the threshold, the acquisition module 61 is used to acquire a replacement point, where the first three-dimensional trajectory point is one of the at least partially continuous three-dimensional trajectory points, and the curvature of the curve formed by the replacement point and the two three-dimensional trajectory points before and after the first three-dimensional trajectory point is less than the curvature of the three-dimensional trajectory at the first three-dimensional trajectory point; the preprocessing unit is used to replace the first three-dimensional trajectory point with the replacement point. When the acquisition module 61 acquires the replacement point, it specifically is used to: acquire a first intermediate point between the first three-dimensional trajectory point and the previous three-dimensional trajectory point of the first three-dimensional trajectory point, and acquire a second intermediate point between the first three-dimensional trajectory point and the next three-dimensional trajectory point of the first three-dimensional trajectory point, and the first intermediate point and the second intermediate point are the replacement point; or, acquire the center or centroid of the triangle formed by the first three-dimensional trajectory point, the previous three-dimensional trajectory point of the first three-dimensional trajectory point, and the next three-dimensional trajectory point of the first three-dimensional trajectory point, and the center or centroid of the triangle is the replacement point.

[0181] When the determination module 62 determines the three-dimensional trajectory point set according to the height of the imaging device from the ground when shooting the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device, the determination module 62 specifically is used to: determine the back-projection point of the pixel point on the ground, and the back-projection point is the intersection of the projection ray passing through the optical center of the camera lens of the imaging device and the pixel point and the ground; determine the coordinate position of the back-projection point in the camera coordinate system according to the coordinates of the pixel point in the image coordinate system where the specific image is located and the focal length of the imaging device; determine the coordinate position of the back-projection point in the ground coordinate system according to the coordinate position of the back-projection point in the camera coordinate system; determine the three-dimensional trajectory point corresponding to the pixel point in the ground coordinate system according to the height of the imaging device from the ground when shooting the specific image and the coordinate position of the back-projection point in the ground coordinate system. When the determination module 62 determines the coordinate position of the back-projection point in the ground coordinate system according to the coordinate position of the back-projection point in the camera coordinate system, it specifically is used to: determine the external parameters of the camera coordinate system relative to the ground coordinate system according to the height of the imaging device from the ground when shooting the specific image and the angle of the imaging device relative to the ground; determine the coordinate position of the back-projection point in the ground coordinate system according to the coordinate position of the back-projection point in the camera coordinate system and the external parameters of the camera coordinate system relative to the ground coordinate system.

[0182] Optionally, the trajectory generation algorithm includes: a minimum jerk trajectory generation algorithm.

[0183] In addition, the control device 60 further includes a detection module 66, a start module 67, and a control module 68. The detection module 66 is configured to detect whether there is an obstacle in a part of the flight trajectory in front of the unmanned aerial vehicle when the unmanned aerial vehicle is flying along the flight trajectory; the start module 67 is configured to activate the obstacle avoidance function of the unmanned aerial vehicle when there is an obstacle in a part of the flight trajectory in front of the unmanned aerial vehicle; the control module 68 is configured to control the unmanned aerial vehicle to return to the flight trajectory after the unmanned aerial vehicle bypasses the obstacle.

[0184] Furthermore, the control device 60 further includes a sending module 69, and the sending module 69 is configured to upload the flight trajectory to a specific server. Alternatively, the control device is a first ground station, and the control device further includes: a sending module, configured to send the flight trajectory to a second ground station.

[0185] In this embodiment, a specific curve drawn on a specific image is used to generate a flight trajectory for controlling the unmanned aerial vehicle. The specific curve can be a specific curve set by the user on a static picture, or a specific curve set on one frame or multiple frames of images in a dynamic video. Correspondingly, the specific image can be a static picture, or one frame or multiple frames of images in a dynamic video. The specific curve drawn by the user on the specific image can be used to control the flight trajectory of the unmanned aerial vehicle, that is, the unmanned aerial vehicle can fly according to the specific curve designed by the user's personalization, realizing the personalized design of the flight mode of the unmanned aerial vehicle. Compared with the flight modes such as point-to-point flight and intelligent following in the prior art, the flexibility of the flight mode of the unmanned aerial vehicle is improved.

[0186] An embodiment of the present invention provides an unmanned aerial vehicle. Figure 7 The following is a structural diagram of the unmanned aerial vehicle provided by the embodiment of the present invention, as Figure 7 shown, the unmanned aerial vehicle 100 includes: a fuselage, a power system, and a flight controller 118. The power system includes at least one of the following: a motor 107, a propeller 106, and an electronic speed controller 117. The power system is installed on the fuselage and is used to provide flight power; the flight controller 118 is communicatively connected to the power system and is used to control the flight of the unmanned aerial vehicle; wherein, the flight controller 118 includes an inertial measurement unit and a gyroscope. The inertial measurement unit and the gyroscope are used to detect the acceleration, pitch angle, roll angle, yaw angle, etc. of the unmanned aerial vehicle.

[0187] In addition, as Figure 7As shown in the figure, the unmanned aerial vehicle 100 further includes: a sensing system 108, a communication system 110, a support device 102, and an imaging device 104. Among them, the support device 102 may specifically be a gimbal. The communication system 110 may specifically include a receiver, and the receiver is used to receive the wireless signal sent by the antenna 114 of the ground station 112. 116 represents the electromagnetic wave generated during the communication process between the receiver and the antenna 114.

[0188] The specific principle and implementation manner of the flight controller 118 provided in the embodiments of the present invention are similar to those of the control device described in the above embodiments, and will not be elaborated here.

[0189] In this embodiment, a specific curve drawn on a specific image is used to generate a flight trajectory for controlling the unmanned aerial vehicle. The specific curve may be a specific curve set by the user on a static picture, or a specific curve set on one or more frames of an image in a dynamic video. Correspondingly, the specific image may be a static picture, or one or more frames of an image in a dynamic video. The specific curve drawn by the user on the specific image can be used to control the flight trajectory of the unmanned aerial vehicle, that is, the unmanned aerial vehicle can fly according to the specific curve designed by the user's personalization, realizing the personalized design of the flight mode of the unmanned aerial vehicle. Compared with the flight modes such as point-to-point flight and intelligent following in the prior art, the flexibility of the flight mode of the unmanned aerial vehicle is improved.

[0190] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical or other forms.

[0191] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0192] In addition, in each embodiment of the present invention, the functional units may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware, or in the form of a hardware plus software functional unit.

[0193] The above-mentioned integrated unit implemented in the form of a software functional unit may be stored in a computer-readable storage medium. The above-mentioned software functional unit stored in a storage medium includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute some steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0194] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the device described above can refer to the corresponding process in the foregoing method embodiments and will not be repeated here.

[0195] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for generating a flight trajectory, characterized in that, it includes: Obtain a specific image and a specific curve, wherein the specific curve is a curve drawn on the specific image, and the specific image is multiple frames of images in a real-time dynamic video captured by an imaging device of a first unmanned aerial vehicle; Determine a three-dimensional trajectory point set according to the specific image and the specific curve, where the three-dimensional trajectory point set includes three-dimensional trajectory points corresponding to multiple pixel points corresponding to the specific curve on the specific image in a ground coordinate system; and Preprocess the three-dimensional trajectory point set to obtain a preprocessed three-dimensional trajectory point set; Generate a first flight trajectory according to the preprocessed three-dimensional trajectory point set, and the first flight trajectory is used to control the first unmanned aerial vehicle to fly along the first flight trajectory.

2. The method according to claim 1, characterized in that, The obtaining of the specific image and the specific curve includes: Obtain a real-time image captured by an imaging device carried on the first unmanned aerial vehicle; Display the real-time image on a display screen; Receive a specific curve drawn on the real-time image displayed on the display screen; Obtain a specific image, where the specific image includes at least part of the real-time image where the specific curve is located.

3. The method according to claim 1, characterized in that, The obtaining of the specific image and the specific curve includes: Download the specific image and the specific curve from a cloud platform; Or, The obtaining of the specific image and the specific curve includes: receiving the specific image and the specific curve sent by a second ground station.

4. The method according to claim 2, characterized in that, The obtaining of the specific image and the specific curve includes: Obtain the height of the imaging device from the ground when capturing the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device; The determining of the three-dimensional trajectory point set according to the specific image and the specific curve includes: According to the height of the imaging device from the ground when capturing the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device, translate the back-projection point of each pixel point on the specific curve on the ground to the flight height of the first unmanned aerial vehicle to obtain a three-dimensional trajectory point set.

5. The method according to claim 4, characterized in that, The generating of the first flight trajectory according to the preprocessed three-dimensional trajectory point set includes: According to the preprocessed three-dimensional trajectory point set, adopt a trajectory generation algorithm to determine the first flight trajectory, and the first flight trajectory satisfies the kinematic constraints of the first unmanned aerial vehicle.

6. The method according to claim 5, characterized in that, The preprocessing of the three-dimensional trajectory point set includes at least one of the following: Obtain the maximum flight distance of the first unmanned aerial vehicle, and preprocess the three-dimensional trajectory point set according to the maximum flight distance; Obtain the density of at least some consecutive three-dimensional trajectory points in the three-dimensional trajectory point set, and preprocess the at least some consecutive three-dimensional trajectory points according to the density; Obtain the jitter degree of specific three-dimensional trajectory points in the three-dimensional trajectory point set, and preprocess the specific three-dimensional trajectory points according to the jitter degree; Generate a three-dimensional trajectory according to at least some consecutive three-dimensional trajectory points in the three-dimensional trajectory point set, and preprocess the at least some consecutive three-dimensional trajectory points according to the curvature of the three-dimensional trajectory.

7. The method according to claim 6, wherein, The preprocessing of the three-dimensional trajectory point set according to the maximum flight distance includes: Calculate the length of the three-dimensional trajectory formed by the three-dimensional trajectory point set; If the length of the three-dimensional trajectory formed by the three-dimensional trajectory point set is greater than the maximum flight distance, delete some three-dimensional trajectory points in the three-dimensional trajectory point set so that the length of the three-dimensional trajectory formed by the remaining three-dimensional trajectory points in the three-dimensional trajectory point set is less than the maximum flight distance of the first unmanned aerial vehicle.

8. The method according to claim 6, wherein, The preprocessing of the at least some consecutive three-dimensional trajectory point set according to the density includes: Determine the number of three-dimensional trajectory points in the three-dimensional trajectory point set that are within a preset range; If the number of three-dimensional trajectory points within the preset range is greater than a threshold, reduce the number of three-dimensional trajectory points within the preset range, or obtain replacement points within the preset range, and replace all three-dimensional trajectory points within the preset range with the replacement points within the preset range.

9. The method according to claim 6, wherein, The preprocessing of the specific three-dimensional trajectory points according to the jitter degree includes: If the jitter degree of the specific three-dimensional trajectory point is less than a threshold, remove the specific three-dimensional trajectory point; and / or, If the jitter degree of the specific three-dimensional trajectory point is not less than a threshold, retain the specific three-dimensional trajectory point.

10. The method according to claim 9, wherein, The jitter degree of the specific three-dimensional trajectory point is determined according to the distance from the next three-dimensional trajectory point of the specific three-dimensional trajectory point to the straight line where the specific three-dimensional trajectory point and the previous three-dimensional trajectory point of the specific three-dimensional trajectory point are located.

11. The method according to claim 6, wherein, The preprocessing of the at least some consecutive three-dimensional trajectory points according to the curvature of the three-dimensional trajectory includes: If the curvature of the three-dimensional trajectory at a first three-dimensional trajectory point is greater than a threshold, obtain a replacement point, where the first three-dimensional trajectory point is one of the at least some consecutive three-dimensional trajectory points, and the curvature of the curve formed by the replacement point and the two three-dimensional trajectory points before and after the first three-dimensional trajectory point at the replacement point is less than the curvature of the three-dimensional trajectory at the first three-dimensional trajectory point; Replace the first three-dimensional trajectory point with the replacement point.

12. The method according to claim 11, wherein, The obtaining of the replacement point includes: Obtain a first intermediate point between the first three-dimensional trajectory point and the previous three-dimensional trajectory point of the first three-dimensional trajectory point, and obtain a second intermediate point between the first three-dimensional trajectory point and the next three-dimensional trajectory point of the first three-dimensional trajectory point. The first intermediate point and the second intermediate point are the replacement points; or, Obtain the center or centroid of a triangle formed by the first three-dimensional trajectory point, the previous three-dimensional trajectory point of the first three-dimensional trajectory point, and the next three-dimensional trajectory point of the first three-dimensional trajectory point. The center or centroid of the triangle is the replacement point.

13. The method according to claim 4, wherein, Based on the height of the imaging device from the ground when shooting the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device, translate the back-projection point of each pixel point on the specific curve on the ground to the flight height of the first unmanned aerial vehicle, obtaining a three-dimensional trajectory point set, including: Determine the back-projection point of the pixel point on the ground. The back-projection point is the intersection of the projection ray passing through the optical center of the camera lens of the imaging device and the pixel point and the ground; Based on the coordinates of the pixel point in the image coordinate system where the specific image is located and the focal length of the imaging device, determine the coordinate position of the back-projection point in the camera coordinate system; Based on the coordinate position of the back-projection point in the camera coordinate system, determine the coordinate position of the back-projection point in the ground coordinate system; Based on the height of the imaging device from the ground when shooting the specific image and the coordinate position of the back-projection point in the ground coordinate system, determine the corresponding three-dimensional trajectory point of the pixel point in the ground coordinate system.

14. The method according to claim 13, wherein, The determining the coordinate position of the back-projection point in the ground coordinate system based on the coordinate position of the back-projection point in the camera coordinate system includes: Based on the height of the imaging device from the ground when shooting the specific image and the angle of the imaging device relative to the ground, determine the external parameters of the camera coordinate system relative to the ground coordinate system; Based on the coordinate position of the back-projection point in the camera coordinate system and the external parameters of the camera coordinate system relative to the ground coordinate system, determine the coordinate position of the back-projection point in the ground coordinate system.

15. The method according to any one of claims 5-12, wherein, The trajectory generation algorithm includes: a minimum oscillation trajectory generation algorithm.

16. The method according to claim 1, wherein, The controlling the first unmanned aerial vehicle to fly along the first flight trajectory includes: When the first unmanned aerial vehicle is flying along the first flight trajectory, detect whether there are obstacles in the part of the first flight trajectory in front of the first unmanned aerial vehicle; If there are obstacles in the part of the first flight trajectory in front of the first unmanned aerial vehicle, activate the obstacle avoidance function of the first unmanned aerial vehicle; After the first unmanned aerial vehicle bypasses the obstacle, control the first unmanned aerial vehicle to return to the first flight trajectory.

17. The method according to claim 1, wherein, the method further includes: uploading the first flight trajectory to a specific server; alternatively, the method further includes: sending the first flight trajectory to a second ground station.

18. The method according to claim 1, wherein, the execution subject of the method for generating the first flight trajectory is a first ground station, and the first ground station can transmit the specific curve to the second ground station in a direct communication or indirect communication manner, so that the second ground station can control a second unmanned aerial vehicle to fly based on the first flight trajectory. The first ground station and the second ground station are two different ground stations, and the first unmanned aerial vehicle and the second unmanned aerial vehicle are two different unmanned aerial vehicles.

19. The method according to claim 18, wherein, the second ground station can control the second unmanned aerial vehicle to fly based on the first flight trajectory, including: generating a second flight trajectory based on the first flight trajectory; the second ground station can control the second unmanned aerial vehicle to fly along the second flight trajectory.

20. The method according to claim 19, wherein, wherein, the first flight trajectory and the second flight trajectory are the same, or the projections of the first flight trajectory and the second flight trajectory on the horizontal plane coincide, but their heights are different.

21. The method according to claim 19 or 20, wherein, wherein, the first flight trajectory and the second flight trajectory are executed simultaneously, or the first flight trajectory and the second flight trajectory are executed at different times.

22. A control device, wherein, including one or more processors, working alone or in cooperation, and the one or more processors are used for: acquiring a specific image and a specific curve, wherein the specific curve is a curve drawn on the specific image, and the specific image is multiple frames of images in a real-time dynamic video captured by an imaging device of a first unmanned aerial vehicle; determining a three-dimensional trajectory point set according to the specific image and the specific curve, the three-dimensional trajectory point set including three-dimensional trajectory points corresponding to multiple pixel points corresponding to the specific curve on the specific image in a ground coordinate system; and performing preprocessing on the three-dimensional trajectory point set to obtain a preprocessed three-dimensional trajectory point set; generating a first flight trajectory according to the preprocessed three-dimensional trajectory point set, and the first flight trajectory is used to control the first unmanned aerial vehicle to fly along the first flight trajectory.

23. The control device according to claim 22, wherein, the control device is a ground station or a flight controller.

24. The control device according to claim 23, wherein, The control device is a flight controller, or the flight controller includes the control device; the control device further includes: a receiver communicatively connected to the one or more processors, the receiver being configured to receive a flight trajectory sent by a ground station, and the one or more processors are further configured to control the first unmanned aerial vehicle to fly along the first flight trajectory; Or, The control device is a ground station, or the ground station includes the control device; the control device further includes: a transmitter communicatively connected to the one or more processors, the transmitter being configured to send the first flight trajectory to the flight controller of the unmanned aerial vehicle.

25. The control device according to claim 22, wherein, The control device is a ground station, or the ground station includes the control device; The one or more processors are configured to: Obtain a real-time image captured by an imaging device carried on the first unmanned aerial vehicle; The control device further includes: A display screen configured to display the real-time image; and sense a specific curve drawn on the real-time image displayed on the display screen; The one or more processors are configured to: obtain the specific curve and a specific image, the specific image including at least a part of the real-time image where the specific curve is located.

26. The control device according to claim 22 or 23, wherein, The one or more processors are configured to: download the specific image and the specific curve from a cloud platform; Or, The control device is a first ground station, or the first ground station includes the control device; the control device further includes: a receiver communicatively connected to the one or more processors, the receiver being configured to receive the specific image and the specific curve sent by a second ground station.

27. The control device according to claim 25, wherein, When the one or more processors obtain the specific image and the specific curve, specifically: Obtain the height of the imaging device from the ground when capturing the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device; When the one or more processors determine a three-dimensional trajectory point set according to the specific image and the specific curve, specifically: According to the height of the imaging device from the ground when capturing the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device, translate the back-projection point of each pixel point on the specific curve on the ground to the flight height of the unmanned aerial vehicle, to obtain a three-dimensional trajectory point set, the three-dimensional trajectory point set including the three-dimensional trajectory points corresponding to each pixel point on the specific curve in the specific image in the ground coordinate system.

28. The control device according to claim 27, wherein, When the one or more processors generate a first flight trajectory according to the preprocessed three-dimensional trajectory point set, specifically: Based on the preprocessed three-dimensional trajectory point set, a trajectory generation algorithm is used to determine the first flight trajectory, and the first flight trajectory satisfies the kinematic constraints of the first unmanned aerial vehicle.

29. The control device according to claim 28, wherein, when the one or more processors preprocess the three-dimensional trajectory point set, they are specifically configured to perform at least one of the following: Obtain the maximum flight distance of the first unmanned aerial vehicle, and preprocess the three-dimensional trajectory point set according to the maximum flight distance; Obtain the density of at least some consecutive three-dimensional trajectory points in the three-dimensional trajectory point set, and preprocess the at least some consecutive three-dimensional trajectory points according to the density; Obtain the jitter degree of specific three-dimensional trajectory points in the three-dimensional trajectory point set, and preprocess the specific three-dimensional trajectory points according to the jitter degree; Generate a three-dimensional trajectory according to at least some consecutive three-dimensional trajectory points in the three-dimensional trajectory point set, and preprocess the at least some consecutive three-dimensional trajectory points according to the curvature of the three-dimensional trajectory.

30. The control device according to claim 29, wherein, when the one or more processors preprocess the three-dimensional trajectory point set according to the maximum flight distance, they are specifically configured to: Calculate the length of the three-dimensional trajectory formed by the three-dimensional trajectory point set; If the length of the three-dimensional trajectory formed by the three-dimensional trajectory point set is greater than the maximum flight distance, delete some three-dimensional trajectory points in the three-dimensional trajectory point set so that the length of the three-dimensional trajectory formed by the remaining three-dimensional trajectory points in the three-dimensional trajectory point set is less than the maximum flight distance of the first unmanned aerial vehicle.

31. The control device according to claim 29, wherein, when the one or more processors preprocess the at least some consecutive three-dimensional trajectory point set according to the density, they are specifically configured to: Determine the number of three-dimensional trajectory points within a preset range in the three-dimensional trajectory point set; If the number of three-dimensional trajectory points within the preset range is greater than a threshold, reduce the number of three-dimensional trajectory points within the preset range, or obtain replacement points within the preset range and replace all three-dimensional trajectory points within the preset range with the replacement points within the preset range.

32. The control device according to claim 29, wherein, when the one or more processors preprocess the specific three-dimensional trajectory points according to the jitter degree, they are specifically configured to: When the jitter degree of the specific three-dimensional trajectory point is less than the threshold, remove the specific three-dimensional trajectory point; and / or, When the jitter degree of the specific three-dimensional trajectory point is not less than the threshold, retain the specific three-dimensional trajectory point.

33. The control device according to claim 32, wherein, the jitter degree of the specific three-dimensional trajectory point is determined according to the distance from the next three-dimensional trajectory point of the specific three-dimensional trajectory point to the straight line where the specific three-dimensional trajectory point and the previous three-dimensional trajectory point of the specific three-dimensional trajectory point are located.

34. The control device according to claim 29, wherein, When preprocessing the at least partially continuous three-dimensional trajectory points, the one or more processors specifically are configured to: When the curvature of the three-dimensional trajectory at a first three-dimensional trajectory point is greater than a threshold, obtain a replacement point, where the first three-dimensional trajectory point is one of the at least partially continuous three-dimensional trajectory points, and the curvature of the curve formed by the replacement point and the two three-dimensional trajectory points before and after the first three-dimensional trajectory point is less than the curvature of the three-dimensional trajectory at the first three-dimensional trajectory point; replace the first three-dimensional trajectory point with the replacement point.

35. The control device according to claim 33, wherein, when the one or more processors obtain a replacement point, they are specifically configured to: obtain a first intermediate point between the first three-dimensional trajectory point and the three-dimensional trajectory point before the first three-dimensional trajectory point, and obtain a second intermediate point between the first three-dimensional trajectory point and the three-dimensional trajectory point after the first three-dimensional trajectory point, where the first intermediate point and the second intermediate point are the replacement point; or, obtain the center or centroid of a triangle formed by the first three-dimensional trajectory point, the three-dimensional trajectory point before the first three-dimensional trajectory point, and the three-dimensional trajectory point after the first three-dimensional trajectory point, where the center or centroid of the triangle is the replacement point.

36. The control device according to claim 27, wherein, when the one or more processors translate the back-projection point of each pixel point on the specific curve to the flight altitude of the unmanned aerial vehicle to obtain a set of three-dimensional trajectory points according to the height of the imaging device from the ground when shooting the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device, they are specifically configured to: determine the back-projection point of the pixel point on the ground, where the back-projection point is the intersection of the projection ray passing through the optical center of the camera lens of the imaging device and the pixel point and the ground; determine the coordinate position of the back-projection point in the camera coordinate system according to the coordinates of the pixel point in the image coordinate system where the specific image is located and the focal length of the imaging device; determine the coordinate position of the back-projection point in the ground coordinate system according to the coordinate position of the back-projection point in the camera coordinate system; determine the three-dimensional trajectory point corresponding to the pixel point in the ground coordinate system according to the height of the imaging device from the ground when shooting the specific image and the coordinate position of the back-projection point in the ground coordinate system.

37. The control device according to claim 36, wherein, when the one or more processors determine the coordinate position of the back-projection point in the ground coordinate system according to the coordinate position of the back-projection point in the camera coordinate system, they are specifically configured to: determine the external parameters of the camera coordinate system relative to the ground coordinate system according to the height of the imaging device from the ground when shooting the specific image and the angle of the imaging device relative to the ground; Determine the coordinate position of the back-projected point in the ground coordinate system according to the coordinate position of the back-projected point in the camera coordinate system and the external parameters of the camera coordinate system relative to the ground coordinate system.

38. The control device according to any one of claims 28-35, wherein, the trajectory generation algorithm includes: a minimum jerk trajectory generation algorithm.

39. The control device according to claim 22, wherein, further includes: a sensor communicatively connected to the one or more processors, the sensor is configured to detect obstacles in a portion of the first flight trajectory in front of the first unmanned aerial vehicle, and send the detection result to the one or more processors; the one or more processors determine whether there are obstacles in a portion of the first flight trajectory in front of the first unmanned aerial vehicle according to the detection result; if there are obstacles in a portion of the first flight trajectory in front of the first unmanned aerial vehicle, the one or more processors control the first unmanned aerial vehicle to bypass the obstacles; after the first unmanned aerial vehicle bypasses the obstacles, the one or more processors control the first unmanned aerial vehicle to return to the first flight trajectory.

40. A control device, wherein, includes: an acquisition module, configured to acquire a specific image and a specific curve, wherein the specific curve is a curve drawn on the specific image, and the specific image is multiple frames of images in a real-time dynamic video captured by an imaging device of a first unmanned aerial vehicle; a determination module, configured to determine a three-dimensional trajectory point set according to the specific image and the specific curve, the three-dimensional trajectory point set includes three-dimensional trajectory points corresponding to multiple pixel points corresponding to the specific curve on the specific image in the ground coordinate system; and the determination module includes a preprocessing unit and a determination unit; when the determination module generates a first flight trajectory according to the three-dimensional trajectory point set, the preprocessing unit is configured to preprocess the three-dimensional trajectory point set to obtain a preprocessed three-dimensional trajectory point set; the determination unit is configured to generate a first flight trajectory according to the preprocessed three-dimensional trajectory point set, and the first flight trajectory is used to control the first unmanned aerial vehicle to fly along the first flight trajectory.

41. The control device according to claim 40, wherein, the acquisition module is specifically configured to acquire a real-time image captured by an imaging device carried on the first unmanned aerial vehicle; the control device further includes: a display module, configured to display the real-time image; a receiving module, configured to receive a specific curve drawn on the real-time image; the acquisition module is specifically configured to acquire a specific image, and the specific image includes at least a part of the real-time image where the specific curve is located.

42. The control device according to claim 40, wherein, the acquisition module is configured to download the specific image and the specific curve from a cloud platform; or, the control device is a first ground station; the control device further includes: a receiving module, configured to receive a specific image and a specific curve sent by a second ground station.

43. The control device according to claim 41, wherein, when the acquisition module acquires the specific image and the specific curve, the acquisition module is specifically configured to acquire the height of the imaging device from the ground when shooting the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device; when the determination module determines the three-dimensional trajectory point set according to the specific image and the specific curve, the determination module is specifically configured to translate the back-projection point of each pixel point on the specific curve on the ground to the flight height of the unmanned aerial vehicle according to the height of the imaging device from the ground when shooting the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device, so as to obtain the three-dimensional trajectory point set.

44. The control device according to claim 43, wherein, the determination unit is specifically configured to determine the first flight trajectory according to the preprocessed three-dimensional trajectory point set by using a trajectory generation algorithm, and the first flight trajectory satisfies the kinematic constraints of the first unmanned aerial vehicle.

45. The control device according to claim 44, wherein, when the preprocessing unit preprocesses the three-dimensional trajectory point set, the acquisition module is further at least configured to: acquire the maximum flight distance of the first unmanned aerial vehicle, acquire the density of at least some consecutive three-dimensional trajectory points in the three-dimensional trajectory point set, and acquire the jitter degree of a specific three-dimensional trajectory point in the three-dimensional trajectory point set; the preprocessing unit is specifically configured to: preprocess the three-dimensional trajectory point set according to the maximum flight distance; preprocess the at least some consecutive three-dimensional trajectory points according to the density; preprocess the specific three-dimensional trajectory point according to the jitter degree; generate a three-dimensional trajectory according to at least some consecutive three-dimensional trajectory points in the three-dimensional trajectory point set, and preprocess the at least some consecutive three-dimensional trajectory points according to the curvature of the three-dimensional trajectory.

46. The control device according to claim 45, wherein, it further includes: a calculation module; when the preprocessing unit preprocesses the three-dimensional trajectory point set according to the maximum flight distance, the calculation module is used to calculate the length of the three-dimensional trajectory formed by the three-dimensional trajectory point set; if the length of the three-dimensional trajectory formed by the three-dimensional trajectory point set is greater than the maximum flight distance, the preprocessing unit is used to delete some three-dimensional trajectory points in the three-dimensional trajectory point set so that the length of the three-dimensional trajectory formed by the remaining three-dimensional trajectory points in the three-dimensional trajectory point set is less than the maximum flight distance of the first unmanned aerial vehicle.

47. The control device according to claim 45, wherein, when the preprocessing unit preprocesses the at least some consecutive three-dimensional trajectory point set according to the density, the determination unit is used to determine the number of three-dimensional trajectory points in the three-dimensional trajectory point set that are located within a preset range. If the number of three-dimensional trajectory points within the preset range is greater than the threshold, the preprocessing unit is used to reduce the number of three-dimensional trajectory points within the preset range, or the acquisition module is used to acquire replacement points within the preset range, and the preprocessing unit replaces all the three-dimensional trajectory points within the preset range with the replacement points within the preset range.

48. The control device according to claim 45, wherein, when the preprocessing unit preprocesses the specific three-dimensional trajectory points according to the jitter degree, if the jitter degree of the specific three-dimensional trajectory points is less than the threshold, the preprocessing unit is used to remove the specific three-dimensional trajectory points; and / or, if the jitter degree of the specific three-dimensional trajectory points is not less than the threshold, the preprocessing unit is used to retain the specific three-dimensional trajectory points.

49. The control device according to claim 48, wherein, the jitter degree of the specific three-dimensional trajectory points is determined according to the distance from the next three-dimensional trajectory point of the specific three-dimensional trajectory points to the straight line where the specific three-dimensional trajectory points and the previous three-dimensional trajectory point of the specific three-dimensional trajectory points are located.

50. The control device according to claim 45, wherein, when the preprocessing unit preprocesses the at least partially continuous three-dimensional trajectory points according to the curvature of the three-dimensional trajectory, if the curvature of the three-dimensional trajectory at the first three-dimensional trajectory point is greater than the threshold, the acquisition module is used to acquire a replacement point, where the first three-dimensional trajectory point is one of the at least partially continuous three-dimensional trajectory points, and the curvature of the curve formed by the replacement point and the two three-dimensional trajectory points before and after the first three-dimensional trajectory point at the replacement point is less than the curvature of the three-dimensional trajectory at the first three-dimensional trajectory point; the preprocessing unit is used to replace the first three-dimensional trajectory point with the replacement point.

51. The control device according to claim 50, wherein, when the acquisition module acquires a replacement point, it specifically is used for: acquiring a first intermediate point between the first three-dimensional trajectory point and the previous three-dimensional trajectory point of the first three-dimensional trajectory point, and acquiring a second intermediate point between the first three-dimensional trajectory point and the next three-dimensional trajectory point of the first three-dimensional trajectory point, and the first intermediate point and the second intermediate point are the replacement points; or, acquiring the center or centroid of the triangle formed by the first three-dimensional trajectory point, the previous three-dimensional trajectory point of the first three-dimensional trajectory point, and the next three-dimensional trajectory point of the first three-dimensional trajectory point, and the center or centroid of the triangle is the replacement point.

52. The control device according to claim 44, wherein, when the determination module translates the back-projection points of each pixel point on the specific curve to the flight altitude of the unmanned aerial vehicle to obtain a three-dimensional trajectory point set according to the height of the imaging device from the ground when shooting the specific image, the angle of the imaging device relative to the ground, the coordinates of each pixel point on the specific curve in the image coordinate system where the specific image is located, and the focal length of the imaging device, it specifically is used for: Determine the back-projection point of the pixel point on the ground, where the back-projection point is the intersection of the projection ray passing through the optical center of the camera lens of the imaging device and the pixel point with the ground; Determine the coordinate position of the back-projection point in the camera coordinate system according to the coordinates of the pixel point in the image coordinate system where the specific image is located and the focal length of the imaging device; Determine the coordinate position of the back-projection point in the ground coordinate system according to the coordinate position of the back-projection point in the camera coordinate system; Determine the three-dimensional trajectory point corresponding to the pixel point in the ground coordinate system according to the height of the imaging device from the ground when taking the specific image and the coordinate position of the back-projection point in the ground coordinate system.

53. The control device according to claim 52, wherein, when the determination module determines the coordinate position of the back-projection point in the ground coordinate system according to the coordinate position of the back-projection point in the camera coordinate system, it is specifically used for: Determine the external parameters of the camera coordinate system relative to the ground coordinate system according to the height of the imaging device from the ground when taking the specific image and the angle of the imaging device relative to the ground; Determine the coordinate position of the back-projection point in the ground coordinate system according to the coordinate position of the back-projection point in the camera coordinate system and the external parameters of the camera coordinate system relative to the ground coordinate system.

54. The control device according to any one of claims 44-51, wherein, the trajectory generation algorithm includes: a minimum-swing trajectory generation algorithm.

55. The control device according to claim 40, wherein, further includes: a detection module, configured to detect whether there is an obstacle in the part of the first flight trajectory in front of the first unmanned aerial vehicle when the first unmanned aerial vehicle flies along the first flight trajectory; a start module, configured to start the obstacle avoidance function of the first unmanned aerial vehicle when there is an obstacle in the part of the first flight trajectory in front of the first unmanned aerial vehicle; a control module, configured to control the first unmanned aerial vehicle to return to the first flight trajectory after the first unmanned aerial vehicle bypasses the obstacle.

56. The control device according to claim 40, wherein, further includes: a sending module, configured to upload the first flight trajectory to a specific server; or, the control device is a first ground station, and the control device further includes: a sending module, configured to send the first flight trajectory to a second ground station.

57. An unmanned aerial vehicle, wherein, includes: a fuselage; a power system, installed on the fuselage, for providing flight power; a flight controller, communicatively connected to the power system, for controlling the flight of the unmanned aerial vehicle; the flight controller includes the control device according to any one of claims 24-39; or, the flight controller includes the control device according to any one of claims 43-56.

Images