A flight control method and system for an unmanned aerial vehicle

Abstract

An embodiment of the present specification provides a flight control method for an unmanned aerial vehicle, the method comprising: acquiring a pattern; acquiring one or more pixel positions corresponding to the pattern on an image plane of a terminal; determining a target distance, wherein the target distance represents the distance from a target plane in a three-dimensional space where the terminal is located to a reference point in the three-dimensional space; determining a mapping path corresponding to the pattern in the target plane based on the one or more pixel positions corresponding to the pattern, the target distance, and a preset mapping relationship, wherein the preset mapping relationship represents a mapping relationship between a position in the image plane of the terminal and a mapping position of the position in the target plane in the three-dimensional space with respect to the reference point; and determining a flight trajectory of the unmanned aerial vehicle based on the mapping path.

Description

Technical Field

[0001] This specification relates to the field of UAV flight control, and in particular to a UAV flight control method and system. Background Art

[0002] Drones can be used in a variety of fields, including power inspection, agricultural insurance, environmental protection, film and television production, property rights verification, street scenes, express delivery, and disaster relief. In these fields, drone flight control can achieve various tasks or goals. For example, drone flight control can be used to create patterns and perform performances in physical space. In some application scenarios (such as drone performances, signal transmission, film and television production, and flight paths for reconnaissance or data collection), it is necessary to control the drone to fly according to a user-specified pattern within a specific plane in space.

[0003] Therefore, it is necessary to provide a method and system for flight control of a drone, so that the drone can fly in a certain plane in space according to a pattern specified by a user. Summary of the Invention

[0004] One of the embodiments of the present specification provides a flight control method for an unmanned aerial vehicle, the method comprising: obtaining a pattern; obtaining one or more pixel positions corresponding to the pattern on an image plane of a terminal; determining a target distance, wherein the target distance represents the distance from a target plane in a three-dimensional space where the terminal is located to a reference point in the three-dimensional space; determining a mapping path corresponding to the pattern in the target plane based on the one or more pixel positions corresponding to the pattern, the target distance, and a preset mapping relationship, wherein the preset mapping relationship represents a mapping relationship between a position in the image plane of the terminal and a mapping position of the position in the target plane in the three-dimensional space with respect to the reference point; and determining a flight trajectory of the unmanned aerial vehicle based on the mapping path.

[0005] One of the embodiments of the present specification provides a flight control system for an unmanned aerial vehicle, comprising: a pattern acquisition module for acquiring a pattern; a pattern conversion module for acquiring one or more pixel positions corresponding to the pattern on the image plane of the terminal; a target distance determination module for determining a target distance, wherein the target distance represents the distance from the target plane in the three-dimensional space where the terminal is located to a reference point in the three-dimensional space; a mapping path determination module for determining a mapping path corresponding to the pattern in the target plane based on the one or more pixel positions corresponding to the pattern, the target distance and a preset mapping relationship, wherein the preset mapping relationship represents a mapping relationship between the position of the terminal in the image plane and the mapping position of the position in the target plane in the three-dimensional space with respect to the reference point; and a flight trajectory determination module for determining the flight trajectory of the unmanned aerial vehicle based on the mapping path.

[0006] One of the embodiments of this specification provides a flight control device for an unmanned aerial vehicle, comprising at least one storage medium and at least one processor, wherein the at least one storage medium is used to store computer instructions; and the at least one processor is used to execute the computer instructions to implement a flight control method for the unmanned aerial vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] This specification will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, like numbers represent like structures, wherein:

[0008] Figure 1 is a schematic diagram of an application scenario of a flight control system for a drone according to some embodiments of this specification;

[0009] Figure 2 is a module diagram of a flight control system for a drone according to some embodiments of this specification;

[0010] Figure 3 is an exemplary flow chart of flight control of a drone according to some embodiments of this specification;

[0011] Figure 4 is a schematic diagram of a three-dimensional space where a terminal is located according to some embodiments of this specification;

[0012] Figure 5 is an exemplary flow chart of determining a mapping path corresponding to a pattern in a screen of a terminal in a target plane according to some embodiments of this specification;

[0013] Figure 6 It is a schematic diagram of the preset mapping relationship shown in some embodiments of this specification. DETAILED DESCRIPTION

[0014] To more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly describes the drawings required for describing the embodiments. Obviously, the drawings described below are merely examples or embodiments of this specification. Those skilled in the art can apply this specification to other similar scenarios based on these drawings without inventive effort. Unless otherwise apparent from the context or otherwise noted, the same reference numerals in the figures represent the same structure or operation.

[0015] It should be understood that the terms "system," "device," "unit," and / or "module" used herein are a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other terms can achieve the same purpose, the terms may be replaced by other expressions.

[0016] As used in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not refer to the singular but also include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0017] Flowcharts are used throughout this specification to illustrate the operations performed by systems according to embodiments of this specification. It should be understood that preceding or following operations do not necessarily need to be performed in exact order. Instead, the steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0018] Figure 1 1 is a schematic diagram of an application scenario of a drone flight control system according to some embodiments of this specification. In some embodiments, the drone flight control scenario 100 may include a drone 110, a processing device 120, a terminal 130, and a network 140.

[0019] The drone 110 refers to an unmanned aerial vehicle that can be operated using a radio remote control device and / or a self-contained program control device. The drone 110 may include an unmanned fixed-wing aircraft, an unmanned vertical take-off and landing aircraft, an unmanned airship, an unmanned helicopter, an unmanned multi-rotor aircraft, an unmanned paraglider, etc. In some embodiments, the drone 110 may include one or more drones. In some embodiments, a light-emitting device (not shown in the figure) may be provided on the drone 110. In some embodiments, the light-emitting device may include various light-emitting devices such as LED lights, lasers, fluorescent lamps, etc. In some embodiments, the light-emitting device may be controlled by a control signal of a processing device (such as a circuit signal, a wireless signal, etc.) to control its light switch, light-emitting time, light intensity, light direction, light-emitting mode, etc. In some embodiments, the drone 110 can fly according to the control instructions of the processing device.

[0020] The processing device 120 can be a system with computing and processing capabilities. The processing device 120 can include various computers, such as servers, personal computers, or a computing platform composed of multiple computers connected in various structures. In some embodiments, the processing device 120 can be implemented on a cloud platform. For example, the cloud platform can include one or more combinations of private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, cross-cloud, multi-cloud, etc. The processing device 120 can include one or more sub-processing devices (for example, a single-core processing device or a multi-core multi-core processing device). By way of example only, the processing device 120 can include various common general-purpose central processing units (CPUs), graphics processing units (GPUs), microprocessors, application-specific integrated circuits (ASICs), or other types of integrated circuits.

[0021] Terminal 130 may include a mobile device 130-1, a tablet computer 130-2, a laptop computer 130-3, a camera (not shown), or any combination thereof. In some embodiments, terminal 130 may include a processing device. In some embodiments, processing device 120 may be integrated into terminal 130. In some embodiments, terminal 130 may be used by one or more users, including those directly controlling the flight of the drone or other related users.

[0022] Network 140 may include any suitable network that facilitates information and / or data exchange. For example, network 140 may include a public network (e.g., the Internet), a private network (e.g., a local area network (LAN), a wide area network (WAN), etc.), a wired network (e.g., an Ethernet network), a wireless network (e.g., an 802.11 network, a Wi-Fi network, etc.), a cellular network (e.g., a Long Term Evolution (LTE) network), a frame relay network, a virtual private network (VPN), a satellite network, a telephone network, or a combination thereof. In some embodiments, components in scene 100 (e.g., drone 110, processing device 120, and terminal 130) may exchange information and / or data via network 140.

[0023] In some embodiments, processing device 120 may receive the pattern determined by terminal 130 and parameters of terminal 130 (e.g., the terminal's position, the terminal's image plane position, the first parameter, the second parameter, etc.) via network 140. In some embodiments, processing device 120 may receive information related to the color, touch pressure, line type (e.g., line thickness), and other points in the pattern from terminal 130 via network 140. In some embodiments, processing device 120 may obtain data and / or information directly from drone 110 and / or terminal 130.

[0024] In some embodiments, a user can send and / or receive information related to drone flight control to processing device 120 via a user interface on terminal 130. In some embodiments, the user interface can be in the form of an application for implementing drawing on terminal 130 to obtain a pattern in a picture. The user interface can be configured to facilitate communication between terminal 130 and a user associated with terminal 130. In some embodiments, the user interface can receive input from the user requesting flight control of the drone via, for example, a user interface screen. The user can send a request to execute flight control of the drone to processing device 120 via the user interface on terminal 130 to control the flight of the drone.

[0025] In some embodiments, processing device 120 can be used to obtain information about other components in scene 100 (e.g., drone 110, terminal 130) and analyze and process the collected information to perform one or more functions described herein. For example, processing device 120 can obtain information related to terminal 130, such as a pattern (e.g., a pattern determined by terminal 130), parameters of terminal 130 (e.g., the terminal's position, the terminal's image plane position, a first parameter, a second parameter, etc.). For another example, processing device 120 can obtain the target position of a drone (e.g., drone 110) in three-dimensional space. For another example, processing device 120 can determine the target distance. For another example, processing device 120 can obtain one or more pixel positions corresponding to the pattern on the image plane of terminal 130. For another example, processing device 120 can determine a mapping path corresponding to the pattern in the target plane based on the one or more pixel positions corresponding to the pattern, the target distance, and a preset mapping relationship. For another example, processing device 120 can determine a flight trajectory for the drone based on the mapping path. For another example, processing device 120 can control drone 110 to fly according to the determined flight trajectory.

[0026] In some embodiments, the drone can fly according to the control instructions of the processing device 120. In some embodiments, the drone 110 can fly according to the flight trajectory determined by the processing device 120 based on the pattern in the terminal screen.

[0027] In some embodiments, scene 100 may further include a shooting terminal (not shown). In some embodiments, the shooting terminal may shoot a flight trajectory video of drone 110. In some embodiments, terminal 130 may also serve as a shooting terminal.

[0028] Figure 2 This is a module diagram of a flight control system for a drone according to some embodiments of this specification. Figure 2 As shown, the flight control system 200 of the UAV may include an acquisition module 210 , a pattern conversion module 220 , a target distance determination module 230 , a mapping path determination module 240 , and a flight trajectory determination module 250 .

[0029] In some embodiments, the pattern acquisition module 210 may be used to acquire a pattern.

[0030] In some embodiments, the pattern conversion module 220 may be configured to obtain one or more pixel positions corresponding to the pattern on the image plane of the terminal.

[0031] In some embodiments, the target distance determination module 230 can be configured to determine a target distance, where the target distance represents the distance from a target plane in the three-dimensional space where the terminal resides to a reference point in the three-dimensional space. In some embodiments, the target distance determination module 230 can also be configured to obtain the target position of the drone in the three-dimensional space, determine the distance from the target position to the reference point in the three-dimensional space, and determine the distance as the target distance. In some embodiments, the reference point is the optical center of the terminal.

[0032] In some embodiments, the mapping path determination module 240 can be used to determine the mapping path corresponding to the pattern in the target plane based on one or more pixel positions corresponding to the pattern, the target distance and a preset mapping relationship, wherein the preset mapping relationship represents the mapping relationship between the position of the terminal in the image plane and the mapping position of the position in the target plane in the three-dimensional space with respect to the reference point.

[0033] In some embodiments, the flight trajectory determination module 250 may be configured to determine the flight trajectory of the drone based on the mapped path.

[0034] In some embodiments, the flight control system 200 of the drone may further include a flight control module (not shown). In some embodiments, the flight control module may be used to control the drone to fly according to the flight trajectory.

[0035] For more details about the acquisition module 210, the pattern conversion module 220, the target distance determination module 230, the mapping path determination module 240, the flight trajectory determination module 250, the flight control module and the preset mapping relationship, see Figure 3-Figure 6 and its related descriptions.

[0036] It should be understood that Figure 2 The system and its modules shown can be implemented in various ways. It should be noted that the above description of the candidate display and determination system and its modules is only for the convenience of description and does not limit this specification to the scope of the embodiments. It is understandable that for those skilled in the art, after understanding the principles of the system, it is possible to arbitrarily combine the modules or form subsystems to connect with other modules without deviating from this principle. In some embodiments, Figure 2 The acquisition module, target distance determination module, mapping path determination module, and flight trajectory determination module disclosed in the present disclosure may be separate modules within a single system, or a single module may implement the functions of two or more of the aforementioned modules. For example, each module may share a storage module, or each module may have its own storage module. Such variations are within the scope of protection of this specification.

[0037] Figure 3 is an exemplary flow chart of a flight control method for a drone according to some embodiments of this specification. In some embodiments, process 300 may be executed by the processing device 120. Figure 3 As shown, process 300 may include the following steps:

[0038] Step 310 , acquiring a pattern. In some embodiments, this step may be performed by the pattern acquisition module 210 .

[0039] A pattern is a graphic design that can be composed of geometric shapes such as points, lines, and surfaces. For example, a pattern could be a sketch of a person. Another example is a line drawing.

[0040] In some embodiments, patterns can be acquired through various feasible methods. For example, patterns can be acquired by selecting from an existing pattern library, downloading from elsewhere via a network, importing patterns stored in other devices, etc. In some embodiments, patterns can be acquired through terminal 130. In some embodiments, patterns can be acquired by a user through the aforementioned methods, or automatically by a processing device (e.g., automatically acquired according to a rule or requirement).

[0041] In some embodiments, a desired image (e.g., a photograph, a composite image, etc.) can be obtained using a method similar to the aforementioned pattern acquisition method, and the image can be processed to generate a corresponding pattern. For example, the image can be stylized using a deep learning model (e.g., a Generative Adversarial Network (GAN), a Convolutional Neural Network (CNN), etc.) to obtain the desired pattern.

[0042] In some embodiments, the user may draw on a terminal (eg, terminal 130 or other terminals) to obtain a desired pattern.

[0043] Step 320 , obtaining one or more pixel positions corresponding to the pattern on the image plane of the terminal. In some embodiments, this step is performed by the pattern conversion module 220 .

[0044] The terminal here may be the aforementioned terminal 130. For details about the terminal, please refer to Figure 1 and its related descriptions.

[0045] A terminal may include an optical system (also called an imaging system), which may include components such as a lens, an imaging medium (e.g., a CCD, CMOS, or other imaging sensor, and film). Generally speaking, a lens focuses light onto the imaging medium, thereby capturing an image. The terminal's image plane refers to the plane where the imaging medium resides, and may also be called the imaging plane or pixel plane.

[0046] In some embodiments, the terminal may be placed at a certain location in the physical world. In some embodiments, the location and posture of the terminal may be determined by positioning with a positioning device, measuring with an inertial measurement unit, or other various feasible methods.

[0047] In some embodiments, the pattern can be converted into one or more corresponding pixel positions on the image plane of the terminal by various feasible image processing methods. For example, the picture where the pattern is located can be pixelated by various existing methods, wherein the pixel value of one or more pixels constituting the pattern can be 1, and the pixel values ​​of the pixels corresponding to the remaining positions in the picture can be 0; the image plane can be composed of multiple pixel arrangements (for example, the image plane is composed of an m*n pixel matrix), and further, the one or more pixels corresponding to the pattern can be represented by one or more pixel positions on the image plane. For example, each pixel is represented by a two-dimensional array (or two-dimensional vector) [u, v], wherein u represents the horizontal coordinate of the pixel in the image plane (for example, the pixel row number), and v represents the vertical coordinate of the pixel in the image plane (for example, the pixel column number). As an example, if the pixel position corresponding to a certain pixel of the image on the image plane is [2, 3], it means that in the pixel matrix of the image plane, the position corresponding to the pixel is in the 2nd row and the 3rd column.

[0048] Step 330 : Determine a target distance, where the target distance represents the distance from a target plane in the three-dimensional space where the terminal is located to a reference point in the three-dimensional space. In some embodiments, this step is performed by the target distance determination module 230 .

[0049] The three-dimensional space where the terminal is located may refer to an objectively existing physical space, in which a three-dimensional space coordinate system may be established and the position in the space may be represented by the coordinates of the three-dimensional space coordinate system.

[0050] Figure 4 is a schematic diagram of a three-dimensional space where a terminal is located according to some embodiments of this specification, wherein the terminal 420, the drone 410, the target plane 430, the three-dimensional space coordinate system {x i -y i -z i}、Terminal coordinate system of terminal 420 {x c -y c -z c}.

[0051] The origin of the three-dimensional space coordinate system {xi-yi-zi} can be any point in the three-dimensional space and can be set according to needs or experience.

[0052] The terminal coordinate system refers to the coordinate system associated with the terminal. The relative position and direction of each coordinate axis of the terminal coordinate system follow a fixed rule (for example, the right-hand rule). The origin of the terminal coordinate system is determined based on the relative position of the terminal. Generally, the origin can be fixed to a certain point on the terminal. For example, the origin can be the position of the optical center of the terminal's lens. For example, Figure 4 As shown, C: {x c -y c -z c} is the terminal coordinate system of the terminal 420, where the origin is at the optical center of the lens of the terminal 420, x c is consistent with the direction of the main optical axis of the terminal 420, z c The direction perpendicular to the main optical axis and parallel to the image plane of the terminal 420, y c Perpendicular to z c direction and is parallel to the image plane of terminal 420.

[0053] In some embodiments, the position of the optical center of the lens of the terminal 420 (for example, the coordinates of the optical center in the three-dimensional space coordinate system {xi-yi-zi}) and the posture information of the terminal can be obtained through sensors (for example, a GPS module, an inertial measurement unit, etc.), thereby determining the terminal coordinate system.

[0054] In some embodiments, a plane can be specified in the three-dimensional space where the terminal is located, and the specified plane can be the target plane. The target plane can be used as the flight plane of the drone. For example, Figure 4 As shown, the target plane 430 may be the flight plane of the drone 410. In some embodiments, the target plane is perpendicular to the main optical axis of the terminal. Figure 4 As shown, the target plane 430 is perpendicular to the x c direction.

[0055] In some embodiments, a reference point may be specified in the three-dimensional space where the terminal is located. For example, it may be the optical center of a lens of the terminal, which may be represented by point o.

[0056] In some embodiments, the distance from the target plane to the reference point o may be referred to as the target distance, which may be represented by xc. The distance xc from the target plane to the reference point o may refer to the distance along a specified direction (e.g. Figure 4 Center axis x j direction), the distance from the reference point o to the target plane.

[0057] In some embodiments, the target distance may be determined in a variety of ways.

[0058] In some embodiments, the target distance xc can be set as required. For example, the user can manually input the target distance xc through the terminal.

[0059] In some embodiments, the drone can first be flown to a certain position in the three-dimensional space, and the position is determined to be the target position. Then, the coordinates of the target position of the drone in the three-dimensional space (for example, the target position in the three-dimensional space coordinate system {x i -y i -zi}); As can be seen from the above, the posture of the terminal can be determined. Based on the posture of the terminal, the posture transformation matrix (also called rotation matrix) R between the three-dimensional space coordinate system and the terminal coordinate system of the terminal can be determined (the coordinates of a position in the three-dimensional space coordinate system are multiplied by R to obtain the coordinates of the position in the terminal coordinate system); then, based on R and the target position in the three-dimensional space coordinate system {x i -y i -z i} determines the distance (i.e., target distance) from the target position to the reference point o (i.e., the origin of the terminal coordinate system - the optical center of the lens of the terminal 420). For example, the target position is in the three-dimensional space coordinate system {x i -y i -z i} is p1, and the coordinate position of the terminal in the three-dimensional space coordinate system {xi-yi-zi} is p0. The coordinate position pc of the target position in the terminal coordinate system (which can be expressed as [xc, y1, z1]) can be determined as pc=R(p1-p0), and then the target distance xc can be determined based on pc([xc, y1, z1]).

[0060] In some embodiments, the drone used to determine the target location may be a drone flying in a flight plane, or may be another drone not flying in a flight plane.

[0061] Step 340: Determine a mapping path corresponding to the pattern in the target plane based on the one or more pixel positions corresponding to the pattern, the target distance, and a preset mapping relationship, where the preset mapping relationship represents a mapping relationship between the position of the terminal in the image plane and its mapped position in the target plane in the three-dimensional space with respect to the reference point. In some embodiments, this step may be performed by mapping path determination module 240.

[0062] The mapping position refers to the position of the terminal in the image plane mapped to the position in the target plane. Figure 4 As shown, the position p' of the terminal 420 in the image plane is mapped to the position p in the target plane 430.

[0063] In some embodiments, the preset mapping relationship is related to the target distance and the relevant parameters of the terminal, and can be predetermined. For more details about the preset mapping relationship, please refer to Figure 5 and its related descriptions.

[0064] The mapping path refers to the path that the pattern is mapped to in the target plane. Figure 4 As shown, the path 430 - 1 corresponding to the pattern 420 - 1 mapped in the target plane 430 is 430 - 1 .

[0065] In some embodiments, after determining one or more pixel positions and target distances corresponding to the pattern, a mapping path corresponding to the pattern in the target plane can be determined based on the one or more pixel positions, target distances, and a preset mapping relationship.

[0066] For more information on determining the mapping path of the pattern in the target plane, see Figure 5 The related descriptions will not be repeated here.

[0067] Step 350 : Determine the flight trajectory of the drone based on the mapped path. In some embodiments, this step is performed by the flight trajectory determination module 250 .

[0068] A flight trajectory is a sequence of consecutive positions of a drone in three-dimensional space. In some embodiments, a drone can fly along a flight trajectory consisting of a series of time-sequential three-dimensional points. The drone can fly to each of the three-dimensional points in the flight trajectory in sequence, thereby achieving flight along the flight trajectory.

[0069] In some embodiments, the flight trajectory of the drone can be determined based on the mapped path corresponding to the pattern in the target plane. In some embodiments, the mapped path can be processed using various feasible drone flight trajectory processing algorithms to obtain a flight trajectory comprising a series of spatial three-dimensional points with time sequence.

[0070] In some embodiments, after determining the flight trajectory, a starting point can be specified in the flight trajectory as the first point to be flown to by the drone, and the drone can then continue to fly to various three-dimensional points according to the time sequence of each point in the flight trajectory. In some embodiments, no starting point is required, and the drone can use any three-dimensional point in the flight trajectory (e.g., the point closest to the drone) as the first point to be flown to.

[0071] In some embodiments, the drone can be controlled to fly along a predetermined flight trajectory using various drone automatic flight algorithms (e.g., trajectory tracking algorithms). Since the flight trajectory is located on a target plane in three-dimensional space, it is understood that the drone will always remain in the target plane when flying along the flight trajectory.

[0072] In some embodiments, the drone may be provided with a light emitting device. For more details about the light emitting device, please refer to Figure 1 In some embodiments, the light emitting device may be controlled to emit light while the drone is flying along the flight trajectory. For example, the light emitting device may be controlled to continuously emit light while the drone is flying along the flight trajectory.

[0073] In some embodiments, while the drone is flying along a flight trajectory, a video of the drone's flight trajectory can be captured by a filming terminal (e.g., terminal 130 or another filming terminal). In some embodiments, as previously described, while the drone is flying along the flight trajectory, the light-emitting device can be controlled to emit light. It is understood that the flight trajectory video captured by the filming terminal can display the light trajectory of the drone.

[0074] In some embodiments, when the shooting terminal shoots the flight trajectory video of the drone, shooting parameters such as exposure shutter time, focus, sensitivity ISO, etc. can be set to achieve the best shooting effect.

[0075] In some embodiments, after obtaining a drone's flight trajectory video, various processing operations may be performed on the video to obtain a desired image (e.g., synthesizing multiple frames in the video to obtain a light trail image) and / or a processed video (e.g., a processed short video). The multi-frame synthesis method may include extracting frames from a suitable time period, pixel overlay, and other methods.

[0076] In some embodiments, the processing device may further determine the color of each of the multiple points in the pattern, where each point in the pattern may correspond to a pixel position in the aforementioned image plane. In some embodiments, the processing device may further determine, based on the color of the point in the pattern, the color of the light-emitting device when the drone flies to the mapping position corresponding to the pixel position corresponding to that point. For example, if the color of a point in the pattern is red, then when the drone flies to the mapping position corresponding to the pixel position corresponding to that point, the light-emitting device will emit red light.

[0077] In some embodiments, a pattern can be drawn by a user on a pressure-sensitive touch screen (e.g., the pressure-sensitive touch screen of terminal 130). In some embodiments, the processing device can obtain the touch pressure of each of the multiple points of the pattern. In some embodiments, the processing device can also determine, based on the touch pressure of the point, the luminous intensity of the light-emitting device when the drone flies to the mapped position corresponding to the pixel position corresponding to the point. For example, the greater the touch pressure on a point of the pattern, the stronger the luminous intensity of the light-emitting device when the corresponding drone flies to the mapped position corresponding to the pixel position corresponding to the point.

[0078] In some embodiments, the processing device may obtain the line thickness corresponding to each of the multiple points of the pattern (which may refer to the line thickness of the line where the point is located). In some embodiments, the processing device may also determine the luminous intensity of the light-emitting device when the drone flies to the mapping position corresponding to the pixel position corresponding to the point based on the line thickness of the point. The line thickness may include 0.5 pounds, 1 pound, 1.5 pounds, etc. In some embodiments, different line thicknesses may correspond to different luminous intensities of the light-emitting device, for example, 1 pound corresponds to luminous intensity a of the light-emitting device, and 0.5 pounds corresponds to luminous intensity b of the light-emitting device, and luminous intensity a may be greater than luminous intensity b.

[0079] In some embodiments, a processing device can determine multiple flight trajectories for multiple drones and control the multiple drones to fly along their corresponding flight trajectories according to methods described elsewhere in this specification. Each drone can determine a target distance xc, and the multiple xc values ​​corresponding to the multiple drones are similar in size, with only minor differences. A corresponding flight trajectory can be determined for each drone according to method 300. It will be appreciated that, according to this embodiment, the three-dimensional points [xc, yc, zc] in the flight trajectories of the multiple drones differ only slightly in xc, while yc and zc are the same. In some embodiments, as the multiple drones fly along their respective flight trajectories, the light-emitting devices on the multiple drones can be controlled to scatter light (where the light-emitting devices on different drones emit different colors, and the colors can be determined as needed, such as from the three primary colors), thereby achieving a color-matching effect for the lights on the multiple drones.

[0080] In some embodiments, the processing device can determine multiple mapping paths corresponding to multiple patterns (or multiple partial patterns included in a single pattern) in a target plane according to method 300, thereby determining multiple flight paths in the target plane. In some embodiments, each of the multiple flight paths can correspond to a drone, and the processing device can control the multiple drones to fly in the same target plane according to the corresponding flight paths according to the methods described elsewhere in this specification. This makes the patterns presented by the drone flight paths more vivid and varied, and can meet more flight requirements and mission objectives.

[0081] Through the drone flight control method described in method 300, the drone can fly in a set flight plane in the air according to the pattern on the screen of the shooting terminal, so as to achieve the goal of presenting the pattern in the set flight plane in the air, and the drone remains flying in the set flight plane, which can make the shooting angle and shooting range of the shooting terminal more stable when shooting the flight trajectory video, and the shooting effect is also better.

[0082] It should be noted that the above description of the process drone's flight control is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art will be able to make various modifications and changes to the process drone's flight control under the guidance of this specification. However, such modifications and changes remain within the scope of this specification.

[0083] Figure 5 This is an exemplary flow chart of determining the mapping path of a pattern in a terminal screen corresponding to a target plane according to some embodiments of this specification. In some embodiments, process 500 may be executed by the processing device 120. Figure 5 As shown, process 500 includes the following steps:

[0084] Step 510 : Based on the one or more pixel positions, the target distance, and the preset mapping relationship, determine that the one or more pixel positions correspond to one or more mapping positions in the target plane. In some embodiments, this step is performed by the mapping path determination module 240 .

[0085] For each of the one or more pixel positions corresponding to the pattern, the processing device can determine the corresponding mapping position of the pixel position in the target plane based on the pixel position, the target distance, and the preset mapping relationship. The preset mapping relationship and the determination of the corresponding mapping position of the pixel position in the target plane based on the pixel position, the target distance, and the preset mapping relationship are described in detail below.

[0086] Figure 6 is a schematic diagram of a preset mapping relationship according to some embodiments of this specification. Figure 6 The terminal image plane 610, the target plane 630 in the three-dimensional space, the reference plane 620 (ie, the plane where the reference point o is located), the terminal coordinate system {x c -y c -z c}, the image coordinate system {xy} in the image plane, and the pixel coordinate system {uv} in the image plane.

[0087] The origin o' of the image coordinate system {xy} can be defined at any point in the image plane, as an example, Figure 6 As shown, o' can be defined at the center of the image plane and on the principal optical axis of the terminal.

[0088] The origin o0 of the pixel coordinate system {uv} can be defined at any point in the image plane, as an example, Figure 6As shown, o0 can be defined at a vertex of the pixel matrix in the image plane. The pixel coordinate system {uv} represents the coordinate position in pixels. For example, the image plane is composed of a plurality of pixels, and the coordinate [2, 3] in the pixel coordinate system {uv} represents the position corresponding to the pixel in the 2nd row and 3rd column. It can be understood that if the length of a pixel in the image plane is α and the width is β, then the coordinate (1, 1) in the pixel coordinate system {uv} corresponds to the coordinate (1*α-c x , 1*β-c y ), where c x is the offset of the origin of the image coordinate system relative to the origin of the pixel coordinate system on the x-axis, c y The offset of the origin of the image coordinate system relative to the origin of the pixel coordinate system on the y-axis.

[0089] The center of the image plane (i.e. Figure 6 The distance from (o') to the optical center of the terminal is the focal length f of the optical system of the terminal, and the focal length f can be determined based on the parameters of the terminal itself.

[0090] In some embodiments, the preset mapping relationship includes a preset formula (1) (also referred to as a first preset formula). In some embodiments, if the coordinates [x, y] of a position p' in the image plane in the image coordinate system and the target distance xc are known, the coordinates [xc, yc, zc] of the mapping position p corresponding to the position in the target plane in the terminal coordinate system can be determined by the preset formula (1):

[0091] x / yc=y / zc=f / xc (1)

[0092] Where x is the x-axis coordinate value of the position p' in the image plane in the image coordinate system, y is the y-axis coordinate value of the position p' in the image plane in the image coordinate system, f is the focal length of the terminal's optical system, and xc is the x-axis coordinate value of the mapped position p in the target plane corresponding to the position p' in the terminal coordinate system. c Axis coordinate value, yc is the y coordinate of the position p' corresponding to the mapping position p in the target plane in the terminal coordinate system c Axis coordinate value, zc is the z coordinate of the position p' corresponding to the mapping position p in the target plane in the terminal coordinate system c Axis coordinate values.

[0093] Thus, for each of the one or more pixel positions corresponding to the pattern, the coordinates of the pixel position in the image coordinate system in the image plane can be determined, and then the processing device can determine the coordinates of the mapping position corresponding to the pixel position in the target plane in the terminal coordinate system based on the coordinates of the pixel position in the image coordinate system in the image plane, the target distance, and the preset formula (1).

[0094] In some embodiments, after determining the coordinates [xc, yc, zc] of the mapping position in the terminal coordinate system, the coordinates [xc, yc, zc] in the terminal coordinate system can be converted into a three-dimensional space coordinate system {x i -y i -z i}, thereby obtaining the mapping position in the three-dimensional space coordinate system {x i -y i -z i} in the coordinates.

[0095] In some embodiments, the preset mapping relationship may further include a preset formula (2) (also referred to as a second preset formula). In some embodiments, if the coordinates [u, v] of a position p' in the image plane in the pixel coordinate system and the target distance xc are known, the coordinates [x, y] of the position p' in the image coordinate system can be determined by formula (2), and then the coordinates [xc, yc, zc] of the mapping position p in the target plane corresponding to the position p' in the terminal coordinate system can be determined based on formula (1):

[0096] u=x / α+c x , v=y / β+c y (2)

[0097] Where u is the u-axis coordinate value of position p' in the image plane in the pixel coordinate system, v is the v-axis coordinate value of position p' in the image in the pixel coordinate system, α is the length of a pixel in the x-axis direction in the pixel coordinate system, and β is the length of a pixel in the y-axis direction in the pixel coordinate system. a, β, c x 、c y Both can be determined based on existing parameters of the terminal itself.

[0098] Thus, for each of the one or more pixel positions corresponding to the pattern, the coordinates of the pixel position in the pixel coordinate system in the image plane can be determined, and then the processing device can determine the coordinates of the mapping position corresponding to the pixel position in the target plane in the terminal coordinate system based on the coordinates of the pixel position in the pixel coordinate system in the image plane, the target distance, the preset formula (1) and the preset formula (2).

[0099] In some embodiments, the preset mapping relationship may include a preset formula (3). In some embodiments, if the coordinates [u, v] of a position p' in the image plane in the pixel coordinate system and the target distance xc are known, the coordinates [xc, yc, zc] of the mapping position p in the target plane corresponding to the position p' in the terminal coordinate system can be determined by the preset formula (3) (in this case, the origin o' of the image coordinate system coincides with the origin o0 of the pixel coordinate system):

[0100]

[0101] Among them, f y and f z can be referred to as the first and second parameters of the optical system of the terminal, and f y =(1 / α*f) / xc,f z =(1 / β*f) / xc.

[0102] Step 520 : Determine the mapping path based on the one or more mapping locations. In some embodiments, this step is performed by the mapping path determination module 240 .

[0103] According to the aforementioned steps, for each of the one or more pixel positions corresponding to the pattern, the processing device can determine the mapping position corresponding to the pixel position in the target plane, thereby obtaining the one or more mapping positions corresponding to the pattern.

[0104] In some embodiments, the obtained one or more mapping positions may constitute a path, which may be referred to as a mapping path.

[0105] It should be noted that the above description of the mapping path of the process-determined pattern in the target plane is for illustrative purposes only and does not limit the scope of application of this specification. Those skilled in the art, guided by this specification, may make various modifications and alterations to the mapping path of the process-determined pattern in the target plane. However, such modifications and alterations remain within the scope of this specification.

[0106] While the basic concepts have been described above, it will be apparent to those skilled in the art that the detailed disclosure is merely illustrative and does not limit this specification. Although not explicitly stated herein, various modifications, improvements, and revisions to this specification may be made by those skilled in the art. Such modifications, improvements, and revisions are suggested in this specification and remain within the spirit and scope of the exemplary embodiments of this specification.

[0107] This specification also uses specific terms to describe the embodiments of this specification. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "one embodiment," "an embodiment," or "an alternative embodiment" two or more times in different locations in this specification do not necessarily refer to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of this specification may be appropriately combined.

[0108] In addition, unless expressly stated in the claims, the order of the processing elements and sequences, the use of alphanumeric characters, or the use of other names described in this specification are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some of the invention embodiments currently considered useful through various examples, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the spirit and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0109] Similarly, it should be noted that, in order to simplify the presentation of this specification and thus facilitate understanding of one or more embodiments of the invention, the foregoing descriptions of the embodiments of this specification sometimes combine multiple features into a single embodiment, figure, or description thereof. However, this disclosure method does not imply that the subject matter of this specification requires more features than those recited in the claims. In fact, an embodiment may have fewer features than all of the features of a single disclosed embodiment.

[0110] In some embodiments, numbers are used to describe the quantity of components and attributes. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise stated, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the description and claims are approximate values, which may change according to the required characteristics of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the settings of such numerical values ​​are as accurate as possible within the feasible range.

[0111] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, and documents, cited in this specification is hereby incorporated by reference in its entirety. This includes application history documents that are inconsistent with or conflict with the content of this specification, as well as documents (currently or subsequently attached to this specification) that limit the broadest scope of the claims of this specification. It should be noted that if the descriptions, definitions, and / or terminology used in the accompanying materials are inconsistent or conflicting with the content of this specification, the descriptions, definitions, and / or terminology used in this specification will control.

[0112] Finally, it should be understood that the embodiments described in this specification are intended only to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly described and illustrated in this specification.

Claims

1. A flight control method for an unmanned aerial vehicle, comprising: Get the pattern; Obtaining one or more pixel positions corresponding to the pattern on an image plane of the terminal; Determine a target distance, where the target distance represents a distance from a target plane to a reference point, the target plane and the reference point being located in a three-dimensional space where the terminal is located, the reference point being the optical center of the terminal, the target plane being the flight plane of the UAV, the target plane being perpendicular to the principal optical axis of the terminal, and the posture of the terminal being determined by a sensor; determining a mapping path corresponding to the pattern in the target plane based on one or more pixel positions corresponding to the pattern, the target distance, and a preset mapping relationship, wherein the preset mapping relationship represents a mapping relationship between a position of the terminal in the image plane and a mapping position of the position in the target plane in the three-dimensional space with respect to the reference point; A flight trajectory of the UAV is determined based on the mapped path.

2. The method according to claim 1, wherein acquiring a pattern comprises: The pattern is obtained by a user drawing on the terminal.

3. The method of claim 1 , wherein determining the target distance comprises: Obtaining a target position of the UAV in the three-dimensional space; The distance from the target position to the reference point in the three-dimensional space is determined and determined as the target distance.

4. The method of claim 1 , wherein determining a mapping path corresponding to the pattern in the target plane based on one or more pixel positions corresponding to the pattern, the target distance, and a preset mapping relationship comprises: Determining, based on the one or more pixel positions, the target distance, and the preset mapping relationship, that the one or more pixel positions correspond to one or more mapping positions in the target plane; Based on the one or more mapping locations, the mapping path is determined.

5. The method according to claim 1, wherein the preset mapping relationship comprises a first preset formula: ， in, is a pixel position in the image plane in the image coordinate system preset in the image plane Axis coordinate value, y is the pixel position in the image coordinate system Axis coordinate values, is the distance from the reference point to the image plane, The x value of the mapping position of the pixel position corresponding to the target plane in the preset terminal coordinate system c Axis coordinate values, The y coordinate system of the terminal coordinate system corresponds to the mapping position of the certain pixel position in the target plane c Axis coordinate values, The z coordinate of the mapping position of the pixel position in the target plane in the terminal coordinate system c Axis coordinate values; in the preset image coordinate system Axis and The axis is determined by the preset; the x in the preset terminal coordinate system c Axis, y c Axis, z c The axis is determined by preset; is equal to the target distance.

6. The method according to claim 5, wherein the preset mapping relationship further comprises a second preset formula: ， ， in, u is the u-axis coordinate value of the pixel position in the image plane in the preset pixel coordinate system, v is the v-axis coordinate value of the pixel position in the image plane in the preset pixel coordinate system, and α is the coordinate value of a pixel in the preset pixel coordinate system. The length in the axis direction, β is the length of a pixel in the preset pixel coordinate system. Length in the axial direction, c x The origin of the preset image coordinate system and the origin of the preset pixel coordinate system are Axis offset, c y The origin of the preset image coordinate system and the origin of the preset pixel coordinate system are The offset in the axis direction; the u-axis and v-axis in the preset pixel coordinate system are determined by the preset, and the u-axis is axis parallel to the v axis and axis parallel; α and β are determined based on the relevant parameters of the terminal.

7. The method according to claim 1, wherein the preset mapping relationship comprises a third preset formula: in, u is the u-axis coordinate value of a certain pixel position in the image plane in the preset pixel coordinate system, v is the v-axis coordinate value of the certain pixel position in the image plane in the preset pixel coordinate system, The x value of the mapping position of the pixel position corresponding to the target plane in the preset terminal coordinate system c Axis coordinate values, The y value of the mapping position of the pixel position in the target plane in the terminal coordinate system c Axis coordinate values, The z coordinate of the mapping position of the pixel position in the target plane in the terminal coordinate system c Axis coordinate values; the u-axis and v-axis in the preset pixel coordinate system are determined by the preset; the x-axis in the preset terminal coordinate system c Axis, y c Axis, z c The axis is determined by preset; Equal to the target distance; as well as ;in, is the distance from the reference point to the image plane, α is the distance of a pixel in the image plane in a preset pixel coordinate system. The length in the axial direction, β is the distance of a pixel in the image plane in the preset pixel coordinate system. Length in the axial direction; Axis and The axis is determined by the preset, and the u axis is axis parallel to the v axis and axis parallel; α and β are determined based on the relevant parameters of the terminal.

8. The method of claim 1, further comprising: Controlling the UAV to fly according to the flight trajectory; Obtain the flight trajectory video of the drone.

9. A flight control system for an unmanned aerial vehicle, comprising: A pattern acquisition module, used to acquire a pattern; A pattern conversion module, configured to obtain one or more pixel positions corresponding to the pattern on an image plane of the terminal; a target distance determination module, configured to determine a target distance, wherein the target distance represents the distance from a target plane to a reference point, the target plane and the reference point being located in the three-dimensional space where the terminal is located, the reference point being the optical center of the terminal, the target plane being the flight plane of the UAV, the target plane being perpendicular to the principal optical axis of the terminal, and the terminal's posture being determined by a sensor; a mapping path determining module, configured to determine a mapping path corresponding to the pattern in the target plane based on one or more pixel positions corresponding to the pattern, the target distance, and a preset mapping relationship, wherein the preset mapping relationship represents a mapping relationship between a position of the terminal in the image plane and a mapping position of the position in the target plane in the three-dimensional space with respect to the reference point; A flight trajectory determination module is used to determine the flight trajectory of the UAV based on the mapped path.

10. A flight control device for an unmanned aerial vehicle, comprising at least one storage medium and at least one processor, wherein the at least one storage medium is used to store computer instructions; and the at least one processor is used to execute the computer instructions to implement the method according to any one of claims 1 to 8.

Images