Control method and device of unmanned aerial vehicle and unmanned aerial vehicle

Abstract

The embodiment of the present application provides a kind of unmanned plane control method.This method includes: the image that image device is photographed is acquired;Determine the position of the hand of target object in the image in the image;According to the position of the hand in the image, the position information of the hand of the target object is determined, wherein, position information is three-dimensional position information;According to the position information of the hand of the target object, the flight of unmanned plane is controlled.Therefore, the embodiment can control the flight of unmanned plane according to the hand of target object in the image that image device is photographed.Avoid the situation that user must operate control equipment to control unmanned plane, overcome the defect that user is not familiar with control equipment and cannot control unmanned plane, simplify the control mode and operation process of control unmanned plane, improve the efficiency of control unmanned plane flight, enhance the entertainment of man-machine interaction.In addition, the embodiment of the present application also provides a kind of unmanned plane control device.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of unmanned aerial vehicle, and particularly relate to a control method and device of unmanned aerial vehicle and unmanned aerial vehicle. BACKGROUND

[0002] With the more and more extensive application of unmanned aerial vehicle, the control methods of unmanned aerial vehicle become more and more various. From the beginning of using remote controller handle to control unmanned aerial vehicle, the touch device such as mobile phone and tablet computer is also evolved to control unmanned aerial vehicle. However, these control methods of unmanned aerial vehicle need to rely on additional devices other than unmanned aerial vehicle, which not only increases the cost, but also requires users to learn to operate these devices to control unmanned aerial vehicle, and the control method is complex, the requirement for users is high, and the human-computer interaction is poor. SUMMARY

[0003] Embodiments of the present application provide a control method and device of unmanned aerial vehicle and unmanned aerial vehicle to simplify the control method of unmanned aerial vehicle.

[0004] In a first aspect, embodiments of the present application provide a control method of unmanned aerial vehicle, comprising:

[0005] obtaining an image captured by a shooting device;

[0006] determining the position of a hand of a target object in the image;

[0007] determining position information of the hand of the target object according to the position of the hand in the image;

[0008] controlling the flight of the unmanned aerial vehicle according to the position information of the hand of the target object.

[0009] In a second aspect, embodiments of the present application provide a control method of unmanned aerial vehicle, comprising:

[0010] obtaining an image captured by a shooting device;

[0011] determining a feature part of a target object in the image;

[0012] recognizing a hand in the image;

[0013] determining the hand of the target object from the hand in the image according to the feature part of the target object;

[0014] when the hand gesture of the hand of the target object is a control gesture, controlling the unmanned aerial vehicle to perform an action indicated by the gesture.

[0015] In a third aspect, embodiments of the present application provide a control method of unmanned aerial vehicle, comprising:

[0016] obtaining an image captured by a shooting device;

[0017] identify a feature part of the target object in the image;

[0018] identify a hand of the target object in the image;

[0019] when the feature part of the target object is identified and the hand of the target object is not identified, control the UAV to track the target object so that the target object is in a shooting frame of the shooting device.

[0020] In a fourth aspect, an embodiment of the present application provides a control device of a UAV, comprising a memory and a processor;

[0021] the memory is configured to store program code;

[0022] the processor is configured to invoke the program code to perform:

[0023] obtain an image shot by a shooting device;

[0024] determine a position of a hand of a target object in the image;

[0025] determine position information of the hand of the target object according to the position of the hand in the image;

[0026] control flight of the UAV according to the position information of the hand of the target object.

[0027] In a fifth aspect, an embodiment of the present application provides a control device of a UAV, comprising a memory and a processor;

[0028] the memory is configured to store program code;

[0029] the processor is configured to invoke the program code to perform:

[0030] obtain an image shot by a shooting device;

[0031] determine a feature part of a target object in the image;

[0032] identify a hand in the image;

[0033] determine the hand of the target object from the hand identified in the image according to the feature part of the target object;

[0034] when a hand gesture of the hand of the target object is a control gesture, control the UAV to perform an action indicated by the gesture.

[0035] In a sixth aspect, an embodiment of the present application provides a control device of a UAV, comprising a memory and a processor;

[0036] The memory is configured to store program code.

[0037] The processor is configured to invoke the program code to perform:

[0038] acquire an image captured by a photographing device;

[0039] identify a feature part of a target object in the image;

[0040] identify a hand of the target object in the image;

[0041] when the feature part of the target object is identified and the hand of the target object is not identified, control the unmanned aerial vehicle to track the target object so that the target object is in a photographing frame of the photographing device.

[0042] In a seventh aspect, an embodiment of the present application provides an unmanned aerial vehicle, comprising:

[0043] a control device of the unmanned aerial vehicle as described in at least one of the fourth aspect, the fifth aspect, and the sixth aspect;

[0044] a photographing device configured to capture an image;

[0045] and a power system configured to provide flight power.

[0046] In an eighth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon; the computer program, when executed, implements the control method of the unmanned aerial vehicle as described in at least one of the first aspect, the second aspect, and the third aspect.

[0047] The control method, the control device, and the unmanned aerial vehicle provided by the embodiments of the present application acquire an image captured by a photographing device, determine a position of a hand of a target object in the image, determine position information of the hand of the target object according to the position of the hand in the image, and control flight of the unmanned aerial vehicle according to the position information of the hand of the target object. Therefore, the embodiments can control flight of the unmanned aerial vehicle according to the hand of the target object in the image captured by the photographing device. This avoids the situation that a user must operate a control device to control the unmanned aerial vehicle, overcomes the defect that the user cannot control the unmanned aerial vehicle because the user is not familiar with the control device, simplifies a control method and an operation process for controlling the unmanned aerial vehicle, improves efficiency of controlling flight of the unmanned aerial vehicle, and enhances entertainment of human-computer interaction. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 FIG. 1 is a schematic architectural diagram of an unmanned aerial vehicle according to an embodiment of the present application;

[0049] Figure 2 FIG. 4 is a flowchart of a control method of an unmanned aerial vehicle according to an embodiment of the present application.

[0050] Figure 3 A schematic view of a target object in an image captured by a photographing device according to an embodiment of the present application;

[0051] Figure 4 A flowchart of a control method of a UAV according to another embodiment of the present application;

[0052] Figure 5 A schematic view of controlling the height of a UAV according to an embodiment of the present application;

[0053] Figure 6 A flowchart of a control method of a UAV according to another embodiment of the present application;

[0054] Figure 7 A schematic view of controlling the height of a UAV according to another embodiment of the present application;

[0055] Figure 8 A schematic view of controlling the height of a UAV according to another embodiment of the present application;

[0056] Figure 9 A flowchart of a control method of a UAV according to another embodiment of the present application;

[0057] Figure 10 A schematic view of controlling a UAV to fly around a target object according to an embodiment of the present application;

[0058] Figure 11 A schematic view of controlling a UAV to fly around a target object according to another embodiment of the present application;

[0059] Figure 12 A flowchart of a control method of a UAV according to another embodiment of the present application;

[0060] Figure 13 A schematic view of controlling a UAV to fly away from or approach a target object according to an embodiment of the present application;

[0061] Figure 14 A schematic view of controlling a UAV to fly away from or approach a target object according to another embodiment of the present application;

[0062] Figure 15 A flowchart of a control method of a UAV according to another embodiment of the present application;

[0063] Figure 16 A schematic view of controlling a UAV to take off according to an embodiment of the present application;

[0064] Figure 17 A schematic view of controlling a UAV to land according to an embodiment of the present application;

[0065] Figure 18 A flow chart of a control method of a UAV according to another embodiment of the present application;

[0066] Figure 19 A schematic diagram of a control device of a UAV according to an embodiment of the present application;

[0067] Figure 20 A schematic diagram of a UAV according to an embodiment of the present application. DETAILED DESCRIPTION

[0068] Embodiments of the present application provide a control method of a UAV, a control device of a UAV and a UAV. The UAV can be a rotorcraft, for example, a multicopter, and embodiments of the present application are not limited thereto.

[0069] Figure 1 A schematic diagram of a UAV according to an embodiment of the present application. The embodiment is described with respect to a rotorcraft.

[0070] The UAV 100 can include a power system 150, a flight control system 160 and a frame.

[0071] The frame can include a body and a leg (also referred to as a landing gear). The body can include a central frame and one or more arms connected to the central frame, the one or more arms extending radially from the central frame. The leg is connected to the body and serves as a support when the UAV 100 lands.

[0072] The power system 150 can include one or more electronic speed controllers (abbreviated as ESCs) 151, one or more propellers 153 and one or more motors 152 corresponding to the one or more propellers 153, wherein the motor 152 is connected between the ESC 151 and the propeller 153, and the motor 152 and the propeller 153 are arranged on the arm of the UAV 100; the ESC 151 is configured to receive a driving signal generated by the flight control system 160 and provide a driving current to the motor 152 according to the driving signal, so as to control the rotation speed of the motor 152. The motor 152 is configured to drive the propeller to rotate, thereby providing power for the flight of the UAV 100, and the power enables the UAV 100 to realize movement in one or more degrees of freedom. In some embodiments, the UAV 100 can rotate around one or more rotation axes. For example, the rotation axes can include a roll axis (Roll), a yaw axis (Yaw) and a pitch axis (pitch). It should be understood that the motor 152 can be a direct current motor or an alternating current motor. In addition, the motor 152 can be a brushless motor or a brushed motor.

[0073] The flight control system 160 can include a flight controller 161 and a sensing system 162. The sensing system 162 is configured to measure attitude information of the UAV 100, i.e. position information and state information of the UAV 100 in space, such as three-dimensional position, three-dimensional angle, three-dimensional velocity, three-dimensional acceleration, three-dimensional angular velocity, etc. The sensing system 162 can include at least one of a gyroscope, an ultrasonic sensor, an electronic compass, an inertial measurement unit (IMU), a vision sensor, a global navigation satellite system, and a barometer, etc. For example, the global navigation satellite system can be a global positioning system (GPS). The flight controller 161 is configured to control the flight of the UAV 100, for example, the flight of the UAV 100 can be controlled according to the attitude information measured by the sensing system 162. It should be understood that the flight controller 161 can control the UAV 100 according to pre-programmed instructions, or control the UAV 100 by capturing images.

[0074] The UAV 100 further includes a gimbal 120, which can include a motor 122. The gimbal is configured to carry a photographing device 123. The flight controller 161 can control the movement of the gimbal 120 by the motor 122. Alternatively, as another embodiment, the gimbal 120 can further include a controller configured to control the movement of the gimbal 120 by controlling the motor 122. It should be understood that the gimbal 120 can be independent of the UAV 100, or can be part of the UAV 100. It should be understood that the motor 122 can be a direct current motor, or an alternating current motor. In addition, the motor 122 can be a brushless motor, or a brushed motor. It should be further understood that the gimbal can be located at the top of the UAV, or at the bottom of the UAV.

[0075] The photographing device 123 can be a camera or a video camera, etc. for capturing images. The photographing device 123 can communicate with the flight controller, and capture images under the control of the flight controller. The flight controller can also control the UAV 100 according to the images captured by the photographing device 123. The photographing device 123 of the present embodiment at least includes a photosensitive element, such as a complementary metal oxide semiconductor (CMOS) sensor or a charge-coupled device (CCD) sensor. It should be understood that the photographing device 123 can also be directly fixed to the UAV 100, so that the gimbal 120 can be omitted.

[0076] It should be understood that the above naming of the components of the UAV is only for the purpose of identification, and should not be understood as a limitation on the embodiments of the present application.

[0077] Figure 2 The flowchart of the control method of the UAV provided by an embodiment of the present application is shown in Figure 2 The method of the embodiment can include:

[0078] S201, obtaining an image captured by a photographing device.

[0079] The photographing device is an important component of the UAV and can be used to capture images around the UAV. When the UAV enters the gesture control mode, the UAV can control itself according to the image captured by the photographing device. After the photographing device captures an image, the UAV obtains the image captured by the photographing device. The photographing device can be the photographing device 123 described in the foregoing part, which will not be described here.

[0080] S202, determining the position of the hand of a target object in the image.

[0081] This embodiment mainly controls the UAV according to the hand in the image. The user can be located in a place where the photographing device can capture images. The user can be referred to as a target object. The image captured by the photographing device has the target object. Therefore, after obtaining the image captured by the photographing device, this embodiment determines the position of the hand of the target object in the image. The position can be represented by pixel coordinates in the image coordinate system UOV. As shown in Figure 3 Figure 3 The schematic diagram of the target object in the image captured by the photographing device provided by an embodiment of the present application is shown in the figure. The upper left corner of the image can be regarded as the origin (0, 0). The length and width of the image are represented by the number of pixels. Based on this, the pixel coordinates of the hand of the target object in the image can be determined. After obtaining the image of the target object captured by the photographing device, the hand of the target object can be recognized by a neural network that has been trained to recognize the hand. Specifically, the neural network can return the position of the hand of the target object in the image. In some cases, the neural network can return the coordinates of the upper left corner and the lower right corner of the image region corresponding to the hand of the target object in the image.

[0082] After entering the gesture control mode, if multiple people are captured in the image, this embodiment can determine the person closest to the center of the image as the target object.

[0083] S203, determining the position information of the hand of the target object according to the position of the hand in the image.

[0084] ​In this embodiment, after determining the position of the hand of the target object in the image, the position information of the hand of the target object is determined according to the position of the hand in the image. The position information can be represented by three-dimensional coordinates (x, y, z), which can be coordinates in a navigation coordinate system of the UAV, for example, and the origin O of the navigation coordinate system is the take-off point of the UAV, wherein the positive axis of the X axis of the navigation coordinate system points to the north direction, the positive axis of the Y axis of the navigation coordinate system points to the east direction, and the Z axis of the navigation coordinate system is perpendicular to the XOY plane and away from the ground. In some embodiments, the three-dimensional coordinates can also be coordinates in other coordinate systems, which are not limited here.

[0085] S204, controlling the flight of the UAV according to the position information of the hand of the target object.

[0086] In this embodiment, after determining the position information of the hand of the target object, the flight of the UAV is controlled according to the position information of the hand of the target object, for example, the UAV can be controlled to perform various flight operations: controlling the flight height of the UAV, or the UAV can be controlled to fly around the target object, or the UAV can be controlled to fly away from or close to the target object, wherein the flight trajectory of the UAV and the functions implemented by the UAV can be different in different flight operations. Different position information of the hand of the target object can correspond to different flight operations, so the target object can control the UAV to perform different flight operations by controlling the hand to be in different positions. This embodiment will control the UAV to perform the flight operation corresponding to the position information of the hand of the target object.

[0087] The control method of the UAV provided in this embodiment determines the position of the hand of the target object in the image by obtaining the image captured by the photographing device, determines the position information of the hand of the target object according to the position of the hand in the image, and controls the flight of the UAV according to the position information of the hand of the target object. Therefore, this embodiment can control the flight of the UAV according to the hand of the target object in the image captured by the photographing device. This avoids the situation that the user must operate the control device to control the UAV, overcomes the defect that the user cannot control the UAV because the user is not familiar with the control device, simplifies the control method and operation process of controlling the UAV, improves the efficiency of controlling the flight of the UAV, and enhances the entertainment of human-computer interaction.

[0088] In some embodiments, one possible implementation of S203 is to determine the position information of the hand of the target object according to the position of the hand in the image, the attitude of the holder of the camera, the horizontal distance between the target object and the UAV, and the position information of the UAV. The attitude of the holder of the camera determines the attitude of the camera, i.e., determines the attitude angle of the camera, such as the pitch angle and the yaw angle. Therefore, different attitudes of the holder of the camera will result in different position information of the hand of the target object. The horizontal distance between the target object and the UAV determines the horizontal distance between the hand and the UAV, thereby affecting the position information of the hand. Therefore, different horizontal distances between the target object and the UAV will result in different position information of the hand of the target object. In addition, the position information of the hand of the target object is determined by referring to the position information of the UAV, which can be obtained by a positioning sensor configured on the UAV, such as a GPS receiver or a Beidou receiver. In some cases, the positioning sensor can also include an inertial measurement unit, a vision sensor, etc. In summary, the position information of the hand of the target object is determined according to the position of the hand in the image, the attitude of the holder of the camera, the horizontal distance between the target object and the UAV, and the position information of the UAV.

[0089] Specifically, the position of the hand of the target object is determined according to the position of the hand in the image, the attitude of the holder of the camera, the horizontal distance between the target object and the UAV, and the position information of the UAV. The field of view (FOV) of the camera is known, and the angle of the hand relative to the optical axis of the camera can be determined according to the position of the hand in the image. For example, if the hand is at the center of the image, it means that the angle of the hand relative to the optical axis of the camera is 0. If the FOV of the camera is 20 degrees in the horizontal direction, and if the hand is at the leftmost edge of the image, it means that the horizontal angle of the hand relative to the optical axis of the camera is 10 degrees. The vertical direction is similar, which will not be described here. Moreover, the attitude of the holder of the camera also determines the orientation of the optical axis of the camera. The orientation of the hand relative to the UAV can be obtained by combining the angle of the hand relative to the optical axis of the camera and the orientation of the optical axis. The horizontal distance between the hand and the UAV can be obtained according to the horizontal distance between the target object and the UAV, for example, by subtracting an empirical value (e.g., 0.65 meters) from the horizontal distance between the target object and the UAV. The position information of the hand can be determined according to the above-obtained orientation, the distance between the hand and the UAV, and the position information of the UAV.

[0090] In some embodiments, the horizontal distance between the target object and the UAV can be determined in the following manner:

[0091] A possible way is as follows: the position of the feet of the target object in the image is determined, the orientation of the feet of the target object relative to the UAV is determined according to the position information of the feet in the image and the posture of the holder of the camera, the angle of the feet of the target object relative to the UAV in the pitch direction is determined according to the orientation, then, the height value measured by the distance sensor arranged on the UAV is obtained, and the horizontal distance between the target object and the UAV is determined according to the angle in the pitch direction and the height value measured by the distance sensor.

[0092] Another possible way is as follows: the positions of the feet and the head of the target object in the image are determined, the orientation of the feet and the head of the target object relative to the UAV is determined according to the positions of the feet and the head of the target object in the image and the posture of the holder of the camera, the angle of the feet and the head of the target object relative to the UAV in the pitch direction is determined according to the orientation, the height of the target object can be set as an empirical value, for example, 1.75 m, and the horizontal distance between the target object and the UAV is determined according to the angle of the feet and the head of the target object relative to the UAV in the pitch direction and the set height of the target object.

[0093] It can be understood that the horizontal distance between the target object and the UAV can be the horizontal distance obtained by fusing the horizontal distances determined by the above two possible ways.

[0094] In some embodiments, S204 can be as follows: the flight height of the UAV is controlled according to the position information of the hand of the target object, for example, the flight height of the UAV can be adjusted to be higher or lower.

[0095] In a possible implementation, the embodiment can determine the angle of the hand relative to the UAV in the pitch direction according to the position information of the hand of the target object and the position information of the UAV determined in S203, and then control the flight height of the UAV according to the angle.

[0096] wherein the angle of the hand relative to the UAV in the pitch direction is determined according to the position information of the hand of the target object and the position information of the UAV,

[0097] Figure 4 A flowchart of a control method of a UAV provided by another embodiment of the application is shown in FIG. 6. Figure 4 The method of the embodiment can include, for example, controlling the flight height of the UAV according to the position information of the hand.

[0098] S301, obtaining an image captured by a camera.

[0099] S302, determining the position of the hand of the target object in the image.

[0100] S303. Determine the position information of the hand of the target object based on the position of the hand in the image.

[0101] In this embodiment, the specific implementation process of S303 can be found in [reference needed]. Figure 2 The relevant descriptions in the illustrated embodiments will not be repeated here.

[0102] S304. Determine the angle of the hand relative to the drone in the pitch direction based on the position information of the target object's hand and the position information of the drone.

[0103] S305. Control the flight altitude of the drone according to the angle.

[0104] In this embodiment, after determining the position information of the target object's hand, the angle of the hand relative to the drone in the pitch direction is determined based on the hand's position information and the drone's position information. Then, the desired altitude of the drone is determined based on this angle, and the drone is controlled to fly to that desired altitude. The desired altitude of the drone can be determined based on this angle and the horizontal distance between the target object and the drone. For example, an empirical value can be subtracted from the horizontal distance between the target object and the drone to obtain the horizontal distance between the hand and the drone. Then, the desired altitude of the drone is determined based on this angle and the horizontal distance between the hand and the drone. The desired altitude of the drone can be at the same height as the target object's hand. Figure 5 This is a schematic diagram of controlling the altitude of a drone according to an embodiment of the present invention, as shown below. Figure 5 As shown, based on the position information of the target object's hand and the position information of the drone, the angle of the target object's hand relative to the drone in the pitch direction can be determined as α1. The horizontal distance between the target object and the drone is D1, so the horizontal distance between the target object's hand and the drone can be obtained as d1, where d1 = D1 - 0.65m. Additionally, the current altitude of the drone is Pc1, which is determined by... Figure 5 The altitude difference between the drone's current altitude Pc1 and its desired altitude Pt1 can be obtained as h1, where h1 = d1 * tanα1. Therefore, the desired altitude of the drone can be obtained as Pt1 = Pc1 + h1. Figure 5 As can be seen, this embodiment can control the drone to fly at the same height as the hand.

[0105] In some embodiments, the S304 is executed when the state parameter of the target object meets a first preset requirement. The state parameter of the target object meeting the first preset requirement includes that a size proportion of the target object in the image is greater than or equal to a preset first proportion threshold, and / or a distance between the target object and the UAV is less than or equal to a preset first distance. In the embodiment, it can be determined whether the size proportion of the target object in the image is less than the preset first proportion threshold. When the size proportion of the target object in the image is greater than or equal to the preset first proportion threshold, the S304 is executed. The greater the size proportion of the target object in the image, the closer the distance between the target object and the UAV. It can also be determined whether the distance between the target object and the UAV is greater than the preset first distance. When the distance between the target object and the UAV is less than or equal to the preset first distance, the S304 is executed. The distance between the target object and the UAV can be obtained by a distance measuring method of a binocular camera arranged on the UAV. Therefore, when the state parameter of the target object meets the first preset requirement, it indicates that the distance between the target object and the UAV is close, and the target object is in a near-field state. At this time, the UAV can accurately obtain the angle of the hand relative to the UAV in the pitch direction, so as to accurately control the flight height of the UAV.

[0106] In the embodiment, the flight height of the UAV is controlled by the photographed image, the user does not need to operate the control device to control the UAV, the user who is not familiar with the control device can control the UAV, the control method of the UAV is simplified, the operation process of the UAV is simplified, the efficiency of controlling the UAV is improved, and the entertainment of human-computer interaction is enhanced.

[0107] Figure 6 The flowchart of the control method of the UAV provided by another embodiment of the present application is shown in Figure 6 The method of the embodiment takes controlling the flight height of the UAV according to the position information of the hand as an example. The method of the embodiment can include:

[0108] S401, obtaining an image photographed by a photographing device.

[0109] S402, determining the position of the hand of a target object in the image.

[0110] S403, determining the position information of the hand of the target object according to the position of the hand in the image.

[0111] In the embodiment, the specific implementation process of S403 can refer to the related description in the embodiment shown in Figure 2

[0112] ​S404, determining an angle of the hand relative to the preset part in a pitch direction according to the position information of the hand of the target object and the position information of the preset part of the target object.

[0113] S405, controlling a flight height of the unmanned aerial vehicle according to the angle.

[0114] In the embodiment, after the position information of the hand of the target object is determined, the angle of the hand relative to the preset part in a pitch direction is determined according to the position information of the hand of the target object and the position information of the preset part of the target object, and then the expected height of the unmanned aerial vehicle is determined according to the angle, and the unmanned aerial vehicle is controlled to fly to the expected height.

[0115] The preset part may, for example, be at least one of a head, a shoulder and a chest. The position information of the preset part of the target object may, for example, be determined according to the position information of the target object. For example, when the preset part is the head, the uppermost fifth of the target object is taken as the head, and thus the position information of the uppermost fifth of the target object is determined according to the position information of the target object.

[0116] In some embodiments, after the image captured by the photographing device is acquired, the position of the target object in the image is determined, and then the position information of the target object is determined according to the position of the target object in the image. Specifically, the position information of the target object may, for example, be determined according to the position of the target object in the image, the posture of the holder of the photographing device, the horizontal distance between the target object and the unmanned aerial vehicle, and the position information of the unmanned aerial vehicle. The process of determining the position information of the target object is similar to that of determining the position information of the hand, and thus is not described herein.

[0117] The expected height of the unmanned aerial vehicle may, for example, be determined according to the angle and the horizontal distance between the target object and the unmanned aerial vehicle. For the convenience of description, the preset part is taken as the head.

[0118] In one implementation manner, Figure 7 A schematic diagram for controlling the height of the unmanned aerial vehicle is provided for another embodiment of the present application, as shown in the figure. Figure 7 The angle of the hand of the target object relative to the preset part in a pitch direction may, for example, be determined according to the position information of the hand of the target object and the position information of the preset part of the target object, and the angle is α2. The horizontal distance between the target object and the unmanned aerial vehicle is D2, and thus the height difference between the expected height of the unmanned aerial vehicle and the preset part of the target object is D2*tanα2. The height difference between the current height Pc2 of the unmanned aerial vehicle and the preset part of the target object is △h, and thus Figure 7The altitude difference between the current altitude Pc2 and the desired altitude Pt2 of the drone can be obtained as h2, where h2 = Δh + D2 * tanα2. Therefore, the desired altitude of the drone can be obtained as Pt2 = Pc2 + h2. Figure 7 As can be seen, this embodiment can control the drone to fly so that it is on the same line as the target object's hand and a preset part.

[0119] In another implementation, the drone's flight altitude can be controlled based on hand movements. Figure 8 A schematic diagram of controlling the altitude of a drone provided in another embodiment of the present invention, as shown below. Figure 8 As shown, based on the position information of the target object's hand before movement and the position information of the target object's preset part, the angle of the target object's hand in the pitch direction relative to the preset part before movement can be determined to be α3. Based on the position information of the target object's hand after movement and the position information of the target object's preset part, the angle of the target object's hand in the pitch direction relative to the preset part after movement can be determined to be α3'. Therefore, the change in the angle of the target object's hand relative to the preset part in the pitch direction is α3 + α3'. It should be noted that if the hand is located below or above the preset part both before and after movement, the change in the above angle is |α3 - α3'|. Wherein, the horizontal distance between the target object and the drone is D3. Therefore, based on the change in this angle and the horizontal distance between the target object and the drone, the height difference between the intersection point P0 of the line connecting the hand and the target object before movement and the intersection point P0' of the line connecting the hand and the target object after movement and the intersection point P0' of the line connecting the hand and the target object in the vertical direction of the drone can be obtained as h3 = P0' - P0. And this height difference h3 is taken as the height difference between the UAV's expected height Pt3 and its current height Pc3. Therefore, the expected height of the UAV can be obtained as Pt3 = Pc3 + h3, where, as Figure 8 As shown, h3=D2*(tanα3+tanα3'), therefore Pt3=Pc3+D2*(tanα3+tanα3'). From Figure 8 As can be seen, this embodiment can map the change in hand height to the change in height in the vertical direction of the drone, thereby controlling the drone's flight altitude based on the change in height in the vertical direction of the drone.

[0120] In some embodiments, the S404 is executed when the state parameter of the target object meets a second preset requirement. The state parameter of the target object meeting the second preset requirement includes that a size proportion of the target object in the image is less than or equal to a preset second proportion threshold, and / or a distance between the target object and the UAV is greater than or equal to a preset second distance. In the embodiments, it can be determined whether the size proportion of the target object in the image is greater than the preset second proportion threshold. When the size proportion of the target object in the image is less than or equal to the preset second proportion threshold, the S404 is executed. The smaller the size proportion of the target object in the image, the farther the distance between the target object and the UAV. It can also be determined whether the distance between the target object and the UAV is less than the preset second distance. When the distance between the target object and the UAV is greater than or equal to the preset second distance, the S404 is executed. Therefore, when the state parameter of the target object meets the second preset requirement, it indicates that the distance between the target object and the UAV is far, and the target object is in a far field state. At this time, the UAV can accurately obtain the angle of the hand relative to the preset part of the target object in the pitch direction, so as to accurately control the flight height of the UAV.

[0121] In some embodiments, the preset second proportion threshold can be equal to the preset first proportion threshold. The preset second distance can be equal to the preset first distance. Accordingly, Figure 4 With Figure 6 the embodiments shown can be combined, that is, when the state parameter of the target object meets the second preset requirement, the S404 is executed, and when the state parameter of the target object meets the first preset requirement, the S402 is executed. Figure 6 Figure 4

[0122] In the embodiments, by the above scheme, the flight height of the UAV is controlled by shooting the image, which avoids the situation that the user must operate the control device to control the UAV, overcomes the defect that the user cannot control the UAV due to unfamiliarity with the control device, simplifies the control method and operation process of controlling the UAV, improves the efficiency of controlling the UAV to fly, and enhances the entertainment of human-computer interaction.

[0123] Figure 9 The flowchart of the control method of the UAV provided by another embodiment of the present application is shown in FIG. 8. The method of the present embodiment takes the UAV performing a surrounding flight on a target object according to the position information of the hand as an example. The method of the present embodiment can include: Figure 9

[0124] S501, acquiring an image shot by a shooting device.

[0125] S502, determining the position of the hand of the target object in the image. ​​​

[0126] S503, determining position information of the hand of the target object according to the position of the hand in the image.

[0127] In this embodiment, the specific implementation process of S501-S503 can be referred to the related description in the embodiment shown in Figure 2 and will not be described here again.

[0128] S504, determining an angle of the hand relative to the target object in the yaw direction according to the position information of the hand of the target object and the position information of the target object.

[0129] S505, controlling the unmanned aerial vehicle to fly around the target object according to the angle.

[0130] In this embodiment, after determining the position information of the hand of the target object, an angle of the hand relative to the target object in the yaw direction is determined according to the position information of the hand of the target object and the position information of the target object, and then a desired angle of the unmanned aerial vehicle relative to the target object in the yaw direction is determined according to the angle, and then the unmanned aerial vehicle is controlled to fly around the target object to the desired angle. In some embodiments, after determining the position information of the hand of the target object, an angle of the hand relative to the target object in the yaw direction is determined according to the position information of the hand of the target object and the position information of the target object, and then desired position information of the unmanned aerial vehicle is determined according to the angle, and then the unmanned aerial vehicle is controlled to fly around the target object to the desired position information.

[0131] In one implementation manner, Figure 10 A schematic diagram of controlling the unmanned aerial vehicle to fly around the target object is provided for an embodiment of the present application, as shown in Figure 10 According to the position information of the hand of the target object and the position information of the target object, an angle of the hand of the target object relative to the target object in the yaw direction can be determined as β1, and this embodiment can take the angle β1 as a desired angle of the unmanned aerial vehicle relative to the target object in the yaw direction, and then this embodiment controls the unmanned aerial vehicle to fly around the target object from the current position Pc4 to the desired position Pt4, and when the unmanned aerial vehicle flies around to the desired position Pt4, the angle of the unmanned aerial vehicle relative to the target object in the yaw direction is β1. Wherein, the current angle of the unmanned aerial vehicle relative to the target object in the yaw direction is β2, since the desired angle of the unmanned aerial vehicle relative to the target object in the yaw direction has been determined as β1, the angle of the unmanned aerial vehicle flying around the target object can be obtained as Δβ=β1-β2. From Figure 10 It can be known that this embodiment can control the unmanned aerial vehicle to fly around the target object to the same yaw direction as the hand of the target object relative to the target object.

[0132] In another implementation manner, the unmanned aerial vehicle can be controlled to fly around the target object according to the movement of the hand.Figure 11 A schematic diagram of controlling the UAV to fly around the target object is provided for another embodiment of the present application, as shown in Figure 11 According to the position information of the hand of the target object before moving and the position information of the target object, the angle of the hand of the target object relative to the target object in the yaw direction before moving can be determined as β3, and according to the position information of the hand of the target object after moving and the position information of the target object, the angle of the hand of the target object relative to the target object in the yaw direction after moving can be determined as β3', therefore, the change amount Δβ of the angle of the hand of the target object relative to the target object in the yaw direction is β3'-β3. The angle change amount Δβ can be taken as the angle of the UAV flying around the target object in this embodiment. Wherein, the current angle of the UAV relative to the target object in the yaw direction is β4, and then this embodiment controls the UAV to fly around the target object from the current position Pc5 to the desired position Pt5, when the UAV flies around to the desired position Pt5, the angle of the UAV relative to the target object in the yaw direction is β4', wherein β4'=β4+Δβ. From Figure 11 It can be known that the angle of the UAV flying around the target object can be controlled according to the angle change of the hand relative to the target object in the yaw direction in this embodiment. Wherein, the direction of the UAV flying around the target object can be the same as the direction of the hand moving relative to the target object.

[0133] In this embodiment, through the above scheme, the UAV is controlled to fly around the target object by shooting the image, avoiding the situation that the user must operate the control device to control the UAV, overcoming the defect that the user cannot control the UAV due to unfamiliarity with the control device, simplifying the control method and operation process of controlling the UAV, improving the efficiency of controlling the UAV to fly, and enhancing the entertainment of human-computer interaction.

[0134] Figure 12 A flowchart of a control method of a UAV provided for another embodiment of the present application is shown in Figure 12 The method of this embodiment takes the UAV flying away from or close to the target object according to the position information of the hand as an example, and the method of this embodiment can include:

[0135] S601, acquiring an image shot by a shooting device.

[0136] In this embodiment, the specific implementation process of S601-S603 can refer to the related description in the embodiment shown in Figure 2

[0137] S602, determining the positions of two hands of a target object in an image.

[0138] ​In this embodiment, the positions of the two hands (left hand and right hand) of the target object in the image are determined. Since the scheme of how to determine the position of a hand in the image has been described in the above embodiment, the determination of the position of each of the two hands in the image can refer to the description in the above embodiment, which will not be repeated here.

[0139] S603, determining the position information of the two hands of the target object according to the positions of the two hands in the image.

[0140] After the positions of the two hands in the image are determined, the position information of each hand is determined according to the position of each hand in the image. The specific implementation process can refer to the related description of determining the position information of the hand of the target object according to the position of the hand in the image in the above embodiment, which will not be repeated here.

[0141] S604, controlling the unmanned aerial vehicle to fly away from or close to the target object according to the position information of the two hands.

[0142] In this embodiment, after the position information of the two hands of the target object is determined, the unmanned aerial vehicle is controlled to fly away from or close to the target object according to the position information of the two hands. In this embodiment, the distance between the two hands can be determined according to the position information of the two hands, which can be the distance between the two hands in the horizontal direction. Then, the unmanned aerial vehicle is controlled to fly away from or close to the target object according to the distance, for example, the unmanned aerial vehicle can be controlled to fly away from or close to the target object along the direction of the line connecting the unmanned aerial vehicle and the target object, or the unmanned aerial vehicle can be controlled to fly away from or close to the target object along the horizontal direction while keeping the height of the unmanned aerial vehicle unchanged.

[0143] In a possible implementation manner, Figure 13 A schematic diagram of controlling the unmanned aerial vehicle to fly away from or close to the target object is provided for an embodiment of the present application, as shown in Figure 13 According to the distance d2 between the two hands of the target object, the expected distance D4' between the unmanned aerial vehicle and the target object is determined, where d2 and D4' satisfy a certain functional relationship, for example, D4' = d2 * C1, C1 being a preset value. If the current distance D4 between the unmanned aerial vehicle and the target object is greater than the expected distance D4', the distance difference ΔD = D4 - D4', so the embodiment controls the unmanned aerial vehicle to fly a distance ΔD from the current position Pc6 to the expected position Pt6 in the direction of close to the target object. If the current distance D4 between the unmanned aerial vehicle and the target object is less than the expected distance D4', the distance difference ΔD = D4' - D4, so the embodiment controls the unmanned aerial vehicle to fly a distance ΔD from the current position Pc6 to the expected position Pt6 in the direction of away from the target object. Wherein, Figure 13 The unmanned aerial vehicle flying close to the target object is shown in

[0144] In another implementation, the relative movement of the two hands can be used to control the UAV to fly away from or approach the target object. Figure 14 The schematic diagram for controlling the UAV to fly away from or approach the target object is provided for another embodiment of the present application, as shown in Figure 14 According to the position information of the two hands of the target object before the relative movement, the distance between the two hands before the relative movement is determined; and according to the position information of the two hands of the target object after the relative movement, the distance between the two hands after the relative movement is determined. If the two hands move towards each other, i.e. the distance between the two hands decreases, as shown in Figure 14 the distance between the two hands before the relative movement is d3, and the distance between the two hands after the relative movement is d4, then the change amount of the distance between the two hands △d = d3-d4 can be determined according to d3 and d4, and then the change amount of the distance between the UAV and the target object △D can be determined according to the change amount of the distance between the two hands △d, wherein △D and △d satisfy a certain function relationship, for example: △D = △d*C2, C2 is a preset value; and then the UAV is controlled to approach the target object by △D. If the two hands move away from each other, i.e. the distance between the two hands increases, as shown in Figure 14 the distance between the two hands before the relative movement is d4, and the distance between the two hands after the relative movement is d3, then the change amount of the distance between the two hands △d = d3-d4 can be determined according to d3 and d4, and then the change amount of the distance between the UAV and the target object △D can be determined according to the change amount of the distance between the two hands △d, wherein △D and △d satisfy a certain function relationship, for example: △D = △d*C2, C2 is a preset value; and then the UAV is controlled to fly away from the target object by △D. In some embodiments, when the distance between the two hands decreases, the UAV can be controlled to fly away from the target object, and when the distance between the two hands increases, the UAV can be controlled to approach the target object.

[0145] In some embodiments, the present embodiment also limits the maximum distance of the UAV away from the target object, and the minimum distance of the UAV approaching the target object. When the UAV is controlled to fly away from the target object, the distance between the UAV and the target object is also detected, and if the distance is greater than or equal to the maximum distance, the UAV is controlled to stop flying away from the target object. When the UAV is controlled to approach the target object, the distance between the UAV and the target object is also detected, and if the distance is less than or equal to the minimum distance, the UAV is controlled to stop approaching the target object.

[0146] In this embodiment, the user can control the flight of the UAV by shooting the image, avoiding the situation that the user has to operate the control device to control the UAV, overcoming the defect that the user cannot control the UAV due to unfamiliarity with the control device, simplifying the control mode and operation process of the UAV, improving the efficiency of controlling the flight of the UAV, and enhancing the entertainment of human-computer interaction.

[0147] In some embodiments, on the basis of the above-mentioned embodiments, the hand gesture of the target object in the image shot by the shooting device can be recognized in this embodiment. Accordingly, one way of controlling the flight of the UAV according to the position information of the hand of the target object in this embodiment is that when the hand gesture of the target object is a preset gesture, the flight of the UAV is controlled according to the position information of the hand of the target object. When the recognized hand gesture of the target object is a preset gesture, the flight height of the UAV is controlled according to the position information of the hand of the target object, the UAV is controlled to fly around the target object, the UAV is controlled to fly away from or close to the target object, and the like. The preset gesture is, for example, ok, yeah, and the gesture of stretching out the palm.

[0148] Figure 15 The flowchart of the control method of the UAV provided by another embodiment of the present application is shown in Figure 15 The method of this embodiment can include:

[0149] S701, acquiring an image shot by a shooting device.

[0150] In this embodiment, the specific implementation process of S701 can refer to the related description in the embodiment shown in Figure 2 The specific implementation process of S701 can refer to the related description in the embodiment shown in

[0151] S702, determining a feature part of a target object in the image.

[0152] In this embodiment, the target object needs to be recognized from the image, wherein the recognition of the target object can be realized by recognizing the feature part of the body of the target object. Therefore, the feature part of the target object in the image shot by the shooting device is determined, which can be used to represent the target object. When the target object is a person, the feature part can be the head of the human body, the head and shoulder of the human body, or at least one of the human body.

[0153] In some embodiments, after entering the gesture control mode, the feature part in the image can be identified first, and then the feature part of the target object is determined from the feature part in the image. In some cases, there can be multiple persons in the image captured, and when the feature part is identified, the feature part of these persons can also be identified, therefore, in this embodiment, the feature part of the target object is determined from the feature part in the image, for example, the feature part closest to the center of the image can be determined as the feature part of the target object, so that the target object is closest to the center of the image. In this way, after entering the gesture control mode, the feature part of the target object in the image can be found. After the feature part of the target object in the image is found, when the new image is acquired by the photographing device, the feature part of the target object in the new image can be found by using the tracking algorithm. For example, the position of the feature part of the target object in the previous image determines a target image region, and an image region most similar to the feature part of the target object in the previous image in the target image region in the next image is determined as the feature part of the target object in the next image.

[0154] S703, identify the hand in the image.

[0155] In this embodiment, the hand in the image is also identified from the image captured by the photographing device. The execution sequence of S702 and S703 is not distinguished.

[0156] S704, determine the hand of the target object from the hand in the image according to the feature part of the target object.

[0157] In this embodiment, after the feature part of the target object in the image is determined and the hand in the image is identified, the hand of the target object is determined from the hand in the image according to the feature part of the target object. There can be multiple hands in the image captured, and when the hand is identified, these hands can also be identified, but some of them are not the hand of the target object, therefore, in this embodiment, the hand of the target object is determined from the hand in the image according to the feature part of the target object. In this embodiment, the joint of the target object can be determined according to the feature part of the target object, which includes the joint of the hand, the joint of the arm, the joint of the head, the joint of the shoulder, etc., and then the hand of the target object is determined from the hand in the image according to the joint of the target object. In this embodiment, the joint of the hand of the target object can be determined from the joint of the target object, and the hand closest to the joint of the hand of the target object can be determined from the hand in the image, and then the hand closest to the joint of the hand of the target object is determined as the hand of the target object.

[0158] S705, when the gesture of the hand of the target object is a control gesture, control the drone to perform the action indicated by the gesture.

[0159] In this embodiment, after recognizing the target object's hand, when the target object's hand gesture is a control gesture, the drone is controlled to execute the gesture-instructed action, such as: controlling the drone to take off, land, control the drone's flight altitude, control the drone to circle the target object, control the drone to fly away from or towards the target object, take photos or record videos, etc. The control gesture can be, for example, "ok," "yeah," or an extended palm.

[0160] The control of the drone's flight altitude, the control of the drone's orbit around the target object, and the control of the drone's flight away from or close to the target object can be found in the descriptions in the above embodiments.

[0161] The following describes the control of drone takeoff and landing.

[0162] In some embodiments, when the hand gesture of the target object is a takeoff gesture, the takeoff gesture is a control gesture that instructs the drone to take off. In this embodiment, the drone is controlled to take off based on the gesture being a takeoff gesture. Optionally, this embodiment controls the drone to take off and hover at a preset height, wherein, as... Figure 16 As shown, Figure 16 This is a schematic diagram illustrating the control of a drone takeoff according to an embodiment of the present invention. In some embodiments, after executing S701, this embodiment further detects a first user operation. Upon detecting the first user operation, the gimbal carrying the shooting device is controlled to drive the shooting device to scan within a preset angle range. The rotation of the gimbal causes the rotation of the shooting device, thereby enabling the shooting device to scan within the preset angle range to capture images within that range, so as to execute subsequent operations S701-S705. The first operation includes at least one of: clicking or double-clicking the battery switch, shaking the drone, or issuing a voice command to the drone. In this embodiment, these operations enable the drone to enter a gesture recognition mode. After the drone enters the gesture recognition mode, the gimbal carrying the shooting device is controlled to drive the shooting device to scan within the preset angle range to recognize the takeoff gesture of the target object.

[0163] In some embodiments, after the take-off gesture is identified, the UAV is further controlled to take off after the take-off gesture is a stable gesture. In this embodiment, S701 is performed to obtain multiple images of the photographing apparatus. After the multiple images are obtained, the position of the hand of the target object in each of the multiple images is determined. The position of the hand of the target object in each of the multiple images can be determined according to the description of the above embodiments, which will not be repeated here. When the hand of the target object is the take-off gesture, it is determined whether the position of the hand of the target object in each of the multiple images is within a preset range of the reference position. When the position of the hand of the target object in each of the multiple images is within the preset range of the reference position, it is determined that the take-off gesture is a stable gesture, and the target object wants to control the UAV to take off. Then, the UAV is controlled to take off. When the position of the hand of the target object in each of the multiple images is not within the preset range of the reference position, it is determined that the take-off gesture is not a stable gesture, and the target object is misoperated and does not want to control the UAV to take off. Then, the gesture is ignored, and the UAV is not controlled to take off. In some embodiments, the reference position is the position of the hand of the target object in the last image, which also indicates that the position of the hand in each of the multiple images is stable.

[0164] In some embodiments, when the gesture of the hand of the target object is a landing gesture, the landing gesture is a control gesture, and the landing gesture indicates that the UAV is to land. In this embodiment, the UAV is controlled to land according to the gesture being the landing gesture. As shown in FIG. 8, Figure 17 Figure 17 FIG. 8 is a schematic diagram of controlling the UAV to land according to an embodiment of the present application.

[0165] In the first possible implementation, the height value measured by the distance sensor is further obtained, and the height value represents the flight height of the UAV. In this embodiment, whether the UAV can land is further determined according to the height value. Specifically, when the gesture of the hand of the target object is a landing gesture and the height value is less than or equal to a preset height threshold, the UAV is controlled to land. When the height value is greater than the preset height threshold, it is determined that the flight height of the UAV is high, and the UAV is not suitable for landing under the current condition. In order to ensure the flight safety of the UAV, the landing gesture is ignored when the height value is greater than the preset height threshold, and the UAV is not controlled to land.

[0166] ​In a second possible implementation, the embodiment also detects the flatness of the ground below the UAV, wherein the flatness can be detected by the binocular camera. The embodiment further determines whether the UAV can land according to the flatness, and the specific process is as follows: when the hand gesture of the target object is the landing gesture and the flatness is greater than or equal to a preset flatness threshold, the UAV is controlled to land. If the flatness is less than the preset flatness threshold, it indicates that the ground below the UAV is not flat enough to ensure the safe landing of the UAV, and therefore, when the flatness is less than the preset flatness threshold, the landing gesture is ignored, that is, the UAV is not controlled to land.

[0167] In a third possible implementation, the embodiment also detects whether there is water below the UAV. The embodiment further determines whether the UAV can land according to whether there is water below the UAV, and the specific process is as follows: when the hand gesture of the target object is the landing gesture and there is no water below the UAV, the UAV is controlled to land. If there is water below the UAV, the UAV will fall into the water after stopping on the water surface, causing damage to the UAV, and therefore, when there is water below the UAV, the landing gesture is ignored, that is, the UAV is not controlled to land.

[0168] In a fourth possible implementation, the embodiment also detects the flight speed of the UAV, which can be detected by a speed sensor. The embodiment further determines whether the UAV lands according to the flight speed of the UAV, and the specific process is as follows: when the hand gesture of the target object is the landing gesture and the flight speed of the UAV is less than or equal to a preset speed threshold, the UAV is controlled to land. If the flight speed of the UAV is greater than the preset speed threshold, in order to avoid damage to the UAV caused by the flight speed of the UAV after landing on the ground, therefore, when the flight speed of the UAV is greater than the preset speed threshold, the landing gesture is ignored, that is, the UAV is not controlled to land.

[0169] In a fifth possible implementation, the embodiment also detects whether the height of the hand of the target object is lower than the height of the head of the target object, wherein the height of the hand can be determined by the position information of the hand, and the height of the head of the target object can be determined by the position information of the target object. How to determine the position information of the hand and the position information of the target object can be referred to the related description in the above embodiments, which will not be described here. The embodiment further determines whether the UAV lands according to the relationship between the height of the hand and the height of the head, and the specific process is as follows: when the hand gesture of the target object is the landing gesture and the height of the hand of the target object is lower than the height of the head of the target object, the UAV is controlled to land. If the hand gesture of the target object is the landing gesture and the height of the hand of the target object is not lower than the height of the head of the target object, the landing gesture is ignored, that is, the UAV is not controlled to land.

[0170] It should be noted that at least two of the above first to fifth possible implementation manners can be combined to control the UAV to land.

[0171] Figure 18 A flowchart of a control method of a UAV according to another embodiment of the present application is shown in FIG. 8. The method of the present embodiment can include the following steps. Figure 18

[0172] S801, acquiring an image captured by a photographing device.

[0173] In the present embodiment, the specific implementation process of S801 can refer to the related description in the embodiment shown in FIG. 1, which will not be repeated here. Figure 2

[0174] S802, identifying a feature part of a target object in the image.

[0175] In the present embodiment, the specific implementation process of identifying a feature part of a target object in the image can refer to the related description in the embodiment shown in FIG. 2, which will not be repeated here. Figure 15

[0176] S803, identifying a hand of the target object in the image.

[0177] In the present embodiment, the specific implementation process of identifying a hand of the target object in the image can refer to the related description in the embodiment shown in FIG. 3, which will not be repeated here. The execution order of S802 and S803 is not limited. Figure 15

[0178] S804, when the feature part of the target object is identified and the hand of the target object is not identified, controlling the UAV to track the target object so that the target object is in the shooting picture of the photographing device.

[0179] In the present embodiment, when the feature part of the target object is identified through S802, and the hand of the target object is not identified through S803, it means that the target object exists in the captured image but the hand does not exist, and the UAV does not need to be controlled according to the hand, and then the UAV is controlled to perform a hand-free following mode, that is, the UAV is controlled to track the target object so that the photographing device can capture the target object, and the target object is in the shooting picture of the photographing device.

[0180] In some embodiments, in order to make the target object in the shooting picture of the photographing device, the present embodiment can control the UAV to track the target object by adjusting at least one of the position information of the UAV, the attitude of the UAV, and the attitude of a gimbal bearing the photographing device.

[0181] In some embodiments, Figure 18 ​​​​The embodiments shown can be combined with the above Figures 2-15 In combination with any embodiment, if the hand of the target object is not recognized, the above Figures 2-15 In combination with any embodiment of the above scheme, if the hand of the target object is recognized, the above Figure 19 In combination with any embodiment of the above scheme.

[0182] In the embodiment, by the above scheme, the user can control the drone to follow the target object by shooting an image, avoiding the situation that the user must operate the control device to control the drone, overcoming the defect that the user cannot control the drone due to unfamiliarity with the control device, simplifying the control method and operation process of the drone, improving the efficiency of controlling the drone to fly, and enhancing the entertainment of human-computer interaction.

[0183] In some embodiments, the feature part in each of the above embodiments can refer to: a head of a human body, a head and a shoulder of a human body, or at least one of a human body.

[0184] In some embodiments, when the state parameter of the target object meets a preset first state parameter condition, the feature part is the head and the shoulder of the human body. The preset first state parameter condition includes: a size proportion of the target object in the image is greater than or equal to a preset first proportion threshold, and / or a distance between the target object and the drone is less than or equal to a preset first distance. In the embodiment, it can be determined whether the size proportion of the target object in the image is less than the preset first proportion threshold, and when the size proportion of the target object in the image is greater than or equal to the preset first proportion threshold, the feature part in each of the above embodiments is the head and the shoulder of the human body. The greater the size proportion of the target object in the image, the closer the distance between the target object and the drone. It can also be determined whether the distance between the target object and the drone is greater than the preset first distance, and when the distance between the target object and the drone is less than or equal to the preset first distance, the feature part in each of the above embodiments is the head and the shoulder of the human body, wherein the distance between the target object and the drone can be obtained by a distance measuring mode of the binocular camera. Therefore, when the state parameter of the target object meets the first preset requirement, it indicates that the distance between the target object and the drone is close, and the target object is in a near-field state, and at this time the drone can accurately recognize the head and the shoulder of the target object.

[0185] In some embodiments, when the state parameters of the target object meet a preset second state parameter condition, the feature part is a human body. The preset second state parameter condition includes: the size proportion of the target object in the image is less than or equal to a preset second proportion threshold; and / or, the distance between the target object and the drone is greater than or equal to a preset second distance. In this embodiment, it can be determined whether the size proportion of the target object in the image is greater than the preset second proportion threshold. When the size proportion of the target object in the image is less than or equal to the preset second proportion threshold, the feature part in the above embodiments is a human body. The smaller the size proportion of the target object in the image, the farther the distance between the target object and the drone. Alternatively, it can be determined whether the distance between the target object and the drone is less than the preset second distance. When the distance between the target object and the drone is greater than or equal to the preset second distance, the feature part in the above embodiments is a human body. Therefore, when the state parameters of the target object meet the second preset requirement, it indicates that the distance between the target object and the drone is relatively far, and the target object is in a far-field state. At this time, the drone can identify the human body of the target object.

[0186] In some embodiments, the aforementioned preset first proportion threshold may be equal to the aforementioned preset second proportion threshold. The aforementioned preset first distance may be equal to the aforementioned preset second distance.

[0187] In summary, the embodiments of the present invention allow direct control of the drone's flight based on the hand image captured by the shooting device, including a series of processes such as takeoff, flight altitude, circling flight, moving away or approaching, following, and landing. This avoids the situation where the user must operate the control device to control the drone, overcomes the defect that the user is unfamiliar with the control device and therefore cannot control the drone, simplifies the control method and operation process of the drone, improves the efficiency of controlling the drone's flight, and enhances the entertainment value of human-computer interaction.

[0188] This invention also provides a computer storage medium storing program instructions, which, when executed, may include some or all of the steps of the UAV control method described in the above embodiments.

[0189] Figure 19 This is a schematic diagram of a control device for a drone provided in an embodiment of the present invention, as shown below. Figures 2-12 As shown, the control device 1900 of the UAV in this embodiment may include a memory 1901 and a processor 1902. The memory 1901 and the processor 1902 are connected via a bus. The memory 1901 may include read-only memory and random access memory, and provides instructions and data to the processor 1902. A portion of the memory 1901 may also include non-volatile random access memory.

[0190] The processor 1902 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0191] The memory 1901 is configured to store program codes.

[0192] The processor 1902 is configured to invoke the program codes to perform the following steps.

[0193] An image captured by a photographing device is acquired.

[0194] A position of a hand of a target object in the image is determined.

[0195] Position information of the hand of the target object is determined according to the position of the hand in the image.

[0196] The flight of a drone is controlled according to the position information of the hand of the target object.

[0197] Optionally, the processor 1902 is specifically configured to:

[0198] The position information of the hand of the target object is determined according to the position of the hand in the image, a posture of a holder carrying the photographing device, a horizontal distance between the target object and the drone, and position information of the drone.

[0199] Optionally, the processor 1902 is specifically configured to:

[0200] An orientation of the hand relative to the drone is determined according to the position of the hand in the image and the posture of the holder carrying the photographing device.

[0201] The position information of the hand of the target object is determined according to the orientation, the horizontal distance between the target object and the drone, and the position information of the drone.

[0202] Optionally, the processor 1902 is specifically configured to control a flight height of the drone according to the position information of the hand of the target object.

[0203] Optionally, the processor 1902 is specifically configured to:

[0204] determine an angle of the hand relative to the UAV in a pitch direction according to the position information of the hand of the target object and the position information of the UAV;

[0205] control a flight height of the UAV according to the angle.

[0206] Optionally, the processor 1902 is specifically configured to:

[0207] when the state parameter of the target object meets a first preset requirement, determine an angle of the hand relative to the UAV in a pitch direction according to the position of the hand of the target object and the position information of the UAV.

[0208] Optionally, the state parameter of the target object meeting the first preset requirement comprises:

[0209] a size proportion of the target object in the image is greater than or equal to a preset first proportion threshold; and / or,

[0210] a distance between the target object and the UAV is less than or equal to a preset first distance.

[0211] Optionally, the processor 1902 is specifically configured to:

[0212] determine an angle of the hand relative to a preset part in a pitch direction according to the position information of the preset part of the target object and the position information of the hand;

[0213] control a flight height of the UAV according to the angle.

[0214] Optionally, the preset part comprises at least one part of a head, a shoulder and a chest.

[0215] Optionally, the processor 1902 is specifically configured to:

[0216] when the state parameter of the target object meets a second preset requirement, determine an angle of the hand relative to a preset part in a pitch direction according to the position information of the preset part of the target object and the position information of the hand.

[0217] Optionally, the state parameter of the target object meeting the second preset requirement comprises:

[0218] a size proportion of the target object in the image is less than or equal to a preset second proportion threshold; and / or,

[0219] a distance between the target object and the UAV is greater than or equal to a preset second distance.

[0220] Optionally, the position information of the preset position is determined according to the position information of the target object.

[0221] Optionally, the processor 1902 is specifically configured to:

[0222] control the unmanned aerial vehicle to fly around the target object according to the position information of the hand of the target object.

[0223] Optionally, the processor 1902 is specifically configured to:

[0224] determine an angle of the hand relative to the target object in a yaw direction according to the position information of the hand of the target object and the position information of the target object;

[0225] control the unmanned aerial vehicle to fly around the target object according to the angle.

[0226] Optionally, the processor 1902 is specifically configured to:

[0227] control the unmanned aerial vehicle to fly away from or close to the target object according to the position information of the hand of the target object.

[0228] Optionally, the processor 1902 is specifically configured to:

[0229] determine positions of two hands of the target object in the image;

[0230] determine position information of the two hands of the target object according to the positions of the two hands in the image;

[0231] control the unmanned aerial vehicle to fly away from or close to the target object according to the position information of the two hands.

[0232] Optionally, the processor 1902 is specifically configured to:

[0233] determine a distance between the two hands according to the position information of the two hands;

[0234] control the unmanned aerial vehicle to fly away from or close to the target object according to the distance.

[0235] Optionally, the processor is further configured to:

[0236] determine a position of the target object in the image;

[0237] determine position information of the target object according to the position of the target object in the image.

[0238] Optionally, the processor 1902 is specifically configured to:

[0239] The position information of the target object is determined according to a position of the target object in the image, a posture of a holder of a camera holder, a horizontal distance between the target object and the unmanned aerial vehicle, and position information of the unmanned aerial vehicle.

[0240] Optionally, the processor 1902 is further configured to identify a hand gesture of a hand of the target object in the image.

[0241] When the processor controls the flight of the unmanned aerial vehicle according to the position information of the hand of the target object, the processor is specifically configured to control the flight of the unmanned aerial vehicle according to the position information of the hand of the target object when the hand gesture of the hand is a preset hand gesture.

[0242] The device of the embodiment can be used to execute the technical solutions of the above-mentioned embodiments and corresponding embodiments, and the implementation principles and technical effects are similar, which will not be repeated here. Figure 15 The technical solutions of the above-mentioned embodiments and corresponding embodiments, and the implementation principles and technical effects are similar, which will not be repeated here.

[0243] In some embodiments, the processor 1902 is configured to call the program code to execute:

[0244] Obtaining an image captured by a camera;

[0245] Determining a feature part of a target object in the image;

[0246] Identifying a hand in the image;

[0247] Determining a hand of the target object from the hand in the image according to the feature part of the target object;

[0248] When the hand gesture of the hand of the target object is a control gesture, controlling the unmanned aerial vehicle to perform an action indicated by the gesture.

[0249] Optionally, the processor 1902 is specifically configured to:

[0250] When the gesture is a take-off gesture, controlling the unmanned aerial vehicle to take off.

[0251] Optionally, the processor 1902 is specifically configured to:

[0252] Controlling the unmanned aerial vehicle to take off and hover at a preset height.

[0253] Optionally, the processor 1902 is further configured to control the camera holder to drive the camera to scan within a preset angle range after detecting a first operation of a user.

[0254] Optionally, the first operation includes at least one of single-clicking or double-clicking a battery switch, shaking the unmanned aerial vehicle, and issuing a voice instruction to the unmanned aerial vehicle.

[0255] Optionally, the processor 1902 is specifically configured to:

[0256] acquire a plurality of images captured by the photographing apparatus;

[0257] determine a position of a hand of the target object in each of the plurality of images;

[0258] when the gesture of the hand of the target object is a take-off gesture and the position of the hand of the target object in each of the plurality of images is within a preset range of a reference position, control the UAV to take off.

[0259] Optionally, the reference position is a position of the hand of the target object in a last image.

[0260] Optionally, the processor 1902 is specifically configured to:

[0261] when the gesture is a landing gesture, control the UAV to land.

[0262] Optionally, the processor 1902 is further configured to:

[0263] acquire a height value measured by a distance sensor;

[0264] The processor 1902 is configured to, when the gesture is a landing gesture, control the UAV to land, including: the processor is configured to, when the gesture is a landing gesture and the height value is less than or equal to a preset height threshold, control the UAV to land.

[0265] Optionally, the processor 1902 is further configured to detect flatness of a ground below the UAV.

[0266] The processor 1902 is configured to, when the gesture is a landing gesture, control the UAV to land, including:

[0267] The processor 1902 is configured to, when the gesture is a landing gesture and the flatness is greater than or equal to a preset flatness threshold, control the UAV to land.

[0268] Optionally, the processor 1902 is further configured to detect whether a water surface exists below the UAV.

[0269] The processor 1902 is configured to, when the gesture is a landing gesture, control the UAV to land, including:

[0270] The processor 1902 is configured to, when the gesture is a landing gesture and no water surface exists below the UAV, control the UAV to land.

[0271] Optionally, the processor 1902 is further configured to detect a flight speed of the UAV.

[0272] The processor 1902 is configured to control the UAV to land when the gesture is a landing gesture.

[0273] The processor 1902 is configured to control the UAV to land when the gesture is a landing gesture and the speed is less than or equal to a preset speed threshold.

[0274] Optionally, the processor 1902 is specifically configured to:

[0275] identify a feature part in the image;

[0276] determine the feature part of the target object from the feature part identified in the image.

[0277] Optionally, the processor 1902 is specifically configured to:

[0278] determine the feature part of the target object as the feature part closest to the center of the image.

[0279] Optionally, the processor 1902 is specifically configured to:

[0280] determine a joint of the target object according to the feature part of the target object;

[0281] determine the hand of the target object from the hand identified in the image according to the joint of the target object.

[0282] Optionally, the processor 1902 is specifically configured to:

[0283] determine the hand of the target object as the hand closest to the joint of the hand from the hand identified in the image.

[0284] determine the hand of the target object as the hand closest to the joint of the hand.

[0285] Optionally, the feature part includes at least one of a head of a human body, a head and a shoulder of a human body, and a human body.

[0286] Optionally, when a state parameter of the target object satisfies a preset first state parameter condition, the feature part is the head and the shoulder.

[0287] Optionally, the preset first state parameter condition includes that a size ratio of the target object in the image is greater than or equal to a preset first ratio threshold, and / or a distance between the target object and the UAV is less than or equal to a preset first distance.

[0288] Optionally, when a state parameter of the target object satisfies a preset second state parameter condition, the feature part is the human body.

[0289] Optionally, the preset second state parameter condition comprises that a size proportion of the target object in the image is less than or equal to a preset second proportion threshold, and / or a distance between the target object and the unmanned aerial vehicle is greater than or equal to a preset second distance.

[0290] The device of the embodiment can be used to execute the method of the embodiment. Figure 18 The technical solutions of the corresponding embodiments and the implementation principles and technical effects thereof are similar, and details are not described herein.

[0291] In some embodiments, the processor 1902 is configured to execute the program code to:

[0292] obtain an image captured by a photographing device;

[0293] identify a feature part of a target object in the image;

[0294] identify a hand of the target object in the image;

[0295] when the feature part of the target object is identified and the hand of the target object is not identified, control the unmanned aerial vehicle to track the target object so that the target object is in a photographing field of view of the photographing device.

[0296] Optionally, the processor 1902 is specifically configured to:

[0297] adjust at least one of position information of the unmanned aerial vehicle, a posture of the unmanned aerial vehicle, and a posture of a holder of the photographing device to track the target object so that the target object is in the photographing field of view of the photographing device.

[0298] Optionally, the feature part comprises at least one of a head of a human body, the head of the human body and a shoulder of the human body, and the human body.

[0299] Optionally, when a state parameter of the target object meets a preset first state parameter condition, the feature part is the head and the shoulder.

[0300] Optionally, the preset first state parameter condition comprises that a size proportion of the target object in the image is greater than or equal to a preset first proportion threshold, and / or a distance between the target object and the unmanned aerial vehicle is less than or equal to a preset first distance.

[0301] Optionally, when a state parameter of the target object meets a preset second state parameter condition, the feature part is the human body.

[0302] Optionally, the preset second state parameter conditions include: the size proportion of the target object in the image is less than or equal to a preset second proportion threshold, and / or, the distance between the target object and the drone is greater than or equal to a preset second distance.

[0303] The device in this embodiment can be used to perform... Figure 20 The technical solutions of the above and their corresponding embodiments are similar in principle and technical effect, and will not be described in detail here.

[0304] Figure 20 This is a schematic diagram of a drone provided in an embodiment of the present invention, as shown below. Figure 19 As shown, the drone in this embodiment may include: a drone control device 2001, a camera device 2002, and a power system (not shown in the figure). The drone control device 2001, camera device 2002, and power system are connected via a bus. The drone control device 2201 is used to control the flight of the drone and may employ... Figures 2-18 The structure of the illustrated embodiment, correspondingly, can be executed ​ The technical solutions of any of the method embodiments and their corresponding embodiments are similar in implementation principle and technical effect, and will not be described again here. The imaging device 2002 is used to capture images. The power system is used to provide flight power to drive the UAV, wherein the power system includes an electronic speed controller, a motor, a propeller, etc. In some embodiments, the UAV may also include a gimbal 2003 for supporting the imaging device 2002. In some embodiments, the UAV may also include: a positioning sensor, a distance sensor, a speed sensor, etc.

[0305] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0306] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of controlling a drone, the method comprising: The method comprises: acquiring an image captured by a photographing device; determining the position of a hand of a target object in the image; determining position information of the hand of the target object according to the position of the hand in the image, wherein the position information is three-dimensional position information; controlling the flight of a drone according to the position information of the hand of the target object; controlling the flight of the drone according to the position information of the hand of the target object comprises: controlling the drone to fly around the target object according to the position information of the hand of the target object; controlling the drone to fly around the target object according to the position information of the hand of the target object comprises: determining an expected angle of the hand relative to the target object in a yaw direction according to the position information of the hand of the target object and the position information of the target object; and controlling the drone to fly around the target object according to the expected angle; or, determining an angle change of the hand relative to the target object in the yaw direction according to the position information of the hand of the target object and the position information of the target object; and controlling the drone to fly around the target object according to the angle change.

2. The method of claim 1, wherein controlling the flight of the drone according to the position information of the hand of the target object comprises: controlling the flight height of the drone according to the position information of the hand of the target object.

3. The method of claim 1, wherein controlling the flight of the drone according to the position information of the hand of the target object comprises: controlling the drone to fly away from or close to the target object according to the position information of the hand of the target object.

4. The method of claim 3, wherein determining the position of the hand of the target object in the image comprises: determining the positions of two hands of the target object in the image; determining the position information of the hands of the target object according to the positions of the hands in the image comprises: determining the position information of the two hands of the target object according to the positions of the two hands in the image; controlling the drone to fly away from or close to the target object according to the position information of the hands of the target object comprises: controlling the drone to fly away from or close to the target object according to the position information of the two hands.

5. The method of claim 1, wherein controlling the drone to fly away from or close to the target object according to the position information of the two hands comprises: determining the distance between the two hands according to the position information of the two hands; controlling the drone to fly away from or close to the target object according to the distance.

6. The method of claim 1, wherein, The method further comprises: recognizing a hand gesture of the hand of the target object in the image; controlling the flight of the drone according to the position information of the hand of the target object comprises: controlling the flight of the drone according to the position information of the hand of the target object when the hand gesture of the hand is a preset hand gesture.

7. A control device of a drone, characterized by, The device comprises: a memory and a processor; the memory is configured to store program code; the processor is configured to invoke the program code to execute: acquiring an image captured by a photographing device; determining the position of a hand of a target object in the image; determine position information of the hand of the target object according to the position of the hand in the image, wherein the position information is three-dimensional position information; control flight of the unmanned aerial vehicle according to the position information of the hand of the target object; the control of the flight of the unmanned aerial vehicle according to the position information of the hand of the target object comprises: control the unmanned aerial vehicle to fly around the target object according to the position information of the hand of the target object; the control of the unmanned aerial vehicle to fly around the target object according to the position information of the hand of the target object comprises: determine an expected angle of the hand relative to the target object in a yaw direction according to the position information of the hand of the target object and the position information of the target object; and control the unmanned aerial vehicle to fly around the target object according to the expected angle; or, determine an angle change of the hand relative to the target object in the yaw direction according to the position information of the hand of the target object and the position information of the target object; and control the unmanned aerial vehicle to fly around the target object according to the angle change.

8. A control method of a drone, characterized by, comprise: obtain an image captured by a photographing device; identify a feature part of a target object in the image; identify a hand of the target object in the image; when the feature part of the target object is identified and the hand of the target object is not identified, control the unmanned aerial vehicle to track the target object so that the target object is in a photographing field of view of the photographing device; when the hand of the target object is identified, determine a position of the hand of the target object in the image; determine position information of the hand of the target object according to the position of the hand in the image, wherein the position information is three-dimensional position information; control flight of the unmanned aerial vehicle according to the position information of the hand of the target object; the control of the flight of the unmanned aerial vehicle according to the position information of the hand of the target object comprises: control the unmanned aerial vehicle to fly around the target object according to the position information of the hand of the target object; the control of the unmanned aerial vehicle to fly around the target object according to the position information of the hand of the target object comprises: determine an expected angle of the hand relative to the target object in a yaw direction according to the position information of the hand of the target object and the position information of the target object; and control the unmanned aerial vehicle to fly around the target object according to the expected angle; or, determine an angle change of the hand relative to the target object in the yaw direction according to the position information of the hand of the target object and the position information of the target object; and control the unmanned aerial vehicle to fly around the target object according to the angle change.

9. The method of claim 8, wherein the feature part comprises at least one of a head of a human body, the head of the human body and a shoulder, and the human body.

10. The method of claim 8 or 9, wherein when a state parameter of the target object meets a preset first state parameter condition, the feature part is the head and the shoulder; and when the state parameter of the target object meets a preset second state parameter condition, the feature part is the human body.

11. A control device of a drone, characterized by, comprise: a memory and a processor; the memory is configured to store program code; the processor is configured to invoke the program code to perform: obtain an image captured by a photographing device; identifying a feature part of a target object in the image; identifying a hand of the target object in the image; when the feature part of the target object is identified and the hand of the target object is not identified, controlling the UAV to track the target object so that the target object is in the shooting frame of the shooting device; when the hand of the target object is identified, determining the position of the hand of the target object in the image; determining the position information of the hand of the target object according to the position of the hand in the image, wherein the position information is three-dimensional position information; controlling the flight of the UAV according to the position information of the hand of the target object; controlling the flight of the UAV according to the position information of the hand of the target object includes: controlling the UAV to fly around the target object according to the position information of the hand of the target object; controlling the UAV to fly around the target object according to the position information of the hand of the target object includes: determining the expected angle of the hand relative to the target object in the yaw direction according to the position information of the hand of the target object and the position information of the target object; and controlling the UAV to fly around the target object according to the expected angle; or, determining the angle change of the hand relative to the target object in the yaw direction according to the position information of the hand of the target object and the position information of the target object; and controlling the UAV to fly around the target object according to the angle change.

12. The control device according to claim 11, wherein the feature part includes at least one of a head of a human body, a head and a shoulder of a human body, and a part of a human body.

Images