Method, device, drone, system and storage medium for obtaining landing position

Abstract

A method for acquiring a landing position, a device for acquiring a landing position, and a drone (12) can more effectively determine a target landing area to enable the drone (12) to land safely and at high speed. The method comprises: acquiring depth information of a target area, the target area being a landing area in an environment below the drone (12) determined based on semantic segmentation of an image, the image being obtained by photographing the environment with a photographing device (S201); and determining a target landing area in the target area based on the depth information (S202).

Description

Technical Field

[0001] The present invention relates to the field of control technology, and in particular to a method, device, unmanned aerial vehicle, system and storage medium for obtaining a landing position. Background Art

[0002] With the development of science and technology, drones are being used in more and more scenarios. With the improvement of wireless transmission technology, drones can fly farther and farther. Since the drone's shooting device is very small, it is difficult to see the drone beyond a certain distance, and it is completely invisible at a farther distance. Usually, drones flying at this distance can be called beyond visual range flight.

[0003] At present, the process of returning to land of drones is usually based on the take-off position located by the Global Positioning System (GPS). However, if the take-off environment itself is relatively complex, if the drone returns to the take-off position located by GPS, it may not be able to determine the appropriate landing area, and there are certain risks in the landing process. Therefore, how to control the landing of drones more effectively is of great significance. Summary of the invention

[0004] The embodiments of the present invention provide a method, device, drone, system and storage medium for obtaining a landing position, which can more effectively determine a target landing area to enable a drone to land at a high speed and safely.

[0005] In a first aspect, an embodiment of the present invention provides a method for obtaining a landing position, which is applied to a drone, wherein the drone is equipped with a shooting device, and the method includes:

[0006] Acquire depth information of a target area, where the target area is a landing area in an environment below the drone determined based on semantic segmentation of an image, where the image is obtained by photographing the environment with the photographing device;

[0007] Based on the depth information, a target landing area is determined in the target area.

[0008] In a second aspect, an embodiment of the present invention provides a device for obtaining a landing position, including a memory and a processor;

[0009] The memory is used to store programs;

[0010] The processor is used to call the program, and when the program is executed, is used to perform the following operations:

[0011] Acquire depth information of a target area, where the target area is a landing area in an environment below the drone determined based on semantic segmentation of an image, where the image is obtained by photographing the environment with a photographing device;

[0012] Based on the depth information, a target landing area is determined in the target area.

[0013] In a third aspect, an embodiment of the present invention provides a drone, including:

[0014] body;

[0015] A power system configured on the fuselage, used to provide power for the UAV to move;

[0016] A device for acquiring a landing position as described in the second aspect above.

[0017] In a fourth aspect, an embodiment of the present invention provides a system for obtaining a landing position, including:

[0018] A device for acquiring a landing position is used to acquire depth information of a target area, wherein the target area is a landing area in an environment below the drone determined based on semantic segmentation of an image, wherein the image is obtained by photographing the environment by the photographing device; and a target landing area is determined in the target area based on the depth information;

[0019] The unmanned aerial vehicle is used to fly to the target landing area.

[0020] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method described in the first aspect above.

[0021] In an embodiment of the present invention, the device for obtaining the landing position can obtain depth information of the target area, and determine the target landing area in the target area based on the depth information, wherein the target area is a landing area in the environment below the drone determined after semantic segmentation of the image, and the image is obtained by the shooting device shooting the environment. Through this implementation, the target landing area can be determined more effectively to achieve high-speed and safe landing of the drone. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 is a structural schematic diagram of a system for obtaining a landing position provided by an embodiment of the present invention;

[0024] Figure 2 is a flow chart of a method for obtaining a landing position provided by an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of determining a landing area provided by an embodiment of the present invention;

[0026] Figure 4 It is a structural schematic diagram of a device for obtaining a landing position provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present invention will be described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0029] The method for obtaining a landing position provided in an embodiment of the present invention can be performed by a system for obtaining a landing position, wherein the system for obtaining a landing position includes a device for obtaining a landing position and a drone. In some embodiments, the device for obtaining a landing position can be installed on a drone. In some embodiments, the device for obtaining a landing position can be spatially independent of the drone. In some embodiments, the device for obtaining a landing position can be a component of a drone, that is, the drone includes a device for obtaining a landing position. In other embodiments, the method for obtaining a landing position can also be applied to other movable devices, such as robots, unmanned vehicles, unmanned ships, and other movable devices that can move autonomously. In some embodiments, a shooting device is mounted on the drone; in some embodiments, a gimbal can be mounted on the drone, and the shooting device can be mounted on the gimbal of the drone. In some embodiments, the shooting device can include but is not limited to a monocular shooting device, a binocular shooting device, and the like.

[0030] The system for obtaining a landing position provided by an embodiment of the present invention is schematically described below with reference to the accompanying drawings.

[0031] See also Figure 1 , Figure 1: is a structural schematic diagram of a system for obtaining a landing position provided by an embodiment of the present invention. The system for obtaining a landing position includes: a device 11 for obtaining a landing position and an unmanned aerial vehicle 12. The unmanned aerial vehicle 12 includes a power system 121, and the power system 121 is used to provide power for the movement of the unmanned aerial vehicle 12. In some embodiments, the device 11 for obtaining a landing position is arranged in the unmanned aerial vehicle 12, and a communication connection can be established with other devices in the unmanned aerial vehicle (such as the power system 121) through a wired communication connection. In other embodiments, the unmanned aerial vehicle 12 and the device 11 for obtaining a landing position are independent of each other. For example, the device 11 for obtaining a landing position is arranged in a cloud server, and a communication connection is established with the unmanned aerial vehicle 12 through a wireless communication connection. In some embodiments, the device 11 for obtaining a landing position can be a flight controller.

[0032] In an embodiment of the present invention, the device 11 for obtaining the landing position can obtain depth information of the target area, and determine the target landing area in the target area based on the depth information, wherein the target area is a landing area in the environment below the drone 12 determined after semantic segmentation of the image, and the image is obtained by the shooting device shooting the environment.

[0033] The method for obtaining a landing position provided by an embodiment of the present invention is schematically described below with reference to the accompanying drawings.

[0034] For details, please see Figure 2 , Figure 2 The present invention provides a flowchart of a method for obtaining a landing position provided by an embodiment of the present invention. The method can be executed by a device for obtaining a landing position, wherein the specific explanation of the device for obtaining a landing position is as described above. Specifically, the method of the embodiment of the present invention includes the following steps.

[0035] S201: Acquire depth information of a target area, where the target area is a landing area in an environment below the drone determined based on semantic segmentation of an image, and the image is obtained by photographing the environment with a photographing device.

[0036] In an embodiment of the present invention, a device for acquiring a landing position can acquire depth information of a target area, where the target area is a landing area in an environment below the drone determined based on semantic segmentation of an image, and the image is obtained by photographing the environment with a photographing device.

[0037] Semantic segmentation is a classification at the pixel level. Pixels belonging to the same category are classified into one category. Therefore, semantic segmentation understands images at the pixel level. For example, when looking at the environment below from the perspective of a drone, pixels belonging to the lawn are classified into one category, pixels belonging to the road are also classified into one category, and in addition, pixels belonging to the roof are also classified into one category. Specifically, the semantic segmentation model can be trained by deep learning methods, such as Patch classification, full convolution method (FCN), etc. This is only an example, and the embodiments of the present invention do not limit the specific method of semantic learning.

[0038] In some embodiments, the image is captured when the drone is in a height range greater than a first height value; and the depth information is acquired when the drone is in a height range not greater than the first height value.

[0039] In one embodiment, when the drone is in a height range greater than the first height value, the device for acquiring the landing position can acquire an image of the environment below the drone taken by a shooting device on the drone, and determine the landing area in the environment below the drone by semantically segmenting the image. In some embodiments, the height range of the first height value belongs to a high-altitude area. In one example, the first height value can be 50m; in one example, the height range of the first height value can be 50m to 200m. Through this embodiment, when the drone is in a high-altitude area, the landing area of ​​the drone can be roughly determined to control the drone to fly to the landing area.

[0040] In one embodiment, when the device for obtaining the landing position determines the landing area in the environment below the drone by semantically segmenting the image, it can perform semantic analysis on the image to determine image areas of different semantic categories, and segment the image areas of different semantic categories to determine the landing area.

[0041] In some embodiments, the image areas of different semantic categories include, but are not limited to, houses, vehicles, woods, lawns, playgrounds, roads, and other areas. In some embodiments, the landing area may include, but are not limited to, lawns, playgrounds, roads, and other areas. In some embodiments, image areas of different semantic categories may be marked with different identification information; in some embodiments, the different identification information may include, but are not limited to, numbers, colors, shapes, and the like; in one example, image areas of different semantic categories may be marked with different colors, such as marking houses with red, lawns with green, roads with yellow, and the like. Through this implementation, areas of different semantic types can be distinguished, which helps determine the landing area of ​​the drone.

[0042] Specific can Figure 3 Take the example of determining the landing area in the environment below the drone as an example. Figure 3 is a schematic diagram of determining a landing area provided by an embodiment of the present invention, such as Figure 3 As shown, triangle 31 represents a camera (i.e., a shooting device) mounted on a drone and shooting vertically downward, wherein the focal length of the camera is f, the height of the camera from the ground measured by the barometer is z, and the camera shoots vertically downward to obtain image 32. The device for obtaining the landing position can determine the landing area 331 and the unsuitable landing area 332 on the ground 33 by semantically segmenting the image 32, and determine the position information of the nearest safe landing area 333 in the landing area 331.

[0043] In some embodiments, the position information of the safe landing area 333 is converted to the world coordinate system through the attitude information of the drone, such as the yaw angle yaw, to obtain the position information of the safe landing area 333. In some embodiments, the yaw angle yaw of the drone can be obtained by integrating the inertial measurement unit (IMU) with the electronic compass and GPS; in some embodiments, the process of the drone flying to the safe landing area is a process of landing and translating at the same time. Through this implementation, the drone can be controlled to fly to the safe landing area while landing.

[0044] In one embodiment, the area of ​​the landing area 331 can be calculated using the following formula (1) according to the positional relationship between the center points of the two areas on the acquired image 32, the focal length, and the height above the ground measured by a barometer.

[0045]

[0046] Wherein, P is the position of the safe landing point in the camera coordinate system, f is the focal length of the camera (in pixels), and z is the height above the ground measured by the barometer. In some embodiments, the optical center of the camera and the installation position of the barometer are fixed, and the distance between the camera and the ground can be obtained through the design drawings.

[0047] In one embodiment, when acquiring the depth information of the target area, the device for acquiring the landing position may acquire the depth information through a monocular camera and / or a binocular camera.

[0048] In one embodiment, when acquiring the depth information of the target area, the device for acquiring the landing position may perform a multi-frame shooting operation on the target area through the monocular shooting device when the drone is in a height range greater than a second height value, and determine the depth information based on the image obtained by the multi-frame shooting operation. In some embodiments, the second height value is lower than the first height value.

[0049] In one embodiment, when the drone is in a height range greater than the second height value, the device for acquiring the landing position may acquire a multi-frame shooting operation of the target area through a monocular shooting device, and determine the depth information based on the image obtained by the multi-frame shooting operation. In some embodiments, when the drone is in a height range greater than the second height value, the device for acquiring the landing position may also determine the depth information based on the multi-frame image obtained by radar detection. In one example, the height range of the second height value may be 10m-50m.

[0050] Through this implementation, the boundaries can be made more accurate and the depth information in the same category can be smoother. When optimizing the depth map, the smoothing items of the same category are added and the distinction between different categories is increased, which further helps to determine the image areas of different semantic categories.

[0051] In one embodiment, when acquiring the depth information of the target area, the device for acquiring the landing position can determine the depth information by using the parallax of the image captured by the binocular camera device on the target area when the drone is in a height range not greater than a second height value.

[0052] In one embodiment, when the drone is in a height range not greater than the second height value, the device for acquiring the landing position can acquire an image of the target area captured by the binocular camera device, and determine the depth information through the parallax of the image captured by the binocular camera device on the target area. This embodiment helps to more accurately determine image areas of different semantic categories, thereby more accurately determining the target area.

[0053] In one example, when the drone is at a height range less than 10 m, the device for acquiring the landing position may determine the depth information through the parallax of an image captured by a binocular camera mounted on the drone of the target area.

[0054] In one embodiment, the shooting device is connected to the drone body via a gimbal, and the device for obtaining the landing position can control the gimbal to turn to a specified direction, and control the shooting device on the gimbal to shoot the ground in the specified direction to obtain the image. In some embodiments, the specified direction can be the bottom of the drone, or a direction with a certain angle to the bottom of the drone, which is not specifically limited in the embodiments of the present invention.

[0055] In one embodiment, the device for obtaining the landing position can control the gimbal to turn to a designated direction corresponding to the ground and having a certain angle with the bottom of the drone during the shooting process of the shooting device, and control the shooting device on the gimbal to shoot the ground in the designated direction to obtain an image of the ground in the designated direction. Through this embodiment, the shooting device can shoot the ground more comprehensively, and the obtained image is more effective, which helps to more accurately determine the target area where landing can be carried out.

[0056] In one example, the device for obtaining the landing position can control the gimbal to turn toward the ground house and form a 45-degree direction with the drone, and control the shooting device on the gimbal to shoot the wall of the ground house to obtain an image including the wall of the house.

[0057] In one embodiment, when determining the landing area, the device for obtaining the landing position can also determine the image area in the image whose semantics meet preset semantic conditions based on semantic segmentation, and determine the mapping relationship between the image area and the geographic location coordinates in the environment based on the camera parameters when taking the image and the posture information of the drone, and determine the corresponding location coordinates of the landing area based on the mapping relationship.

[0058] In one example, a device for obtaining a landing position can determine an image area in the image whose semantics meet preset semantic conditions based on semantic segmentation, and determine a mapping relationship between the image area and the geographic location coordinates in the environment based on camera parameters when the image was taken and the posture information of the drone, and based on the mapping relationship, convert the geographic location coordinates of the image area into a world coordinate system, thereby determining the location coordinates of the corresponding landing area.

[0059] S202: Determine a target landing area in the target area based on the depth information.

[0060] In the embodiment of the present invention, the device for acquiring the landing position may determine the target landing area in the target area based on the depth information.

[0061] In one embodiment, when the device for acquiring the landing position determines the target landing area in the target area based on the depth information, it can determine a candidate area in the target area that meets a preset flatness condition based on the depth information, and determine the target landing area in the candidate area based on the size information of the drone. In some embodiments, the preset flatness condition includes depth information of the same semantics or different semantics within a preset range.

[0062] In one example, if the lawn and the playground have different semantics, but the depth information is within a preset range, it is determined that the preset leveling condition is met. In another example, if the house and the lawn have different semantics, and the depth information exceeds the preset range, it is determined that the preset leveling condition is not met. This implementation method can more effectively determine the selected area from the target area.

[0063] In one example, assuming that the size information of the drone is relatively small, if the determined candidate area includes a lawn and a road, the road can be determined as the target landing area based on the size information of the drone. This implementation can avoid the problem of being unable to find the drone in the target landing area due to the drone being too small to land in a larger target landing area, thereby improving the flexibility of determining the target landing area.

[0064] In one embodiment, the target area includes sub-areas divided into different semantics; when the device for acquiring the landing position determines the target landing area in the target area based on the depth information, it can determine whether the boundary position between adjacent sub-areas meets the preset flatness condition based on the depth information; if so, it is determined that the selected area includes the boundary position.

[0065] In one example, assuming that the target area is divided into three sub-areas with different semantics, namely, lawn, playground, and house, if the lawn sub-area is adjacent to the playground sub-area, and the depth information of the boundary position meets the preset range, it can be determined that the preset flatness condition is met, and the area to be selected includes the boundary position of the lawn sub-area and the playground sub-area. Through this implementation, the area to be selected can be further determined more accurately.

[0066] In one embodiment, the device for acquiring the landing position can control the drone to descend while controlling the drone to move in the horizontal direction, so that the drone approaches the target landing area in the horizontal direction. Through this implementation, the drone can be controlled to move in the horizontal direction while controlling the drone to land, so as to more effectively adjust the position of the drone and achieve a smooth transition landing.

[0067] In an embodiment of the present invention, the device for obtaining the landing position can obtain depth information of the target area, and determine the target landing area in the target area based on the depth information, wherein the target area is a landing area in the environment below the drone determined after semantic segmentation of the image, and the image is obtained by the shooting device shooting the environment. Through this implementation, the target landing area can be determined more effectively to achieve high-speed and safe landing of the drone.

[0068] See also Figure 4 , Figure 4401 and 402. The device for acquiring the landing position is a schematic diagram of a structure of a device for acquiring the landing position provided by an embodiment of the present invention. Specifically, the device for acquiring the landing position includes: a memory 401 and a processor 402.

[0069] In one embodiment, the device for acquiring the landing position further includes a data interface 403, and the data interface 403 is used to transmit data information between the device for acquiring the landing position and other devices.

[0070] The memory 401 may include a volatile memory; the memory 401 may also include a non-volatile memory; the memory 401 may also include a combination of the above-mentioned types of memories. The processor 402 may be a central processing unit (CPU). The processor 402 may further include a hardware chip. The above-mentioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA) or any combination thereof.

[0071] The memory 401 is used to store programs, and the processor 402 can call the program stored in the memory 401 to perform the following steps:

[0072] Acquire depth information of a target area, where the target area is a landing area in an environment below the drone determined based on semantic segmentation of an image, where the image is obtained by photographing the environment with the photographing device;

[0073] Based on the depth information, a target landing area is determined in the target area.

[0074] Further, the image is captured when the drone is in a height range greater than a first height value;

[0075] The depth information is acquired when the drone is in a height range not greater than the first height value.

[0076] Furthermore, when the processor 402 obtains the depth information of the target area, it is specifically used to:

[0077] The depth information is acquired through a monocular camera and / or a binocular camera.

[0078] Furthermore, when the processor 402 obtains the depth information of the target area, it is specifically used to:

[0079] When the drone is in a height range greater than a second height value, performing a multi-frame shooting operation on the target area through the monocular shooting device, and determining the depth information according to images obtained by the multi-frame shooting operation;

[0080] When the drone is in a height range not greater than a second height value, the depth information is determined by parallax of an image captured by the binocular camera device on the target area.

[0081] Furthermore, the second height value is lower than the first height value.

[0082] Further, when determining a target landing area in the target area based on the depth information, the processor 402 is specifically configured to:

[0083] Based on the depth information, determining a candidate area in the target area that meets a preset flatness condition;

[0084] Based on the size information of the UAV, the target landing area is determined in the candidate area.

[0085] Furthermore, the target area includes sub-areas divided into different semantics; when the processor 402 determines the candidate area that meets the preset flatness condition in the target area based on the depth information, it is specifically used to:

[0086] Based on the depth information, determining whether a boundary position between adjacent sub-areas meets the preset flatness condition;

[0087] If it meets the conditions, it is determined that the to-be-selected area includes the boundary position.

[0088] Furthermore, the shooting device is connected to the drone body via a gimbal, and the processor 402 is further configured to:

[0089] Control the pan / tilt head to turn to a specified direction;

[0090] The shooting device on the pan-tilt platform is controlled to shoot the ground in the specified direction to obtain the image.

[0091] Furthermore, the processor 402 is further configured to:

[0092] Determine, based on semantic segmentation, an image region in the image whose semantics meet a preset semantic condition;

[0093] Determine a mapping relationship between the image area and the geographic location coordinates in the environment based on camera parameters and the attitude information of the drone when the image is taken;

[0094] Based on the mapping relationship, the position coordinates of the corresponding landing area are determined.

[0095] Furthermore, the processor 402 is further configured to:

[0096] While controlling the drone to descend, the drone is controlled to move in a horizontal direction so that the drone approaches the target landing area in the horizontal direction.

[0097] In an embodiment of the present invention, the device for obtaining the landing position can obtain depth information of the target area, and determine the target landing area in the target area based on the depth information, wherein the target area is a landing area in the environment below the drone determined based on semantic segmentation of the image, and the image is obtained by the shooting device shooting the environment. Through this implementation, the target landing area can be determined more effectively to achieve high-speed and safe landing of the drone.

[0098] The embodiment of the present invention also provides a drone, which includes: a fuselage; a power system configured on the fuselage, which is used to provide power for the drone to move; a shooting device, which is used to shoot the environment below the drone to obtain an image; a processor, which is used to obtain depth information of a target area, wherein the target area is a landing area in the environment below the drone determined based on semantic segmentation of the image, and the image is obtained by the shooting device shooting the environment; based on the depth information, a target landing area is determined in the target area. And the above-mentioned device for obtaining the landing position.

[0099] Further, the image is captured when the drone is in a height range greater than a first height value;

[0100] The depth information is acquired when the drone is in a height range not greater than the first height value.

[0101] Furthermore, when the processor 402 obtains the depth information of the target area, it is specifically used to:

[0102] The depth information is acquired through a monocular camera and / or a binocular camera.

[0103] Furthermore, when the processor 402 obtains the depth information of the target area, it is specifically used to:

[0104] When the drone is in a height range greater than a second height value, performing a multi-frame shooting operation on the target area through the monocular shooting device, and determining the depth information according to images obtained by the multi-frame shooting operation;

[0105] When the drone is in a height range not greater than a second height value, the depth information is determined by parallax of an image captured by the binocular camera device on the target area.

[0106] Furthermore, the second height value is lower than the first height value.

[0107] Further, when determining a target landing area in the target area based on the depth information, the processor 402 is specifically configured to:

[0108] Based on the depth information, determining a candidate area in the target area that meets a preset flatness condition;

[0109] Based on the size information of the UAV, the target landing area is determined in the candidate area.

[0110] Furthermore, the target area includes sub-areas divided into different semantics; when the processor 402 determines the candidate area that meets the preset flatness condition in the target area based on the depth information, it is specifically used to:

[0111] Based on the depth information, determining whether a boundary position between adjacent sub-areas meets the preset flatness condition;

[0112] If it meets the conditions, it is determined that the to-be-selected area includes the boundary position.

[0113] Furthermore, the shooting device is connected to the drone body via a gimbal, and the processor 402 is further configured to:

[0114] Control the pan / tilt head to turn to a specified direction;

[0115] The shooting device on the pan-tilt platform is controlled to shoot the ground in the specified direction to obtain the image.

[0116] Furthermore, the processor 402 is further configured to:

[0117] Determine, based on semantic segmentation, an image region in the image whose semantics meet a preset semantic condition;

[0118] Determine a mapping relationship between the image area and the geographic location coordinates in the environment based on camera parameters and the attitude information of the drone when the image is taken;

[0119] Based on the mapping relationship, the position coordinates of the corresponding landing area are determined.

[0120] Furthermore, the processor 402 is further configured to:

[0121] While controlling the drone to descend, the drone is controlled to move in a horizontal direction so that the drone approaches the target landing area in the horizontal direction.

[0122] In an embodiment of the present invention, the drone can obtain depth information of a target area, and determine a target landing area in the target area based on the depth information, wherein the target area is a landing area in the environment below the drone determined based on semantic segmentation of an image, and the image is obtained by the shooting device shooting the environment. Through this implementation, the target landing area can be determined more effectively to achieve high-speed and safe landing of the drone.

[0123] An embodiment of the present invention further provides a system for obtaining a landing position, the system comprising:

[0124] A device for acquiring a landing position is used to acquire depth information of a target area, wherein the target area is a landing area in an environment below the drone determined based on semantic segmentation of an image, wherein the image is obtained by photographing the environment by the photographing device; and a target landing area is determined in the target area based on the depth information;

[0125] The unmanned aerial vehicle is used to fly to the target landing area.

[0126] Further, the image is captured when the drone is in a height range greater than a first height value;

[0127] The depth information is acquired when the drone is in a height range not greater than the first height value.

[0128] Furthermore, when the device for acquiring the landing position acquires the depth information of the target area, it is specifically used to:

[0129] The depth information is acquired through a monocular camera and / or a binocular camera.

[0130] Furthermore, when the device for acquiring the landing position acquires the depth information of the target area, it is specifically used to:

[0131] When the drone is in a height range greater than a second height value, performing a multi-frame shooting operation on the target area through the monocular shooting device, and determining the depth information according to images obtained by the multi-frame shooting operation;

[0132] When the drone is in a height range not greater than a second height value, the depth information is determined by parallax of an image captured by the binocular camera device on the target area.

[0133] Furthermore, the second height value is lower than the first height value.

[0134] Further, the device for acquiring the landing position is specifically used to determine the target landing area in the target area based on the depth information:

[0135] Based on the depth information, determining a candidate area in the target area that meets a preset flatness condition;

[0136] Based on the size information of the UAV, the target landing area is determined in the candidate area.

[0137] Furthermore, the target area includes sub-areas divided into different semantics; when the device for acquiring the landing position determines the candidate area that meets the preset flatness condition in the target area based on the depth information, it is specifically used to:

[0138] Based on the depth information, determining whether a boundary position between adjacent sub-areas meets the preset flatness condition;

[0139] If it meets the conditions, it is determined that the to-be-selected area includes the boundary position.

[0140] Furthermore, the shooting device is connected to the drone body via a gimbal, and the device for obtaining the landing position is also used for:

[0141] Control the pan / tilt head to turn to a specified direction;

[0142] The shooting device on the pan-tilt platform is controlled to shoot the ground in the specified direction to obtain the image.

[0143] Furthermore, the device for obtaining the landing position is also used for:

[0144] Determine, based on semantic segmentation, an image region in the image whose semantics meet a preset semantic condition;

[0145] Determine a mapping relationship between the image area and the geographic location coordinates in the environment based on camera parameters and the attitude information of the drone when the image is taken;

[0146] Based on the mapping relationship, the position coordinates of the corresponding landing area are determined.

[0147] Furthermore, the device for obtaining the landing position is also used for:

[0148] While controlling the drone to descend, the drone is controlled to move in a horizontal direction so that the drone approaches the target landing area in the horizontal direction.

[0149] In an embodiment of the present invention, the device for obtaining the landing position can obtain depth information of the target area, and determine the target landing area in the target area based on the depth information, wherein the target area is a landing area in the environment below the drone determined based on semantic segmentation of the image, and the image is obtained by the shooting device shooting the environment. Through this implementation, the target landing area can be determined more effectively to achieve high-speed and safe landing of the drone.

[0150] The embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the present invention is implemented. Figure 2 The method described in the corresponding embodiment can also be implemented Figure 4 The equipment corresponding to the embodiment of the present invention will not be described in detail here.

[0151] The computer-readable storage medium may be an internal storage unit of the device described in any of the foregoing embodiments, such as a hard disk or memory of the device. The computer-readable storage medium may also be an external storage device of the device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0152] The above disclosure is only part of the embodiments of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for obtaining a landing position, characterized in that: Applied to a mobile device, the mobile device is equipped with a photographing device, and the method includes: Acquire depth information of a target area, where the target area is a landing area in an environment below the movable device determined based on semantic segmentation of an image, where the image is obtained by photographing the environment by the photographing device; Based on the depth information, the movable device is controlled to land in a target landing area in the target area, where the target landing area is an area that meets a preset flatness condition, and the preset flatness condition is related to the depth information.

2. The method according to claim 1, characterized in that The image is captured when the movable device is in a height range greater than a first height value; The depth information is acquired when the movable device is in a height range not greater than the first height value.

3. The method according to claim 2, characterized in that The obtaining of depth information of the target area includes: The depth information is acquired through a monocular camera and / or a binocular camera.

4. The method according to claim 3, characterized in that The obtaining of depth information of the target area includes: When the movable device is in a height range greater than a second height value, performing a multi-frame shooting operation on the target area through the monocular shooting device, and determining the depth information according to images obtained by the multi-frame shooting operation; When the movable device is in a height range not greater than a second height value, the depth information is determined by parallax of an image captured by the binocular camera device on the target area.

5. The method according to claim 4, characterized in that The second height value is lower than the first height value.

6. The method according to claim 1, characterized in that The controlling the movable device to land in a target landing area in the target area based on the depth information includes: Based on the depth information, determining a candidate area in the target area that meets the preset flatness condition; The target landing area is determined in the candidate area based on the size information of the movable device.

7. The method according to claim 6, characterized in that The target area includes sub-areas divided into different semantics; and determining, based on the depth information, a candidate area in the target area that meets the preset flatness condition includes: Based on the depth information, determining whether a boundary position between adjacent sub-areas meets the preset flatness condition; If it meets the conditions, it is determined that the to-be-selected area includes the boundary position.

8. The method according to claim 1, characterized in that: The shooting device is connected to the body of the movable device via a pan-tilt platform, and the method further includes: Control the pan / tilt head to turn to a specified direction; The shooting device on the pan-tilt platform is controlled to shoot the ground in the specified direction to obtain the image.

9. The method according to claim 1, characterized in that: Also includes: Determine, based on semantic segmentation, an image region in the image whose semantics meet a preset semantic condition; Determine a mapping relationship between the image area and the geographic location coordinates in the environment based on camera parameters when the image is taken and posture information of the movable device; Based on the mapping relationship, the position coordinates of the corresponding landing area are determined.

10. The method according to claim 1, characterized in that Also includes: While controlling the movable device to descend, the movable device is controlled to move in a horizontal direction so that the movable device approaches the target landing area in the horizontal direction.

11. A device for obtaining a landing position, characterized in that: including memory and processor; The memory is used to store programs; The processor is used to call the program, and when the program is executed, is used to perform the following operations: Acquire depth information of a target area, where the target area is a landing area in an environment below the movable device determined based on semantic segmentation of an image, where the image is obtained by photographing the environment with a photographing device; Based on the depth information, the movable device is controlled to land in a target landing area in the target area, where the target landing area is an area that meets a preset flatness condition, and the preset flatness condition is related to the depth information.

12. The device according to claim 11, characterized in that The image is captured when the movable device is in a height range greater than a first height value; The depth information is acquired when the movable device is in a height range not greater than the first height value.

13. The device according to claim 12, characterized in that When the processor obtains the depth information of the target area, it is specifically used to: The depth information is acquired through a monocular camera and / or a binocular camera.

14. The device according to claim 13, characterized in that When the processor obtains the depth information of the target area, it is specifically used to: When the movable device is in a height range greater than a second height value, performing a multi-frame shooting operation on the target area through the monocular shooting device, and determining the depth information according to images obtained by the multi-frame shooting operation; When the movable device is in a height range not greater than a second height value, the depth information is determined by parallax of an image captured by the binocular camera device on the target area.

15. The device according to claim 14, characterized in that The second height value is lower than the first height value.

16. The device according to claim 11, characterized in that When the processor controls the movable device to land in a target landing area in the target area based on the depth information, the processor is specifically configured to: Based on the depth information, determining a candidate area in the target area that meets the preset flatness condition; The target landing area is determined in the candidate area based on the size information of the movable device.

17. The device according to claim 16, characterized in that The target area includes sub-areas divided into different semantics; when the processor determines, based on the depth information, a candidate area in the target area that meets the preset flatness condition, the processor is specifically used to: Based on the depth information, determining whether a boundary position between adjacent sub-areas meets the preset flatness condition; If it meets the conditions, it is determined that the to-be-selected area includes the boundary position.

18. The device according to claim 11, characterized in that The shooting device is connected to the mobile device body via a pan-tilt platform, and the processor is further used for: Control the pan / tilt head to turn to a specified direction; The shooting device on the pan-tilt platform is controlled to shoot the ground in the specified direction to obtain the image.

19. The device according to claim 11, characterized in that The processor is further configured to: Determine, based on semantic segmentation, an image region in the image whose semantics meet a preset semantic condition; Determine a mapping relationship between the image area and the geographic location coordinates in the environment based on camera parameters when the image is taken and posture information of the movable device; Based on the mapping relationship, the position coordinates of the corresponding landing area are determined.

20. The device according to claim 11, characterized in that The processor is further configured to: While controlling the movable device to descend, the movable device is controlled to move in a horizontal direction so that the movable device approaches the target landing area in the horizontal direction.

21. A movable device, characterized in that: include: body; A power system configured on the fuselage, used to provide power for the movable device to move; A shooting device, used for shooting the environment below the movable device to obtain an image; A processor is used to obtain depth information of a target area, wherein the target area is a landing area in an environment below a movable device determined based on semantic segmentation of an image, wherein the image is obtained by photographing the environment by the photographing device; based on the depth information, control the movable device to land in a target landing area in the target area, wherein the target landing area is an area that meets a preset flatness condition, and the preset flatness condition is related to the depth information.

22. The movable device according to claim 21, characterized in that: The image is captured when the movable device is in a height range greater than a first height value; The depth information is acquired when the movable device is in a height range not greater than the first height value.

23. The movable device according to claim 22, characterized in that: When the processor obtains the depth information of the target area, it is specifically used to: The depth information is acquired through a monocular camera and / or a binocular camera.

24. The movable device according to claim 23, characterized in that: When the processor obtains the depth information of the target area, it is specifically used to: When the movable device is in a height range greater than a second height value, performing a multi-frame shooting operation on the target area through the monocular shooting device, and determining the depth information according to images obtained by the multi-frame shooting operation; When the movable device is in a height range not greater than a second height value, the depth information is determined by parallax of an image captured by the binocular camera device on the target area.

25. The movable device according to claim 24, characterized in that: The second height value is lower than the first height value.

26. The movable device according to claim 21, characterized in that: When the processor controls the movable device to land in a target landing area in the target area based on the depth information, the processor is specifically configured to: Based on the depth information, determining a candidate area in the target area that meets the preset flatness condition; The target landing area is determined in the candidate area based on the size information of the movable device.

27. The movable device according to claim 26, characterized in that: The target area includes sub-areas divided into different semantics; when the processor determines, based on the depth information, a candidate area in the target area that meets the preset flatness condition, the processor is specifically used to: Based on the depth information, determining whether a boundary position between adjacent sub-areas meets the preset flatness condition; If it meets the conditions, it is determined that the to-be-selected area includes the boundary position.

28. The movable device according to claim 21, characterized in that: The shooting device is connected to the mobile device body via a pan-tilt platform, and the processor is further used for: Control the pan / tilt head to turn to a specified direction; The shooting device on the pan-tilt platform is controlled to shoot the ground in the specified direction to obtain the image.

29. The movable device according to claim 21, characterized in that: The processor is further configured to: Determine, based on semantic segmentation, an image region in the image whose semantics meet a preset semantic condition; Determine a mapping relationship between the image area and the geographic location coordinates in the environment based on camera parameters when the image is taken and posture information of the movable device; Based on the mapping relationship, the position coordinates of the corresponding landing area are determined.

30. The movable device according to claim 21, characterized in that: The processor is further configured to: While controlling the movable device to descend, the movable device is controlled to move in a horizontal direction so that the movable device approaches the target landing area in the horizontal direction.

31. A system for obtaining a landing position, characterized in that: include: A device for acquiring a landing position is used to acquire depth information of a target area, wherein the target area is a landing area in an environment below a movable device determined based on semantic segmentation of an image, wherein the image is obtained by photographing the environment by a photographing device; based on the depth information, the movable device is controlled to land in a target landing area in the target area, wherein the target landing area is an area that meets a preset flatness condition, and the preset flatness condition is related to the depth information; The movable device is used to fly to the target landing area.

32. The system according to claim 31, characterized in that The image is captured when the movable device is in a height range greater than a first height value; The depth information is acquired when the movable device is in a height range not greater than the first height value.

33. The system according to claim 32, characterized in that When the device for acquiring the landing position acquires the depth information of the target area, it is specifically used to: The depth information is acquired through a monocular camera and / or a binocular camera.

34. The system according to claim 33, characterized in that When the device for acquiring the landing position acquires the depth information of the target area, it is specifically used to: When the movable device is in a height range greater than a second height value, performing a multi-frame shooting operation on the target area through the monocular shooting device, and determining the depth information according to images obtained by the multi-frame shooting operation; When the movable device is in a height range not greater than a second height value, the depth information is determined by parallax of an image captured by the binocular camera device on the target area.

35. The system according to claim 34, characterized in that The second height value is lower than the first height value.

36. The system according to claim 31, characterized in that When the device for acquiring the landing position controls the movable device to land in the target landing area in the target area based on the depth information, the device is specifically used to: Based on the depth information, determining a candidate area in the target area that meets the preset flatness condition; The target landing area is determined in the candidate area based on the size information of the movable device.

37. The system according to claim 36, characterized in that The target area includes sub-areas divided into different semantics; when the device for acquiring the landing position determines the candidate area that meets the preset flatness condition in the target area based on the depth information, it is specifically used to: Based on the depth information, determining whether a boundary position between adjacent sub-areas meets the preset flatness condition; If it meets the conditions, it is determined that the to-be-selected area includes the boundary position.

38. The system according to claim 31, characterized in that The shooting device is connected to the body of the movable device via a pan-tilt platform, and the device for obtaining the landing position is also used for: Control the pan / tilt head to turn to a specified direction; The shooting device on the pan-tilt platform is controlled to shoot the ground in the specified direction to obtain the image.

39. The system according to claim 31, characterized in that The device for obtaining the landing position is also used for: Determine, based on semantic segmentation, an image region in the image whose semantics meet a preset semantic condition; Determine a mapping relationship between the image area and the geographic location coordinates in the environment based on camera parameters when the image is taken and posture information of the movable device; Based on the mapping relationship, the position coordinates of the corresponding landing area are determined.

40. The system according to claim 31, characterized in that The device for obtaining the landing position is also used for: While controlling the movable device to descend, the movable device is controlled to move in a horizontal direction so that the movable device approaches the target landing area in the horizontal direction.

41. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

42. The method according to any one of claims 1-10, the device according to any one of claims 11-20, the removable device according to any one of claims 21-30, the system according to any one of claims 31-40, or the computer-readable storage medium according to claim 41, characterized in that: The movable device includes a drone.

Images