An Unmanned Device Landing Processing Method, Device, Equipment and Storage Medium

Abstract

The embodiments of the present application disclose a method, apparatus, device, and storage medium for handling the landing of an unmanned device. The technical solution provided by the embodiments of the present application determines ground information and obstacle information based on point cloud data in a flight area, determines one or more ground flat areas in the flight area according to the ground information, determines candidate landing areas from the one or more ground flat areas according to the obstacle information, and determines a target landing area from the candidate landing areas, reducing the situation that the safety of the selected candidate landing area is relatively low due to the relatively complex on-site environment of the flight area, resulting in accidents during the landing of the unmanned device, and ensuring the safety of the device and personnel when the unmanned device lands.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of unmanned device control, and in particular, to a method, device, equipment and storage medium for handling the landing of an unmanned device. Background Art

[0002] During the operation of an unmanned device, there are often situations where the unmanned device needs to land, for example, when the unmanned device receives a forced landing or return signal during flight, or when it encounters an emergency (such as low battery, equipment failure, etc.), the unmanned device needs to perform an emergency landing.

[0003] When an unmanned device performs a landing operation, it needs to plan a landing area by referring to the ground information and obstacle information in the flight area. The existing unmanned device landing solutions mainly rely on the map information of the flight area. For example, a ground empty area suitable for landing is found according to the map information. However, the flight site environment of the unmanned device is relatively complex, and it is difficult to determine a suitable landing area only relying on the map information, and the landing safety of the unmanned device cannot be guaranteed. Summary of the Invention

[0004] The embodiments of the present application provide a method, device, equipment and storage medium for handling the landing of an unmanned device, so as to solve the technical problem in the prior art that the landing area of the unmanned device depends on the map information and it is difficult to ensure the landing safety of the unmanned device, and ensure the safety of the unmanned device by selecting a suitable landing area for landing.

[0005] In a first aspect, the embodiments of the present application provide a method for handling the landing of an unmanned device, including:

[0006] Based on the point cloud data in the flight area, determine the ground information and obstacle information of the flight area;

[0007] Determine one or more ground flat areas in the flight area according to the ground information;

[0008] Determine a candidate landing area from one or more of the ground flat areas according to the obstacle information;

[0009] Determine a target landing area from the candidate landing areas.

[0010] In a second aspect, the embodiments of the present application provide a device for handling the landing of an unmanned device, including a flight area analysis module, a flat area analysis module, a candidate area determination module and a target area determination module, where:

[0011] The flight area analysis module is used to determine the ground information and obstacle information of the flight area based on the point cloud data in the flight area;

[0012] The flat area analysis module is configured to determine one or more ground flat areas within the flight area according to the ground information;

[0013] The candidate area determination module is configured to determine a candidate landing area from one or more of the ground flat areas according to the obstacle information;

[0014] The target area determination module is configured to determine a target landing area from the candidate landing areas.

[0015] In a third aspect, an embodiment of the present application provides an unmanned device landing processing device, including: a memory and one or more processors;

[0016] The memory is configured to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the unmanned device landing processing method as described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a storage medium storing computer-executable instructions, where the computer-executable instructions are used to execute the unmanned device landing processing method as described in the first aspect when executed by a computer processor.

[0019] In the embodiments of the present application, by determining the ground information and obstacle information according to the point cloud data within the flight area, determining one or more ground flat areas within the flight area according to the ground information, determining a candidate landing area from one or more ground flat areas according to the obstacle information, and determining a target landing area from the candidate landing areas, it reduces the situation that the safety of the selected candidate landing area is relatively low due to the relatively complex on-site environment of the flight area, resulting in accidents during the landing process of the unmanned device, and ensures the safety of the device and personnel when the unmanned device lands. Description of the Drawings

[0020] Figure 1 is a flowchart of an unmanned device landing processing method provided by an embodiment of the present application;

[0021] Figure 2 is a flowchart of another unmanned device landing processing method provided by an embodiment of the present application;

[0022] Figure 3 is an example diagram of a spiral landing route provided by an embodiment of the present application;

[0023] Figure 4 is a flowchart of another unmanned device landing processing method provided by an embodiment of the present application;

[0024] Figure 5It is an example diagram of a gliding landing route provided by an embodiment of the present application;

[0025] Figure 6 It is a schematic diagram for extracting a landing slide provided by an embodiment of the present application;

[0026] Figure 7 It is a flowchart of another method for processing the landing of an unmanned device provided by an embodiment of the present application;

[0027] Figure 8 It is a schematic structural diagram of a device for processing the landing of an unmanned device provided by an embodiment of the present application;

[0028] Figure 9 It is a schematic structural diagram of a device for processing the landing of an unmanned device provided by an embodiment of the present application. Detailed implementation manners

[0029] In order to make the objectives, technical solutions and advantages of the present application clearer, the following further describes in detail specific embodiments of the present application with reference to the accompanying drawings. It can be understood that the specific embodiments described herein are only used to explain the present application, rather than limiting the present application. Additionally, it should be noted that for the sake of description, only parts related to the present application are shown in the drawings rather than all the content. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. When the operations are completed, the above-mentioned process can be terminated, but there may also be additional steps not included in the drawings. The above-mentioned process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0030] Figure 1 A flowchart of a method for processing the landing of an unmanned device provided by an embodiment of the present application is given. The method for processing the landing of an unmanned device provided by an embodiment of the present application can be executed by a device for processing the landing of an unmanned device. The device for processing the landing of an unmanned device can be implemented in a hardware and / or software manner and integrated in a device for processing the landing of an unmanned device.

[0031] The following describes by taking the device for processing the landing of an unmanned device executing the method for processing the landing of an unmanned device as an example.

[0032] Refer to Figure 1 , the method for processing the landing of an unmanned device includes:

[0033] S101: Based on the point cloud data in the flight area, determine the ground information and obstacle information of the flight area.

[0034] Among them, the flight area can be understood as the area where the unmanned device (such as a drone) conducts flight operations. The point cloud data within the flight area can be collected by a shooting device (such as a lidar scanning device, a depth camera, a binocular camera, etc.) mounted on the unmanned device during the flight process of the unmanned device, or can also be collected before performing the flight mission (for example, collected in advance by an unmanned device or other mobile vehicle equipped with a shooting device for the flight area).

[0035] Exemplarily, obtain the point cloud data within the flight area of the unmanned device, and determine the ground information and obstacle information of the flight area according to the point cloud data. Among them, the ground information and obstacle information can respectively characterize the ground and obstacles (such as trees, houses, wires, utility poles, etc.) within the flight area. Optionally, determining the ground information and obstacle information of the flight area according to the point cloud data can be carried out in a manner of multi-modal fusion based on point cloud segmentation technology and image segmentation technology (multi-modal fusion of segmenting the point cloud data and the digital model of the flight area), or can also be based on a semantic segmentation algorithm.

[0036] S102: Determine one or more ground flat areas within the flight area according to the ground information.

[0037] Exemplarily, after determining the ground information and obstacle information within the flight area, determine one or more ground flat areas within the flight area according to the ground information. Among them, the ground flat area can be understood as an area with relatively flat terrain, and the flat parameters (such as a combination of one or more of height difference, average height, and height variance) of the ground flat area are within the corresponding height threshold.

[0038] S103: Determine the candidate landing area from one or more ground flat areas according to the obstacle information.

[0039] Exemplarily, after determining one or more ground flat areas within the flight area, further combine the obstacle information to determine the candidate landing area from the above-determined one or more ground flat areas. Among them, the candidate landing area can be understood as a ground flat area where the landing route is not interfered by obstacles when landing according to the landing method of the unmanned device.

[0040] For example, the landing route corresponding to each flat ground area where the unmanned device lands is determined according to the landing method of the unmanned device. For each landing route, it is determined whether there are obstacles in the landing route according to the obstacle information. If there are obstacles in the landing route (for example, there is obstacle information intersecting the landing route), it is considered that the unmanned device will be interfered by the obstacles during the landing process, and then the flat ground area corresponding to this landing route can be ignored. If there are no obstacles in the landing route (for example, no obstacle information intersecting the landing route is detected), it is considered that the unmanned device will not be interfered by the obstacles during the landing process, and then the flat ground area corresponding to this landing route can be used as a candidate landing area.

[0041] In a possible embodiment, the determination of the candidate landing area can be made before the unmanned device executes the flight mission, or can also be made during the execution of the flight mission. And the determination of the candidate landing area can be made by the unmanned device itself, or can also be determined and provided by the cloud server.

[0042] S104: Determine the target landing area from the candidate landing areas.

[0043] Exemplarily, after determining the candidate landing areas from the flat ground areas according to the obstacle information, the target landing area for the unmanned device to perform the landing operation can be determined from these candidate landing areas.

[0044] In a possible embodiment, the determination of the target landing area is made based on a combination of one or more of the distance between each candidate landing area and the unmanned device, the distance between the candidate landing area and the control terminal (such as the controller of the unmanned device) and / or the operator, and the area of the candidate landing area.

[0045] In a possible embodiment, the target landing area can also be determined according to the landing reason (normal landing or emergency landing) and / or the landing method (such as the hovering landing or gliding landing of a vertical takeoff and fixed-wing unmanned aerial vehicle) of the unmanned device, in combination with a combination of one or more of the distance between each candidate landing area and the unmanned device, the distance between the candidate landing area and the control terminal and / or the operator, and the area of the candidate landing area.

[0046] As described above, by determining the ground information and obstacle information according to the point cloud data in the flight area, determining one or more flat ground areas in the flight area according to the ground information, determining the candidate landing areas from the one or more flat ground areas according to the obstacle information, and determining the target landing area from the candidate landing areas, it reduces the situation that the safety of the selected candidate landing area is relatively low due to the relatively complex on-site environment of the flight area, resulting in accidents during the landing process of the unmanned device, and ensures the safety of the device and personnel when the unmanned device lands.

[0047] Based on the above embodiments, Figure 2 A flowchart of another method for processing the landing of an unmanned device provided by an embodiment of the present application is given. This method for processing the landing of an unmanned device is a specific implementation of the above method for processing the landing of an unmanned device. Referring to Figure 2 , this method for processing the landing of an unmanned device includes:

[0048] S201: Obtain point cloud data and digital models within the flight area.

[0049] S202: Based on point cloud segmentation technology and image segmentation technology, determine ground information and obstacle information within the flight area according to the point cloud data and digital models.

[0050] Exemplarily, obtain point cloud data and digital models within the flight area. Among them, the digital model can be a digital surface model (DSM, Digital Orthophoto Map) and / or a digital orthophoto map (DOM, Digital Orthophoto Map) corresponding to the flight area. The point cloud data provided in this embodiment can be the original point cloud data and / or dense point cloud data within the flight area.

[0051] Further, use a multi-modal fusion processing method of point cloud segmentation technology and image segmentation technology to perform segmentation processing on the point cloud data and digital models of the flight area to determine ground information and obstacle information within the flight area. For example, extract ground information corresponding to the ground area and obstacle information corresponding to obstacles from the digital model.

[0052] S203: By performing two-dimensional grid division on the ground information, divide one or more flat candidate areas in the ground information.

[0053] Exemplarily, perform two-dimensional grid division on the above-determined ground information, and divide one or more areas with a set size and shape (such as a rectangle with a size of 1m * 1m) in the ground information as flat candidate areas.

[0054] In a possible embodiment, after dividing one or more areas with a set size and shape from the ground information, flat candidate areas can be screened out from these areas by using a threshold screening method according to the height information of these areas. Based on this, when this solution divides one or more flat candidate areas in the ground information by performing two-dimensional grid division on the ground information, it includes:

[0055] S2031: By performing two-dimensional grid division on the ground information, divide one or more flat area blocks in the ground information.

[0056] S2032: Screen out one or more flat candidate regions from the flat region blocks according to the flat parameters of the flat region blocks.

[0057] Specifically, divide the ground information into two-dimensional grids according to the set size and shape, and divide one or more flat region blocks from the ground information. Calculate the flat parameters of each flat region block according to the height information of each point on the upper surface of each flat region block. Among them, the flat parameter can be used to represent the flatness of the flat region. Optionally, the flat parameter includes one or more of the height difference, average height, and height variance of the flat region block. This solution is described by taking the flat parameter including the height difference, average height, and height variance of the flat region block as an example.

[0058] Further, compare the flat parameters of each flat region with the set flat parameter threshold, screen out the flat region blocks whose flat parameters are within the flat parameter threshold, and use the screened flat region blocks as flat candidate regions. For example, compare the height difference, average height, and height variance of each flat region block with the set height difference threshold, average height threshold, and height variance threshold respectively, screen out the flat region blocks whose height difference, average height, and height variance are within the set height difference threshold, average height threshold, and height variance threshold respectively, and use the screened flat region blocks as flat candidate regions.

[0059] S204: Conduct adjacent region search for each flat candidate region, and fuse the flat candidate regions according to the adjacent region search results to obtain one or more ground flat regions within the flight region.

[0060] Exemplarily, for each flat candidate region, centered on itself, conduct adjacent region search based on the central coordinates to search for adjacent flat candidate regions (flat candidate regions with adjacent boundaries or the central distance within the set distance), and obtain the adjacent region search results, that is, obtain other flat candidate regions adjacent to each flat candidate region.

[0061] Further, determine the adjacent flat candidate regions according to the adjacent region search results, and perform fusion processing on the adjacent flat candidate regions to obtain one or more ground flat regions. Among them, a ground flat region can be composed of one flat candidate region (in this case, there are no other adjacent flat candidate regions for this flat candidate region), or it can be formed by fusing multiple adjacent flat candidate regions. Optionally, the fusion processing of adjacent flat candidate regions can be performed during the process of searching for adjacent regions, or it can be performed uniformly after determining the adjacent regions of each flat candidate region. This solution obtains a larger area of ground flat region by performing fusion processing on adjacent flat candidate regions, which is convenient for obtaining a more suitable target landing region subsequently.

[0062] S205: Determine the landing route fitting space in each flat ground area according to the hovering landing route of the unmanned device.

[0063] In this solution, a vertical fixed-wing unmanned aerial vehicle (UAV) is taken as an example of the unmanned device. The landing methods of the vertical fixed-wing UAV include hovering landing and gliding landing, and different landing methods correspond to different landing routes (landing paths). As Figure 3 Shown in the provided exemplary diagram of a hovering landing route, the hovering landing route (L1 in the figure, with the arrow direction being the forward direction) is a path that hovers downward and has a gradually decreasing radius. The area enclosed by the hovering landing route is a cone-like space with the tip pointing downward.

[0064] Exemplarily, taking hovering landing as the landing method of the unmanned device, after determining one or more flat ground areas, a landing route fitting space (cone-like space) is fitted on each flat ground area according to the hovering landing route of the unmanned device, and the corresponding radius information (the maximum radius of the landing route fitting space) of the landing route fitting space is determined.

[0065] S206: Determine the candidate landing areas from one or more flat ground areas according to the intersection of the obstacle information and the landing route fitting space.

[0066] Exemplarily, calculate the intersection of each landing route fitting space and the obstacle information, determine the intersection of each landing route fitting space and the obstacle information, and determine the candidate landing areas from the one or more flat ground areas determined above according to the intersection situation of the landing route fitting space and the obstacle information.

[0067] Specifically, when determining the candidate landing areas from one or more flat ground areas, for each landing route fitting space, when its intersection with the obstacle information is empty, determine the point cloud data corresponding to the absence of obstacles within the landing route fitting space. When the unmanned device lands according to the hovering landing route corresponding to the landing route fitting space, it will not be interfered by obstacles. Then, the flat ground area corresponding to the landing route fitting space can be determined as a candidate landing area, and continue to judge whether the next flat ground area can be used as a candidate landing area. When the intersection of the landing route fitting space and the obstacle information is not empty, determine the point cloud data corresponding to the presence of obstacles within the landing route fitting space. When the unmanned device lands according to the hovering landing route corresponding to the landing route fitting space, it will be interfered by obstacles. Then, ignore or discard the flat ground area corresponding to the landing route fitting space, and continue to judge whether the next flat ground area can be used as a candidate landing area.

[0068] In a possible embodiment, when determining a candidate landing area from one or more flat ground areas according to the intersection of obstacle information and the landing route fitting space, it may also be to determine the first point cloud data included in the flat ground area within the space enclosed by taking the center of the flat ground area as the origin and the radius information of the corresponding landing route fitting space as the radius, determine the second point cloud data included in the corresponding landing route fitting space, and determine the intersection of the first point cloud data and the second point cloud data. When the intersection of the first point cloud data and the second point cloud data is empty, the flat ground area corresponding to the landing route fitting space can be determined as a candidate landing area.

[0069] S207: Determine the target landing area from the candidate landing areas.

[0070] Exemplarily, when the unmanned device needs to perform a landing operation, the target landing area can be selected from the candidate landing areas for landing. When the unmanned device determines the target landing area for landing, it can perform a spiral landing according to the spiral landing route corresponding to the target landing area, reduce the interference caused by obstacles during the landing process, and ensure the safe landing of the unmanned device.

[0071] Among them, the selection of the target landing area can be determined according to the flight state of the unmanned device during landing, that is, when the flight state is relatively urgent, the nearest candidate landing area is preferentially selected for forced landing, and when the flight state is normal, the appropriate target landing area can be determined according to the scoring situation of different candidate landing areas for landing. Based on this, when determining the target landing area from the candidate landing areas in this solution, it includes:

[0072] S2071: When the unmanned device lands in the first flight state, determine the first distance between each candidate landing area and the unmanned device, the second distance between each candidate landing area and the operation terminal and / or the operator, and the area information of each candidate landing area, and determine the target landing area from the candidate landing areas according to the first distance, the second distance, and the area information; and / or

[0073] S2072: When the unmanned device lands in the second flight state, determine the first distance between each candidate landing area and the unmanned device, and determine the target landing area from the candidate landing areas according to the first distance.

[0074] The flight states provided by this solution include a first flight state and a second flight state, and the (landing) urgency of the second flight state is higher than that of the first flight state. That is, when the unmanned device is in the second flight state, it needs to complete the landing operation more quickly. For example, the first flight state can be a normal flight state. For example, when the unmanned device receives a return signal indicating a normal return for landing or a landing signal for normal landing during flight, the flight state of the unmanned device at this time is the first flight state. The second flight state can be an abnormal flight state. For example, when the unmanned device detects insufficient power, a flight fault, or needs to perform a breakpoint landing task after completing part of the task during flight, the flight state of the unmanned device at this time is the second flight state.

[0075] Exemplarily, when the unmanned device lands in the first flight state, calculate the first distance between each of the above-determined candidate landing areas and the unmanned device (which can be calculated based on the coordinates of the center point of the candidate landing area and the positioning coordinates of the unmanned device), the second distance between each candidate landing area and the operation terminal and / or the operator (which can be calculated based on the coordinates of the center point of the candidate landing area and the positioning coordinates of the operation terminal and / or the operator), and the area information of each candidate landing area. Further, calculate the landing point score of each candidate landing area based on the first distance, second distance, and area information calculated above, and determine the candidate landing area with the highest landing point score as the target landing area. The calculation of the landing point score can be obtained by performing a weighted calculation on the first distance, second distance, and area information according to the set weighting coefficients.

[0076] When the unmanned device returns and lands in the second flight state, calculate the first distance between each of the above-determined candidate landing areas and the unmanned device, and determine the candidate landing area with the smallest first distance as the target landing area to accelerate the landing speed of the unmanned device and ensure the safety of the unmanned device.

[0077] As described above, by determining ground information and obstacle information based on the point cloud data in the flight area, determining one or more ground flat areas in the flight area according to the ground information, determining candidate landing areas from the one or more ground flat areas according to the obstacle information, and determining the target landing area from the candidate landing areas, the situation that the safety of the selected candidate landing area is relatively low due to the relatively complex on-site environment in the flight area, resulting in accidents during the landing process of the unmanned device, is reduced, and the safety of the device and personnel during the landing of the unmanned device is ensured. At the same time, through the multi-modal segmentation technology fusion method based on the point cloud segmentation technology and the image segmentation technology, the ground information and obstacle information in the flight area can be accurately determined. And by means of two-dimensional grid division and adjacent area fusion, a larger and flatter ground flat area can be obtained, and an area more suitable for the unmanned device to land can be obtained, ensuring the landing safety of the unmanned device. The candidate landing areas are also screened through the intersection of the obstacles and the hovering landing route in each ground flat area, reducing the situation of the unmanned device colliding with obstacles and damaging the unmanned device during the landing process. And corresponding target landing area determination strategies are determined for different flight states during the landing of the unmanned device, and the landing point scores of different candidate landing areas are calculated according to different target landing area determination strategies to determine the optimal target landing area, ensuring the safe landing of the unmanned device and the safety of relevant personnel.

[0078] Based on the above embodiments, Figure 4 The flowchart of another unmanned device landing processing method provided by the embodiment of the present application is given. This unmanned device landing processing method is a concretization of the above unmanned device landing processing method. Refer to Figure 4 and this unmanned device landing processing method includes:

[0079] S301: Obtain the point cloud data in the flight area.

[0080] S302: Determine the ground information and obstacle information from the point cloud data based on the semantic segmentation algorithm, and divide the ground information into multiple ground candidate areas.

[0081] Exemplarily, obtain the point cloud data (raw point cloud data and / or dense point cloud data) in the flight area, and distinguish the ground information and obstacle information in the point cloud data based on the point cloud semantic segmentation algorithm (where the point cloud data corresponding to the ground under the obstacle is classified into the obstacle information).

[0082] After determining the ground information and obstacle information in the point cloud data, cluster processing is performed on the ground information by means of point cloud density clustering, so as to divide the ground information into multiple ground candidate areas.

[0083] S303: Determine one or more ground flat areas within the flight area from multiple ground candidate areas according to the height difference, length distance, and / or width distance of the ground candidate areas.

[0084] Exemplarily, calculate the boundary areas (such as the maximum inscribed rectangle of the ground candidate area) of the multiple determined ground candidate areas respectively, determine the length distance and / or width distance of the ground candidate area according to the boundary area, and calculate the height difference of the ground candidate area.

[0085] Furthermore, compare the height difference, length distance, and / or width distance of the ground candidate area with the set height difference threshold, length threshold, and / or width threshold respectively, and use the ground candidate areas that respectively meet the corresponding threshold requirements as the ground flat areas within the flight area.

[0086] S304: Determine one or more landing slides in each ground flat area according to the slide distance requirement of the unmanned device.

[0087] In this embodiment, the gliding landing of a vertical fixed-wing unmanned aerial vehicle is taken as an example of the landing method of the unmanned device for description. As Figure 5 Shown in an example gliding landing route diagram provided, the gliding landing route of the gliding landing includes a landing slide on the ground (L21 in the figure, and the arrow direction is the forward direction), and a gliding path that slopes downward towards the landing slide (L22 in the figure, and the arrow direction is the forward direction). The gliding path is the path above the extension direction of the landing slide.

[0088] Exemplarily, after determining one or more ground flat areas within the flight area, further determine one or more landing slides for the unmanned device to glide and land in each ground flat area according to the slide distance requirement of the unmanned device (the slide on the ground for the unmanned device to decelerate after gliding to the ground).

[0089] Among them, the slide distance corresponding to the slide distance requirement of the unmanned device can be determined according to the current flight speed of the unmanned device, and the faster the flight speed of the unmanned device, the longer the required slide distance.

[0090] In a possible embodiment, multiple candidate slides can be determined on each ground flat area, and then the appropriate landing slides are determined from these candidate slides. Based on this, when this solution determines one or more landing slides in each ground flat area according to the slide distance requirement of the unmanned device, it includes: determining candidate slides corresponding to multiple angles in each ground flat area, and determining one or more landing slides from the multiple candidate slides according to the slide distance requirement of the unmanned device.

[0091] Exemplarily, after determining one or more ground flat areas within the flight area, for each ground flat area, rotate it at multiple angles with the midpoint of the ground flat area as the rotation center, and generate a candidate runway for each rotation angle. Then, according to the runway distance requirement of the unmanned device, one or more landing runways that meet the runway distance requirement are selected from these candidate runways. As Figure 6 As shown in the schematic diagram of a landing runway extraction provided, in the ground flat area A, rotate it at multiple angles with the center O of the ground flat area A as the rotation center, generate a candidate runway for each rotation angle, and select a landing runway that meets the runway distance requirement (two landing runways L31 and L41 at rotation angles of 0° and 90° are schematically shown in the figure).

[0092] S305: Determine candidate landing areas from one or more ground flat areas according to the obstacle information in the extending direction of each landing runway.

[0093] Exemplarily, for each of the above-determined landing runways, determine the obstacle information in the extending direction of the landing runway to determine the obstacle height in the extending direction of each landing runway, and determine whether the landing runway meets the sliding and landing requirement of the unmanned device according to the obstacle height. For example, compare the obstacle height in the extending direction of the landing runway with a set obstacle height threshold. If there is an obstacle with an obstacle height reaching the set obstacle height threshold in the extending direction of the landing runway, determine that there is point cloud data corresponding to the obstacle in the gliding landing route corresponding to the landing runway. When the unmanned device lands according to the gliding landing route, it will be interfered by the obstacle, then ignore or discard the landing runway, and continue to judge the intersection situation between the extending direction of the next landing runway and the obstacle information.

[0094] Further, for a flat ground area, if there are no obstacles with a height reaching the set obstacle height threshold in the extension direction of the landing slide, it is considered that when the unmanned device glides and lands along the gliding landing route corresponding to the landing slide and its extension direction, it will not be interfered by obstacles. The flat ground area can be determined as a candidate landing area, and the landing slide without obstacles with a height reaching the set obstacle height threshold in its corresponding extension direction can be used as the gliding and decelerating path for the unmanned device during gliding landing. If there are obstacles with a height reaching the set obstacle height threshold in the extension direction of all landing slides on the flat ground area, it is considered that when the unmanned device glides and lands along any landing slide on the flat ground area and the gliding landing route corresponding to its extension direction, it will be interfered by obstacles. Then, this flat ground area is ignored, and it continues to be judged whether the next flat ground area can be used as a candidate landing area. Optionally, the set obstacle height threshold can be dynamically set according to the distance from the extension direction to the landing slide, and the greater the distance from the extension direction to the landing slide, the greater the set obstacle height threshold.

[0095] S306: Determine the target landing area from the candidate landing areas.

[0096] Exemplarily, when the unmanned device needs to perform a landing operation, the target landing area can be selected from the candidate landing areas provided above for landing. When the unmanned device determines the target landing area for landing, it can circle and land along the gliding landing route corresponding to the target landing area, reduce the interference caused by obstacles during the landing process, and ensure the safe landing of the unmanned device.

[0097] In a possible embodiment, when this solution determines the target landing area from the candidate landing areas, it includes:

[0098] S3061: Perform weighted fusion calculation based on one or more combinations of the flatness, slide length, slide width, obstacle height of the candidate landing area, and the third distance between each candidate landing area and the unmanned device.

[0099] S3062: Determine the target landing area from the candidate landing areas according to the weighted fusion calculation result.

[0100] Exemplarily, when the unmanned device needs to perform a landing operation, determine the flatness of the landing slide corresponding to each candidate landing area (which can be represented by one or a combination of the flat area blocks or the height difference, average height, and height variance of the landing slide), the slide length, and the slide width, the obstacle height corresponding to the obstacle information in the extending direction of the landing slide, and one or a combination of the third distances between each candidate landing area and the unmanned device, and perform a weighted fusion calculation on one or a combination of the flatness, slide length, slide width, obstacle height, and third distance to obtain a weighted fusion calculation result reflecting the slide scores corresponding to each landing slide.

[0101] Further, determine the landing slide corresponding to the weighted fusion calculation result with the highest slide score, and determine the candidate landing area corresponding to the landing slide as the target landing area. The unmanned device will glide and land on the target landing area according to the gliding landing route corresponding to the landing slide.

[0102] As described above, by determining the ground information and obstacle information based on the point cloud data in the flight area, determining one or more ground flat areas in the flight area according to the ground information, determining candidate landing areas from one or more ground flat areas according to the obstacle information, and determining the target landing area from the candidate landing areas, it reduces the situation that the safety of the selected candidate landing area is relatively low due to the complex on-site environment of the flight area, resulting in accidents during the landing process of the unmanned device, and ensures the safety of the device and personnel when the unmanned device lands. At the same time, based on the semantic segmentation algorithm, the ground information and obstacle information in the flight area can be accurately obtained, and multiple ground candidate areas can be more accurately divided. The ground flat areas can be accurately screened according to the boundary areas, height differences, length distances, and / or width distances of each ground candidate area. According to the obstacle information and the intersection situation of the extending direction of each landing slide, more suitable candidate landing areas for the unmanned device to land can be screened to ensure the landing safety of the unmanned device. And analyze the obstacle height around the candidate landing area, determine multiple landing slides that meet the gliding and landing requirements of the unmanned device in the candidate landing area. Subsequently, the unmanned device can determine the optimal landing slide and the target landing area based on the weighted fusion calculation results of the flatness, slide length, slide width, obstacle height of different landing slides, and the third distance between each candidate landing area and the unmanned device, and the landing location of the unmanned device is determined more reasonably.

[0103] Based on the above embodiments, Figure 7 The flowchart of another unmanned device landing processing method provided by the embodiment of the present application is given. This unmanned device landing processing method is a concretization of the above unmanned device landing processing method. Refer to Figure 7 and this unmanned device landing processing method includes:

[0104] S401: Determine the ground information and obstacle information of the flight area based on the point cloud data within the flight area.

[0105] S402: Determine one or more flat ground areas within the flight area according to the ground information.

[0106] S403: Determine candidate landing areas from one or more flat ground areas according to the obstacle information.

[0107] S404: Calculate the landing safety score for each candidate landing area based on a combination of one or more of the safety factor corresponding to the landing method of the unmanned device, the device condition, the landing reason, the fourth distance between the candidate landing area and the unmanned device, and the area information of each candidate landing area.

[0108] S405: Determine the target landing area from the candidate landing areas based on the landing safety score.

[0109] In this embodiment, a vertical fixed-wing unmanned device is taken as an example of the unmanned device for description. Among them, the candidate landing areas include a hovering landing area determined based on the hovering landing method (the candidate landing area determined by referring to steps S201 - S206 in the above embodiment), and a gliding landing area determined based on the gliding landing method (the candidate landing area determined by referring to steps S301 - S306 in the above embodiment), and the safety factor of the gliding landing method is higher than that of the hovering landing method.

[0110] Exemplarily, when the unmanned device needs to perform a landing operation, determine the current device condition and landing reason of the unmanned device, determine the safety factor corresponding to the current landing reason, as well as the fourth distance between the candidate landing area and the unmanned device and the area information of each candidate landing area.

[0111] Perform weighted fusion calculation on a combination of one or more of the safety factor, device condition, landing reason, fourth distance, and area information to obtain the landing safety score for each candidate landing area. Further, determine the candidate landing area with the highest landing safety score as the target landing area.

[0112] As described above, by determining the ground information and obstacle information based on the point cloud data in the flight area, determining one or more ground flat areas in the flight area according to the ground information, determining candidate landing areas from the one or more ground flat areas according to the obstacle information, and determining the target landing area from the candidate landing areas, it reduces the situation that the safety of the selected candidate landing area is relatively low due to the complex on-site environment of the flight area, resulting in accidents during the landing of the unmanned device, and ensures the safety of the device and personnel when the unmanned device lands. By calculating the landing safety scores of each candidate landing area in combination with multiple parameters such as the device situation of the unmanned device, the reason for landing, the distance from the candidate landing area, and the area of each candidate landing area, and flexibly selecting a suitable target landing area for landing based on the landing safety scores, it assists the safe landing of the unmanned device, ensures the safety of relevant personnel, and reduces the situation of collisions during the landing of the unmanned device.

[0113] Figure 8 FIG. shows a schematic structural diagram of a landing processing device for an unmanned device provided by an embodiment of the present application. Refer to Figure 8 and the landing processing device for the unmanned device includes a flight area analysis module 81, a flat area analysis module 82, a candidate area determination module 83, and a target area determination module 84.

[0114] Among them, the flight area analysis module 81 is configured to determine the ground information and obstacle information of the flight area based on the point cloud data in the flight area; the flat area analysis module 82 is configured to determine one or more ground flat areas in the flight area according to the ground information; the candidate area determination module 83 is configured to determine candidate landing areas from the one or more ground flat areas according to the obstacle information; the target area determination module 84 is configured to determine the target landing area from the candidate landing areas.

[0115] As described above, by determining the ground information and obstacle information based on the point cloud data in the flight area, determining one or more ground flat areas in the flight area according to the ground information, determining candidate landing areas from the one or more ground flat areas according to the obstacle information, and determining the target landing area from the candidate landing areas, it reduces the situation that the safety of the selected candidate landing area is relatively low due to the complex on-site environment of the flight area, resulting in accidents during the landing of the unmanned device, and ensures the safety of the device and personnel when the unmanned device lands.

[0116] Based on the above embodiment, the flight area analysis module 81 is specifically configured to:

[0117] Obtain the point cloud data and digital model in the flight area;

[0118] Based on point cloud segmentation technology and image segmentation technology, ground information and obstacle information within the flight area are determined according to point cloud data and digital models.

[0119] Based on the above embodiments, the flat area analysis module 82 is specifically configured to:

[0120] By performing two-dimensional grid division on the ground information, one or more flat candidate areas are divided in the ground information;

[0121] Perform adjacent area search on each flat candidate area, and fuse the flat candidate areas according to the adjacent area search results to obtain one or more ground flat areas within the flight area.

[0122] Based on the above embodiments, when the flat area analysis module 82 divides one or more flat candidate areas in the ground information by performing two-dimensional grid division on the ground information, it includes:

[0123] By performing two-dimensional grid division on the ground information, one or more flat area blocks are divided in the ground information;

[0124] According to the flat parameters of the flat area blocks, one or more flat candidate areas are selected from the flat area blocks, and the flat parameters include one or more of height difference, average height, and height variance.

[0125] Based on the above embodiments, the flight area analysis module 81 is specifically configured to:

[0126] Obtain the point cloud data within the flight area;

[0127] Based on the semantic segmentation algorithm, determine the ground information and obstacle information from the point cloud data.

[0128] Based on the above embodiments, the flat area analysis module 82 is specifically configured to:

[0129] Divide the ground information into multiple ground candidate areas;

[0130] According to the height difference, length distance, and / or width distance of the ground candidate areas, determine one or more ground flat areas within the flight area from the multiple ground candidate areas.

[0131] Based on the above embodiments, the candidate area determination module 83 is specifically configured to:

[0132] According to the hovering landing route of the unmanned device, determine the landing route fitting space in each ground flat area;

[0133] According to the intersection situation between the obstacle information and the landing route fitting space, determine the candidate landing areas from one or more ground flat areas.

[0134] Based on the above embodiments, the target area determination module 84 is specifically configured to:

[0135] When the unmanned device lands in the first flight state, determine the first distance between each candidate landing area and the unmanned device, the second distance between each candidate landing area and the operation terminal and / or the operator, and the area information of each candidate landing area, and determine the target landing area from the candidate landing areas according to the first distance, the second distance, and the area information; and / or

[0136] When the unmanned device lands in the second flight state, determine the first distance between each candidate landing area and the unmanned device, and determine the target landing area from the candidate landing areas according to the first distance.

[0137] Based on the above embodiments, the candidate area determination module 83 is specifically configured to:

[0138] Determine one or more landing slides in each flat ground area according to the slide distance requirement of the unmanned device;

[0139] Determine the candidate landing areas from one or more flat ground areas according to the obstacle information in the extending direction of each landing slide.

[0140] Based on the above embodiments, when the candidate area determination module 83 determines one or more landing slides in each flat ground area according to the slide distance requirement of the unmanned device, it includes:

[0141] Determine candidate slides corresponding to multiple angles in each flat ground area, and determine one or more landing slides from the multiple candidate slides according to the slide distance requirement of the unmanned device.

[0142] Based on the above embodiments, the target area determination module 84 is specifically configured to:

[0143] Perform weighted fusion calculation based on one or more combinations of the flatness, slide length, slide width, obstacle height of the candidate landing area, and the third distance between each candidate landing area and the unmanned device;

[0144] Determine the target landing area from the candidate landing areas according to the weighted fusion calculation result.

[0145] Based on the above embodiments, the unmanned device is a vertical fixed-wing unmanned device, the candidate landing areas include a spiral landing area determined based on the spiral landing method and a gliding landing area determined based on the gliding landing method, and the safety factor of the gliding landing method is higher than that of the spiral landing method;

[0146] The target area determination module 84 is specifically configured to:

[0147] Calculate the landing safety score for each candidate landing area based on one or a combination of the safety factor corresponding to the landing method of the unmanned device, the device condition, the reason for landing, the fourth distance between the candidate landing area and the unmanned device, and the area information of each candidate landing area;

[0148] Determine the target landing area from the candidate landing areas based on the landing safety score.

[0149] It should be noted that in the embodiments of the above-mentioned unmanned device landing processing device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the embodiments of the present invention.

[0150] The embodiments of the present application also provide an unmanned device landing processing device, and this unmanned device landing processing device can integrate the unmanned device landing processing device provided by the embodiments of the present application. Figure 9 It is a schematic structural diagram of an unmanned device landing processing device provided by the embodiments of the present application. Refer to Figure 9 , this unmanned device landing processing device includes: an input device 93, an output device 94, a memory 92, and one or more processors 91; the memory 92 is used to store one or more programs; when the one or more programs are executed by the one or more processors 91, the one or more processors 91 implement the unmanned device landing processing method provided by the above embodiments. Among them, the input device 93, the output device 94, the memory 92, and the processor 91 can be connected through a bus or other means, Figure 9 taking the connection through the bus as an example.

[0151] The memory 92, being a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the unmanned device landing processing method provided in any embodiment of the present application (for example, the flight area analysis module 81, flat area analysis module 82, candidate area determination module 83, and target area determination module 84 in the unmanned device landing processing device). The memory 92 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the device, etc. In addition, the memory 92 can include high-speed random access memory and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 92 can further include a memory remotely set relative to the processor 91, and these remote memories can be connected to the device through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and their combinations.

[0152] The input device 93 can be used to receive input digital or character information and generate key signal inputs related to the user settings and function controls of the device. The output device 94 can include display devices such as a display screen.

[0153] The processor 91 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 92, that is, implements the above-mentioned unmanned device landing processing method.

[0154] The above-provided unmanned device landing processing device, device, and computer can be used to execute the unmanned device landing processing method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0155] The embodiment of the present application also provides a storage medium storing computer-executable instructions. When the computer-executable instructions are executed by a computer processor, they are used to execute the unmanned device landing processing method provided in the above embodiment. The unmanned device landing processing method includes: determining the ground information and obstacle information of the flight area based on the point cloud data in the flight area; determining one or more ground flat areas in the flight area according to the ground information; determining candidate landing areas from the one or more ground flat areas according to the obstacle information; and determining a target landing area from the candidate landing areas.

[0156] Storage Medium - Any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media such as CD-ROMs, floppy disks or tape drives; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory such as flash memory, magnetic media (such as hard disks or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. Additionally, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system that is connected to the first computer system via a network (such as the Internet). The second computer system may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (e.g., in different computer systems connected via a network). The storage medium may store program instructions executable by one or more processors (e.g., embodied as a computer program).

[0157] Of course, for a storage medium provided in an embodiment of the present application that stores computer-executable instructions, the computer-executable instructions are not limited to the unmanned device landing processing method provided above, and may also execute related operations in the unmanned device landing processing method provided in any embodiment of the present application.

[0158] The unmanned device landing processing apparatus, device, and storage medium provided in the above embodiments can execute the unmanned device landing processing method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, reference may be made to the unmanned device landing processing method provided in any embodiment of the present application.

[0159] The above is only the preferred embodiment of the present application and the technical principles applied. The present application is not limited to the specific embodiments provided here. Various obvious changes, re-adjustments, and substitutions that can be made by those skilled in the art will not depart from the protection scope of the present application. Therefore, although the present application has been described in relatively detail through the above embodiments, the present application is not limited to the above embodiments. Without departing from the concept of the present application, it may also include more other equivalent embodiments, and the scope of the present application is determined by the scope of the claims.

Claims

1. A method for handling the landing of an unmanned device, characterized in that, Including: Determine the ground information and obstacle information of the flight area based on the point cloud data within the flight area; Determine one or more ground flat areas within the flight area according to the ground information; Determine a candidate landing area from one or more of the ground flat areas according to the obstacle information. The determining of the candidate landing area from one or more of the ground flat areas according to the obstacle information includes: determining one or more landing slides in each of the ground flat areas according to the slide distance requirement of the unmanned device; determining the candidate landing area from one or more of the ground flat areas according to the obstacle information in the extending direction of each of the landing slides; Determine the target landing area from the candidate landing areas.

2. The method for handling the landing of an unmanned device according to claim 1, wherein The determining of the ground information and obstacle information of the flight area based on the point cloud data within the flight area includes: Obtain the point cloud data and digital model within the flight area; Based on the point cloud segmentation technology and image segmentation technology, determine the ground information and obstacle information within the flight area according to the point cloud data and the digital model.

3. The method for handling the landing of an unmanned device according to claim 2, wherein, The determining of one or more ground flat areas within the flight area according to the ground information includes: By performing two-dimensional grid division on the ground information, divide one or more flat candidate areas in the ground information; Perform adjacent area search on each of the flat candidate areas, and fuse the flat candidate areas according to the adjacent area search results to obtain one or more ground flat areas within the flight area.

4. The method for processing the landing of an unmanned device according to claim 3, wherein, The performing of two-dimensional grid division on the ground information to divide one or more flat candidate areas in the ground information includes: By performing two-dimensional grid division on the ground information, divide one or more flat area blocks in the ground information; According to the flat parameters of the flat area blocks, screen out one or more flat candidate areas from the flat area blocks. The flat parameters include one or more of the height difference, average height, and height variance.

5. The method for processing the landing of an unmanned device according to claim 1, wherein The determining of the ground information and obstacle information of the flight area based on the point cloud data within the flight area includes: Obtain the point cloud data within the flight area; Determine the ground information and obstacle information from the point cloud data based on the semantic segmentation algorithm.

6. The method for handling the landing of an unmanned device according to claim 5, wherein, The determining of one or more ground flat areas within the flight area according to the ground information includes: Divide the ground information into multiple ground candidate areas; Determine one or more ground flat areas within the flight area from the multiple ground candidate areas according to the height difference, length distance, and / or width distance of the ground candidate areas.

7. The method for handling the landing of an unmanned device according to claim 1, wherein The determining of the candidate landing area from one or more of the ground flat areas according to the obstacle information further includes: Determine the landing route fitting space in each of the ground flat areas according to the hovering landing route of the unmanned device; Determine the candidate landing area from one or more of the ground flat areas according to the intersection situation between the obstacle information and the landing route fitting space.

8. The method for landing processing of the unmanned device according to claim 7, wherein Determining a target landing area from the candidate landing areas includes: When the unmanned device lands in the first flight state, determining the first distance between each of the candidate landing areas and the unmanned device, the second distance between each of the candidate landing areas and the operation terminal and / or the operator, and the area information of each of the candidate landing areas, and determining the target landing area from the candidate landing areas according to the first distance, the second distance, and the area information; and / or When the unmanned device lands in the second flight state, determining the first distance between each of the candidate landing areas and the unmanned device, and determining the target landing area from the candidate landing areas according to the first distance.

9. The method for handling the landing of an unmanned device according to claim 1, wherein Determining one or more landing runways in each of the flat ground areas according to the runway distance requirement of the unmanned device includes: Determining candidate runways corresponding to multiple angles in each of the flat ground areas, and determining one or more landing runways from the multiple candidate runways according to the runway distance requirement of the unmanned device.

10. The method for handling the landing of an unmanned device according to claim 1, wherein Determining a target landing area from the candidate landing areas includes: Performing weighted fusion calculation based on a combination of one or more of the flatness, runway length, runway width, obstacle height of the candidate landing area, and the third distance between each of the candidate landing areas and the unmanned device; Determining the target landing area from the candidate landing areas according to the weighted fusion calculation result.

11. The method for handling the landing of an unmanned device according to any one of claims 1-10, characterized in that, The unmanned device is a vertical fixed-wing unmanned device, the candidate landing areas include a hovering landing area determined based on the hovering landing method and a gliding landing area determined based on the gliding landing method, and the safety factor of the gliding landing method is higher than that of the hovering landing method; Determining a target landing area from the candidate landing areas includes: Calculating the landing safety score of each candidate landing area based on a combination of one or more of the safety factor corresponding to the landing method of the unmanned device, the device condition, the landing reason, the fourth distance between the candidate landing area and the unmanned device, and the area information of each candidate landing area; Determining the target landing area from the candidate landing areas based on the landing safety score.

12. A landing processing device for an unmanned device, characterized in that, Including a flight area analysis module, a flat area analysis module, a candidate area determination module, and a target area determination module, wherein: The flight area analysis module is configured to determine the ground information and obstacle information of the flight area based on the point cloud data in the flight area; The flat area analysis module is configured to determine one or more flat ground areas in the flight area according to the ground information; The candidate area determination module is configured to determine a candidate landing area from one or more of the ground flat areas according to the obstacle information. The determining of the candidate landing area from one or more of the ground flat areas according to the obstacle information includes: determining one or more landing slides in each of the ground flat areas according to the slide distance requirement of the unmanned device; and determining a candidate landing area from one or more of the ground flat areas according to the obstacle information in the extending direction of each of the landing slides. The target area determination module is configured to determine a target landing area from the candidate landing areas.

13. A landing processing device for an unmanned device, characterized in that Comprising: A memory and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the unmanned device landing processing method according to any one of claims 1-11.

14. A storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are used to execute the unmanned device landing processing method according to any one of claims 1-11 when executed by a computer processor.

Images