Dynamic and static combined unmanned aerial vehicle safe landing area sensing method and system and medium

Abstract

The invention discloses a dynamic and static combined unmanned aerial vehicle safe landing area sensing method and system and a medium. The method comprises the steps of performing semantic segmentation on static map data of a current position of an unmanned aerial vehicle to obtain a plurality of ground objects; for each potential landing area in the area range corresponding to the static map data, calculating a static risk value according to each ground object contained in the potential landing area; determining a target landing area according to the static risk value of each potential landing area; generating a safety value distribution image through the dynamic visual data of the target landing area and a preset Bayesian segmentation network; the safety value distribution image is used for reflecting safety values of different position points of the target landing area; and determining a safe landing position of the unmanned aerial vehicle according to the safety value distribution image. According to the invention, the static map data and the dynamic visual data are combined to identify the safe landing area, and the accurate sensing and safe landing of the unmanned aerial vehicle in a high-speed rotation state can be effectively ensured through a mode of fusing the static data and the dynamic data.

Description

Technical Field

[0001] The present invention relates to the field of drone sensing technology, and in particular to a dynamic and static combined drone safe landing zone sensing method, system and medium. Background Art

[0002] With the development of low-altitude economy, drones are widely used in various fields. Current drone landing zone recognition technologies are mainly designed for the operation of drones in normal flight state. These technologies usually assume that the flight attitude of drones is stable and regular flight path planning can be performed.

[0003] However, in actual applications, drones may rotate at high speeds due to technical failures or external factors. Existing drone landing zone recognition technology fails to take into account the high-speed rotation of drones. Since the orientation and stability of the visual system are affected when the drone rotates at high speed, conventional image recognition and data processing technologies are difficult to accurately process rapidly changing visual inputs, resulting in inaccurate or delayed recognition results.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0005] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a dynamic and static combined drone safe landing area perception method, system and medium are provided, aiming to solve the problem that the existing drone landing area recognition technology is difficult to provide accurate recognition results when the drone is in high-speed rotation or other abnormal flight states.

[0006] The technical solution adopted by the present invention to solve the problem is as follows:

[0007] In a first aspect, an embodiment of the present invention provides a method for sensing a safe landing area of ​​a drone by combining dynamic and static methods, the method comprising:

[0008] Obtain static map data corresponding to the current position of the drone, perform semantic segmentation on the static map data, and obtain a number of ground objects of different types;

[0009] For each potential landing area within the area corresponding to the static map data, a static risk value is calculated according to each ground object contained in the potential landing area; and a target landing area is determined according to the static risk value of each potential landing area;

[0010] Acquire dynamic visual data corresponding to the target landing area, and generate a safety value distribution image corresponding to the target landing area according to the dynamic visual data and a preset Bayesian segmentation network; wherein the safety value distribution image is used to reflect the safety values ​​corresponding to different position points of the target landing area;

[0011] A safe landing position of the UAV is determined according to the safety value distribution image.

[0012] In one embodiment, the step of calculating the static risk value according to each ground object included in the potential landing area includes:

[0013] For each ground object included in the potential landing area, obtain the number of pixel points corresponding to each ground object in the static map data, and determine the weight value corresponding to each ground object according to the number of pixel points;

[0014] The preset risk degrees corresponding to each ground object are obtained, and weighted summation is performed according to the preset risk degrees of each ground object and the weight values ​​to obtain the static risk value of the potential landing area.

[0015] In one embodiment, the dynamic visual data includes laser radar point cloud data, and the acquiring of the dynamic visual data corresponding to the target landing area and the generating of a safety value distribution image corresponding to the target landing area according to the dynamic visual data and a preset Bayesian segmentation network include:

[0016] Acquire original laser radar point cloud data of the target landing area, perform noise reduction and downsampling on the original laser radar point cloud data, and obtain the laser radar point cloud data;

[0017] The laser radar point cloud data is input into a preset Bayesian segmentation network, the safety value of each point cloud is calculated through the Bayesian segmentation network, and the safety value distribution image is generated according to the safety value of each point cloud.

[0018] In one implementation, generating the security value distribution image according to the security value of each point cloud includes:

[0019] According to the safety value of each point cloud in the laser radar point cloud data, the safety value of each point cloud deleted after downsampling is calculated by inverse distance weighted interpolation;

[0020] The security values ​​of all point clouds are binarized, and the security value distribution image is generated according to the security values ​​of all point clouds after the binarization.

[0021] In one embodiment, the method further comprises:

[0022] Divide the laser radar point cloud data into a triangular mesh, and for each triangle in the triangular mesh, calculate the predicted elevation value of the centroid of the triangle according to the weighted sum of the elevation values ​​corresponding to the three endpoints of the triangle; wherein the weights corresponding to the three endpoints are determined based on the corresponding centroid coordinates, and the sum of the centroid coordinates of the three endpoints is 1;

[0023] The marking information of the predicted elevation value of each center of gravity point is added to the safety value distribution image.

[0024] In one embodiment, the method further comprises:

[0025] Obtaining a global surface normal of the laser radar point cloud data, and determining a normal vector corresponding to each point cloud according to the global surface normal;

[0026] For each point cloud, a neighborhood of the point cloud is determined with the point cloud as the center and a preset value as the radius, and the surface roughness corresponding to the point cloud is determined according to the difference between the normal vectors of the point cloud and other point clouds in the neighborhood; the slope corresponding to the point cloud is determined according to the Z-axis component of the normal vector of the point cloud and the normal vector;

[0027] The marking information of the surface roughness and the slope corresponding to each point cloud is added to the safety value distribution image.

[0028] In one embodiment, the method further comprises:

[0029] When the drone lands, the new posture of the drone after the preset time increment is obtained, and interpolation processing is performed according to the new posture at each time point to obtain the interpolated posture between each time point;

[0030] Correcting the laser radar point cloud data according to the interpolation posture to obtain corrected point cloud data;

[0031] A new safety value distribution image is generated according to the corrected point cloud data, and the safe landing position is adjusted according to the new safety value distribution image.

[0032] In one embodiment, the method further comprises:

[0033] The positions of the UAV in the x, y, and z directions at different time points are calculated based on the safe landing position through a polynomial curve equation to obtain a safe landing trajectory of the UAV.

[0034] In a second aspect, an embodiment of the present invention further provides a dynamic and static combined drone safe landing zone perception system, the system comprising:

[0035] A static analysis module is used to obtain static map data corresponding to the current position of the UAV, and perform semantic segmentation on the static map data to obtain a number of ground objects of different types;

[0036] For each potential landing area within the area corresponding to the static map data, a static risk value is calculated according to each ground object contained in the potential landing area; and a target landing area is determined according to the static risk value of each potential landing area;

[0037] A dynamic analysis module, used to obtain dynamic visual data corresponding to the target landing area, and generate a safety value distribution image corresponding to the target landing area according to the dynamic visual data and a preset Bayesian segmentation network; wherein the safety value distribution image is used to reflect the safety values ​​corresponding to different position points of the target landing area;

[0038] The static-dynamic combination module is used to determine the safe landing position of the UAV according to the safety value distribution image.

[0039] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium having a plurality of instructions stored thereon, wherein the instructions are suitable for being loaded and executed by a processor to implement the steps of the dynamic and static combined drone safe landing zone perception method as described in any of the above.

[0040] Beneficial effects of the present invention: The embodiment of the present invention combines static map data and dynamic visual data to identify a safe landing area. The fusion of static data and dynamic data can effectively ensure that the drone can accurately perceive and land safely even in a high-speed rotating state. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 It is a flow chart of a dynamic and static combined UAV safe landing zone perception method provided by an embodiment of the present invention.

[0043] Figure 2 It is a schematic diagram of a process for safely landing a dynamically rotating UAV provided by an embodiment of the present invention.

[0044] Figure 3 It is a module schematic diagram of a dynamic and static combined drone safe landing area perception system provided in an embodiment of the present invention.

[0045] Figure 4 It is a principle block diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The present invention discloses a method, system and medium for sensing a safe landing zone of a drone in combination with dynamic and static conditions. In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0048] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as herein.

[0049] In view of the above-mentioned defects of the prior art, the present invention provides a method for sensing a safe landing area of ​​a drone that combines static and dynamic aspects. The method obtains static map data corresponding to the current position of the drone, performs semantic segmentation on the static map data, and obtains several ground objects of different types; for each potential landing area within the area corresponding to the static map data, a static risk value is calculated according to each ground object contained in the potential landing area; a target landing area is determined according to the static risk value of each potential landing area; dynamic visual data corresponding to the target landing area is obtained, and a safety value distribution image corresponding to the target landing area is generated according to the dynamic visual data and a preset Bayesian segmentation network; wherein the safety value distribution image is used to reflect the safety values ​​corresponding to different position points of the target landing area; and the safe landing position of the drone is determined according to the safety value distribution image. The present invention combines static map data and dynamic visual data to identify a safe landing area, and the fusion of static data and dynamic data can effectively ensure that the drone can accurately sense and land safely even in a high-speed rotating state.

[0050] like Figure 1 As shown, the method specifically comprises the following steps:

[0051] Step S100: Obtain static map data corresponding to the current position of the drone, perform semantic segmentation on the static map data, and obtain a number of ground objects of different types.

[0052] Specifically, this embodiment first needs to obtain large-scale static map data (i.e., GIS static images) through the current position of the drone in combination with the geographic information system (GIS) to perform preliminary perception and identification of potential landing areas. In order to better perceive the information contained in the static map data, this embodiment will perform semantic segmentation on the static map data to segment different types of ground objects.

[0053] For example, the Fast-SCNN model is used to perform semantic segmentation on static map data to segment different types of ground objects, such as ground, road, roof, water surface, and tree.

[0054] The semantic segmentation process is as follows:

[0055] Define the pixel matrix F of the static map data, mapping each pixel in the image to a real value:

[0056] F: {1,…,H}×{1,…,W}→R;

[0057] Where F represents the pixel matrix of static map data; H represents the image height (number of pixels); W represents the image width (number of pixels);

[0058] Define a set of predefined categories C for the semantic segmentation model, where each category represents a ground object type:

[0059] C={c1,c2,…,c k};

[0060] Where C represents the predefined category set of the semantic segmentation model;

[0061] Through the semantic segmentation model S, each pixel is mapped to the corresponding category to generate a category labeling matrix:

[0062] S: {1,…,H}×{1,…,W}→C;

[0063] Where S represents the category labeling matrix of the image after being processed by the semantic segmentation model;

[0064] The set of all pixels belonging to a class k in the class set C is represented as P k :

[0065] P k ={)i,j)∈{1,…,H}×{1,…,W}|P k (i, j) = c k};

[0066] In the formula, K represents the total number of categories; P k Represents the set of all pixels belonging to a certain class k in the class set C.

[0067] Step S200: for each potential landing area within the area corresponding to the static map data, a static risk value is calculated according to each ground object contained in the potential landing area; and a target landing area is determined according to the static risk value of each potential landing area.

[0068] Specifically, since the area corresponding to the static map data is relatively large, there may be multiple potential landing areas for the drone. In order to select the best potential landing area for more refined analysis, this embodiment needs to comprehensively determine the static risk value of each potential landing area based on the number and type of ground objects contained therein, the area size corresponding to each ground object, and other factors. The high or low static risk value can indirectly reflect the landing safety of the area. Then, the best target landing area is selected from the static risk values ​​of each potential landing area, thereby completing the preliminary perception and judgment of the potential landing area of ​​the drone through the static map data.

[0069] In one implementation, calculating the static risk value according to each ground object included in the potential landing area includes:

[0070] For each ground object included in the potential landing area, obtain the number of pixel points corresponding to each ground object in the static map data, and determine the weight value corresponding to each ground object according to the number of pixel points;

[0071] The preset risk degrees corresponding to each ground object are obtained, and weighted summation is performed according to the preset risk degrees of each ground object and the weight values ​​to obtain the static risk value of the potential landing area.

[0072] Specifically, this embodiment needs to calculate the static risk value of the potential landing area of ​​the drone (i.e., the ground of the achievable flight zone) through the semantic segmentation results. This embodiment pre-sets the risk levels corresponding to different types of ground objects. In actual application, taking a potential landing area as an example, the potential landing area includes several types of ground objects. For each ground object, the weighted risk of the ground object is calculated based on the preset risk level of the ground object and the corresponding number of pixels in the static map data. The weighted risks of all ground objects are added together to obtain the static risk value of the potential landing area.

[0073] For example, calculate the static risk value OL of the potential landing area s (X), which is determined by the risk value τ of each category k Weighted average.

[0074]

[0075] In the formula, OL s (X) represents the static risk value of the Xth potential landing area; τ k represents the risk value of the kth type of object on the ground; p k (i, j) indicates that the pixel point (i, j) belongs to a category in the category set C; F (i, j) indicates the pixel point in the pixel matrix corresponding to the static map data.

[0076] Define the risk value of each category of ground objects, such as vegetation, land, buildings, roads and water bodies:

[0077] {τ0, τ1, τ2, τ3, τ4}={0, 1, 2, 5, 10}.

[0078] Step S300, obtaining dynamic visual data corresponding to the target landing area, and generating a safety value distribution image corresponding to the target landing area based on the dynamic visual data and a preset Bayesian segmentation network; wherein the safety value distribution image is used to reflect the safety values ​​corresponding to different position points of the target landing area.

[0079] Specifically, in order to more accurately perceive and identify the safe landing zone, this embodiment will further obtain dynamic visual data of the target landing zone, such as small-scale ground point cloud data and / or video data collected by the laser radar. The dynamic visual data is analyzed by a Bayesian segmentation network pre-constructed based on a computer vision neural network algorithm, and a safety value distribution image of the target landing zone is output based on the analysis results. The safety value distribution image can be used to obtain safety information at different locations in the target landing zone, thereby facilitating accurate selection of a safe landing location.

[0080] For example, the calculation process of using a Bayesian segmentation network to perceive the ground safety of a landing zone based on dynamic visual data is as follows:

[0081] For each data point, the predicted mean of the point in the Monte Carlo samples is calculated to evaluate the safety of the point. The following takes 50 Monte Carlo samples as an example. In actual use, you can choose other numbers of Monte Carlo samples:

[0082]

[0083] In the formula, μ(p Li ) represents point p Li The predicted mean over 50 Monte Carlo samples;

[0084] Calculate the prediction variance of the point in 50 Monte Carlo samples to assess the uncertainty of the prediction results:

[0085]

[0086] In the formula, σ 2 (p Li ) represents point p Li The prediction variance in 50 Monte Carlo samples; y k (p Li ) represents the midpoint p of the kth Monte Carlo sampling Li The predicted value of

[0087] Normalize the standard deviation of the point for comparison on a relative scale:

[0088]

[0089] In the formula, σ norm (p Li ) represents point p Li The standard deviation normalized result;

[0090] Binarize the standard deviation of the point and determine whether its uncertainty is below the threshold:

[0091]

[0092] σ binary (p Li ) represents point p Li The binarization result of

[0093] Combine the binarization results of the mean and standard deviation to determine the safety value of the point for safety indication:

[0094] S(p Li )=σ binary (p Li )∧(μ(p Li )>0.5);

[0095] In the formula, S(p Li ) represents point p Li The binary security value.

[0096] In one implementation, the dynamic visual data includes laser radar point cloud data, and the acquiring of the dynamic visual data corresponding to the target landing area, and the generating of a safety value distribution image corresponding to the target landing area according to the dynamic visual data and a preset Bayesian segmentation network, include:

[0097] Acquire original laser radar point cloud data of the target landing area, perform noise reduction and downsampling on the original laser radar point cloud data, and obtain the laser radar point cloud data;

[0098] The laser radar point cloud data is input into a preset Bayesian segmentation network, the safety value of each point cloud is calculated through the Bayesian segmentation network, and the safety value distribution image is generated according to the safety value of each point cloud.

[0099] Specifically, in order to save computational overhead, after the raw data is collected by the laser radar in this embodiment, it is necessary to perform noise reduction and downsampling preprocessing. For example, the raw laser radar point cloud data can be preprocessed using methods such as Kalman filtering and voxel downsampling, wherein Kalman filtering is used to reduce noise, and voxel downsampling reduces the amount of point cloud data and improves processing efficiency. The preprocessed point cloud data is then input into the Bayesian segmentation network, and the safety value distribution image of the target landing area is calculated by the Bayesian segmentation network.

[0100] In one implementation, generating the security value distribution image according to the security value of each point cloud includes:

[0101] According to the safety value of each point cloud in the laser radar point cloud data, the safety value of each point cloud deleted after downsampling is calculated by inverse distance weighted interpolation;

[0102] The security values ​​of all point clouds are binarized, and the security value distribution image is generated according to the security values ​​of all point clouds after the binarization.

[0103] Specifically, since some point clouds are excluded by downsampling during preprocessing, there may be some point clouds in the original LiDAR point cloud data that do not have corresponding safety values. This embodiment will calculate the safety value of such point clouds by inverse distance weighted interpolation, binarize the safety value, and finally summarize the safety values ​​of all point clouds to generate a complete safety value distribution image.

[0104] For example, for each position without a safety value (i.e., the point cloud deleted after downsampling), the safety value of the position is calculated by inverse distance weighted interpolation to integrate the safety information of the surrounding points:

[0105]

[0106] Where S(x, y) represents the safety value of the position (x, y) on the plane; d(x, y, p Li ) represents the position (x, y) and the point p Li The distance between them, p represents the power parameter of the interpolation;

[0107] Binarize the safety value of the position (x, y) to determine whether it is safe:

[0108]

[0109] In the formula, S plane (x, y) represents a binary safety indication of the position (x, y) on the plane.

[0110] In one implementation, the method further includes:

[0111] Divide the laser radar point cloud data into a triangular mesh, and for each triangle in the triangular mesh, calculate the predicted elevation value of the centroid of the triangle according to the weighted sum of the elevation values ​​corresponding to the three endpoints of the triangle; wherein the weights corresponding to the three endpoints are determined based on the corresponding centroid coordinates, and the sum of the centroid coordinates of the three endpoints is 1;

[0112] The marking information of the predicted elevation value of each center of gravity point is added to the safety value distribution image.

[0113] Specifically, in order to enrich the information contained in the safety value distribution image and improve the accuracy of identifying the safe landing position, the present embodiment can also mark the elevation values ​​of some key points in the safety value distribution image. In practical applications, the laser radar point cloud data can be divided into triangular meshes by triangulation, and the centroid of the triangle is used as the key point, and the elevation value of the centroid of the triangle is jointly predicted by the elevation values ​​of the three vertices of the triangle.

[0114] For example, the terrain surface reconstruction of this embodiment can be performed by linear interpolation reconstruction of the ground point cloud data (i.e., the laser radar point cloud data) through the Delaunay triangulation method, so as to obtain the elevation values ​​of some key points. These key points correspond to the centroid points of the triangles, and are defined as the points to be estimated. The calculation formula for the predicted elevation value of the points to be estimated is:

[0115] z p =λ A z A +λ B z B +λ C z C ;

[0116] In the formula, z p Indicates the elevation value of the point to be estimated; A , B , C Represents the centroid coordinates of the point relative to the triangle vertices A, B, and C; z A 、z B 、z C : The elevation values ​​of triangle vertices A, B, and C;

[0117] Make sure the sum of the interpolated barycentric coordinates is 1 to ensure the accuracy of the interpolation result:

[0118] λ A +λ B +λ C =1.

[0119] In one implementation, the method further includes:

[0120] Obtaining a global surface normal of the laser radar point cloud data, and determining a normal vector corresponding to each point cloud according to the global surface normal;

[0121] For each point cloud, a neighborhood of the point cloud is determined with the point cloud as the center and a preset value as the radius, and the surface roughness corresponding to the point cloud is determined according to the difference between the normal vectors of the point cloud and other point clouds in the neighborhood; the slope corresponding to the point cloud is determined according to the Z-axis component of the normal vector of the point cloud and the normal vector;

[0122] The marking information of the surface roughness and the slope corresponding to each point cloud is added to the safety value distribution image.

[0123] Specifically, this embodiment will analyze the roughness of the surface, terrain changes and other dynamic obstacle information through the small-scale ground point cloud data collected from the lidar in real time, and conduct a more detailed safety assessment of the landing area. In practical applications, this embodiment can extract the global surface normal of the landing area terrain through the multi-scale core estimation method, and determine the normal vector of each point cloud based on the global surface normal. Taking a point cloud as an example, the neighborhood of the point cloud is determined by a preset radius, and the surface roughness corresponding to the point cloud is calculated by the difference between the normal vectors of the point cloud and other point clouds in the neighborhood, and the slope corresponding to the point cloud is calculated by the component of the normal vector of the point cloud on the Z axis.

[0124] For example, calculate the surface roughness Rou(p Li ), which is used to assess the smoothness of the terrain surface.

[0125]

[0126] In the formula, Rou(p Li ) represents the surface roughness of the i-th point; N r (p Li ) represents point p Li The point set in the neighborhood with radius r; n(p Li )、n(p Lj ) represents point p Li 、p Lj The normal vector of ; ||·|| represents the norm of the vector;

[0127] Calculate the slope of the i-th point to evaluate the slope of the terrain.

[0128]

[0129] In the formula, θ(p Li ) represents the slope of the i-th point; n z (p Li ) represents the component of the normal vector in the z-axis direction.

[0130] In one implementation, the aforementioned processing algorithms related to the LiDAR point cloud data can be integrated into an algorithm processing framework, which is specially designed to stably and quickly process the point cloud data collected by the LiDAR when the UAV is rotating at high speed, ensuring that potential safe landing areas can be effectively identified even under extreme conditions. The present invention accurately evaluates the dynamic environment of the landing area through point cloud data preprocessing, terrain surface reconstruction, and Bayesian segmentation network, thereby improving the perception accuracy and decision-making ability of the UAV in complex environments. Secondly, the present invention not only improves the safety of the UAV under special working conditions, but also significantly improves the reliability and efficiency of operations, providing strong technical support for the widespread use of UAVs in various complex environments.

[0131] Step S400: Determine a safe landing position for the UAV according to the safety value distribution image.

[0132] Solutions that rely solely on static maps lack flexibility in the face of dynamic environmental changes and cannot update map information in real time to reflect new obstacles or changes in the environment. Solutions that rely solely on real-time visual data may be affected by changes in lighting, weather conditions, and environmental interference (such as rain, fog, dust, etc.), further exacerbating the difficulty of identification in abnormal flight conditions. In view of the problems of flexibility, real-time, and accuracy in the landing area identification method that relies on a single data source, this embodiment adopts a combination of large-scale GIS static maps (i.e., static map data) and small-scale dynamic visual data (i.e., lidar and video data) to achieve safe landing area identification for drones in high-speed rotation. Through the extensive geographic information provided by static maps and the dynamic update capability of real-time radar point cloud data, a comprehensive assessment of potential landing areas in complex urban environments is carried out to ensure efficient perception and safe operation of drones in complex environments. Optimize the rapid decision-making and accurate identification capabilities of drones in emergency situations.

[0133] In one implementation, the method further includes:

[0134] When the drone lands, the new posture of the drone after the preset time increment is obtained, and interpolation processing is performed according to the new posture at each time point to obtain the interpolated posture between each time point;

[0135] Correcting the laser radar point cloud data according to the interpolation posture to obtain corrected point cloud data;

[0136] A new safety value distribution image is generated according to the corrected point cloud data, and the safe landing position is adjusted according to the new safety value distribution image.

[0137] Specifically, since the point cloud data is easily distorted when the drone rotates at high speed, this embodiment needs to correct the point cloud data by real-time detection and analysis of the drone's posture when the drone lands, eliminating the point cloud data distortion caused by the drone's rotation, thereby improving the accuracy of the data. Then, a new safety value distribution image is regenerated by correcting the point cloud data to further identify a more accurate safe landing position.

[0138] For example, the quaternion update formula is used to calculate the new attitude of the drone after the time increment, which is used to track the rotation state of the drone in real time:

[0139]

[0140] Where q(t) represents the current attitude quaternion; δt represents the time increment; ω(t) represents the current angular velocity;

[0141] Use spherical linear interpolation (Slerp) to calculate the interpolated pose between time points to ensure the smoothness and accuracy of the pose interpolation:

[0142]

[0143] In the formula, q interp Represents the interpolated attitude, that is, the interpolated attitude quaternion;

[0144] According to the interpolated posture, each lidar point is corrected to generate the corrected point cloud data p corrected p_{corrected} (the purpose of correction is to eliminate the point cloud data distortion caused by the rotation of the drone, thereby improving the accuracy of the data):

[0145] p corrected =R(q interp ) raw +t(q interp );

[0146] In the formula, p raw represents the laser radar point, that is, the original laser radar point cloud data; p corrected Represents the corrected point cloud data; R(q int erp ) represents the quaternion q int erp The transformed rotation matrix; t(q int erp ) represents the translation vector estimated by the IMU.

[0147] In one implementation, the method further includes:

[0148] The positions of the UAV in the x, y, and z directions at different time points are calculated based on the safe landing position through a polynomial curve equation to obtain a safe landing trajectory of the UAV.

[0149] Specifically, Figure 2 As shown, this embodiment will integrate static map data and dynamic perception data to identify a safe landing position, then plan the flight trajectory based on a polynomial curve, calculate the best landing path, and optimize the landing trajectory of the drone while ensuring the highest safety standards, so that the drone can accurately search for a safe landing area and land safely even in a dynamic rotation state.

[0150] For example, the positions of the drone in the x, y, and z directions are calculated by the polynomial curve equation (determined based on the aforementioned safety value distribution image) to generate the flight trajectory of the drone. i , b i , c i Determined by a trajectory planning algorithm to ensure that the drone can land along a safe path.

[0151] x(t)=a0+a1t+a2t 2 +...a n t n

[0152] y(t)=b0+b1t+b2t 2 +...+b n t n

[0153] z(t)=c0+c1t+c2t 2 +...+c n t n ;;

[0154] Where x(t), y(t), z(t) represent the spatial coordinates of the drone at time t; a0, a1, a2, ..., a n represents the polynomial coefficients in the x direction; b0, b1, b2, ..., b n represents the polynomial coefficients in the y direction; c0, c1, c2, ..., c n Represents the polynomial coefficients in the z direction.

[0155] Summary of the technical advantages of the present invention:

[0156] 1. Ability to accurately identify safe landing areas when the drone is rotating at high speed:

[0157] Technical means: This invention adopts a method of fusing multi-scale static map perception with small-scale dynamic vision and radar data. By combining large-scale GIS static maps and small-scale dynamic data (lidar and video data) collected in real time, it can accurately identify the landing area of ​​the UAV in a high-speed rotating state.

[0158] Specific implementation: Large-scale static map perception uses the Fast-SCNN model to perform semantic segmentation on GIS static images and preliminarily determine the safe area; small-scale dynamic data fusion uses point cloud data preprocessing, terrain surface reconstruction and Bayesian segmentation network to identify ground roughness and slope, providing fine dynamic environment perception. Compared with existing technologies, this technical means significantly improves the landing recognition accuracy of drones in emergency situations.

[0159] 2. Able to effectively correct the point cloud data distortion when the drone is rotating at high speed:

[0160] Technical means: The present invention uses quaternion interpolation and attitude update algorithm to correct the distortion of laser radar point cloud data under high-speed rotation state in real time.

[0161] Specific implementation: Use the quaternion interpolation method (Slerp) to calculate the attitude interpolation, and combine the attitude quaternion update formula and the rotation matrix to correct the original point cloud data. This method can eliminate the data distortion caused by the rotation process, improve the accuracy and stability of the point cloud data, and enable the drone to maintain high-precision perception capabilities in complex dynamic environments.

[0162] 3. Ability to optimize safe landing paths for drones in complex dynamic environments:

[0163] Technical means: The present invention adopts a trajectory planning and optimization algorithm based on polynomial curves, combined with real-time perception data for trajectory tracking, to ensure the safe landing of the UAV in a complex dynamic environment.

[0164] Specific implementation: The flight trajectory of the drone in the x, y, and z directions is generated through polynomial curve equations, and the polynomial coefficients are adjusted using optimization algorithms to meet safety and feasibility requirements. At the same time, the real-time error feedback control algorithm continuously adjusts the flight trajectory to cope with environmental changes. Compared with existing technologies, this technical means significantly improves the autonomous decision-making and safe landing capabilities of drones in complex environments.

[0165] 4. Improve the operational safety and reliability of drones in complex environments such as cities:

[0166] Technical means: The present invention provides a comprehensive assessment of potential landing areas in complex urban environments by fusing large-scale static prior maps and small-scale dynamic visual data.

[0167] Specific implementation: Use GIS static maps to calculate large-scale risk, combine lidar and video data for dynamic real-time perception, and generate a dynamic safety map of the safe landing area. This method can not only accurately identify safe areas, but also update ground status information in real time, improving the operational safety and reliability of drones in complex environments such as cities.

[0168] Based on the above embodiments, the present invention also provides a dynamic and static combined drone safe landing zone perception system, such as Figure 3 As shown, the system comprises:

[0169] Static analysis module 01 is used to obtain static map data corresponding to the current position of the drone, perform semantic segmentation on the static map data, and obtain a number of ground objects of different types;

[0170] For each potential landing area within the area corresponding to the static map data, a static risk value is calculated according to each ground object contained in the potential landing area; and a target landing area is determined according to the static risk value of each potential landing area;

[0171] The dynamic analysis module 02 is used to obtain dynamic visual data corresponding to the target landing area, and generate a safety value distribution image corresponding to the target landing area according to the dynamic visual data and a preset Bayesian segmentation network; wherein the safety value distribution image is used to reflect the safety values ​​corresponding to different position points of the target landing area;

[0172] The static-dynamic combination module 03 is used to determine the safe landing position of the UAV according to the safety value distribution image.

[0173] Based on the above embodiment, the present invention can also provide a terminal, whose principle block diagram can be as follows: Figure 4 As shown. The terminal includes a processor, a memory, a network interface, and a display screen connected through a system bus. Among them, the processor of the terminal is used to provide computing and control capabilities. The memory of the terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a dynamic and static combined drone safe landing area perception method is implemented. The display screen of the terminal can be a liquid crystal display or an electronic ink display.

[0174] Those skilled in the art will understand that Figure 4 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0175] In one implementation, the terminal has one or more programs stored in its memory and is configured to be executed by one or more processors. The one or more programs include instructions for performing a dynamic and static combined drone safe landing zone perception method.

[0176] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0177] In summary, the present invention discloses a method, system and medium for sensing a safe landing area of ​​a drone that combines static and dynamic elements. The method acquires static map data corresponding to the current position of the drone, performs semantic segmentation on the static map data, and obtains several ground objects of different types; for each potential landing area within the area corresponding to the static map data, calculates a static risk value according to each ground object contained in the potential landing area; determines a target landing area according to the static risk value of each potential landing area; acquires dynamic visual data corresponding to the target landing area, and generates a safety value distribution image corresponding to the target landing area according to the dynamic visual data and a preset Bayesian segmentation network; wherein the safety value distribution image is used to reflect the safety values ​​corresponding to different position points of the target landing area; and determines the safe landing position of the drone according to the safety value distribution image. The present invention combines static map data and dynamic visual data to identify a safe landing area, and can effectively ensure that the drone can accurately sense and land safely even in a high-speed rotating state by fusing static data and dynamic data.

[0178] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A dynamic and static combined method for sensing safe landing areas of unmanned aerial vehicles, characterized in that: The method comprises: Obtain static map data corresponding to the current position of the drone, perform semantic segmentation on the static map data, and obtain a number of ground objects of different types; For each potential landing area within the area corresponding to the static map data, a static risk value is calculated according to each ground object contained in the potential landing area; and a target landing area is determined according to the static risk value of each potential landing area; Acquire dynamic visual data corresponding to the target landing area, and generate a safety value distribution image corresponding to the target landing area according to the dynamic visual data and a preset Bayesian segmentation network; wherein the safety value distribution image is used to reflect the safety values ​​corresponding to different position points of the target landing area; A safe landing position of the UAV is determined according to the safety value distribution image.

2. The method for sensing a safe landing area of ​​a UAV by combining dynamic and static methods according to claim 1, characterized in that: The step of calculating the static risk value according to each ground object included in the potential landing area includes: For each ground object included in the potential landing area, obtain the number of pixel points corresponding to each ground object in the static map data, and determine the weight value corresponding to each ground object according to the number of pixel points; The preset risk degrees corresponding to each ground object are obtained, and weighted summation is performed according to the preset risk degrees of each ground object and the weight values ​​to obtain the static risk value of the potential landing area.

3. The method for sensing a safe landing area of ​​a UAV by combining dynamic and static methods according to claim 1, characterized in that: The dynamic visual data includes laser radar point cloud data, and the acquiring of the dynamic visual data corresponding to the target landing area, and the generating of a safety value distribution image corresponding to the target landing area according to the dynamic visual data and a preset Bayesian segmentation network, include: Acquire original laser radar point cloud data of the target landing area, perform noise reduction and downsampling on the original laser radar point cloud data, and obtain the laser radar point cloud data; The laser radar point cloud data is input into a preset Bayesian segmentation network, the safety value of each point cloud is calculated through the Bayesian segmentation network, and the safety value distribution image is generated according to the safety value of each point cloud.

4. The dynamic and static combined UAV safe landing zone perception method according to claim 3 is characterized in that: The step of generating the security value distribution image according to the security value of each point cloud includes: According to the safety value of each point cloud in the laser radar point cloud data, the safety value of each point cloud deleted after downsampling is calculated by inverse distance weighted interpolation; The security values ​​of all point clouds are binarized, and the security value distribution image is generated according to the security values ​​of all point clouds after the binarization.

5. The method for sensing a safe landing area of ​​a UAV by combining dynamic and static methods according to claim 1, characterized in that: The method further comprises: Divide the laser radar point cloud data into a triangular mesh, and for each triangle in the triangular mesh, calculate the predicted elevation value of the centroid of the triangle according to the weighted sum of the elevation values ​​corresponding to the three endpoints of the triangle; wherein the weights corresponding to the three endpoints are determined based on the corresponding centroid coordinates, and the sum of the centroid coordinates of the three endpoints is 1; The marking information of the predicted elevation value of each center of gravity point is added to the safety value distribution image.

6. The method for sensing a safe landing area of ​​a UAV by combining dynamic and static methods according to claim 1, characterized in that: The method further comprises: Obtaining a global surface normal of the laser radar point cloud data, and determining a normal vector corresponding to each point cloud according to the global surface normal; For each point cloud, a neighborhood of the point cloud is determined with the point cloud as the center and a preset value as the radius, and the surface roughness corresponding to the point cloud is determined according to the difference between the normal vectors of the point cloud and other point clouds in the neighborhood; the slope corresponding to the point cloud is determined according to the Z-axis component of the normal vector of the point cloud and the normal vector; The marking information of the surface roughness and the slope corresponding to each point cloud is added to the safety value distribution image.

7. The method for sensing a safe landing area of ​​a UAV by combining dynamic and static methods according to claim 1, characterized in that: The method further comprises: When the drone lands, the new posture of the drone after the preset time increment is obtained, and interpolation processing is performed according to the new posture at each time point to obtain the interpolated posture between each time point; Correcting the laser radar point cloud data according to the interpolation posture to obtain corrected point cloud data; A new safety value distribution image is generated according to the corrected point cloud data, and the safe landing position is adjusted according to the new safety value distribution image.

8. The method for sensing a safe landing area of ​​a UAV by combining dynamic and static methods according to claim 1, characterized in that: The method further comprises: The positions of the UAV in the x, y, and z directions at different time points are calculated based on the safe landing position through a polynomial curve equation to obtain a safe landing trajectory of the UAV.

9. A dynamic and static combined drone safe landing zone perception system, characterized in that: The system comprises: A static analysis module is used to obtain static map data corresponding to the current position of the UAV, and perform semantic segmentation on the static map data to obtain a number of ground objects of different types; For each potential landing area within the area corresponding to the static map data, a static risk value is calculated according to each ground object contained in the potential landing area; and a target landing area is determined according to the static risk value of each potential landing area; A dynamic analysis module, used to obtain dynamic visual data corresponding to the target landing area, and generate a safety value distribution image corresponding to the target landing area according to the dynamic visual data and a preset Bayesian segmentation network; wherein the safety value distribution image is used to reflect the safety values ​​corresponding to different position points of the target landing area; The static-dynamic combination module is used to determine the safe landing position of the UAV according to the safety value distribution image.

10. A computer-readable storage medium having a plurality of instructions stored thereon, characterized in that: The instructions are suitable for being loaded and executed by a processor to implement the steps of the dynamic and static combined drone safe landing area perception method as described in any one of claims 1 to 8 above.

Images