A multi-sensor fusion-based rapid unmanned aerial vehicle landing area detection method

Abstract

The application relates to a kind of fast unmanned aerial vehicle landing area detection methods based on multi-sensor fusion, camera and laser radar are used to actively perceive the same area, obtain camera image data and laser radar data at the same time;Through the semantic segmentation of image layer, the candidate landing area in the camera image data is pre-screened;The laser radar data is matched, and the best landing area is screened from the candidate landing area by calculating in the radar point cloud layer.The application detects the ground by fusing camera image data and laser radar data, improves the perception information dimension, replaces the evaluation of all point cloud data by using the method of processing radar point cloud in the region of interest, reduces the calculation amount;In addition, IMU data is not used during data fusion, the influence of multi-source noise caused by too many sensors is reduced, so that the unmanned aerial vehicle can perform fast perception task in unknown environment, and the unmanned aerial vehicle landing area is found.

Description

Technical Field

[0001] This invention relates to the field of autonomous landing and emergency landing technology for unmanned aerial vehicles (UAVs), specifically a rapid UAV landing area detection method based on multi-sensor fusion. Background Technology

[0002] With the continuous development of artificial intelligence technology and multi-sensor fusion technology, more intelligent agents are choosing active perception technology to achieve automated functions, such as autonomous driving technology in the automotive field and intelligent perception technology in the industrial robot field. Among them, image processing technology, as an important development direction in the field of artificial intelligence, has been applied in many engineering fields.

[0003] Existing image semantic segmentation techniques have evolved from traditional threshold- or edge-based segmentation techniques to segmentation methods based on neural network deep learning. These neural network deep learning-based segmentation methods can not only achieve end-to-end category output, but also improve the accuracy and precision of deep learning models through the construction of large datasets and iterative training. They can perform real-time image semantic segmentation for single images or video streams.

[0004] Equipping intelligent agents with more active sensors not only allows them to proactively detect environmental information like human sensory organs, but also enables them to combine different types of sensors to perceive the same object, enriching the information about the perceived object and improving perception stability. Typically, multi-sensor data fusion involves spatial and temporal unification. This can be achieved by actively triggering the capture of multi-source data at the same timestamp through hardware, or by using software to achieve multi-source data fusion based on nearby timestamp matching, tracking, and prediction algorithms. This technology can enhance the perception capabilities of intelligent agents, reduce errors in multi-source time-series data, and improve the accuracy of processing results.

[0005] There has been considerable research both domestically and internationally on the development of autonomous landing and emergency landing capabilities for unmanned aerial vehicles (UAVs), including:

[0006] The study on emergency landing mechanisms provides a detailed description of the buffer devices required for emergency landing functions;

[0007] The emergency landing method for UAVs based on radio and laser guidance uses ground markings and target recognition to identify and guide the UAV to land autonomously. However, its shortcomings are that it cannot land autonomously in unknown environments because it requires identifying and searching the target area, specifically by setting ground markings in advance.

[0008] Methods based on image segmentation can identify the ground and guide the UAV to an autonomous landing area; however, their shortcomings are: they use a single image for segmentation, ignoring the three-dimensional information of the ground, resulting in insufficient safety and robustness in finding a suitable landing area.

[0009] In CN113359810A, a method for identifying the landing area of ​​a UAV based on multiple sensors, the landing area is perceived based on the method of multi-sensor fusion. However, the processing involves full processing of image and lidar data, which not only results in a large amount of data but also a greater computational load. IMU data (inertial measurement unit data) is used as part of the data fusion, but IMU data has drift and noise, which can lead to a lot of cumulative errors.

[0010] The information disclosed in this background section is intended only to enhance the understanding of the general background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0011] To address the shortcomings of existing technologies, the present invention aims to provide a rapid UAV landing area detection method based on multi-sensor fusion. Utilizing the UAV's onboard hardware computing capabilities, it detects the ground by fusing camera image data and LiDAR data, thereby enhancing the dimensionality of perceived information. Furthermore, by processing radar point clouds within the region of interest instead of evaluating all point cloud data, the computational load is reduced. Additionally, the data fusion process does not use IMU data, minimizing the impact of multi-source noise introduced by excessive sensors. This allows the UAV to perform rapid perception tasks in unknown environments, enabling the location of the UAV landing area.

[0012] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0013] A rapid UAV landing area detection method based on multi-sensor fusion, characterized by the following specific steps:

[0014] Active sensing of the same area is performed using cameras and LiDAR, acquiring camera image data and LiDAR data at the same time.

[0015] Candidate landing regions in camera image data are pre-screened through semantic segmentation of the image layer;

[0016] The lidar data is matched, and calculations are performed on the radar point cloud layer to select the best landing area from the candidate landing areas.

[0017] Based on the above technical solution, when selecting the best landing area, according to the projection relationship between the coordinate systems of the lidar and the camera image, the pixel coordinates of the pre-selected image layer are matched with the lidar coordinates, and the point cloud around the corresponding point in the lidar layer is filtered to obtain the 3D point cloud of interest, thus narrowing the selection range of candidate landing areas.

[0018] Based on the above technical solution, ground images are acquired using an airborne camera to obtain camera image data. The image semantic segmentation method is used to extract the ground region of interest in the camera image data, and then the landing point is extracted at the image layer.

[0019] The specific steps are as follows:

[0020] Image semantic segmentation processing; the image semantic segmentation model is deployed on the UAV platform to segment the acquired airborne downward view image, and the categories are merged according to the landing attributes of the labeled categories to generate a merged landing area image, thereby achieving a clear division between landable and non-landable areas at the image level;

[0021] Distance transform calculation: The landing area image is merged and binarized. The distance d from the point in the landable area to the boundary of the segmented area is calculated based on the distance transform method.

[0022] After binarization, the landable area is assigned a value of 1, and the non-landable area is assigned a value of 0.

[0023] Image layer landing point extraction; based on the drone's current altitude and physical dimensions, filter the pixel coordinates that meet the requirements within the landable area;

[0024] The method for calculating the coordinates of the pixels that meet the selection requirements is as follows:

[0025] r pixel =α*Rv ehicle *f / Z

[0026] Among them, R vehicle The radius required for the drone's landing is defined by α, a constant used to ensure the size of the surrounding area meets safety requirements during landing, Z being the drone's current altitude above the ground, f being the camera's focal length, and r being the distance between the drone and the ground. pixel This represents the pixel radius of the drone's landing area calculated at the current altitude.

[0027] Iterate through the distance d of each point on the image, compare the distance d with the distance d required for the drone's current altitude projection, and when d > r pixel The image coordinates of the current point will be filtered and retained as the landing point for the image layer.

[0028] Based on the above technical solution, the specific steps for selecting the optimal landing area from the candidate landing areas are as follows:

[0029] Laser point cloud filtering and preprocessing: The amount of LiDAR data per second is enormous. After the LiDAR and camera image data are synchronized, filtering conditions are set after expanding a certain area according to the actual size of the image to reduce the number of point clouds outside the image.

[0030] Calculate the joint calibration projection matrix; unify the coordinate systems of the camera and LiDAR, which can be achieved through joint calibration of the camera and LiDAR, or by determining the installation position and coordinate system orientation using the 3D model of the airborne sensor. This allows us to determine the rotation and translation matrix of the radar point cloud projected onto the image; based on the camera intrinsic parameters and the transformation matrix from the LiDAR coordinate system to the camera coordinate system as shown below:

[0031]

[0032] in, Represents pixel coordinates on the image. [R|T] represents the coordinates of the point cloud detected by the lidar, K represents the intrinsic parameter matrix of the image, and [R|T] represents the extrinsic parameter matrix of the camera, which is the relative rotation and translation matrix between the lidar and camera coordinate systems.

[0033] Right now:

[0034]

[0035] Where i represents the abbreviation for image, l represents the abbreviation for lidar, N represents the Nth corresponding point, R is the rotation matrix, T is the translation matrix, where R is a 3x3 matrix that can be decomposed into r11-r33, T is a 3x1 matrix that can be decomposed into t1-t3, fx and fy are the focal lengths in the horizontal and vertical directions, respectively, and u0 and v0 represent the offsets from the optical axis to the pixel coordinate system (0, 0) in the pixel coordinate system.

[0036] The above transformation matrix projects each point of the LiDAR onto the image layer, establishes the correspondence between image coordinates and LiDAR coordinates, and matches the previously filtered sequence of landing points with the LiDAR to obtain a series of successfully matched radar point clouds, forming a point cloud coordinate list. The point cloud coordinate list stores the mapping points from image points to the LiDAR coordinate system.

[0037] Radar layer neighborhood point cloud extraction; using the point cloud coordinate list established above, each mapped point is used as the coordinates of the landing center, based on the UAV's physical dimensions R. vehicle Find point clouds near the center point as candidate landing point cloud regions; the nearby point clouds are selected through a preset region of interest R. vehicleThe filtering conditions are determined by the value of , and what remains after filtering is the point cloud near the mapped point, also known as the neighborhood point cloud.

[0038] Radar layer landing point cloud evaluation; For the point cloud region extracted in step four, calculate the internal variance, average altitude, point cloud slope, and distance from the current UAV, perform normalization calculations, and assign different weights to calculate the final score; or calculate the landing evaluation function and final score based on the landing conditions.

[0039] The final scores are sorted and filtered to find the optimal landing site.

[0040] Based on the above technical solution, the specific steps for setting filtering conditions after expanding a certain area are as follows: the projection of the lidar on the image is determined in advance using the joint calibration matrix, and point clouds that cannot be projected onto the image are directly discarded.

[0041] Based on the above technical solutions, the optimal landing point location is calculated. This also includes: the output of the optimal landing point location calculation;

[0042] Because the drone's GPS and onboard sensors are installed in different locations, it is necessary to input the drone's GPS coordinates and the GPS device's location to calculate the actual WGS84 coordinate system for the guidance and control program to complete the task. The specific steps are as follows:

[0043] Simultaneous acquisition and analysis of GPS data along with the currently processed camera image data and LiDAR data; on the one hand, based on the GPS and LiDAR coordinates and installation position, using the rotation and translation matrix between the two, determining the optimal landing point for the LiDAR. Switch to GPS device coordinate system

[0044] Points in the GPS device coordinate system The system is switched to the WGS84 coordinate system, and the final calculation results are sent to the guidance and control system to complete the subsequent landing mission.

[0045] The rapid UAV landing area detection method based on multi-sensor fusion described in this invention has the following beneficial effects:

[0046] 1. This method addresses the information deficiency issue inherent in single-sensor perception found in other methods. Furthermore, by optimizing the perception method, it reduces the amount of computational data, improves perception speed, and resolves the computational complexity associated with using raw multi-source data. The raw multi-source data refers to data obtained from sensors, specifically camera image data and LiDAR data in this application.

[0047] In this application, the image is first segmented. After segmentation, the landable areas are merged into categories. Then, the distance of each pixel within the merged area to the boundary is calculated. A group of pixels farther from the boundary is selected. The obtained pixel coordinates are mapped to the position perceived by the LiDAR. Then, the point cloud around the mapped point is filtered in the 3D point cloud for processing, which is the neighborhood point cloud mentioned in the paper. The size of the neighborhood point cloud is generally determined according to the size required for aircraft landing. Therefore, only the point cloud region of interest needs to be processed during the processing, reducing the processing of the global point cloud and thus improving the speed of LiDAR data processing.

[0048] 2. Considering the computing power of the UAV's onboard hardware, not only is the amount of computation reduced, but the dimensionality of perceived information can be improved by fusing camera image data and lidar data to detect the ground.

[0049] By using image segmentation for preprocessing of ground images and filtering for LiDAR, the ground area is pre-screened while reducing the amount of computation required for LiDAR data.

[0050] LiDAR data can provide UAVs with three-dimensional information about the ground during landing. When performing active landing missions, UAVs can select a suitable landing area based on the ground conditions of the area to be landed.

[0051] 3. Data fusion does not use IMU data, which reduces the impact of multi-source noise introduced by too many sensors, enabling UAVs to perform rapid perception tasks in unknown environments and locate UAV landing areas. Attached Figure Description

[0052] The present invention includes the following figures:

[0053] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0054] Figure 1 The flowchart of a rapid UAV landing area detection method based on multi-sensor fusion described in this invention. Detailed Implementation

[0055] The present invention will be further described in detail below with reference to the accompanying drawings. This detailed description is an illustration in conjunction with exemplary embodiments of the invention, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0056] like Figure 1 As shown, this invention presents a rapid UAV landing area detection method based on multi-sensor fusion, with the specific steps as follows:

[0057] Active perception of the same area is achieved using both cameras and LiDAR, acquiring camera image data and LiDAR data (point cloud data) at the same time.

[0058] Candidate landing regions in camera image data are pre-screened through semantic segmentation of the image layer;

[0059] The lidar data is matched, and calculations are performed on the radar point cloud layer to select the best landing area from the candidate landing areas.

[0060] The detection method described in this invention does not use IMU data, thus reducing the drift and noise effects of the IMU sensor;

[0061] The detection method described in this invention combines image semantic information and three-dimensional information through data fusion of camera image data and lidar data, which is of great significance for the autonomous landing and emergency landing of UAVs.

[0062] Based on the above technical solution, when selecting the best landing area, according to the projection relationship between the coordinate systems of the lidar and the camera image, the pixel coordinates of the pre-selected image layer are matched with the lidar coordinates, and the point cloud around the corresponding point in the lidar layer is filtered to obtain the 3D point cloud of interest, thus narrowing the selection range of candidate landing areas.

[0063] Based on the above technical solution, ground images are acquired using an airborne camera to obtain camera image data. The image semantic segmentation method is used to extract the ground region of interest in the camera image data, and then the landing point is extracted at the image layer.

[0064] The specific steps are as follows:

[0065] Image semantic segmentation processing: The image semantic segmentation model is deployed on the UAV platform to segment the acquired airborne downward view images and merge the categories according to the landing attributes of the labeled categories to generate a merged landing area image, thus achieving a clear division of landable and non-landable areas at the image level; generally, in ground image region segmentation, based on the identified content, including grassland, woodland, buildings, farmland, water areas, etc., the grassland and farmland identified results are merged to form the landable area, and other areas are merged to form the non-landable area;

[0066] Distance transformation calculation: The landing area merged image is binarized, and the distance d from the point in the landable area to the boundary of the segmented area is calculated based on the distance transformation method. The distance transformation method is implemented using distanceTransform in OpenCV. This invention does not improve on this method and will not be described in detail.

[0067]

[0068] This formula describes the result of binarization of the image after the landing areas are merged. It assigns a value of 1 to the landable area and a value of 0 to the non-landable area. The result of f(u,v) represents the binarized value of the point (u,v) in the image coordinate system.

[0069] Image layer landing point extraction; based on the drone's current altitude and physical dimensions, filter the pixel coordinates that meet the requirements within the landable area;

[0070] The method for calculating the coordinates of the pixels that meet the selection requirements is as follows:

[0071] r pixel =α*R vehicle *f / Z

[0072] Among them, R vehicle The radius required for the drone's landing is defined by α, a constant used to ensure the size of the surrounding area meets safety requirements during landing, Z being the drone's current altitude above the ground, f being the camera's focal length, and r being the distance between the drone and the ground. pixel This represents the pixel radius of the drone's landing area calculated at the current altitude.

[0073] Iterate through the distance d of each point on the image, compare the distance d with the distance d required for the drone's current altitude projection, and when d > r pixel The image coordinates of the current point will be filtered and retained as the landing point for the image layer.

[0074] Based on the above technical solution, the specific steps for selecting the optimal landing area from the candidate landing areas are as follows:

[0075] Laser point cloud filtering and preprocessing; The amount of LiDAR data per second is enormous. After synchronizing the LiDAR and camera image data, filtering conditions are set after expanding a certain area according to the actual image size to reduce the number of point clouds outside the image. This step can reduce the computational load of the entire process; the synchronization includes:

[0076] Time synchronization is implemented using functions under the ROS framework, which is existing technology. This invention does not improve upon this technology and will not be described in detail here.

[0077] Spatial synchronization involves obtaining coordinate system transformation relationships by acquiring extrinsic parameters through joint calibration. This is existing technology, and this invention does not improve upon it, so it will not be described in detail here.

[0078] The specific steps for setting filtering conditions after expanding a certain area are as follows: the projection of the lidar on the image is determined in advance using the joint calibration matrix, and point clouds that cannot be projected onto the image are directly discarded. The purpose is to reduce the impact of irrelevant lidar point clouds on the overall computational load; the actual size of the image is generally the image resolution.

[0079] Calculate the joint calibration projection matrix; unify the coordinate systems of the camera and LiDAR, which can be achieved through joint calibration of the camera and LiDAR, or by determining the installation position and coordinate system orientation using the 3D model of the airborne sensor. This allows us to determine the rotation and translation matrix of the radar point cloud projected onto the image; based on the camera intrinsic parameters and the transformation matrix from the LiDAR coordinate system to the camera coordinate system as shown below:

[0080]

[0081] in, Represents pixel coordinates on the image. [R|T] represents the coordinates of the point cloud detected by the lidar, K represents the intrinsic parameter matrix of the image, and [R|T] represents the extrinsic parameter matrix of the camera, which is the relative rotation and translation matrix between the lidar and camera coordinate systems.

[0082] Right now:

[0083]

[0084] Where i represents the abbreviation for image (in image coordinate system), l represents the abbreviation for lidar (in radar coordinate system), N represents the Nth corresponding point, R is the rotation matrix, T is the translation matrix, where R is a 3x3 matrix that can be decomposed into r11-r33, T is a 3x1 matrix that can be decomposed into t1-t3, fx and fy are the focal lengths in the horizontal and vertical directions, respectively, and u0 and v0 represent the offset of the optical axis from the pixel coordinate system to the pixel coordinate system point (0, 0).

[0085] The above transformation matrix projects each point of the LiDAR onto the image layer, establishes the correspondence between image coordinates and LiDAR coordinates, and matches the previously filtered sequence of landing points with the LiDAR to obtain a series of successfully matched radar point clouds, forming a point cloud coordinate list. The point cloud coordinate list stores the mapping points from image points to the LiDAR coordinate system.

[0086] Radar layer neighborhood point cloud extraction; using the point cloud coordinate list established above, each mapped point is used as the coordinates of the landing center, based on the UAV's physical dimensions R. vehicleFind point clouds near the center point as candidate landing point cloud regions; the nearby point clouds are selected through a preset region of interest R. vehicle The filtering conditions are determined by the value of , and what remains after filtering is the point cloud near the mapped point, also known as the neighborhood point cloud.

[0087] Radar layer landing point cloud evaluation; For the point cloud region extracted in step four, calculate the internal variance, average altitude, point cloud slope, and distance from the current UAV, perform normalization calculations, and assign different weights to calculate the final score; or calculate the landing evaluation function and final score based on the landing conditions.

[0088] The final scores are sorted and filtered to find the optimal landing site.

[0089] Based on the above technical solutions, the optimal landing point location is calculated. This also includes: the output of the optimal landing point location calculation;

[0090] The optimal landing point location can be calculated using the above steps. Because the drone's GPS and onboard sensors are installed in different locations, it is necessary to input the drone's GPS coordinates and the GPS device's location to calculate the actual WGS84 coordinate system for the guidance and control program to complete the task. The specific steps are as follows:

[0091] Simultaneous acquisition and analysis of GPS data along with the currently processed camera image data and LiDAR data; on the one hand, based on the GPS and LiDAR coordinates and installation position, using the rotation and translation matrix between the two, determining the optimal landing point for the LiDAR. Switch to GPS device coordinate system

[0092] Points in the GPS device coordinate system The system is switched to the WGS84 coordinate system, and the final calculation results are sent to the guidance and control system to complete the subsequent landing mission.

[0093] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

[0094] The above description is only a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. Any equivalent modifications or changes made by those skilled in the art based on the content disclosed in the present invention should be included in the scope of protection set forth in the claims.

Claims

1. A rapid UAV landing area detection method based on multi-sensor fusion, characterized in that, The specific steps are as follows: Active sensing of the same area is performed using cameras and LiDAR, acquiring camera image data and LiDAR data at the same time. Candidate landing regions in camera image data are pre-screened through semantic segmentation of the image layer; The lidar data is matched, calculations are performed on the radar point cloud layer, and the best landing area is selected from the candidate landing areas. The specific steps for selecting the optimal landing area from the candidate landing areas are as follows: Laser point cloud filtering and preprocessing: The amount of LiDAR data per second is enormous. After the LiDAR and camera image data are synchronized, filtering conditions are set after expanding a certain area according to the actual size of the image to reduce the number of point clouds outside the image. Calculate the joint calibration projection matrix; unify the coordinate systems of the camera and LiDAR, which can be achieved through joint calibration of the camera and LiDAR, or by determining the installation position and coordinate system orientation using the 3D model of the airborne sensor. This allows us to determine the rotation and translation matrix of the radar point cloud projected onto the image; based on the camera intrinsic parameters and the transformation matrix from the LiDAR coordinate system to the camera coordinate system as shown below: ， in, Represents pixel coordinates on the image. This represents the coordinates of the point cloud detected by the lidar. The intrinsic parameter matrix of the image. This represents the camera extrinsic parameter matrix, which is the relative rotation and translation matrix between the lidar and camera coordinate systems; Right now: ， Where i represents the abbreviation for image, l represents the abbreviation for lidar, N represents the Nth corresponding point, R is the rotation matrix, T is the translation matrix, where R is a 3x3 matrix that can be decomposed into r11-r33, T is a 3x1 matrix that can be decomposed into t1-t3, fx and fy are the focal lengths in the horizontal and vertical directions, respectively, and u0 and v0 represent the offsets from the optical axis to the pixel coordinate system (0, 0) in the pixel coordinate system. The above transformation matrix projects each point of the LiDAR onto the image layer, establishes the correspondence between image coordinates and LiDAR coordinates, and matches the previously filtered sequence of landing points with the LiDAR to obtain a series of successfully matched radar point clouds, forming a point cloud coordinate list. The point cloud coordinate list stores the mapping points from image points to the LiDAR coordinate system. Radar layer neighborhood point cloud extraction; using the point cloud coordinate list established above, each mapped point is used as the coordinates of the landing center, based on the physical size of the UAV. The point cloud near the center point is identified as a candidate landing point cloud region; the nearby point cloud is selected through a predefined region of interest. The filtering conditions are determined by the value of , and what remains after filtering is the point cloud near the mapped point, also known as the neighborhood point cloud. Radar layer landing point cloud evaluation; For the point cloud region extracted in step four, calculate the internal variance, average altitude, point cloud slope, and distance from the current UAV, perform normalization calculations, and assign different weights to calculate the final score; or calculate the landing evaluation function and final score based on the landing conditions. The final scores are sorted and filtered to find the optimal landing site. .

2. The rapid UAV landing area detection method based on multi-sensor fusion as described in claim 1, characterized in that, When selecting the best landing area, the pixel coordinates of the pre-selected image layer are mapped to the coordinates of the LiDAR and camera image coordinate systems according to the projection relationship. The point cloud around the corresponding point in the LiDAR layer is then filtered to obtain the 3D point cloud of interest, thus narrowing down the selection range of candidate landing areas.

3. The rapid UAV landing area detection method based on multi-sensor fusion as described in claim 1, characterized in that, Ground images are acquired using an airborne camera to obtain camera image data. Image semantic segmentation is used to extract the ground region of interest from the camera image data, and then the landing point is extracted at the image layer. The specific steps are as follows: Image semantic segmentation processing; the image semantic segmentation model is deployed on the UAV platform to segment the acquired airborne downward view image, and the categories are merged according to the landing attributes of the labeled categories to generate a merged landing area image, thereby achieving a clear division between landable and non-landable areas at the image level; Distance transform calculation: The landing area image is merged and binarized. The distance transform method is used to calculate the distance from points within the landable area to the boundary of the segmented region. ; After binarization, the landable area is assigned a value of 1, and the non-landable area is assigned a value of 0. Image layer landing point extraction; based on the drone's current altitude and physical dimensions, filter the pixel coordinates that meet the requirements within the landable area; The method for calculating the coordinates of the pixels that meet the selection requirements is as follows: ， in, Define the radius required for drone landing. It is a constant used to ensure that the size of the reserved area around the drone during landing meets safety requirements. This is the current ground altitude of the drone. For camera focal length, This represents the pixel radius of the drone's landing area calculated at the current altitude. Distance to each point on the image Compare distances The number of pixels and distance required for the drone's current altitude projection. ,when The image coordinates of the current point will be filtered and retained as the landing point for the image layer.

4. The rapid UAV landing area detection method based on multi-sensor fusion as described in claim 1, characterized in that, The specific steps for setting filtering conditions after expanding a certain area are as follows: the projection of the lidar on the image is determined in advance using the joint calibration matrix, and point clouds that cannot be projected onto the image are directly discarded.

5. The rapid UAV landing area detection method based on multi-sensor fusion as described in claim 1, characterized in that, Calculate the optimal landing point location This also includes: the output of the optimal landing point location calculation; Because the drone's GPS and onboard sensors are installed in different locations, it is necessary to input the drone's GPS coordinates and the GPS device's location to calculate the actual WGS84 coordinate system for the guidance and control program to complete the task. The specific steps are as follows: Simultaneous acquisition and analysis of GPS data from the camera and LiDAR systems, along with the currently processed camera image and LiDAR data, are performed. Based on the GPS and LiDAR coordinates and installation positions, the optimal landing point for the LiDAR is determined using the rotation and translation matrix between them. Switch to GPS device coordinate system ; Points in the GPS device coordinate system The system is switched to the WGS84 coordinate system, and the final calculation results are sent to the guidance and control system to complete the subsequent landing mission.

Images