Unmanned aerial vehicle mapping method and system based on multi-source fusion, terminal and storage medium

Abstract

The application relates to a multi-source fusion-based unmanned aerial vehicle mapping method and system, a terminal and a storage medium, and relates to the technical field of unmanned aerial vehicle image processing. The application comprises the following steps: acquiring multi-source detection data, unmanned aerial vehicle pose data and forward running speed; analyzing the multi-source detection data and the unmanned aerial vehicle pose data to determine effective point cloud data; analyzing the forward running speed and the effective point cloud data to determine an optimal voxel size; acquiring an unmanned aerial vehicle center coordinate; and performing confidence fusion update on the effective point cloud data according to the unmanned aerial vehicle center coordinate and the optimal voxel size to determine unmanned aerial vehicle mapping data. The application has the effects of improving the unmanned aerial vehicle mapping efficiency and mapping precision.

Description

Technical Field

[0001] This application relates to the technical field of UAV image processing, and in particular to UAV mapping methods, systems, terminals and storage media based on multi-source fusion. Background Technology

[0002] Multi-source fusion-based UAV mapping refers to the process of mapping the occupation of environmental obstacles during UAV operations using multi-source data.

[0003] In related technologies, when mapping is performed during UAV operations, a single-source data mapping method is usually adopted. During the UAV operation, measurement data from the lidar deployed on the UAV is collected. Based on the lidar point cloud data obtained from the lidar measurement, the obstacle occupancy is determined, and a complete obstacle occupancy map of the operation area is generated.

[0004] Regarding the aforementioned technologies, when using a single-source data mapping method, issues such as dust, water mist, and long-distance ranging can lead to low echo detection confidence in lidar point cloud data, resulting in low mapping accuracy for UAVs. Furthermore, the generation of complete obstacles within the operational range during the mapping process can reduce mapping efficiency, indicating room for improvement. Summary of the Invention

[0005] To improve the efficiency and accuracy of UAV mapping, this application provides a UAV mapping method, system, terminal, and storage medium based on multi-source fusion.

[0006] Firstly, this application provides a UAV mapping method based on multi-source fusion, employing the following technical solution: UAV mapping methods based on multi-source fusion include: Acquire multi-source detection data, UAV pose data, and forward running speed; Analyze multi-source detection data and UAV pose data to determine effective point cloud data; The forward running speed and effective point cloud data are analyzed to determine the optimal voxel size; Obtain the center coordinates of the drone; The effective point cloud data is updated by confidence fusion based on the UAV center coordinates and optimal voxel size to determine the UAV mapping data.

[0007] Optionally, the steps of analyzing multi-source detection data and UAV pose data to determine valid point cloud data include: Data filtering is performed on multi-source detection data to determine valid multi-source data; Interpolation data extraction is performed on the UAV pose data based on valid multi-source data to determine the synchronized pose data; Obtain the static extrinsic parameter matrix; Based on the static extrinsic matrix, coordinate transformation is performed on the effective multi-source data to determine the multi-source data of the organism; Perform matrix transformation on the synchronized pose data to determine the real-time pose matrix; The coordinate transformation of the multi-source data of the body is performed based on the real-time pose matrix to determine the effective point cloud data.

[0008] Optionally, the steps of analyzing the forward running speed and effective point cloud data to determine the optimal voxel size include: Obtain the estimated position and altitude of the drone; Calculate the distance between the drone's estimated position and the effective point cloud data to determine the distance to obstacle points; Numerical analysis of obstacle distances is performed to determine the minimum obstacle distance; The minimum obstacle distance, forward running speed, and estimated drone altitude were analyzed to determine the optimal voxel size.

[0009] Optionally, the steps of analyzing the minimum obstacle distance, forward running speed, and estimated drone altitude to determine the optimal voxel size include: Analyze the forward running speed to determine the map update volume; The target gear voxel is determined based on the minimum obstacle distance and the drone's estimated altitude. Input the map update volume, the preset data mapping time, and the preset single-voxel processing time into the preset voxel matching model to determine the theoretical mapping voxels; The maximum value between the target voxel and the theoretical mapping voxel is taken as the optimal voxel size.

[0010] Optionally, the step of analyzing the forward travel speed to determine the map update volume includes: The forward running speed is decomposed to determine the absolute lateral speed, absolute longitudinal speed, and absolute vertical speed; Calculate the products of absolute lateral speed, absolute longitudinal speed, and absolute vertical speed with the preset control cycle duration to determine the lateral running distance, longitudinal running distance, and vertical running distance; The sum of the lateral running distance, longitudinal running distance, and vertical running distance and the preset safety redundancy distance are calculated respectively to determine the lateral update distance, longitudinal update distance, and vertical update distance; Calculate the product of the horizontal update distance, the vertical update distance, and the longitudinal update distance to determine the map update volume.

[0011] Optionally, the steps of performing confidence fusion updates on the effective point cloud data based on the UAV center coordinates and optimal voxel size to determine the UAV mapping data include: The effective point cloud data is extracted based on the UAV center coordinates, the preset sliding update window, and the preset redundant update distance to determine the window point cloud data. Obtain the point cloud center distance and point cloud echo data corresponding to the window point cloud data; The window point cloud data is evaluated based on the preset confidence assessment rules, point cloud center distance, and point cloud echo data to determine the laser confidence weight and millimeter wave confidence weight. The window point cloud data is divided according to the optimal voxel size to determine the voxel laser data and voxel millimeter wave data; The voxel laser data and voxel millimeter-wave data corresponding to the largest laser confidence weight and the largest millimeter-wave confidence weight are determined as the current laser data and the current millimeter-wave data; Analyze the current laser data and current millimeter-wave data to determine the UAV mapping data.

[0012] Optionally, the steps of analyzing current laser data and current millimeter-wave data to determine UAV mapping data include: Numerical analysis is performed on the current laser data, current millimeter-wave data, preset laser occupancy probability, and preset millimeter-wave occupancy probability to determine the historical overlap probability, logarithmic laser probability, and logarithmic millimeter-wave probability. Interpolate and fuse the historical overlap probability, logarithmic laser probability, and logarithmic millimeter wave probability to determine the fused laser probability and fused millimeter wave probability. The fusion probabilities of laser and millimeter wave are weighted and fused to determine the initial fusion probability. The initial fusion probabilities are stored in a circular cache to determine the initial fusion map; The initial fused map is expanded with obstacle inflation to determine the obstacle inflation data; Euclidean symbolic distance field calculations were performed on the obstacle dilatation data to determine the ESDF distance field data; The obstacle inflation data and ESDF distance field data are integrated to determine the UAV mapping data.

[0013] Secondly, this application provides a UAV mapping system based on multi-source fusion, which adopts the following technical solution: A UAV mapping system based on multi-source fusion includes: The acquisition module is used to acquire multi-source detection data, UAV pose data, forward running speed, and UAV center coordinates. A memory for storing programs for the UAV mapping method based on multi-source fusion as described in any of the preceding items; The processor and the program in the memory can be loaded and executed by the processor to implement the UAV mapping method based on multi-source fusion as described above.

[0014] Thirdly, this application provides a smart terminal, which adopts the following technical solution: A smart terminal includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in any of the preceding claims for a UAV mapping method based on multi-source fusion.

[0015] Fourthly, this application provides a computer storage medium capable of storing corresponding programs, which facilitates improving the efficiency and accuracy of UAV mapping, and adopts the following technical solution: A computer-readable storage medium storing a computer program that can be loaded by a processor and executed any of the above-mentioned UAV mapping methods based on multi-source fusion.

[0016] In summary, this application includes at least one of the following beneficial technical effects: 1. By acquiring multi-source detection data, UAV pose data, and forward running speed, the multi-source detection data and UAV pose data are analyzed to determine effective point cloud data. The forward running speed and effective point cloud data are analyzed to determine the optimal voxel size. The UAV center coordinates are obtained. Based on the UAV center coordinates and the optimal voxel size, the effective point cloud data is updated by confidence fusion to determine the UAV mapping data, thereby improving the UAV mapping accuracy. 2. By acquiring the UAV's estimated position and estimated altitude, the distance between the estimated position and the effective point cloud data is calculated to determine the obstacle distance. Numerical analysis is performed on the obstacle distance to determine the minimum obstacle distance. The minimum obstacle distance, forward running speed, and estimated altitude of the UAV are analyzed to determine the optimal voxel size. Thus, the optimal voxel size is determined based on the obstacle density, improving the accuracy of flight control data generation within the flight control cycle and thereby improving the operational stability of the UAV. 3. By performing numerical analysis on current laser data, current millimeter-wave data, and preset laser occupancy probabilities and millimeter-wave occupancy probabilities, historical overlap probabilities, logarithmic laser probabilities, and logarithmic millimeter-wave probabilities are determined. Interpolation and fusion are then performed on these probabilities to determine fused laser probabilities and fused millimeter-wave probabilities. Weighted fusion of these probabilities is then performed to determine the initial fusion probability. This initial fusion probability is then stored in a circular cache to determine the initial fused map. Obstacle inflation is applied to the initial fused map to determine obstacle inflation data. Euclidean signed distance field calculations are then performed on the obstacle inflation data to determine the ESDF distance field data. Finally, the obstacle inflation data and ESDF distance field data are integrated to determine the UAV mapping data. This circular storage of the mapping data improves the UAV's mapping efficiency and optimizes the mapping storage space. Attached Figure Description

[0017] Figure 1 This is a flowchart of the UAV mapping method based on multi-source fusion in the embodiments of this application.

[0018] Figure 2 This is a flowchart illustrating the analysis of multi-source detection data and UAV pose data in this embodiment of the application to determine effective point cloud data.

[0019] Figure 3 This is a flowchart in this application embodiment of analyzing the forward running speed and effective point cloud data to determine the optimal voxel size.

[0020] Figure 4 This is a flowchart illustrating the analysis of minimum obstacle distance, forward running speed, and estimated drone altitude in this embodiment of the application to determine the optimal voxel size.

[0021] Figure 5 This is a flowchart in this application embodiment that analyzes the forward running speed to determine the map update volume.

[0022] Figure 6 This is a flowchart in this application embodiment of performing confidence fusion update on effective point cloud data based on the center coordinates of the UAV and the optimal voxel size to determine the UAV mapping data.

[0023] Figure 7 This is a flowchart illustrating the analysis of current laser data and current millimeter-wave data in this embodiment of the application to determine UAV mapping data. Detailed Implementation

[0024] To make the purpose, technical solution, and advantages of this application clearer, the following description is provided in conjunction with the appendix. Figures 1 to 7The present application will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the application.

[0025] This application discloses a UAV mapping method, system, terminal, and storage medium based on multi-source fusion. Specifically, it discloses a processing terminal that acquires multi-source detection data, UAV pose data, and forward running speed. The processing terminal analyzes the multi-source detection data and UAV pose data to determine effective point cloud data, analyzes the forward running speed and effective point cloud data to determine the optimal voxel size, obtains the UAV center coordinates, and updates the effective point cloud data with confidence fusion based on the UAV center coordinates and optimal voxel size to determine the UAV mapping data, thereby improving the accuracy of UAV mapping.

[0026] Reference Figure 1 This application discloses a UAV mapping method based on multi-source fusion, including the following steps: Step S100: Acquire multi-source detection data, UAV pose data, and forward running speed.

[0027] Among them, multi-source detection data refers to the lidar detection data and millimeter-wave radar detection data collected in the previous operation control cycle, which are determined by the processing terminal by retrieving the original detection data of the forward lidar and upper millimeter-wave radar deployed on the UAV.

[0028] UAV pose data refers to the six-degree-of-freedom pose data of the UAV's body center in the global coordinate system during the previous operational control cycle. This includes the UAV's three-dimensional global coordinate data, as well as the UAV's roll, pitch, and yaw angles. The processing terminal first retrieves the raw sensor data acquired by the UAV's onboard IMU (Inertial Measurement Unit) and RTK (Real-Time Kinematics) satellite positioning module. Then, the raw sensor data undergoes real-time filtering, error compensation, and spatiotemporal alignment processing to determine the UAV's global three-dimensional position coordinates. The UAV's global three-dimensional position coordinates are obtained by integrating the observations from the RTK satellite positioning module with the relative displacement calculated from the acceleration data acquired by the inertial unit. The roll, pitch, and yaw angle data of the UAV are obtained by integrating the angular acceleration data output by the IMU gyroscope, and then using the Kalman fusion algorithm to compensate for the drift error of the gyroscope through static observation constraints of the accelerometer. Finally, the roll and pitch angle data are calculated and determined. The yaw angle data is then determined by calibrating the yaw offset by combining the geomagnetic observation values ​​obtained by the magnetometer. After determining the calculated data at each time, the observation results of coordinate and angle data are synchronously configured with a unified timestamp and stored in the system along with the lidar detection data and lidar point cloud detection data, providing data support for subsequent coordinate transformation of the original point cloud detection data.

[0029] Forward operating speed refers to the operating speed of the drone in the previous cycle, which is determined by the processing terminal by retrieving the drone's operating control data.

[0030] Step S101: Analyze the multi-source detection data and UAV pose data to determine the effective point cloud data.

[0031] Among them, valid point cloud data refers to the simultaneous spatiotemporal global laser point cloud data and millimeter-wave detection data obtained after removing invalid data. This data is determined by the processing terminal through analysis of multi-source detection data and UAV pose data. Specific analysis steps are detailed below. Figure 2 The steps in the process.

[0032] Step S102: Analyze the forward running speed and effective point cloud data to determine the optimal voxel size.

[0033] The optimal voxel size refers to the voxel size of the updated portion of the map when updating the UAV mapping data in the current cycle. This size is determined by the processing terminal through analysis of the forward travel speed and effective point cloud data. Specific analysis steps are detailed below. Figure 3 The steps in the process.

[0034] Step S103: Obtain the center coordinates of the UAV.

[0035] Among them, the UAV center coordinates refer to the global three-dimensional coordinate data of the UAV body center at the start of the current cycle, which is determined by the processing terminal by extracting the body coordinate data from the UAV pose data based on the start of the current cycle.

[0036] Step S104: Perform confidence fusion update on the effective point cloud data based on the UAV center coordinates and optimal voxel size to determine the UAV mapping data.

[0037] The UAV mapping data refers to a fixed-size 3D occupancy grid map generated with the UAV's body coordinates as the center. This includes obstacle inflation data and ESDF distance field data, used to generate flight control commands for the UAV in the next cycle. The data is determined by the processing terminal through confidence fusion updating of the effective point cloud data based on the UAV's center coordinates and optimal voxel size. Specific methods and steps are detailed in [reference needed]. Figure 6 The steps in the process.

[0038] Reference Figure 2 The steps for analyzing multi-source detection data and UAV pose data to determine valid point cloud data include: Step S200: Perform data filtering on the multi-source detection data to determine the valid multi-source data.

[0039] Among them, effective multi-source data refers to the filtered laser point cloud data and millimeter-wave detection data. The processing terminal performs Boolean judgment on the effectiveness of the laser point cloud data and millimeter-wave data in the multi-source detection data respectively. When the absolute value of the difference between the timestamp of the point cloud data and the timestamp of the pose data in the same frame of point cloud data is not greater than the calibrated maximum time difference value, and the total number of point clouds in each frame of data that meet the maximum time difference value is greater than the calibrated minimum number threshold, the point clouds in that frame of data that meet the maximum time difference value are determined as effective point cloud data. All effective point cloud data in the laser point cloud data and millimeter-wave point cloud data are integrated respectively to form multi-source point cloud data.

[0040] The maximum time difference is determined by the operator through offline measurement. The timestamp deviation between the valid point cloud data and pose data in the same frame is determined by synchronously collecting data under multiple working conditions. The sum of the mean and three times the standard deviation of the time difference is then calculated.

[0041] The minimum number threshold is determined by the operator through multiple rounds of offline pre-experiments under UAV operating conditions. The minimum number of point clouds that can effectively build maps and suppress clutter misjudgments is the minimum number threshold, thereby filtering out invalid data frames with too few point clouds that cannot effectively reflect the characteristics of the UAV flight environment.

[0042] Step S201: Interpolate the UAV pose data based on the valid multi-source data to determine the synchronized pose data.

[0043] Synchronous pose data refers to UAV pose data that matches the timestamps of valid multi-source data. The processing terminal interpolates the UAV pose data based on the timestamps of each frame of valid point cloud data, and then extracts the interpolated UAV pose data based on the point cloud timestamps. The UAV pose interpolation data with the same timestamp corresponding to each frame of point cloud data is the synchronous pose data.

[0044] Step S202: Obtain the static extrinsic parameter matrix.

[0045] Among them, the static extrinsic parameter matrix refers to the coordinate transformation matrix that transforms point cloud data from the original radar coordinate system to the UAV body coordinate system, including the lidar transformation matrix and the millimeter-wave radar transformation matrix. It is determined by the processing terminal directly retrieving the lidar transformation matrix and the millimeter-wave radar transformation matrix stored in the system and determined by coordinate calibration experiments.

[0046] Step S203: Perform coordinate transformation on the effective multi-source data based on the static extrinsic matrix to determine the multi-source data of the organism.

[0047] Among them, the multi-source data of the aircraft refers to the effective multi-source data after the coordinate system is transformed to the aircraft coordinate system, including the aircraft laser point cloud data and the aircraft millimeter-wave radar data, which are determined by the processing terminal by calculating the static extrinsic matrix and the matrix multiplication of the corresponding type of effective multi-source data.

[0048] Step S204: Perform matrix transformation on the synchronized pose data to determine the real-time pose matrix.

[0049] The real-time pose matrix refers to the homogeneous rigid body transformation matrix used to transform a set of multi-source body data corresponding to each pose into a global coordinate system. It is determined by the processing terminal through matrix transformation of each synchronous pose data. The matrix transformation process is as follows: First, the roll Euler angle, pitch Euler angle, and yaw Euler angle data are extracted from the synchronous pose data. Then, the rotation matrices of the roll Euler angle, pitch Euler angle, and yaw Euler angle are calculated respectively. Finally, according to the standard rotation sequence of airborne inertial navigation, the rotation matrix is ​​multiplied in the order of first rotating around the X-axis of the body, then around the Y-axis of the body, and finally around the Z-axis of the body to obtain a 3×3 body rotation matrix. Then, the three-dimensional global coordinate data of the UAV body is extracted from the synchronous pose data. The three-dimensional global coordinate data is converted into column vectors and horizontally concatenated with the body rotation matrix to obtain a 3×4 matrix. The fourth row is then fixed as [0,0,0,1] to obtain a fourth-order homogeneous matrix, which is the real-time pose matrix.

[0050] Step S205: Perform coordinate transformation on the multi-source data of the machine body according to the real-time pose matrix to determine the effective point cloud data.

[0051] The effective point cloud data is consistent with the effective point cloud data in step S101. The processing terminal first converts each point cloud coordinate in the multi-source data of the body into a column vector, and then adds a constant 1 to the end of the vector to expand the column, so as to obtain a four-dimensional point cloud vector. Then, the rigid body transformation operation is completed by left multiplying the four-dimensional point cloud vector by the real-time pose matrix.

[0052] Reference Figure 3 The steps for analyzing the forward running speed and effective point cloud data to determine the optimal voxel size include: Step S300: Obtain the estimated position and estimated altitude of the UAV.

[0053] Among them, the UAV's calculated position refers to the three-dimensional coordinate position of the UAV at the start of the next flight control cycle, which is determined by calculation. The processing terminal calculates and determines the UAV's real-time position at the start of the next flight control cycle based on the UAV's real-time position at the start of the current cycle and the flight control data of the current cycle.

[0054] The estimated altitude of a drone refers to the altitude at the start of the next flight control cycle. It is calculated and determined by the processing terminal based on the drone's altitude at the start of the current cycle and the flight control data of the current cycle.

[0055] Step S301: Calculate the distance between the UAV's estimated position and the effective point cloud data to determine the distance to the obstacle point.

[0056] Among them, obstacle point distance refers to the distance between the effective point cloud data and the UAV's estimated position, which is determined by the processing terminal by calculating the Euclidean distance between each effective point cloud data and the UAV's estimated position.

[0057] Step S302: Perform numerical analysis on the distance to the obstacle point to determine the minimum obstacle distance.

[0058] The minimum obstacle distance refers to the minimum distance between obstacle points, which is determined by the processing terminal traversing the distances between obstacle points. The minimum value among the obstacle point distances is the minimum obstacle distance.

[0059] Step S303: Analyze the minimum obstacle distance, forward running speed, and estimated altitude of the UAV to determine the optimal voxel size.

[0060] The optimal voxel size is consistent with the optimal voxel size in step S102, and is determined by the processing terminal through analysis of the minimum obstacle distance, forward running speed, and estimated altitude of the UAV. Specific analysis steps are detailed below. Figure 4 The steps in the process.

[0061] Reference Figure 4 The steps to determine the optimal voxel size by analyzing the minimum obstacle distance, forward speed, and estimated drone altitude include: Step S400: Analyze the forward running speed to determine the map update volume.

[0062] The map update volume refers to the newly added map volume during the map update within the current cycle. It is determined by the processing terminal through analysis of the forward running speed. Specific analysis steps are detailed below. Figure 5 The steps in the process.

[0063] Step S401: Determine the target gear voxel based on the minimum obstacle distance and the drone's calculated altitude.

[0064] Among them, the target level voxel refers to the fixed level voxel classified according to the flight altitude of the UAV, the distance of the UAV body, and the minimum distance between surrounding obstacles. When the minimum obstacle distance is smaller and the flight altitude of the UAV is lower, the size of the fixed level voxel is smaller. This allows for the accurate reconstruction of obstacle outlines when the obstacle distance is close, the UAV flight altitude is low, and the obstacles are relatively dense, thus improving the accuracy of obstacle avoidance judgment of the UAV. Conversely, a large voxel is selected to reduce the amount of mapping calculation. The processing terminal determines the target level voxel by looking up the minimum obstacle distance and the calculated altitude of the UAV in the preset level mapping table.

[0065] The gear mapping table refers to the mapping relationship between the minimum obstacle distance and the estimated altitude of the UAV to fixed gear voxels. For example, when the minimum obstacle distance is in the AB interval and the estimated altitude of the UAV is in the CD interval, the corresponding fixed gear voxel size is E, which is determined by the operator through multi-group stratified offline pre-experiments. In the pre-experiment process, the operator first divides the minimum obstacle distance interval and the UAV flight altitude interval into multiple gradients according to the UAV's operating conditions. Under different combinations of operating conditions in each gear interval, the operator conducts UAV flight mapping pre-experiments. During the experiment, different voxel sizes are used for mapping to determine the UAV obstacle avoidance accuracy, mapping time, and the completeness of the detection data point cloud modeling corresponding to different combinations of operating conditions and different voxel sizes. Based on the evaluation data comparison, the fixed voxel size that meets the requirements of obstacle avoidance accuracy and computing performance is selected. Then, the fixed size is associated with the corresponding combination of operating condition intervals one by one to determine the gear mapping table.

[0066] Step S402: Input the map update volume, the preset data mapping time and the preset single-voxel processing time into the preset voxel matching model to determine the theoretical mapping voxels.

[0067] Among them, the data mapping time refers to the fixed time used in each flight control cycle to map the point cloud data generated in the previous cycle and update the historical mapping data. The remaining basic available processing time is determined by the operator based on the total flight control cycle time, the point cloud data processing time, and the time required for the flight control module to analyze the new map and generate control commands for the next flight control cycle. Then, the basic available processing time is reduced to a certain safety redundancy based on the data processing speed of the mapping module.

[0068] The single-voxel processing time refers to the time consumed in mapping a single voxel. The operator conducts a mapping pre-experiment on the UAV, inputs multi-gradient batch point cloud data into the UAV mapping module to update the mapping experiment, determines the total computation time corresponding to the mapping results of each group and the total number of voxels involved in the processing, and takes the average of the ratio of the total mapping time to the total number of voxels under multiple groups of different load conditions as the single-voxel processing time.

[0069] The voxel matching model is a formulaic model that determines the theoretical mapping voxels based on map update volume data and mapping time. This model assumes that the total processing time for a single mapping operation is the product of the total number of voxels corresponding to the newly added update volume and the processing time of a single voxel. The total number of newly added voxels in a single operation is the quotient of the map update volume and the volume of a single voxel. Therefore, the model solves for the theoretical voxels with the constraint that the total processing time does not exceed the mapping time. The specific model formula is as follows: .

[0070] In the formula, To construct voxels for theoretical mapping, Update the map volume. For the duration of monomer treatment, Duration for data mapping.

[0071] The theoretical mapping voxel refers to the theoretical lower limit of the side length of the mapping voxel determined based on the data mapping time. It is calculated by the processing terminal by inputting the map update volume, data mapping time, and single voxel processing time into the voxel matching model.

[0072] Step S403: Take the maximum value between the target voxel and the theoretical mapping voxel as the optimal voxel size.

[0073] The optimal voxel size is consistent with the optimal voxel size in step S303. After determining the target voxel and the theoretical mapping voxel, the processing terminal takes the maximum value of the target voxel and the theoretical mapping voxel as the optimal voxel size, thereby avoiding the flight control module's inability to determine flight control data in time due to map mapping update timeout, and improving the stability of the UAV operation process.

[0074] Reference Figure 5 The steps for analyzing the forward velocity to determine the map update volume include: Step S500: Decompose the forward running speed to determine the absolute lateral speed, absolute longitudinal speed, and absolute vertical speed.

[0075] Among them, the absolute lateral speed refers to the absolute value of the horizontal axis component of the forward running speed. It is determined by the processing terminal by first performing vector decomposition on the forward running speed to determine the horizontal axis component of the forward running speed, and then calculating the absolute value of the horizontal axis speed component.

[0076] Absolute longitudinal speed refers to the absolute value of the longitudinal component of the forward running speed. It is determined by the processing terminal by first performing vector decomposition on the forward running speed to determine the longitudinal component of the forward running speed, and then calculating the absolute value of the longitudinal speed component.

[0077] Absolute vertical velocity refers to the absolute value of the vertical component of the forward running velocity. It is determined by the processing terminal by first performing vector decomposition on the forward running velocity to determine the vertical component of the forward running velocity, and then calculating the absolute value of the vertical velocity component.

[0078] Step S501: Calculate the product of the absolute lateral speed, absolute longitudinal speed, and absolute vertical speed with the preset control cycle duration to determine the lateral running distance, longitudinal running distance, and vertical running distance.

[0079] Among them, the control cycle duration refers to the length of the control cycle for UAV flight control, which is determined by the processing terminal based on the UAV flight control cycle.

[0080] Lateral travel distance refers to the total travel distance of the UAV in the horizontal direction during the previous control cycle, which is determined by the processing terminal by calculating the product of the absolute lateral velocity and the control cycle duration.

[0081] The longitudinal running distance refers to the total running distance of the UAV in the longitudinal direction during the previous control cycle, which is determined by the processing terminal by calculating the product of the absolute longitudinal speed and the control cycle duration.

[0082] Vertical operating distance refers to the total operating distance of the UAV in the vertical direction during the previous control cycle, which is determined by the processing terminal by calculating the product of the absolute vertical speed and the control cycle duration.

[0083] Step S502: Calculate the sum of the lateral running distance, longitudinal running distance, and vertical running distance with the preset safety redundancy distance, respectively, to determine the lateral update distance, longitudinal update distance, and vertical update distance.

[0084] The safety redundancy distance refers to the safe buffer distance set for UAV mapping updates. It is also used to compensate for distance deviations caused by flight speed fluctuations, distance positioning errors, and pose interpolation errors. Operators first conduct mapping experiments based on the UAV's maximum flight speed, braking response time, and UAV mapping update cycle to determine the UAV's gliding distance offset and braking delay extra movement distance within a single mapping cycle at different flight speeds. The maximum offset data is taken as the basic redundancy distance. Then, a pre-experiment is conducted on the UAV's offline flight control based on the basic redundancy distance, setting experimental conditions identical to the UAV's actual operating environment. The basic redundancy distance is then fine-tuned based on the measured results. During actual operation, the safety redundancy distance can be adjusted synchronously to increase or decrease according to the density of obstacles in the actual operating site. This allows the safety redundancy distance to be superimposed in each axial direction, ensuring that the sliding mapping window covers the UAV's next cycle of movement and reducing the possibility of missed detection of obstacle point clouds due to positional deviations, thereby improving the stability of the UAV operation process.

[0085] The lateral update distance refers to the lateral operating distance that includes the safety redundancy distance, which is determined by the processing terminal by calculating the sum of the lateral operating distance and the safety redundancy distance.

[0086] The longitudinal update distance refers to the longitudinal operating distance that includes the safety redundancy distance, which is determined by the processing terminal by calculating the sum of the longitudinal operating distance and the safety redundancy distance.

[0087] The vertical update distance refers to the vertical operating distance that includes the safety redundancy distance, which is determined by the processing terminal by calculating the sum of the vertical operating distance and the safety redundancy distance.

[0088] Step S503: Calculate the product of the horizontal update distance, the vertical update distance, and the longitudinal update distance to determine the map update volume.

[0089] The map update volume is consistent with the map update volume in step S400, and is determined by the processing terminal by calculating the product of the horizontal update distance, the vertical update distance, and the longitudinal update distance.

[0090] Reference Figure 6 The steps for determining UAV mapping data include: updating the effective point cloud data through confidence fusion based on the UAV's center coordinates and optimal voxel size. Step S600: Extract effective point cloud data based on the UAV center coordinates, the preset sliding update window, and the preset redundant update distance to determine the window point cloud data.

[0091] The sliding update window refers to the boundary of a cubic space with a fixed size centered on the center point of the UAV body, used to extract point cloud data. The side length of the sliding update window is determined by the operator based on twice the product of the UAV's maximum operating speed and the flight control cycle duration.

[0092] The redundant update distance is consistent with the safe redundancy distance in step S502, and is used to perform redundant updates on the UAV mapping map to improve the stability of the UAV operation process.

[0093] Window point cloud data refers to the effective point cloud data obtained after being captured by a sliding update window. The processing terminal determines the basic data capture range by taking the center coordinates of the UAV as the center and the sliding update window as the window range. Then, the basic data capture range is incrementally expanded in all directions according to the redundant update distance. Finally, the effective point cloud data is captured and determined according to the expanded window range.

[0094] Step S601: Obtain the point cloud center distance and point cloud echo data corresponding to the window point cloud data.

[0095] Among them, the point cloud center distance refers to the distance between each point in the window point cloud data and the center coordinates of the UAV body, which is determined by the processing terminal by calculating the Euclidean distance between each point in the window point cloud data and the center coordinates of the UAV.

[0096] Point cloud echo data refers to the echo intensity and signal-to-noise ratio data of each point cloud in the window point cloud data, which is determined by the processing terminal by retrieving the echo intensity and echo data signal-to-noise ratio of each window point cloud data.

[0097] Step S602: Evaluate the window point cloud data based on the preset confidence evaluation rules, point cloud center distance, and point cloud echo data to determine the laser confidence weight and millimeter wave confidence weight.

[0098] The confidence assessment rule refers to determining a quantitative evaluation function for the confidence level of each point cloud data based on the point cloud center distance, echo intensity, and echo signal-to-noise ratio of the point cloud data. This function is calibrated by operators through offline pre-experiments. During the pre-experiments, various common UAV operating conditions are set, including multiple point cloud ranging intervals, different types of obstacles, different levels of detection data signal-to-noise ratio, and clutter interference. Point cloud detection pre-experiments are performed under multiple operating conditions, simultaneously collecting the original echo, distance data, and high-precision positioning true values ​​from the sensors. The measurement error between the point cloud observation data and the true data under each operating condition is determined. Then, using the point cloud center distance, echo intensity, and echo signal-to-noise ratio of the point cloud data as independent variables, and the confidence level (the inverse mapping of the data true value error) as the output dependent variable, an initial piecewise mapping function for the confidence level is fitted. Multiple fusion mapping pre-experiments are then set up to iteratively optimize the piecewise threshold and coefficients of the confidence level piecewise function until the sensor modeling error converges to the calibration range. Finally, the evaluation function determined after iteration is used as the confidence assessment rule.

[0099] Laser confidence weight refers to the confidence weight of each laser point cloud in the window point cloud data. It is determined by the processing terminal by first inputting the point cloud center distance and point cloud echo data of each laser point cloud into the confidence evaluation rules to determine the confidence of each laser point cloud, and then selecting the highest confidence to normalize the confidence of the laser point cloud.

[0100] Millimeter-wave confidence weight refers to the confidence weight of each millimeter-wave point cloud in the window point cloud data. The processing terminal first inputs the point cloud center distance and point cloud echo data of each millimeter-wave point cloud into the confidence evaluation rules to determine the confidence of each millimeter-wave point cloud, and then selects the highest confidence to normalize the confidence of the millimeter-wave point cloud.

[0101] Step S603: Divide the window point cloud data according to the optimal voxel size to determine the voxel laser data and voxel millimeter wave data.

[0102] Among them, voxel laser data refers to the laser point cloud data in the window point cloud data after being divided into a three-dimensional uniform grid with optimal voxel size. The processing terminal divides the data window by setting a three-dimensional grid with optimal voxel size, traverses all laser point clouds in the window point cloud data, matches the corresponding voxel grid number according to the coordinates of each laser point cloud, integrates all laser point clouds whose coordinates fall into the same grid, and finally forms independent voxel laser data in each voxel grid.

[0103] Voxel millimeter-wave data refers to millimeter-wave point cloud data in window point cloud data after being divided into a three-dimensional uniform grid with optimal voxel size. The processing terminal divides the data window into three-dimensional grids with optimal voxel size, traverses all millimeter-wave point clouds in the window point cloud data, matches the corresponding voxel grid number according to the coordinates of each millimeter-wave point cloud, integrates all millimeter-wave point clouds whose coordinates fall into the same grid, and finally forms independent voxel millimeter-wave data within each voxel grid.

[0104] Step S604: Determine the voxel laser data and voxel millimeter-wave data corresponding to the largest laser confidence weight and the largest millimeter-wave confidence weight as the current laser data and the current millimeter-wave data.

[0105] The current laser data refers to the laser point cloud within each voxel grid in the current point cloud detection data, which is used for the final mapping. The processing terminal selects the voxel laser data corresponding to the largest laser confidence weight in each voxel grid by traversing the laser confidence weight of the laser point cloud in each voxel grid.

[0106] The current millimeter-wave data refers to the millimeter-wave point cloud within each voxel grid in the current point cloud detection data, which is used for the final mapping. The processing terminal selects the voxel millimeter-wave data corresponding to the largest millimeter-wave confidence weight in each voxel grid by traversing the millimeter-wave confidence weight of the millimeter-wave point cloud in each voxel grid, and uses it as the current millimeter-wave data.

[0107] Step S605: Analyze the current laser data and current millimeter-wave data to determine the UAV mapping data.

[0108] The UAV mapping data is consistent with the UAV mapping data in step S104, and is determined by the processing terminal through analysis of the current laser data and current millimeter-wave data. Specific analysis steps are detailed below. Figure 7 The steps in the process.

[0109] Reference Figure 7 The steps for analyzing current laser and millimeter-wave data to determine UAV mapping data include: Step S700: Perform numerical analysis on the current laser data, current millimeter wave data, preset laser occupancy probability, and preset millimeter wave occupancy probability to determine the historical overlap probability, logarithmic laser probability, and logarithmic millimeter wave probability.

[0110] The laser occupancy probability refers to the basic probability of an obstacle's existence when the lidar detects a forward obstacle and determines the laser point cloud data. It is determined by the operator through offline pre-experiments. During the offline experiments, different detection distances, dust interference levels, etc., are set, and multi-condition operation scenarios are stratified for calibration experiments. The obstacle detection result errors corresponding to different states of point cloud data, such as sparse laser point clouds, long-distance laser point clouds, and laser point clouds with low echo intensity and low signal-to-noise ratio, are determined in each scenario. The distance threshold, echo intensity threshold, and signal-to-noise ratio threshold when the detection error is clearly stratified are determined as the point cloud state segmentation boundary. The occupancy probability of point clouds in different states is set according to the different detection error conditions corresponding to different segmentation boundaries, and the occupancy probability is associated with the point cloud state, which is the laser occupancy probability.

[0111] Millimeter-wave occupancy probability refers to the basic probability of an obstacle's existence when a millimeter-wave radar detects a forward obstacle and determines the millimeter-wave point cloud data. It is determined by the operator through offline pre-experiments. During the offline experiments, different detection distances, dust interference levels, etc., are set, and multi-condition operation scenarios are used for layered calibration experiments to determine the obstacle detection error corresponding to different states of point cloud data such as sparse millimeter-wave point clouds, long-distance millimeter-wave point clouds, and millimeter-wave point clouds with low echo intensity and low signal-to-noise ratio in each scenario. The distance threshold, echo intensity threshold, and signal-to-noise ratio threshold when the detection error is clearly layered are determined as the point cloud state segmentation boundary. The occupancy probability of point clouds in different states is set according to the different detection error conditions corresponding to different segmentation boundaries, and the occupancy probability is associated with the point cloud state, which is the millimeter-wave occupancy probability.

[0112] Historical overlap probability refers to the logarithmic probability that the voxel in the laser point cloud data corresponding to the historical voxel data that overlaps with the voxel location of the current point cloud detection data is an obstacle, and the logarithmic probability that the voxel in the millimeter-wave point cloud data is an obstacle. The processing terminal first compares the global coordinates of each voxel grid corresponding to the current laser data and the current millimeter-wave data with the coordinates of each voxel in the historical mapping data to locate the historical voxel boundary of the current voxel data and the historical voxel data, and then extracts each historical overlapping voxel determined in the historical mapping process.

[0113] Logarithmic laser probability refers to the logarithmic value of the probability of obstacle occupancy represented by laser point cloud data in each voxel grid in the real-time voxel data corresponding to the current mapping process. It is determined by the processing terminal by first retrieving the laser confidence weight corresponding to each voxel grid based on the current laser data, then determining the laser occupancy probability of the laser point cloud in the voxel grid based on the echo intensity, point cloud distance, and echo signal-to-noise ratio of the laser data in the voxel grid, and finally calculating the logarithmic value of the product of the laser occupancy probability and the laser confidence weight.

[0114] Logarithmic millimeter-wave probability refers to the logarithmic value of the probability of obstacle occupancy represented by laser point cloud data in each voxel grid in the real-time voxel data corresponding to the current mapping process. It is determined by the processing terminal by first retrieving the millimeter-wave confidence weight corresponding to each voxel grid based on the current millimeter-wave data, then determining the millimeter-wave occupancy probability of the millimeter-wave point cloud in the grid based on the echo intensity, point cloud distance, and echo signal-to-noise ratio of the millimeter-wave data in the grid, and finally calculating the logarithmic value of the product of the millimeter-wave occupancy probability and the millimeter-wave confidence weight.

[0115] Step S701: Interpolate and fuse the historical overlap probability, logarithmic laser probability, and logarithmic millimeter wave probability to determine the fused laser probability and fused millimeter wave probability.

[0116] The fused laser probability refers to the occupancy probability of the laser data corresponding to the newly added voxel grid, obtained by fusing historical overlap probabilities. This includes the logarithmic laser probability of non-overlapping voxel grids and the fused probability of voxel grids at boundary overlaps. It is used to eliminate abrupt changes in laser probability at the junction of old and new maps. The processing terminal first compares historical voxel grids with the newly added voxel grids to determine completely non-overlapping voxel grids, overlapping voxel grids at boundary intersections, and historical voxel grids with boundary overlaps. For completely non-overlapping voxel grids, the current logarithmic laser probability is used as the fused laser probability. Internal voxel grids that completely overlap with historical maps but are not at the boundary are removed. No duplicate modeling is performed in fully overlapping areas. For boundary voxels, interpolation fusion calculation is initiated. During the interpolation operation, the original interpolation data is the logarithmic laser probability of historical overlapping voxels. The current logarithmic laser probability obtained at the boundary determined in real time is used as the interpolation adjustment factor, which is the historical overlap probability of the voxel. During the interpolation operation, the current optimal voxel size is used as the interpolation benchmark. Spatial resampling mapping interpolation is performed on the old size voxels that overlap at the boundary. The logarithmic laser probability of the old voxels is converted through three-dimensional linear spatial interpolation and projected into the boundary voxel grid of the current real-time voxel size. Then, the historical overlap probability is used as the historical data weight, and the complement of the historical overlap probability is used as the current data weight. The historical logarithmic probability is multiplied by the corresponding weight and the current logarithmic probability is multiplied by the corresponding weight to complete the linear interpolation calculation. The output result is the smoothed fused laser probability at the boundary. Thus, the proportion of the two sets of data in the interpolation result is dynamically adjusted according to the historical overlap probability to eliminate the difference between the new and old probability values ​​of voxels at the boundary between the newly built map and the historical map.

[0117] The fusion millimeter-wave probability refers to the occupancy probability of the millimeter-wave data corresponding to the newly added voxel grid obtained by fusing historical overlap probabilities. It includes the logarithmic millimeter-wave probability of non-overlapping voxel grids and the fusion probability of voxel grids at boundary overlaps. It is determined by the processing terminal using the same calculation method as the fusion laser probability, except that the laser data is replaced with millimeter-wave data.

[0118] Step S702: Perform weighted fusion of the laser fusion probability and the millimeter wave fusion probability to determine the initial fusion probability.

[0119] The initial fusion probability refers to the overall obstacle occupancy probability in the newly added voxels after weighted fusion. It is determined by the processing terminal through weighted fusion of the fused laser probability and the fused millimeter-wave probability. The weighting weights during weighted fusion are determined by the operator through offline pre-experiments. In the pre-experiments, multiple common operating conditions of the UAV operation are selected. Under multiple operating conditions, the fused laser probability, fused millimeter-wave probability and actual obstacle occupancy of the corresponding voxel grids of the laser point cloud detection device and the millimeter-wave point cloud detection device are collected simultaneously. The error difference between modeling with laser point cloud alone, modeling with millimeter-wave point cloud alone, and modeling with laser point cloud and millimeter-wave point cloud using different fusion weights are determined. The mapping is compared based on the error difference. The weighted fusion weight with the smallest average error difference determined by multiple experiments under different test operating conditions is the final selected weighted weight.

[0120] Step S703: Store the initial fusion probability in a circular cache to determine the initial fusion map.

[0121] The initial fusion map refers to the newly added voxel initial obstacle occupancy probability calibration map within this cycle. The processing terminal first divides the current mapping data into independent storage blocks in a ring cache. For the generated mapping data, historical data exceeding the ring cache size is deleted. The newly added voxel grid data and the initial fusion probability representing the obstacle occupancy probability corresponding to each voxel grid are stored in the newly added independent storage blocks. For the boundary between the new voxel grid and the historical voxel grid, the historical voxel grids with inconsistent sizes are first resampled and mapped to the optimal voxel grid scale determined in this mapping through three-dimensional spatial interpolation, completing the smooth transition and alignment of the voxel grids in the boundary area between the new and old mappings. Then, the calculated occupancy probability at the boundary is associated with the boundary voxel grid. Finally, the voxel grids with occupancy probabilities stored in the ring, the global coordinates corresponding to each voxel grid, and the obstacle occupancy probability corresponding to each voxel grid are integrated to form the initial fusion map.

[0122] Step S704: Perform obstacle inflation expansion on the initial fused map to determine obstacle inflation data.

[0123] The obstacle expansion data refers to the set of voxel data obtained by expanding the buffer area outward from the initial fusion map. It includes the original voxel spatial coordinates, the buffer area spatial coordinates, and the occupancy probability of the voxels. The processing terminal first filters the initial fusion probability of each voxel grid according to a preset occupancy probability threshold to determine the obstacle voxel grids whose initial fusion probability is greater than the occupancy probability threshold. Then, it expands outward from each obstacle voxel grid as the center according to a preset safe expansion radius. After determining the voxel grids that overlap with the 3D expansion area, the overlapping voxel grids are marked as safe buffer grids. The safe buffer grid is then assigned a value according to the fusion occupancy probability of the obstacle voxel grid closest to it. Finally, the global coordinates of the obstacle voxel grids, the global coordinates of the safe buffer grids, and the fusion occupancy probabilities of the obstacle voxel grids and the safe buffer grids are integrated and determined.

[0124] The occupancy probability threshold refers to the threshold for determining voxel obstacles. If the fusion occupancy is higher than this threshold, the voxel mesh is identified as an obstacle voxel mesh. This threshold is determined by the operator through offline pre-experiments. During the experiment, a mapping pre-experiment is conducted under conditions consistent with the UAV's operating environment to determine the fusion occupancy probability of each voxel mesh determined during the mapping process and the true value of the occupancy status of each voxel mesh. Different thresholds are set for filtering, and the obstacle false detection rate and obstacle missed detection rate between the mapping data obtained under each filtering threshold and the actual obstacle situation are determined. Combined with the UAV's flight control planning, the threshold that maximizes the UAV's flight stability and meets the UAV's flight control accuracy is determined through multiple pre-experiments. This threshold is the occupancy probability threshold.

[0125] The safe expansion radius refers to the expansion radius corresponding to the safe expansion outward from the voxel grid occupied by the obstacle. The operator first determines multiple sets of candidate initial expansion radii based on the width of the UAV body, the rotation range of the UAV propellers, and the positioning deviation of the UAV. Then, based on different initial expansion radius settings, obstacle avoidance flight control pre-experiments are conducted in the same working environment as the actual operation of the UAV to determine the obstacle collision probability and the safe passage ratio of the UAV for different initial expansion radii. After multiple experiments, the initial expansion radius that maximizes the safe passage ratio while meeting the obstacle collision probability standard is determined, which is the safe expansion radius.

[0126] Step S705: Perform Euclidean symbolic distance field calculation on the obstacle expansion data to determine the ESDF distance field data.

[0127] Among them, ESDF distance field data refers to the three-dimensional occupied voxel data of the signed distance field, that is, the shortest Euclidean distance from the center coordinates of all voxel grids to the boundary of the nearest inflated obstacle. If the distance is negative, it means that the voxel is inside the obstacle, and if the distance is positive, it means that the voxel is outside the obstacle. It is used to provide data support for UAV flight path planning. The processing terminal traverses all voxel grids in the map through multiple rounds of three-dimensional iterations, calculates the shortest Euclidean distance from the midpoint coordinates of all voxel grids in the mapping data to the boundary of the obstacle in the obstacle inflated data, and assigns positive and negative values. If the voxel center is inside the obstacle, the distance is negative, and otherwise the distance is positive. Finally, the calculated and determined signed distance data and the voxel grids corresponding to each signed distance data are integrated and determined.

[0128] Step S706: Integrate obstacle inflation data and ESDF distance field data to determine UAV mapping data.

[0129] The UAV mapping data is consistent with the UAV mapping data in step S605. The processing terminal integrates the obstacle expansion data and ESDF distance field data after determining the obstacle expansion data and ESDF distance field data.

[0130] Based on the same inventive concept, embodiments of this application provide a UAV mapping system based on multi-source fusion, including: The acquisition module is used to acquire multi-source detection data, UAV pose data, forward running speed, UAV center coordinates, static extrinsic parameter matrix, UAV estimated position, UAV estimated altitude, point cloud center distance, and point cloud echo data. Memory, used to store the program of the UAV mapping method based on multi-source fusion; The processor and memory can load and execute programs to implement a UAV mapping method based on multi-source fusion.

[0131] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0132] This application provides a computer-readable storage medium storing a computer program that can be loaded and executed by a processor for a UAV mapping method based on multi-source fusion.

[0133] Computer storage media include, for example, USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media that can store program code.

[0134] Based on the same inventive concept, embodiments of this application provide a smart terminal, including a memory and a processor, wherein the memory stores a computer program that can be loaded and executed by the processor for a UAV mapping method based on multi-source fusion.

[0135] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0136] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A UAV mapping method based on multi-source fusion, characterized in that, include: Acquire multi-source detection data, UAV pose data, and forward running speed; Analyze multi-source detection data and UAV pose data to determine effective point cloud data; The forward running speed and effective point cloud data are analyzed to determine the optimal voxel size; Obtain the center coordinates of the drone; The effective point cloud data is updated by confidence fusion based on the UAV center coordinates and optimal voxel size to determine the UAV mapping data.

2. The UAV mapping method based on multi-source fusion according to claim 1, characterized in that, The steps for analyzing multi-source detection data and UAV pose data to determine valid point cloud data include: Data filtering is performed on multi-source detection data to determine valid multi-source data; Interpolation data extraction is performed on the UAV pose data based on valid multi-source data to determine the synchronized pose data; Obtain the static extrinsic parameter matrix; Based on the static extrinsic matrix, coordinate transformation is performed on the effective multi-source data to determine the multi-source data of the organism; Perform matrix transformation on the synchronized pose data to determine the real-time pose matrix; The coordinate transformation of the multi-source data of the body is performed based on the real-time pose matrix to determine the effective point cloud data.

3. The UAV mapping method based on multi-source fusion according to claim 1, characterized in that, The steps for analyzing forward velocity and effective point cloud data to determine the optimal voxel size include: Obtain the estimated position and altitude of the drone; Calculate the distance between the drone's estimated position and the effective point cloud data to determine the distance to obstacle points; Numerical analysis of obstacle distances is performed to determine the minimum obstacle distance; The minimum obstacle distance, forward running speed, and estimated drone altitude were analyzed to determine the optimal voxel size.

4. The UAV mapping method based on multi-source fusion according to claim 3, characterized in that, The steps to determine the optimal voxel size by analyzing the minimum obstacle distance, forward speed, and estimated drone altitude include: Analyze the forward running speed to determine the map update volume; The target gear voxel is determined based on the minimum obstacle distance and the drone's estimated altitude. Input the map update volume, the preset data mapping time, and the preset single-voxel processing time into the preset voxel matching model to determine the theoretical mapping voxels; The maximum value between the target voxel and the theoretical mapping voxel is taken as the optimal voxel size.

5. The UAV mapping method based on multi-source fusion according to claim 4, characterized in that, The steps for analyzing the forward velocity to determine the map update volume include: The forward running speed is decomposed to determine the absolute lateral speed, absolute longitudinal speed, and absolute vertical speed; Calculate the products of absolute lateral speed, absolute longitudinal speed, and absolute vertical speed with the preset control cycle duration to determine the lateral running distance, longitudinal running distance, and vertical running distance; The sum of the lateral running distance, longitudinal running distance, and vertical running distance and the preset safety redundancy distance are calculated respectively to determine the lateral update distance, longitudinal update distance, and vertical update distance; Calculate the product of the horizontal update distance, the vertical update distance, and the longitudinal update distance to determine the map update volume.

6. The UAV mapping method based on multi-source fusion according to claim 1, characterized in that, The steps for updating the effective point cloud data based on the UAV center coordinates and optimal voxel size to determine the UAV mapping data include: The effective point cloud data is extracted based on the UAV center coordinates, the preset sliding update window, and the preset redundant update distance to determine the window point cloud data. Obtain the point cloud center distance and point cloud echo data corresponding to the window point cloud data; The window point cloud data is evaluated based on the preset confidence assessment rules, point cloud center distance, and point cloud echo data to determine the laser confidence weight and millimeter wave confidence weight. The window point cloud data is divided according to the optimal voxel size to determine the voxel laser data and voxel millimeter wave data; The voxel laser data and voxel millimeter-wave data corresponding to the largest laser confidence weight and the largest millimeter-wave confidence weight are determined as the current laser data and the current millimeter-wave data; Analyze the current laser data and current millimeter-wave data to determine the UAV mapping data.

7. The UAV mapping method based on multi-source fusion according to claim 6, characterized in that, The steps for analyzing current laser and millimeter-wave data to determine UAV mapping data include: Numerical analysis is performed on the current laser data, current millimeter-wave data, preset laser occupancy probability, and preset millimeter-wave occupancy probability to determine the historical overlap probability, logarithmic laser probability, and logarithmic millimeter-wave probability. Interpolate and fuse the historical overlap probability, logarithmic laser probability, and logarithmic millimeter wave probability to determine the fused laser probability and fused millimeter wave probability. The fusion probabilities of laser and millimeter wave are weighted and fused to determine the initial fusion probability. The initial fusion probabilities are stored in a circular cache to determine the initial fusion map; The initial fused map is expanded with obstacle inflation to determine the obstacle inflation data; Euclidean symbolic distance field calculations were performed on the obstacle dilatation data to determine the ESDF distance field data; By integrating obstacle inflation data and ESDF distance field data, we can determine the UAV mapping data.

8. A UAV mapping system based on multi-source fusion, characterized in that, include: The acquisition module is used to acquire multi-source detection data, UAV pose data, forward running speed, and UAV center coordinates. A memory for storing the program of the UAV mapping method based on multi-source fusion as described in any one of claims 1 to 7; The processor and the program in the memory can be loaded and executed by the processor to implement the UAV mapping method based on multi-source fusion as described in any one of claims 1 to 7.

9. A smart terminal, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in any one of claims 1 to 7, which is based on the UAV mapping method of multi-source fusion.

10. A computer-readable storage medium, characterized in that, The computer program is stored and can be loaded by a processor and executed as described in any one of claims 1 to 7.

Images