Centimeter-level positioning and intelligent obstacle avoidance system for unmanned aerial vehicle

Abstract

The invention discloses a centimeter-level positioning and intelligent obstacle avoidance system for an unmanned aerial vehicle, relates to the technical field of unmanned aerial vehicles, and solves the technical problems that the centimeter-level positioning requirement is difficult to meet, and the intelligent degree of an obstacle avoidance strategy needs to be improved in a complex and changeable environment. The high-precision positioning module is used for acquiring multi-source sensor data; the self-developed multi-satellite positioning algorithm module generates initial positioning information by adopting a self-developed multi-mode GNSS fusion algorithm and an RTK technology, combines a Kalman filtering algorithm, collaboratively optimizes state estimation by predicting and updating GNSS data, and fuses visual SLAM and laser radar data at the same time; the cloud optimization and large model prediction module uploads unmanned aerial vehicle positioning data to a cloud, and models a self-developed multi-satellite positioning algorithm by using a deep learning model; and the intelligent obstacle avoidance decision module identifies obstacle types and generates an environment perception map through deep learning based on the fused positioning data and the high-precision map.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of unmanned aerial vehicles, and particularly relates to a centimeter-level positioning and intelligent obstacle avoidance system for unmanned aerial vehicles. BACKGROUND

[0002] With the rapid development of technologies such as 5G, artificial intelligence, and the Internet of Things, emerging industries such as intelligent networked vehicles, smart cities, and low-altitude economies are experiencing explosive growth. These emerging industries have higher requirements for the precision, real-time performance, and reliability of positioning technologies.

[0003] Traditional GPS positioning technologies have been unable to meet the needs of these emerging applications. Although existing GPS positioning technologies have achieved positioning to some extent, single-sensor positioning methods cannot meet the demand for centimeter-level positioning, and the intelligence level of obstacle avoidance strategies needs to be improved when facing complex and variable environments.

[0004] Therefore, it is of great practical significance to develop a high-precision and intelligent unmanned aerial vehicle positioning and obstacle avoidance system. SUMMARY

[0005] The present application aims to at least solve one of the technical problems existing in the prior art; for this purpose, the present application proposes a centimeter-level positioning and intelligent obstacle avoidance system for unmanned aerial vehicles to solve the following technical problems:

[0006] Traditional GPS positioning technologies have been unable to meet the needs of these emerging applications. Although existing GPS positioning technologies have achieved positioning to some extent, single-sensor positioning methods cannot meet the demand for centimeter-level positioning, and the intelligence level of obstacle avoidance strategies needs to be improved when facing complex and variable environments.

[0007] To solve the above problems, the present application provides a centimeter-level positioning and intelligent obstacle avoidance system for unmanned aerial vehicles, comprising the following modules:

[0008] High-precision positioning module: including a multi-mode GNSS receiving module, an IMU module, a visual SLAM module, and a laser radar module;

[0009] Self-developed multi-satellite positioning algorithm module: using a self-developed multi-mode GNSS fusion algorithm and RTK technology to generate initial positioning information, combining a Kalman filter algorithm to optimize state estimation through prediction and updating of GNSS data, and simultaneously fusing visual SLAM and laser radar data;

[0010] Cloud optimization and large model prediction module: the unmanned aerial vehicle uploads real-time positioning data to the cloud, uses a deep learning model to model the self-developed multi-satellite positioning algorithm, optimizes model parameters through a GPU cluster, deploys the optimized model and feeds back to the local through an API;

[0011] Intelligent obstacle avoidance decision module: based on the fusion of positioning data and high-precision map, through deep learning to identify the type of obstacle and generate environment perception map, A* algorithm is used to plan the bypass path for static obstacles, and reinforcement learning is used to predict the trajectory of dynamic obstacles and generate real-time obstacle avoidance strategy, model predictive control is used to optimize the smooth trajectory, and the flight control is executed.

[0012] Preferably, the self-developed multi-satellite positioning algorithm module comprises:

[0013] Receiving sensor data of the high-precision positioning module, using the independently developed multi-mode GNSS fusion algorithm and RTK technology for preliminary fusion to generate initial positioning information;

[0014] In the prediction stage of the Kalman filter algorithm, according to the acceleration and angular velocity measured by the IMU module, combined with the state estimation at the last time, the position, velocity and attitude state information at the current time is predicted;

[0015] In the update stage of the Kalman filter algorithm, the measurement data of the multi-mode GNSS receiving module is introduced, the error covariance between the predicted value and the measured value is calculated to determine the optimal fusion weight, and the prediction result is corrected;

[0016] The local three-dimensional map data generated by the visual SLAM module and the point cloud data generated by the laser radar module are fused to optimize the positioning result.

[0017] Preferably, the multi-mode GNSS fusion algorithm comprises:

[0018] Using the multi-mode GNSS receiving module to receive signals of multiple satellite systems, the multi-mode GNSS receiving module receives signals from at least two satellite navigation systems, extracts pseudorange, carrier phase and Doppler shift data;

[0019] Using the IMU module to collect inertial measurement data and perform calibration and denoising processing;

[0020] Preliminary fusion of multi-mode GNSS data and IMU data to generate preliminary positioning results;

[0021] Receiving differential correction numbers through the data link between the reference station and the mobile station, and using the RTK algorithm to perform differential processing on the preliminary positioning results to solve the carrier phase ambiguity;

[0022] Generating initial positioning information according to the data processed by RTK.

[0023] Preferably, the determination of the optimal fusion weight by calculating the error covariance between the predicted value and the measured value comprises:

[0024] In the update stage of the Kalman filter algorithm, the data of the multi-mode GNSS receiving module is introduced;

[0025] The adaptive weighting algorithm dynamically determines the optimal fusion weight.

[0026] The update stage of the Kalman filtering algorithm further includes fusing data of the IMU module to compensate for the GNSS signal interruption.

[0027] Preferably, the adaptive weighting algorithm dynamically determines the optimal fusion weight, including:

[0028] The error covariance between the predicted value and the measured value is calculated, specifically:

[0029]

[0030] wherein, is the GNSS error covariance matrix, and are the measurement noise standard deviations of GNSS position and velocity, respectively, is an identity matrix;

[0031]

[0032] wherein, is the IMU error covariance matrix, and are the measurement noise standard deviations of IMU acceleration and angular velocity, respectively, is an identity matrix;

[0033] The adaptive weighting algorithm is obtained from the GNSS error covariance matrix and the IMU error covariance matrix, specifically:

[0034]

[0035] wherein, and are the corresponding weights of GNSS and IMU, respectively, and are the inverses of the GNSS error covariance matrix and the IMU error covariance matrix, respectively.

[0036] Preferably, the fusion of the local three-dimensional map data generated by the visual SLAM module and the point cloud data generated by the laser radar module includes:

[0037] Align the visual SLAM map coordinate system with the laser radar point cloud coordinate system and perform preprocessing operations;

[0038] ​The pose information of visual SLAM is used to provide an initial pose estimation for the lidar point cloud, and the lidar point cloud is registered with the visual SLAM map using an ICP algorithm;

[0039] According to the dynamic allocation of the sensor noise covariance matrix, the weights of visual SLAM and lidar are allocated, the feature points of visual SLAM are matched with the LiDAR point cloud features, and a hybrid map is generated by weighted fusion;

[0040] The poses of visual SLAM and lidar are jointly optimized using a graph optimization algorithm, redundant point clouds and features are removed, and a globally consistent high-precision map is output.

[0041] Preferably, the cloud optimization and large model prediction module comprises:

[0042] Real-time acquisition of positioning data by a high-precision positioning module carried by a UAV, and uploading of the data to a cloud server through 4G / 5G or satellite communication, wherein the positioning data includes real-time position, attitude and speed data;

[0043] Preprocessing of data from different sensors, including data cleaning, time synchronization and coordinate system alignment, and extraction of key features from the original data, including position offset, speed change, attitude angle and environmental features;

[0044] Storing the received data in a distributed database and classifying them according to the timestamp and UAV ID field;

[0045] Modeling of the self-developed multi-satellite positioning algorithm using a deep learning model, dividing the preprocessed data into training set, validation set and test set, and generating label data as ground truth position;

[0046] Using a GPU cluster on the cloud server, optimizing the model parameters through a loss function, deploying the optimized model to the cloud server, and providing it to the high-precision positioning module through an API interface.

[0047] Preferably, the intelligent obstacle avoidance decision module comprises:

[0048] Outputting of the UAV positioning data by the high-precision positioning module in combination with the high-precision map;

[0049] Identifying the type of obstacles through a deep learning model and generating an environment perception map containing geometric and semantic information;

[0050] Analyzing the obstacle situation around the UAV according to the high-precision positioning results and obstacle information, including:

[0051] For static obstacles, a global map and A* algorithm are used to plan a detour path; for dynamic obstacles, a reinforcement learning model is used to predict the motion trajectory and generate a real-time obstacle avoidance strategy;

[0052] The obstacle avoidance trajectory is optimized by model predictive control, a smooth flight path is generated, and control instructions are issued to the flight control system for execution.

[0053] Preferably, for dynamic obstacles, a reinforcement learning model is used to predict the motion trajectory and generate a real-time obstacle avoidance strategy, comprising:

[0054] Through multi-sensor acquisition of environmental data, a deep learning model is used to identify dynamic obstacles and extract their motion characteristics, and a reinforcement learning model is used to predict the motion trajectory of dynamic obstacles, the reinforcement learning model being a deep deterministic policy gradient model, the state space of which includes the relative speed and position of dynamic obstacles, unmanned aerial vehicle positioning data, static obstacle information in the environmental perception map and historical trajectory data.

[0055] According to the predicted obstacle trajectory and the current state of the unmanned aerial vehicle, a real-time obstacle avoidance strategy is generated.

[0056] The beneficial effects of the present application are:

[0057] The present application realizes centimeter-level positioning of the unmanned aerial vehicle through multi-mode GNSS fusion algorithm, RTK technology, Kalman filtering algorithm and fusion of visual SLAM and laser radar data, effectively improves the accuracy and stability of positioning, and can adapt to various complex environments such as urban high-rise areas, mountainous areas, forests, etc., has strong anti-interference ability and robustness.

[0058] The intelligent obstacle avoidance decision module of the present application can accurately identify the type of obstacle, generate an environmental perception map, and adopt different obstacle avoidance strategies for static and dynamic obstacles, and optimize the obstacle avoidance trajectory through model predictive control to ensure safe flight of the unmanned aerial vehicle in complex environments.

[0059] The cloud optimization and large model prediction module of the present application uses a deep learning model to model and optimize the positioning algorithm, accelerates the calculation through a GPU cluster, improves the accuracy and efficiency of the model, and at the same time can feedback the optimized model to the local in a timely manner, improving the overall performance of the system. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 The module flowchart of the present application is shown in the figure;

[0061] Figure 2 The high-precision positioning module of the present application is shown in the figure. DETAILED DESCRIPTION

[0062] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0063] Please refer to Figure 1 As shown in the drawings, the present application is a kind of unmanned aerial vehicle centimeter level positioning and intelligent obstacle avoidance system, comprising the following modules:

[0064] High-precision positioning module: including multi-mode GNSS receiving module, IMU module, visual SLAM module and laser radar module;

[0065] Self-developed multi-satellite positioning algorithm module: using independently developed multi-mode GNSS fusion algorithm and RTK technology to generate initial positioning information, combining Kalman filtering algorithm, optimizing state estimation through predicting and updating GNSS data, and simultaneously fusing visual SLAM and laser radar data;

[0066] Cloud optimization and large model prediction module: the unmanned aerial vehicle uploads the positioning data to the cloud in real time, uses the deep learning model to model the self-developed multi-satellite positioning algorithm, optimizes the model parameters through GPU cluster, deploys the optimized model and feeds back to the local through API;

[0067] Intelligent obstacle avoidance decision module: based on the fusion positioning data and high-precision map, the obstacle type is identified through deep learning and the environment perception map is generated, the A* algorithm is used to plan the bypass path for static obstacles, the reinforcement learning is used to predict the trajectory of dynamic obstacles and generate real-time obstacle avoidance strategy, the model predictive control is used to optimize the smooth trajectory, and the instruction is issued to the flight control for execution.

[0068] In one embodiment of the present application, the self-developed multi-satellite positioning algorithm module comprises:

[0069] Receiving sensor data of the high-precision positioning module, using independently developed multi-mode GNSS fusion algorithm and RTK technology for preliminary fusion to generate initial positioning information;

[0070] In the prediction stage of Kalman filtering algorithm, according to the acceleration and angular velocity measured by the IMU module, the position, velocity and attitude state information at the current time are predicted in combination with the state estimation at the last time;

[0071] In the update stage of Kalman filtering algorithm, the measurement data of multi-mode GNSS receiving module is introduced, the optimal fusion weight is determined by calculating the error covariance between the predicted value and the measured value, and the prediction result is corrected;

[0072] The local three-dimensional map data generated by the visual SLAM module and the point cloud data generated by the lidar module are fused, and the positioning result is optimized.

[0073] Specifically, in the prediction stage of the Kalman filter algorithm, a state vector is defined, which contains the position , velocity and attitude of the last time, and a covariance matrix of the last time is defined, which represents the uncertainty of state estimation; according to the acceleration measured by the IMU and the velocity of the last time, the velocity of the current time is predicted, specifically: wherein, is the time step; according to the predicted velocity of the current time and the position of the last time, the position of the current time is predicted, specifically: wherein, is the time step; according to the angular velocity predicted by the IMU and the attitude of the last time, the attitude of the current time is predicted, specifically: wherein, is the quaternion penalty operation, is the time step; a state transition matrix is defined, which describes the transition relationship of the state vector from time to time, for displacement, velocity and attitude, can be expressed as:

[0074]

[0075] wherein, is the unit matrix, is the time step;

[0076] A process noise matrix is defined, which represents the influence of system noise on the state, and the covariance matrix is updated according to the state transition matrix and the process noise matrix, specifically:

[0077]

[0078] wherein, is the state transition matrix, is the transpose of the state transition matrix, is the process noise matrix;

[0079] The predicted position , velocity and attitude combining into a predicted state vector of a current moment outputting an updated covariance matrix for a subsequent Kalman filter updating stage, the multi-mode GNSS data is specially introduced in the updating stage of the Kalman filter, and the fusion weight is dynamically determined through error covariance, which is an optimization of the traditional Kalman filter algorithm; by introducing the multi-mode GNSS data and calculating the error covariance, the optimal fusion weight is dynamically determined, and the problem that the fixed weight cannot adapt to complex environments is solved.

[0080] In one embodiment of the present application, the multi-mode GNSS fusion algorithm comprises:

[0081] using a multi-mode GNSS receiving module to receive signals of a multi-satellite system, the multi-mode GNSS receiving module receives signals from at least two satellite navigation systems, and extracts pseudorange, carrier phase and Doppler shift data;

[0082] using an IMU module to collect inertial measurement data and perform calibration and denoising processing;

[0083] preliminarily fusing the multi-mode GNSS data and the IMU data to generate a preliminary positioning result;

[0084] receiving differential corrections through a data link between a reference station and a mobile station, and using an RTK algorithm to perform differential processing on the preliminary positioning result to solve carrier phase ambiguity;

[0085] generating initial positioning information according to the data processed by the RTK.

[0086] Specifically, configure a multi-mode GNSS receiver, set up the receiver to support signal reception of satellite systems such as GPS, BeiDou, GLONASS and Galileo, and configure the receiver to track signals at corresponding frequencies according to the frequency characteristics of different satellite systems; the receiver measures the pseudorange between the satellite and the receiver by capturing the C / A code or precision code of the satellite signal, records the carrier phase value of the satellite signal for subsequent RTK processing, calculates the relative speed between the satellite and the receiver through the change of the carrier frequency, extracts satellite orbit parameters and clock correction information, etc. for subsequent positioning solution; IMU data acquisition includes collecting three-axis acceleration to reflect the acceleration of the carrier in the body coordinate system, and collecting three-axis angular velocity to reflect the rotational angular velocity of the carrier; estimate and compensate the zero bias error of the accelerometer and gyroscope through static testing, calibrate the sensor range with a known gravity field, apply a low-pass filter to remove high-frequency noise and retain valid signals; compare the UTC timestamp of the GNSS data with the local time of the IMU data The clocks are aligned to ensure the consistency of the time bases of the two. The attitude angle measured by the IMU is used to convert the acceleration and angular velocity of the body coordinate system to the navigation coordinate system. The GNSS raw observations (pseudorange, carrier phase) are directly fused with the IMU data to construct a joint state vector, and the preliminary fused position, velocity and attitude are output. The differential corrections from the base station are received through radio stations, 4G / 5G networks or the NTRIP protocol. The differential corrections include pseudorange corrections, carrier phase corrections, ionospheric / tropospheric delay corrections, etc. The satellite observations and clock correction information of the base station are extracted. The coarse position and mean difference of the receiver are calculated using the GNSS raw observations and satellite ephemeris. The pseudorange and carrier phase observations of the mobile station are corrected based on the differential corrections from the base station. The double-difference observation equation is constructed, and the carrier phase ambiguity is estimated using the least squares method or Kalman filtering. The ambiguity is fixed through an integer constraint algorithm (such as the LAMBDA method) to obtain a high-precision position solution and output centimeter-level accuracy of position and velocity.

[0087] In one embodiment of the present invention, determining the optimal fusion weight by calculating the error covariance between the predicted value and the measured value includes:

[0088] In the update phase of the Kalman filter algorithm, the multi-mode GNSS receiving module data is introduced;

[0089] The error covariance between the calculated predicted value and the measured value is calculated, and an adaptive weighting algorithm is used to dynamically determine the optimal fusion weight;

[0090] The update phase of the Kalman filter algorithm also includes fusing the data from the IMU module to compensate for GNSS signal interruptions.

[0091] Specifically, obtain the measurement data of the multi-mode GNSS receiving module , the residual between the predicted value and the measured value is calculated, specifically:

[0092] ; ; ;

[0093] wherein, is the position residual, is the velocity residual, is the multi-mode GNSS combined residual, and are the real-time position and velocity measured by the multi-mode GNSS receiving module, and are the position and velocity estimated in the prediction stage of the Kalman filtering algorithm, is the transpose;

[0094] ; ;

[0095] wherein, is the acceleration residual, is the angular velocity residual, is the IMU combined residual, and are the real-time acceleration and angular velocity measured by the IMU module, and are the acceleration and angular velocity estimated in the prediction stage of the Kalman filtering algorithm, is the transpose;

[0096] wherein, if the number of GNSS satellites is less than 4 or the signal quality is lower than a threshold value, the GNSS signal interruption is marked, the IMU data is compensated, and the velocity and position are calculated by integration using the IMU data, specifically: ; ; wherein, and are the velocity and position at the time of , and are the velocity and position at the time of , is the acceleration measured by the IMU, is the time step; the attitude is updated by gyroscope integration, specifically: wherein,

[0097] is the attitude at the time of , is the angular velocity vector measured by the IMU, is a quaternion multiplication operator, representing the rotation update of the attitude; when there is no GNSS signal, only the IMU residual error is used to update the state vector, specifically: ; wherein, and are the residual error vector and the noise covariance matrix of the IMU measurement, and are the residual error vector and the noise covariance matrix of the IMU measurement at the time ; the final positioning result is output, including the position, velocity, attitude and covariance matrix.

[0098] In one embodiment of the present application, the adaptive weighting algorithm dynamically determines the optimal fusion weight, comprising:

[0099] The error covariance between the predicted value and the measured value is calculated, specifically:

[0100]

[0101] wherein, is the GNSS error covariance matrix, and are the measurement noise standard deviations of the GNSS position and velocity respectively, is the unit matrix;

[0102]

[0103] wherein, is the IMU error covariance matrix, and are the measurement noise standard deviations of the IMU acceleration and angular velocity respectively, is the unit matrix;

[0104] The adaptive weighting algorithm is obtained through the GNSS error covariance matrix and the IMU error covariance matrix, specifically:

[0105] ;

[0106] wherein, and are the corresponding weights of GNSS and IMU respectively, and are the inverses of the GNSS error covariance matrix and the IMU error covariance matrix respectively.

[0107] In one embodiment of the present application, the fusion of the local three-dimensional map data generated by the visual SLAM module and the point cloud data generated by the laser radar module comprises:

[0108] align the map coordinate system of visual SLAM with the point cloud coordinate system of LiDAR, and perform preprocessing operations;

[0109] use the pose information of visual SLAM to provide an initial pose estimation for the LiDAR point cloud, and use the ICP algorithm to register the LiDAR point cloud with the visual SLAM map;

[0110] dynamically allocate weights to visual SLAM and LiDAR according to the sensor noise covariance matrix, match the feature points of visual SLAM with the LiDAR point cloud features, and generate a hybrid map through weighted fusion;

[0111] use a graph optimization algorithm to jointly optimize the poses of visual SLAM and LiDAR, remove redundant point clouds and features, and output a globally consistent high-precision map.

[0112] Specifically, the relative pose between the visual SLAM camera and the LiDAR is obtained through calibration, including a rotation matrix and a translation vector, and the point cloud of the LiDAR is converted from the LiDAR coordinate system to the map coordinate system of visual SLAM; the preprocessing operations include point cloud filtering and point cloud segmentation, using voxel filtering to downsample the point cloud, reducing the point cloud density, using statistical filtering to remove outliers, and using plane segmentation or clustering algorithm to divide the point cloud into ground points and non-ground points, and retaining the non-ground points for subsequent processing; the pose of the current frame is obtained from the visual SLAM system, including a rotation matrix and a translation vector, the pose of visual SLAM is used as the initial pose estimation for the LiDAR point cloud, the ICP algorithm is used to register the LiDAR point cloud with the visual SLAM map, the transformation matrix between the point clouds is calculated to align the LiDAR point cloud with the visual SLAM map, the transformation matrix calculated by ICP is multiplied by the initial pose to obtain the accurate pose of the LiDAR; the noise covariance matrix of visual SLAM and the noise covariance matrix of LiDAR, dynamically allocate weights according to the noise covariance matrix, the calculation method is the same as the above GNSS and IMU weight calculation method, the weight reflects the accuracy of the sensor, and the sensor with high accuracy has high weight; feature points are extracted from the visual SLAM map, features are extracted from the LiDAR point cloud, feature matching is performed using feature descriptors, and false matches are removed using nearest neighbor matching or RANSAC algorithm; the feature points of visual SLAM and the feature points of LiDAR are weighted and fused according to the weights, specifically:

[0113]

[0114] wherein, is the fused feature, is the feature point of visual SLAM, is the feature point of LiDAR, and Respectively, the weight corresponding to visual SLAM and lidar;

[0115] The fused feature points are stored as a hybrid map, redundant point clouds are removed using voxel filtering or clustering algorithms, and the hybrid map is optimized using a graph optimization algorithm to ensure global consistency, including taking the poses of visual SLAM and lidar as nodes and feature matching as edges, jointly optimizing the poses using a graph optimization algorithm, minimizing the error function, and obtaining the optimized pose sequence and globally consistent map; all optimized lidar point clouds and visual SLAM feature points are merged to generate a globally consistent high-precision map.

[0116] In one embodiment of the present application, the cloud optimization and large model prediction module comprises:

[0117] Real-time positioning data is collected by a high-precision positioning module carried by a UAV, and the data is uploaded to a cloud server through 4G / 5G or satellite communication, wherein the positioning data includes real-time position, attitude and speed data;

[0118] The data of different sensors is preprocessed, including data cleaning, time synchronization and coordinate system alignment, and key features are extracted from the original data, including position offset, speed change, attitude angle and environmental features;

[0119] The received data is stored in a distributed database and classified according to the timestamp and UAV ID field;

[0120] A deep learning model is used to model the self-developed multi-satellite positioning algorithm, the preprocessed data is divided into training set, validation set and test set, and the label data is generated as ground truth position;

[0121] A GPU cluster is used on the cloud server to optimize the model parameters through a loss function, the optimized model is deployed to the cloud server, and an API interface is provided to the high-precision positioning module.

[0122] In particular, the unmanned aerial vehicle is equipped with a high-precision positioning module, which collects the following data in real time, including position, speed and attitude data. The data collection frequency is set according to the task requirements. The data is uploaded to the cloud server in real time through 4G / 5G or satellite communication. The data is transmitted using MQTT protocol or HTTP protocol. The data is encapsulated in JSON or Protobuf format, including the following fields: timestamp, unique identifier of the unmanned aerial vehicle, position data, speed data and attitude data. The data is preprocessed, including detecting and removing outliers, and interpolating or filling in missing data. Align the timestamps of all sensors to UTC time using NTP or PTP protocol for time synchronization. Align the data of different sensors in time to ensure that the data is in the same time dimension. Convert the data of different sensors to a unified coordinate system, such as the WGS84 coordinate system, using a rotation matrix or quaternion for coordinate system conversion. Align the attitude data of the IMU and visual sensor with the GNSS position data. Extract features including calculating the position offset of the unmanned aerial vehicle from the preset trajectory, calculating the rate of change of the speed to obtain the speed change, extracting the change of the pitch angle, roll angle and yaw angle, and extracting environmental features from visual or lidar data. Use a distributed database to store large-scale data. Store the data according to the timestamp and UAV ID fields. Index the timestamp field to support fast query of data in a certain time period. Index the UAV ID field to support querying data by UAV. Divide the preprocessed data into training set, validation set and test set. Use the ground truth position as label data. Use a deep learning model to model the self-developed multi-satellite positioning algorithm. Input the preprocessed features and output the predicted position. Use mean squared error or smooth L1 loss function as loss function. Use Adam or SGD optimizer. Train on cloud server using GPU cluster. Monitor training loss and validation loss to prevent overfitting. Use TensorRT or ONNX Runtime to quantize and accelerate the model. Deploy the optimized model to the cloud server.

[0123] In one embodiment of the present application, the intelligent obstacle avoidance decision module includes:

[0124] The multi-sensor data collected by the high-precision positioning module is combined with the high-precision map to output the unmanned aerial vehicle positioning data.

[0125] The deep learning model identifies the type of obstacle and generates an environment perception map containing geometric and semantic information.

[0126] According to the high-precision positioning result and the obstacle information, the obstacle situation around the unmanned aerial vehicle is analyzed, including:

[0127] For static obstacles, a global map and A* algorithm are used to plan a detour path; for dynamic obstacles, a reinforcement learning model is used to predict the motion trajectory and generate a real-time obstacle avoidance strategy;

[0128] The obstacle avoidance trajectory is optimized by model predictive control, a smooth flight path is generated, and control instructions are issued to the flight control system for execution.

[0129] Specifically, a high-precision map is loaded, the data of the high-precision positioning module is matched with the high-precision map, and the precise positioning result of the unmanned aerial vehicle is output, including global positioning data and position in the local coordinate system; a pre-trained deep learning model is used to process visual or point cloud data, identify obstacle types, and the obstacle types include static obstacles and dynamic obstacles; the point cloud or image is semantically segmented, and the geometric and semantic information of the obstacle is extracted; the identified obstacle geometric and semantic information is mapped into the high-precision map, the obstacle type is labeled in the map, and an environment perception map containing geometric and semantic information is generated; wherein, based on the high-precision map and the environment perception map, the static obstacles are identified; the dynamic obstacles are identified through visual or radar data; for static obstacle detour, A* algorithm is used to plan a detour path in the global map; for dynamic obstacle avoidance, a reinforcement learning model is used to predict the motion trajectory of the dynamic obstacle, and a real-time obstacle avoidance strategy is generated according to the prediction result; model predictive control (MPC) is used to optimize the obstacle avoidance trajectory, to ensure that the flight path is smooth and meets the unmanned aerial vehicle dynamics constraints, the optimization target is to minimize the path length, avoid collision, and meet the speed and acceleration limit, the current positioning information, obstacle information and target position are input, and the optimized smooth flight path is output, and the control instructions are issued to the flight control system through MAVLink or a custom protocol.

[0130] In one embodiment of the present application, for dynamic obstacles, a reinforcement learning model is used to predict the motion trajectory and generate a real-time obstacle avoidance strategy, which includes:

[0131] The environment data is collected by multiple sensors, the dynamic obstacles are identified by a deep learning model, and the motion characteristics thereof are extracted, a reinforcement learning model is used to predict the motion trajectory of the dynamic obstacles, and the reinforcement learning model is a deep deterministic policy gradient model, and the state space thereof includes the relative speed and position of the dynamic obstacles, the unmanned aerial vehicle positioning data, the static obstacle information in the environment perception map, and the historical trajectory data;

[0132] According to the predicted obstacle trajectory and the current state of the unmanned aerial vehicle, a real-time obstacle avoidance strategy is generated. Specifically, the visual sensor collects image or point cloud data for detecting dynamic obstacles, the ultrasonic sensor collects distance information of close-range obstacles, the GNSS and IMU are used to collect the positioning information of the unmanned aerial vehicle, and the data acquisition frequency is set according to the task requirements; a pre-trained deep learning model is used to process the visual or point cloud data, identify dynamic obstacles, and perform semantic segmentation on the point cloud or image to extract the category and position information of the dynamic obstacles; the relative speed and position of the dynamic obstacles are calculated, the historical trajectory data such as the position sequence in the past few seconds are extracted, and the motion feature vector of the dynamic obstacles is constructed; the reinforcement learning model is a deep deterministic policy gradient model, the state space design includes the relative speed and position of the dynamic obstacles, the positioning data of the unmanned aerial vehicle, the static obstacle information in the environmental perception map, and the historical trajectory data of the dynamic obstacles; the action space is the obstacle avoidance strategy of the unmanned aerial vehicle, and the action space design includes speed adjustment and attitude adjustment; the reward function is designed considering the following factors, including collision avoidance (reward close to 0, high collision penalty), path smoothness (reward trajectory continuity), and task efficiency (reward fast arrival at the target); a large amount of dynamic obstacle trajectory data is generated using a simulation environment, the model is trained using the DDPG algorithm, the policy network and the value network are optimized, and the training process is stabilized through experience replay and target network; the current state is input into the DDPG model, the model outputs the obstacle avoidance strategy, and the future trajectory of the unmanned aerial vehicle is generated according to the obstacle avoidance strategy; according to the output action of the DDPG model, the obstacle avoidance strategy is selected, including turning to avoid obstacles: adjusting the yaw angle of the unmanned aerial vehicle to bypass the dynamic obstacles; speed adjustment: accelerating or decelerating to maintain a safe distance; height adjustment: climbing or descending to avoid obstacles; combined with the dynamic constraints of the unmanned aerial vehicle, such as maximum speed and maximum acceleration, the obstacle avoidance strategy is optimized, and the obstacle avoidance strategy is converted into executable instructions for the flight control system; the flight control system feeds back the state of the unmanned aerial vehicle to the cloud, and dynamically updates the prediction results of the dynamic obstacle trajectory according to the actual flight data of the unmanned aerial vehicle and the changes in the environment.

[0133] The above embodiments are only used to illustrate the technical method of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical method of the present application.

Claims

1. A centimeter-level positioning and intelligent obstacle avoidance system for drones, characterized by: Includes the following modules: High-precision positioning module: including multi-mode GNSS receiver module, IMU module, visual SLAM module and lidar module; Self-developed multi-satellite positioning algorithm module: It uses a self-developed multi-mode GNSS fusion algorithm and RTK technology to generate initial positioning information. Combined with the Kalman filter algorithm, it collaboratively optimizes state estimation by predicting and updating GNSS data, while also integrating visual SLAM and lidar data. Cloud optimization and large-model prediction module: UAVs upload positioning data to the cloud in real time, use deep learning models to build a self-developed multi-satellite positioning algorithm, optimize model parameters through GPU clusters, deploy the optimized model, and feed it back to the local server via API; Intelligent obstacle avoidance decision module: Based on the fusion of positioning data and high-precision maps, it uses deep learning to identify obstacle types and generate environmental perception maps. It uses the A* algorithm to plan detour paths for static obstacles, and uses reinforcement learning to predict trajectories for dynamic obstacles and generate real-time obstacle avoidance strategies. It optimizes smooth trajectories through model predictive control and sends instructions to the flight control for execution.

2. The centimeter-level positioning and intelligent obstacle avoidance system for unmanned aerial vehicles according to claim 1 is characterized in that: The self-developed multi-satellite positioning algorithm module includes: Receive sensor data from the high-precision positioning module, perform preliminary fusion using the independently developed multi-mode GNSS fusion algorithm and RTK technology to generate initial positioning information; In the prediction phase of the Kalman filter algorithm, the current position, velocity, and attitude state information are predicted based on the acceleration and angular velocity measured by the IMU module and the state estimation at the previous moment; In the update phase of the Kalman filter algorithm, the measurement data of the multi-mode GNSS receiving module is introduced. By calculating the error covariance between the predicted value and the measured value, the optimal fusion weight is determined and the prediction result is corrected. The local 3D map data generated by the visual SLAM module and the point cloud data generated by the lidar module are integrated to optimize the positioning results.

3. The centimeter-level positioning and intelligent obstacle avoidance system for unmanned aerial vehicles according to claim 2 is characterized in that: The multi-mode GNSS fusion algorithm includes: Using a multi-mode GNSS receiving module to receive signals from a multi-satellite system, the multi-mode GNSS receiving module receives signals from at least two satellite navigation systems and extracts pseudorange, carrier phase and Doppler shift data; Use the IMU module to collect inertial measurement data and perform calibration and denoising; Initially fuse multi-mode GNSS data with IMU data to generate preliminary positioning results; The base station receives differential corrections via the data link between the base station and the rover, and uses the RTK algorithm to perform differential processing on the preliminary positioning results to resolve the carrier phase ambiguity. Generate initial positioning information based on RTK processed data.

4. The centimeter-level positioning and intelligent obstacle avoidance system for unmanned aerial vehicles according to claim 2, characterized in that: The step of calculating the error covariance between the predicted value and the measured value to determine the optimal fusion weight includes: In the update phase of the Kalman filter algorithm, the multi-mode GNSS receiving module data is introduced; The error covariance between the calculated predicted value and the measured value is calculated, and an adaptive weighting algorithm is used to dynamically determine the optimal fusion weight; The update phase of the Kalman filter algorithm also includes fusing the data from the IMU module to compensate for GNSS signal interruptions.

5. The centimeter-level positioning and intelligent obstacle avoidance system for unmanned aerial vehicles according to claim 4 is characterized in that: The adaptive weighting algorithm dynamically determines the optimal fusion weight, including: Calculate the error covariance between the predicted and measured values, specifically:

6. Among them, is the GNSS error covariance matrix, and are the measurement noise standard deviations of GNSS position and velocity, is the identity matrix; 7. Among them, is the IMU error covariance matrix, and are the measurement noise standard deviations of IMU acceleration and angular velocity, is the identity matrix; The adaptive weighting algorithm is obtained through the GNSS error covariance matrix and the IMU error covariance matrix, specifically: ； 8. Among them, and are the corresponding weights of GNSS and IMU respectively, and are the inverses of the GNSS error covariance matrix and the IMU error covariance matrix, respectively.

9. The centimeter-level positioning and intelligent obstacle avoidance system for unmanned aerial vehicles according to claim 2, characterized in that: The fusing of the local three-dimensional map data generated by the visual SLAM module and the point cloud data generated by the lidar module includes: Align the map coordinate system of visual SLAM with the point cloud coordinate system of lidar and perform preprocessing operations; Use the pose information of visual SLAM to provide an initial pose estimate for the lidar point cloud, and use the ICP algorithm to align the lidar point cloud with the visual SLAM map; Dynamically assign weights to visual SLAM and LiDAR based on the sensor noise covariance matrix, match the feature points of visual SLAM with the LiDAR point cloud features, and generate a hybrid map through weighted fusion; A graph optimization algorithm is used to jointly optimize the poses of visual SLAM and lidar, remove redundant point clouds and features, and output a globally consistent high-precision map.

10. The centimeter-level positioning and intelligent obstacle avoidance system for unmanned aerial vehicles according to claim 1, characterized in that: The cloud optimization and large model prediction module includes: The high-precision positioning module on the drone collects positioning data in real time and uploads the data to the cloud server via 4G / 5G or satellite communication. The positioning data includes real-time position, attitude and speed data. Preprocess the data from different sensors, including data cleaning, time synchronization, and coordinate system alignment, and extract key features from the raw data, including position offset, velocity change, attitude angle, and environmental characteristics; The received data is stored in a distributed database and classified according to the timestamp and drone ID fields; Use a deep learning model to model the self-developed multi-satellite positioning algorithm, divide the pre-processed data into training sets, validation sets, and test sets, and generate labeled data as ground truth positions; Use a GPU cluster on the cloud server to optimize the model parameters through the loss function, deploy the optimized model to the cloud server, and provide it to the high-precision positioning module through the API interface.

11. The centimeter-level positioning and intelligent obstacle avoidance system for unmanned aerial vehicles according to claim 1, characterized in that: The intelligent obstacle avoidance decision module includes: The multi-sensor data collected by the high-precision positioning module is combined with the high-precision map to output the drone positioning data; Identify obstacle types through deep learning models and generate environmental perception maps containing geometric and semantic information; Analyze the obstacles around the drone based on high-precision positioning results and obstacle information, including: For static obstacles, it plans detour paths based on a global map and the A* algorithm. For dynamic obstacles, it uses a reinforcement learning model to predict motion trajectories and generate real-time obstacle avoidance strategies. The obstacle avoidance trajectory is optimized through model predictive control, a smooth flight path is generated, and control instructions are sent to the flight control system for execution.

12. The centimeter-level positioning and intelligent obstacle avoidance system for unmanned aerial vehicles according to claim 8, characterized in that: For dynamic obstacles, a reinforcement learning model is used to predict motion trajectories and generate real-time obstacle avoidance strategies, including: Environmental data is collected through multiple sensors, and a deep learning model is used to identify dynamic obstacles and extract their motion features. The motion trajectory of dynamic obstacles is predicted using a reinforcement learning model. The reinforcement learning model is a deep deterministic policy gradient model, and its state space includes the relative speed and position of dynamic obstacles, drone positioning data, static obstacle information in the environmental perception map, and historical trajectory data. Generate real-time obstacle avoidance strategies based on the predicted obstacle trajectory and the current state of the drone.

Images