Helicopter obstacle avoidance method based on 4D millimeter wave radar and laser radar

Abstract

The invention discloses a helicopter obstacle avoidance method based on a 4D millimeter wave radar and a laser radar, and the technologies are innovatively combined and applied to the field of helicopter obstacle avoidance by combining an MIMO antenna, a waveform agility design, an advanced data processing algorithm and a three-dimensional obstacle avoidance strategy based on machine learning. The obstacle avoidance capability of the helicopter in a complex environment is improved; through space-time registration and data fusion of multiple sensors, the data processing burden of a single sensor is reduced, and meanwhile, the speed and precision of target detection, tracking and classification are improved; through the three-dimensional obstacle avoidance strategy and controller design, the flight path of the helicopter is optimized, the detour distance and energy consumption are reduced, and the obstacle avoidance efficiency is improved. Through multi-sensor fusion, an advanced radar technology, an obstacle avoidance strategy and a data processing algorithm, the autonomous obstacle avoidance capability of the helicopter in a complex environment is improved, and the flight safety and efficiency are ensured.

Description

Technical Field

[0001] The present invention belongs to the technical fields of radar anti-jamming, multi-sensor fusion, and helicopter obstacle avoidance, and particularly relates to a helicopter obstacle avoidance method based on a 4D millimeter-wave radar and a lidar. Background Art

[0002] During the flight of a helicopter, especially during low-altitude flight, in urban environments, or under adverse weather conditions, it faces many obstacle avoidance challenges. Traditional obstacle avoidance systems often rely on a single sensor, such as a lidar or a millimeter-wave radar, and these sensors have certain limitations in detection range, resolution, or environmental adaptability. In addition, existing obstacle avoidance strategies mostly focus on the two-dimensional plane and lack effective planning for obstacle avoidance paths in the vertical direction. Therefore, it is of great significance to develop a helicopter obstacle avoidance system and its data processing method that integrates the advantages of multiple sensors and has three-dimensional obstacle avoidance capabilities. Summary of the Invention

[0003] Object of the Invention: The present invention proposes a helicopter obstacle avoidance method based on a 4D millimeter-wave radar and a lidar to improve the accuracy of target detection, tracking, and classification, and ensure that the helicopter can accurately identify and avoid obstacles during flight.

[0004] Technical Solution: A helicopter obstacle avoidance method based on a 4D millimeter-wave radar and a lidar according to the present invention specifically includes the following steps:

[0005] (1) Adopt MIMO antenna technology to virtually create more receiving antenna channels through a specific array form; adopt waveform design with intra-pulse phase coding and inter-pulse frequency agility to reduce the correlation between target echo signals and interference signals, and suppress deceptive interference and narrowband suppression interference;

[0006] (2) Collect target position point cloud data in the helicopter flight environment based on a multi-sensor system; perform statistical filtering on the point cloud data to remove noise and outliers; and use the DBSCAN clustering algorithm to cluster the point cloud data; the multi-sensor system includes a 4D millimeter-wave radar, a lidar, an IMU, a camera, and an RTK-GPS;

[0007] (3) Adopt a Kalman filter and an IMM-TF algorithm to solve the time synchronization problem between different sensors; solve the coordinate system conversion problem between the 4D millimeter-wave radar, the lidar, and the IMU through a residual estimation registration method;

[0008] (4) Visualize the 4D millimeter-wave radar data as a two-dimensional heat map, use a frame-by-frame calibration tool to accurately calibrate the coordinate systems of the 4D millimeter-wave radar and the lidar to ensure the accuracy of 3D bounding box labels; ensure the consistency of multi-sensor data through the calibration parameters of the lidar and the camera;

[0009] (5) Based on the improved PointPillar network, perform feature encoding and feature extraction on the 4D millimeter-wave radar point cloud to achieve high-precision 3D target detection;

[0010] (6) Adopt the Gaussian mixture probability hypothesis density algorithm GM-PHD, combined with the target model, to achieve accurate tracking of the target;

[0011] (7) Based on the Transformer-based point cloud classification network, extract the local and global features of the point cloud to achieve the classification of the target;

[0012] (8) Introduce the obstacle avoidance strategy in the vertical direction and the machine learning algorithm, design a three-dimensional obstacle avoidance path, predict the movement trend and appearance probability of obstacles in different scenarios, and plan a more reasonable path in advance.

[0013] Furthermore, the waveform design using intra-pulse phase encoding and inter-pulse frequency agility in step (1) is achieved through the following formula:

[0014]

[0015] Among them, each array element emits N pulses within a coherent processing period, the intra-pulse phase encoding length of each pulse is M, the pulse repetition period is T r , the pulse width is T p , the phase encoding symbol width is t p , the symbol width and the pulse width satisfy T p = M·t p ; φ p,n,m is the phase encoding, f p,n is the frequency of the nth pulse emitted by the sub-array, f p,n = f0 + c p,n Δf, n = 0, 1, …, N - 1, c p,n is the frequency encoding of the signal.

[0016] Furthermore, the implementation process of step (2) is as follows:

[0017] The 4D millimeter-wave radar uses the frequency-modulated continuous-wave technology to generate an intermediate-frequency signal by mixing the transmitted signal and the echo signal for ranging, velocity measurement, and angle measurement; the lidar measures the distance by measuring the frequency difference between the transmitted laser and the echo laser, and measures the velocity in combination with the Doppler effect;

[0018] Statistical filtering is performed on the point cloud data of the 4D millimeter-wave radar and lidar. The average distance d between each point and k adjacent points is calculated. Assuming that the average distance distribution after filtering follows a Gaussian distribution, if d exceeds the range (μ - ω·σ, μ + ω·σ), then the point is determined to be an outlier and removed; where μ is the global distance mean, σ is the variance, and ω is the standard deviation amplification factor; the DBSCAN clustering algorithm is used, and the neighborhood radius r and the neighborhood point threshold N are set pt , clustering the point cloud objects with similar characteristics into one category, eliminating false target points, and improving the accuracy of the subsequent processed data.

[0019] Furthermore, the implementation process of step (3) is as follows:

[0020] For multi-sensors, the relative time deviation between sensor i and sensor j at time t is defined as The relative time offset estimation between sensor 1 and other sensors is calculated through a two-stage relative time offset estimation algorithm Then the globally accurate timestamp is obtained using the following formula:

[0021]

[0022] The coordinate transformation problem between the 4D millimeter-wave radar, lidar, and IMU is solved through a residual estimation registration method; first, the maximum likelihood estimation of the complete measurement values in the common coordinate system is performed to obtain the target position estimation Then, based on the sequential filtering technology, the incomplete measurement data is sequentially updated To obtain a more accurate target position estimation

[0023]

[0024] Among them, and are the complete measurement target position estimation and covariance matrix estimation respectively; is the connection Jacobian matrix of the L sensor; is the covariance matrix composed of incomplete measurement data.

[0025] Furthermore, the implementation process of step (5) is as follows:

[0026] The original 4D radar point cloud is cropped along the coordinate axes to a specific range, the plane is divided into grids, and the three-dimensional space is divided into cubic cylinders; the dimension of each radar point cloud is expanded to generate enhanced points; through operations such as linear layers, BN layers, and ReLU, a pseudo-image of size (C, H, W) is obtained, where C represents the number of channels, and H and W represent the height and width of the pseudo-image respectively; at the same time, a coordinate attention module is added to combine the spatial and channel attention mechanisms to improve the attention to the main features;

[0027] Feature extraction is carried out using a backbone network. The sub-network from top to bottom extracts features, and then the features processed with different strides are combined through upsampling and concatenation. A single-shot detector is used to perform 3D object detection, and the prior boxes are matched with the ground truth using 2D union cross, with the bounding box height and elevation as additional regression targets to achieve high-precision 3D object detection.

[0028] Furthermore, the implementation process of step (6) is as follows:

[0029] According to the rectangular target model of the object to be measured, the rectangular extended target state ζ=(γ, x, X) is obtained. The multi-target posterior PHD at time k is represented as a Gaussian mixture, and the multi-target probability hypothesis density is approximated by weighted Gaussian components:

[0030]

[0031] where, J k-1 is the number of Gaussian components at time k - 1, is the weight of the i-th Gaussian component, and N(·; m, P) represents the probability density function of the Gaussian distribution;

[0032] By predicting and updating the target state, the multi-target predicted PHD at time k and the multi-target posterior PHD at time k are respectively represented in the form of Gaussian mixtures as D k|k-1 (x) and D k|k (x):

[0033]

[0034] Using formulas (5)(6) to iterate continuously, generate the target trajectory, ensure continuous tracking of the dynamic target, and master the change of the target's motion state.

[0035] Furthermore, the implementation process of step (7) is as follows:

[0036] The point cloud classification network based on Transformer includes two parts: encoding and decoding, including an abstraction module, a scalar attention module, and a vector attention module; the radar point cloud input contains N points, each point has d-dimensional features, and the point cloud is transformed into embedded features through input embedding operations; in the encoding part, the embedded features extract depth local aggregation information through the set abstraction module, and then extract depth local features through the vector attention module and fuse them with the global features; then the final radar point cloud features are integrated through the scalar attention module; in the decoding part, after passing through the max pooling layer, two LBR layers, and a linear layer, the classification probability is output to determine the category of the target.

[0037] Furthermore, the implementation process of step (8) is as follows:

[0038] Through the angle θ in the spherical coordinate system r (t r ) = [θ xyr (t f ), θ zr (t f )] T Calculate, combine the trend of obstacle point sets in different directions, and obtain the traveling angle θ(t f ) = [θ xy (t f ), θ z (t f )] T , and generate a detouring path for the helicopter; through the traveling angle sequence θ(t) in the time series, generate a virtual traveling path to optimize the obstacle avoidance effect; incorporate machine learning algorithms, use historical flight data and obstacle information to train the model, predict the movement trends and appearance probabilities of obstacles in different scenarios, and plan a more reasonable path in advance.

[0039] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention utilizes data fusion of multiple sensors such as 4D millimeter-wave radar, lidar, and IMU (inertial measurement unit) to improve the accuracy of target detection, tracking, and classification, and ensure that the helicopter can accurately identify and avoid obstacles during flight; through MIMO (multiple input multiple output) antenna technology and waveform agile design, improve the angular resolution and anti-interference ability of the radar, and ensure that the radar can work stably in complex environments; adopt the Gaussian mixture probability hypothesis density (GM-PHD) algorithm and the point cloud classification method based on Transformer to achieve accurate tracking and classification of targets, and ensure that the helicopter can respond to dynamic obstacles in a timely manner; design a three-dimensional obstacle avoidance controller, combine obstacle avoidance strategies in the two-dimensional and vertical directions, optimize the flight path of the helicopter, reduce the detouring distance and energy consumption, and improve the obstacle avoidance efficiency. Description of the Drawings

[0040] Figure 1 It is a schematic diagram of MIMO radar IPFA-PC waveform coding;

[0041] Figure 2 It is a flowchart of the DBSCAN algorithm;

[0042] Figure 3 It is a structural block diagram of a point cloud classification network based on Transformer;

[0043] Figure 4 It is a schematic diagram of two-dimensional obstacle avoidance. Detailed Embodiments

[0044] The present invention will be further described in detail below with reference to the accompanying drawings.

[0045] The present invention proposes a helicopter obstacle avoidance method based on 4D millimeter-wave radar and lidar, which improves the helicopter's obstacle avoidance ability in complex environments by combining MIMO antennas, waveform agile design, and advanced data processing algorithms; through spatio-temporal registration and data fusion of multi-sensors, it reduces the data processing burden of a single sensor while improving the speed and accuracy of target detection, tracking, and classification; through three-dimensional obstacle avoidance strategies and controller design, it optimizes the helicopter's flight path, reduces detour distance and energy consumption, and improves obstacle avoidance efficiency. The specific steps are as follows:

[0046] Step 1: Adopt MIMO antenna technology to virtually create more receiving antenna channels through a specific array layout. For example, in a 2-transmit 4-receive MIMO layout, the number of receiving antenna elements can be increased from 4 in a real aperture to 8, increasing the receiving aperture and improving the radar's angle measurement resolution and spatial resolution. Without increasing the number of physical antennas, it significantly improves the radar's angle measurement resolution and spatial resolution, enabling the radar to more accurately determine the target angle and position.

[0047] Adopt a waveform design of intra-pulse phase coding and inter-pulse frequency agility. The MIMO radar IPFA-PC waveform mathematical model is shown in formula (1), and the IPFA-PC waveform coding method is as Figure 1 shown; the carrier frequency of each pulse of its transmitted signal randomly jumps within the bandwidth, and parameters such as the intra-pulse phase coding length are set according to specific rules. Through this design, the correlation between the target echo signal and the interference signal is reduced, deceptive interference and narrowband suppression interference are suppressed, and the reliability of the radar in a complex electromagnetic environment is improved.

[0048]

[0049] Among them, each element emits N pulses within a coherent processing period, the intra-pulse phase coding length of each pulse is M, the pulse repetition period is T r , the pulse width is T p , the phase coding symbol width is t p , the symbol width and the pulse width satisfy T p = M·t p ; φ p,n,m is the phase coding, f p,n is the frequency of the nth pulse transmitted by the sub-array, f p,n = f0 + c p,n Δf, n = 0, 1, …, N - 1, c p,n is the frequency coding of the signal.

[0050] Step 2: Install a 4D millimeter-wave radar and a lidar at appropriate positions on the helicopter to ensure that the detection ranges of the two overlap effectively. At the same time, use a multi-sensor system such as a 4D millimeter-wave radar, lidar, IMU, camera, and RTK-GPS to collect target information in the flight environment. Among them, the 4D millimeter-wave radar uses frequency-modulated continuous-wave (FMCW) technology to generate an intermediate-frequency signal by mixing the transmitted signal and the echo signal for ranging, velocity measurement, and angle measurement; the lidar measures the range by measuring the frequency difference between the transmitted laser and the echo laser, and combines the Doppler effect for velocity measurement.

[0051] Perform statistical filtering on the point cloud data of the 4D millimeter-wave radar and the lidar. Calculate the average distance d between each point and k adjacent points. Assume that the average distance distribution after filtering follows a Gaussian distribution. If d exceeds the range (μ - ω·σ, μ + ω·σ), where μ is the global distance mean, σ is the variance, and ω is the standard deviation amplification factor, then determine that the point is an outlier and eliminate it. Use the DBSCAN clustering algorithm as shown in Figure 2 Set the neighborhood radius r and the neighborhood point threshold N pt , cluster the point cloud objects with similar characteristics into one category, eliminate false target points, and improve the accuracy of the subsequent processed data.

[0052] Step 3: Use the Kalman filter and the IMM-TF algorithm to solve the time synchronization problem between different sensors. For a multi-sensor system, the relative time deviation between sensors i and j at time t is defined as Calculate the relative time offset estimate between sensor 1 and other sensors through a two-stage relative time offset estimation algorithm. For example, Then use formula (2) to obtain a globally accurate timestamp to achieve time registration of asynchronous measurement information.

[0053]

[0054] Solve the coordinate system transformation problem between the 4D millimeter-wave radar, lidar, and IMU through the residual estimation registration method. First, perform maximum likelihood estimation on the complete measurement values in the common coordinate system to obtain the target position estimate Then, based on the sequential filtering technique, use the incomplete measurement data to sequentially update to obtain a more accurate target position estimate As shown in formula (3), thereby eliminating the sensor system bias and achieving spatial registration.

[0055]

[0056] Among them, and are the complete measurement target position estimate and the covariance matrix estimate respectively; is the connection Jacobian matrix of sensor L; It is the covariance matrix composed of incomplete measurement data.

[0057] Step 4: Collect data through multiple sensors such as high-resolution lidar, 4D millimeter-wave radar, camera, and RTK-GPS, and visualize the 4D millimeter-wave radar data as a 2D heatmap to intuitively display the radar data. Use the frame-by-frame calibration tool to accurately calibrate the coordinate systems of the 4D millimeter-wave radar and lidar to ensure the accuracy of the 3D bounding box labels. Ensure the consistency of multi-sensor data through the calibration parameters of the lidar and camera.

[0058] Step 5: Based on the improved PointPillar network, perform feature encoding on the 4D millimeter-wave radar point cloud. Crop the original 4D radar point cloud along the coordinate axes to a specific range, divide the plane into grids, and divide the three-dimensional space into cubic columns. Generate enhanced points by dimension expansion for each radar point cloud, such as adding information such as the arithmetic mean distance of the point cloud relative to all points within the column and the distance to the center offset of the column. Through operations such as linear layers, BN layers, and ReLU, obtain a pseudo-image of size (C, H, W), where C represents the number of channels, and H and W represent the height and width of the pseudo-image respectively. At the same time, add a coordinate attention module to combine spatial and channel attention mechanisms to improve the attention to main features.

[0059] Adopt a backbone network similar to Voxelnet for feature extraction. Extract features by the top-down sub-network, and then combine the features processed with different strides through upsampling and concatenation. Use the single-shot detector (SSD) setting to perform 3D object detection, match the prior box with the ground truth by 2D intersection over union (IoU), and use the bounding box height and elevation as additional regression targets to achieve high-precision 3D object detection.

[0060] Step 6: Adopt the Gaussian mixture probability hypothesis density (GM-PHD) algorithm, combined with the target model, to achieve accurate tracking of the target. Generate the target trajectory by predicting and updating the target state to ensure continuous tracking of dynamic targets.

[0061] Adopt the Gaussian mixture probability hypothesis density (GM-PHD) algorithm combined with the target model to achieve accurate tracking of the target. According to the rectangular target model of the measured object, obtain the rectangular extended target state ζ = (γ, x, X). Assume that the multi-target posterior PHD at time k can be represented as a Gaussian mixture, and approximate the multi-target probability hypothesis density by weighted Gaussian components, as shown in formula (4).

[0062]

[0063] Among them, J k-1 is the number of Gaussian components at time k-1, is the weight of the i-th Gaussian component, and N(·; m, P) represents the probability density function of the Gaussian distribution.

[0064] By predicting and updating the target state, the multi-target predicted PHD at time k and the multi-target posterior PHD at time k are respectively represented in the form of Gaussian mixtures as D k|k-1 (x) and D k|k (x), as shown in Formulas (5) and (6). By continuously iterating these formulas, the target trajectory is generated to ensure continuous tracking of the dynamic target and timely grasp of the changes in the target's motion state.

[0065]

[0066] Step 7: The point cloud classification network based on Transformer extracts the local and global features of the point cloud to achieve the classification of the target. Through the max pooling layer and the linear layer, the classification probability is output to determine the category of the target. The point cloud classification network based on Transformer consists of two parts: encoding and decoding. The main modules include the set abstraction module, the scalar attention module, and the vector attention module, as Figure 3 shown. Assume that the radar point cloud input contains N points, and each point has d-dimensional features. After the input embedding operation, the point cloud is transformed into the embedded features.

[0067] In the encoding part, the embedded features extract the deep local aggregation information through the set abstraction module, and then extract the deep local features through the vector attention module and fuse them with the global features. Then, the final radar point cloud features are integrated through the scalar attention module. In the decoding part, through the max pooling layer, two LBR layers, and a linear layer, the classification probability is output to determine the category of the target.

[0068] Step 8: On the basis of two-dimensional obstacle avoidance, introduce the obstacle avoidance strategy in the vertical direction to design a three-dimensional obstacle avoidance path. The schematic diagram of two-dimensional obstacle avoidance is as Figure 4 shown.

[0069] Expand the warning circle into a warning sphere with the helicopter as the center and the threshold l d as the radius. At this time, the passing range changes from a planar sector to a spherical sector. Through the angle θ r (t r ) = [θ xyr (t f ), θ zr (t f )] T calculate, and combine the trend of the obstacle point set in different directions to obtain the traveling angle θ(t f ) = [θ xy (t f ), θ z (t f)] T , generate the detour path of the helicopter. Generate a virtual travel path through the sequence of travel angles θ(t) in the time series to optimize the obstacle avoidance effect. Introduce machine learning algorithms. Use historical flight data and obstacle information to train the model, predict the movement trends and appearance probabilities of obstacles in different scenarios, and plan a more reasonable path in advance.

[0070] Taking the SP70 millimeter-wave radar of Hunan Nare Technology as an example, although it can identify wires within a certain distance range and avoid obstacles in advance, its characteristics such as small size, low power consumption, and high reliability are also applicable to some scenarios. However, the millimeter-wave radar has low sensitivity to lateral targets and poor detection effect on small objects. In complex helicopter obstacle avoidance scenarios, in the face of obstacles with irregular shapes and small sizes such as small unmanned aerial vehicles and tree branches, a single millimeter-wave radar may not be able to accurately detect their positions and shapes, affecting the accuracy of obstacle avoidance decisions; some lidars launched by Leishen Intelligence perform excellently in aspects such as long-distance detection and high-precision perception, and can accurately detect and identify small objects such as high-voltage wires, with a maximum detection distance of up to 2000m. However, when the lidar is simply combined with other sensors, their respective advantages are not fully utilized, and the data fusion and collaborative work effects are not ideal. The present invention proposes a helicopter obstacle avoidance system integrating a 4D millimeter-wave radar and a lidar. The 4D millimeter-wave radar can work all-weather and all-time, making up for the disadvantages of the lidar in penetration ability and anti-interference ability in bad weather. The lidar has the advantages of high precision, high resolution, and three-dimensional imaging, and can make up for the defects of the 4D millimeter-wave radar in poor detection effect on small objects and low sensitivity to lateral targets. After the two are integrated, they can detect obstacles of different types and at different distances comprehensively under various weather and lighting conditions. The 4D millimeter-wave radar adds a height detection dimension, has the ability of long-distance detection (up to more than 300 meters), high angular resolution (up to about 1 degree), and the ability to work all-weather and all-time. The lidar has extremely high range resolution (less than 5cm) and angular resolution (can reach 0.1°), and performs excellently especially when detecting close-range fine targets. By integrating the two, this system can make full use of their respective advantages to achieve more comprehensive and accurate obstacle detection in complex environments.

[0071] The present invention introduces MIMO (Multiple-Input Multiple-Output) technology in the design of the radar front end, obtains virtual receiving apertures through waveform diversity, and thus significantly increases the size of the radar's receiving aperture and the angle measurement resolution without increasing the number of physical antennas. Aiming at the problems of limited volume and limited number of channel designs in airborne radars, MIMO technology provides an effective solution and is particularly suitable for miniaturized radar applications. Design an IPFA-PC anti-jamming waveform for MIMO radar based on waveform agility, and reduce the correlation between target echo signals and interference signals through intra-pulse phase coding and inter-pulse frequency agility, thereby suppressing deceptive interference.

[0072] The present invention improves the data quality by performing preprocessing operations such as statistical filtering and clustering on the three-dimensional point cloud data of 4D millimeter-wave radar and lidar. Statistical filtering is performed on the point cloud data of 4D millimeter-wave radar and lidar to remove outliers and gross error points. The DBSCAN clustering algorithm is used to cluster the point cloud data to eliminate false target points. The Kalman filter time registration method is used to perform time registration on the asynchronous measurement information of different sensors. The residual estimation registration method is used to estimate and compensate the detection information of spatial common targets to achieve spatial registration. Combining IMU data, the obstacle avoidance path is corrected and optimized in real time. Improve the accuracy and reliability of the airborne simultaneous localization system, and provide a more accurate obstacle avoidance path planning for the helicopter.

[0073] The three-dimensional obstacle avoidance path planning and controller design method proposed by the present invention extends the two-dimensional obstacle avoidance strategy to the three-dimensional space, achieving effective obstacle avoidance for helicopters in the vertical direction. Compared with the traditional two-dimensional obstacle avoidance strategy, this method can not only effectively avoid obstacles in the vertical direction, but also find the most reasonable flight path in a complex multi-obstacle environment, reducing the detour distance and energy consumption, and significantly improving the obstacle avoidance efficiency of the helicopter. By constructing a three-dimensional obstacle avoidance strategy, the helicopter can compare in different directions simultaneously and select the flight path with the shortest detour, avoiding the problem of too long detour distance and too high energy consumption when encountering large-scale obstacles. Based on the two-dimensional obstacle avoidance schematic diagram, the warning circle is expanded into a warning sphere with the helicopter as the center of the sphere and the threshold as the radius. Combining different two-dimensional obstacle avoidance selection situations, the detour distance and the traveling direction of the helicopter are determined to obtain the traveling angle. The angles of the actual traveling trihedral angle are obtained through the two projection angles, and combined with the traveling angles obtained at different time points, a sequence of traveling angles in the time series is formed. A machine learning algorithm is introduced, and the model is trained using historical flight data and obstacle information to predict the movement trend and appearance probability of obstacles in different scenarios, and plan a more reasonable path in advance. Considering the virtual traveling path of the helicopter under the condition of obstacle avoidance based on this sequence, it is designed as an adaptive controller, which adjusts the control parameters in real time according to the flight state of the helicopter, environmental changes and obstacle avoidance requirements. The adaptive control algorithm is adopted to enable the helicopter to maintain stable flight performance and efficient obstacle avoidance ability under different flight conditions. When encountering strong air flow interference, the adaptive controller can automatically adjust the control parameters to ensure the stable flight of the helicopter and effectively avoid obstacles

[0074] The present invention has been described in detail above in conjunction with specific embodiments, but these descriptions should not be construed as limiting the present invention. Those skilled in the art understand that without departing from the spirit and scope of the present invention, various equivalent substitutions, modifications or improvements can be made to the technical solutions and their implementation manners of the present invention, and all of these fall within the scope of the present invention. The protection scope of the present invention shall be subject to the appended claims

Claims

1. A helicopter obstacle avoidance method based on 4D millimeter wave radar and lidar, characterized in that, It includes the following steps: (1) Adopt MIMO antenna technology to virtually create more receiving antenna channels through a specific array form; adopt waveform design with intra-pulse phase coding and inter-pulse frequency agility to reduce the correlation between target echo signals and interference signals, and suppress deceptive interference and narrowband suppression interference; (2) Collect target position point cloud data in the helicopter flight environment based on a multi-sensor system; perform statistical filtering on the point cloud data to remove noise and outliers; and use the DBSCAN clustering algorithm to cluster the point cloud data; the multi-sensor system includes a 4D millimeter-wave radar, a lidar, an IMU, a camera, and an RTK-GPS; (3) Adopt a Kalman filter and an IMM-TF algorithm to solve the time synchronization problem between different sensors; solve the coordinate transformation problem between the 4D millimeter-wave radar, the lidar, and the IMU through a residual estimation registration method; (4) Visualize the 4D millimeter-wave radar data as a two-dimensional heat map, and use a frame-by-frame calibration tool to accurately calibrate the coordinate systems of the 4D millimeter-wave radar and the lidar to ensure the accuracy of 3D bounding box labels; ensure the consistency of multi-sensor data through the calibration parameters of the lidar and the camera; (5) Based on the improved PointPillar network, perform feature encoding and feature extraction on the 4D millimeter-wave radar point cloud to achieve high-precision 3D object detection; (6) Adopt the Gaussian mixture probability hypothesis density algorithm GM-PHD and combine it with the target model to achieve accurate tracking of the target; (7) Based on the Transformer-based point cloud classification network, extract the local and global features of the point cloud to achieve classification of the target; (8) Introduce an obstacle avoidance strategy in the vertical direction and a machine learning algorithm to design a three-dimensional obstacle avoidance path, predict the movement trend and appearance probability of obstacles in different scenarios, and plan a more reasonable path in advance.

2. The helicopter obstacle avoidance method based on 4D millimeter wave radar and lidar according to claim 1, wherein, The waveform design with intra-pulse phase coding and inter-pulse frequency agility described in step (1) is achieved through the following formula: Among them, each array element emits N pulses within a coherent processing period. The in-pulse phase coding length of each pulse is M, the pulse repetition period is T r , and the pulse width is T p , the phase coding symbol width is t p , and the symbol width and the pulse width satisfy T p = M·t p ; φ p,n,m is the phase coding, f p,n is the frequency of the nth pulse emitted by the sub-array, f p,n = f0 + c p,n Δf, n = 0, 1, …, N - 1, c p,n is the frequency coding of the signal.

3. A helicopter obstacle avoidance method based on 4D millimeter wave radar and lidar according to claim 1, characterized in that, The implementation process of step (2) is as follows: The 4D millimeter-wave radar uses frequency-modulated continuous-wave technology to measure distance, speed, and angle by mixing the transmitted signal and the echo signal to generate an intermediate-frequency signal; the lidar measures the distance by measuring the frequency difference between the transmitted laser and the echo laser, and measures the speed in combination with the Doppler effect; Statistical filtering is performed on the point cloud data of 4D millimeter-wave radar and lidar. Calculate the average distance d between each point and k adjacent points. Assume that the average distance distribution after filtering follows a Gaussian distribution. If d exceeds the range (μ - ω·σ, μ + ω·σ), then determine that the point is an outlier and eliminate it; where μ is the global distance mean, σ is the variance, and ω is the standard deviation amplification factor; Use the DBSCAN clustering algorithm and set the neighborhood radius r and the neighborhood point threshold N pt , cluster the point cloud objects with similar characteristics into one category, eliminate false target points, and improve the accuracy of the subsequent processed data.

4. A helicopter obstacle avoidance method based on 4D millimeter wave radar and lidar according to claim 1, characterized in that, The implementation process of step (3) is as follows: For multiple sensors, the relative time deviation between sensors i and j at time t is defined as The relative time offset estimation between sensor 1 and other sensors is calculated by a two-stage relative time offset estimation algorithm Then, the globally accurate timestamp is obtained using the following formula: Solve the coordinate system conversion problem among 4D millimeter-wave radar, lidar and IMU through the residual estimation registration method; first, perform maximum likelihood estimation on the complete measurement values in the common coordinate system to obtain the target position estimation Then, based on the sequential filtering technique, use the incomplete measurement data to update sequentially Obtain a more accurate target position estimation Among them, and are the full measurement target position estimation and covariance matrix estimation respectively; is the connection Jacobian matrix of the L sensor; is the covariance matrix composed of incomplete measurement data.

5. A helicopter obstacle avoidance method based on a 4D millimeter-wave radar and a lidar according to claim 1, characterized in that The implementation process of step (5) is as follows: Crop the original 4D radar point cloud along the coordinate axes to a specific range, divide the plane into grids, and divide the three-dimensional space into cubic columns; perform dimension expansion on each radar point cloud to generate enhanced points; through operations such as linear layers, BN layers, and ReLU, obtain a pseudo-image with a size of (C, H, W), where C represents the number of channels, and H and W represent the height and width of the pseudo-image respectively; at the same time, add a coordinate attention module to combine spatial and channel attention mechanisms to improve the attention to the main features; Feature extraction is performed using a backbone network. Features are extracted by a top-down sub-network, and then features processed with different strides are combined through upsampling and concatenation. A single-shot detector is used to perform 3D object detection. The prior boxes are matched with the ground truth using 2D union intersection, and the bounding box height and elevation are used as additional regression targets to achieve high-precision 3D object detection.

6. The helicopter obstacle avoidance method based on 4D millimeter wave radar and lidar according to claim 1, characterized in that, The implementation process of step (6) is as follows: According to the rectangular target model of the object to be measured, the rectangular extended target state ζ = (γ, x, X) is obtained. The multi-target posterior PHD at time k is represented as a Gaussian mixture, and the multi-target probability hypothesis density is approximated by weighting the Gaussian components: Among them, J k-1 is the number of Gaussian components at time k - 1, is the weight of the i-th Gaussian component, and N(·; m, P) represents the probability density function of the Gaussian distribution; By predicting and updating the target state, the multi-target predicted PHD at time k and the multi-target posterior PHD at time k are respectively represented in Gaussian mixture form as D k|k-1 (x) and D k|k (x): The formulae (5) and (6) are continuously iterated to generate the target trajectory, ensuring continuous tracking of the dynamic target and mastering the change of the target's motion state.

7. A helicopter obstacle avoidance method based on 4D millimeter wave radar and lidar according to claim 1, characterized in that, The implementation process of step (7) is as follows: The point cloud classification network based on Transformer consists of two parts: encoding and decoding, including an abstraction module, a scalar attention module, and a vector attention module; The radar point cloud input contains N points, each with d-dimensional features. After the input embedding operation, the point cloud is transformed into the embedded features; In the encoding part, the embedded features extract the depth local aggregation information through the set abstraction module, and then extract the depth local features through the vector attention module and fuse them with the global features; After that, the final radar point cloud features are integrated through the scalar attention module. In the decoding part, after passing through the max pooling layer, two LBR layers, and a linear layer, the classification probability is output to determine the category of the target.

8. A helicopter obstacle avoidance method based on a 4D millimeter wave radar and a lidar according to claim 1, characterized in that, The implementation process of step (8) is as follows: Through the angle θ in the spherical coordinate system r (t r ) = [θ xyr (t f ), θ zr (t f )] T Calculate, combine the trend of obstacle point sets in different directions, and obtain the traveling angle θ(t f ) = [θ xy (t f ), θ z (t f )] T , generate the detour path of the helicopter; generate a virtual traveling path through the sequence of traveling angles θ(t) in the time series to optimize the obstacle avoidance effect; Machine learning algorithms are introduced, and the model is trained using historical flight data and obstacle information to predict the motion trend and appearance probability of obstacles in different scenarios, and plan a more reasonable path in advance.

Images