An event camera based bionic perception and obstacle avoidance method for aerial vehicles

Abstract

The present application belongs to the field of autonomous navigation and intelligent perception of aerospace vehicles, and particularly relates to a kind of aerospace vehicle bionic perception and obstacle avoidance method based on event camera, including nonlinear non-warp field rotation filtering operation to initial event stream photographed by event camera, dynamic threshold segmentation and joint constraint clustering to event stream after filtering processing, constructing optical flow field of each dynamic target area, multi-target time to collision TTC fast estimation and constructing airspace hierarchical obstacle avoidance decision based on multi-target time to collision TTC, the present application overcomes the problems of error accumulation, poor generalization and insufficient real-time of existing time to collision TTC estimation method, realizes real-time dynamic target detection, collision risk accurate evaluation and emergency obstacle avoidance decision of aerospace vehicle on-board terminal, and adapts to complex space operation environment of large maneuver, strong disturbance and multi-dynamic target intersection.

Description

Technical Field

[0001] This invention belongs to the field of autonomous navigation and intelligent perception technology for aircraft, specifically relating to a biomimetic perception and obstacle avoidance method for aircraft based on an event camera. Background Technology

[0002] In the autonomous navigation operations of unmanned aerospace platforms such as aircraft and drones, real-time dynamic target perception and collision risk warning are core technical aspects to ensure the safe operation of the platform and the completion of autonomous obstacle avoidance tasks. Existing unmanned platform perception systems mostly use traditional frame-type cameras as visual sensors. These sensors acquire complete image frames at a fixed frame rate, resulting in extremely high data redundancy. In high-dynamic flight, high-speed target intersection, and high dynamic range lighting scenarios, they are prone to problems such as image motion blur, time delay, and data processing lag, making it difficult to meet the real-time perception requirements of aerospace platforms for low latency and high robustness.

[0003] Time-to-Collision (TTC), as an intuitive quantitative indicator of collision risk, does not require complex 3D environment modeling and can directly transform spatial motion relationships into risk measures in the time dimension. It has the advantages of high computational efficiency and intuitive perception and is now widely used in ground intelligent equipment such as autonomous driving active braking and advanced driver assistance systems (ADAS). It also has great application potential in the fields of aerospace transportation networks and multi-agent conflict detection and resolution.

[0004] Existing obstacle avoidance methods based on time-of-collision (TTC) can be mainly divided into two categories: one is the traditional geometric modeling method, which relies on visual geometric features such as optical flow, target bounding box scaling, and monocular depth estimation to calculate the collision time TTC. This type of method is lightweight and highly interpretable, but it has a prominent error accumulation problem and extremely poor robustness in strong disturbances and high dynamic scenes. The other type is the deep learning-based method, which achieves pixel-level collision prediction through self-supervised learning and reinforcement learning. It has high prediction accuracy, but the model has a large number of parameters, high inference latency, relies on a large amount of domain-specific training data, has limited generalization ability, and is difficult to adapt to aircraft.

[0005] Event cameras, as a type of biomimetic visual sensor, borrow from the neuromorphic processing mechanism of biological vision. They trigger asynchronous event output only when the light intensity of a pixel changes, possessing advantages such as microsecond-level temporal resolution, no motion blur, and sparse, low-redundancy data. Event cameras can accurately capture key visual features such as scene edge motion and structural abrupt changes, perfectly adapting to the perception needs of highly dynamic aerospace scenarios. Currently, research on time-of-collision (TTC) estimation and target detection based on event cameras still suffers from several technical shortcomings: existing motion compensation algorithms mostly use linear approximations, ignoring the initial position of event pixels, resulting in significant compensation errors in high-speed three-axis rotation scenarios of aircraft; dynamic target clustering is susceptible to noise interference, with poor accuracy in distinguishing adjacent targets, making it difficult to meet the real-time requirements of airborne terminals; TTC estimation often relies on global threat map construction, resulting in high computational consumption and decision-making latency, failing to achieve rapid response for local emergency obstacle avoidance. In summary, existing event camera technology cannot simultaneously meet the low-latency, high-robustness, and lightweight perception requirements of autonomous aircraft navigation, necessitating a biomimetic perception and obstacle avoidance method that aligns with biological visual mechanisms and integrates multi-source sensor data. Summary of the Invention

[0006] This invention aims to address the perception deficiencies of traditional frame sensors in highly dynamic aerospace scenarios, overcome the problems of error accumulation, poor generalization, and insufficient real-time performance of existing collision time-to-concussion (TTC) estimation methods, and provide a biomimetic perception and obstacle avoidance method for aircraft based on event cameras. This method enables real-time dynamic target detection, accurate collision risk assessment, and emergency obstacle avoidance decision-making on the aircraft's onboard end, adapting to complex aerospace operation environments with strong disturbances and multiple dynamic targets converging.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A biomimetic perception and obstacle avoidance method for aircraft based on an event camera, comprising the following steps: Step 1: Perform nonlinear non-warped field rotation filtering on the initial event stream captured by the airborne event camera: Simultaneously acquire the asynchronous event stream output by the airborne event camera and the attitude data output by the airborne inertial measurement unit to construct a nonlinear motion compensation distortion field; Use the constructed nonlinear motion compensation distortion field to remap and filter the output asynchronous event stream to achieve accurate alignment and aggregation of the dynamic target event stream and the static background event stream; Step 2: Perform dynamic threshold segmentation and joint constraint clustering on the asynchronous event stream after filtering in Step 1: Construct event count images and timestamp images, normalize the event count images and timestamp images, and use dynamic thresholds to distinguish between static background event streams and dynamic target event streams; A joint clustering cost function is constructed, and the DBSCAN algorithm is used to perform target clustering and separation on the spatiotemporal image of the asynchronous event stream after dynamic threshold segmentation. Step 3: Construct the optical flow field for each dynamic target region: The asynchronous event flow within the dynamic target region is sliced ​​into active event surfaces according to time. The spatial gradient of the constructed active event surfaces is solved and optimized using the local plane fitting method to obtain a computable optical flow field. Step 4: Fast estimation of multi-target collision time (TTC): Using the computable optical flow field obtained in Step 3, the radiation center (FOE) of the optical flow field on the image is solved by optimizing the optical flow direction, and the multi-target collision time (TTC) is estimated quickly. Step 5: Construct airspace hierarchical obstacle avoidance decision based on multi-target collision time (TTC): Based on the results of the rapid estimation of multi-target collision time (TTC) in Step 4, the flight safety risk level of the aircraft is divided by multi-target weighted scoring and a progressive obstacle avoidance strategy is executed.

[0008] The further improvement in the above technical solution is that the construction of the nonlinear motion compensation distortion field in step 1 includes adding a nonlinear term of the initial pixel coordinates of the event to the nonlinear distortion field, and performing polynomial fitting compensation for the centrifugal projection deviation.

[0009] A further improvement in the above technical solution lies in the nonlinear torsion field. The model can be represented as: assuming the event camera only orbits... The axis rotates, and the event occurs along the axis. Axis rotation, for time interval The event flow within, ,in It is a rotation matrix obtained by interpolating data from the inertial measurement unit. It is related to the initial pixel position Related nonlinear compensation terms, Represents the spatiotemporal coordinates of an event. Indicates that the event only follows The spacetime coordinates of the axis of rotation.

[0010] The further improvement of the above technical solution is that the joint clustering cost function constraint constructed in step 2 includes the spatial coordinates, time difference and optical flow vector of the event. Through the joint clustering cost function, random noise event streams that do not conform to motion consistency are eliminated, and neighboring dynamic target event streams that are spatially close but have different motion patterns are separated.

[0011] A further improvement to the above technical solution is that the joint clustering cost function is expressed as follows: in, and The event coordinates are the event coordinates. and For event timestamps, and Let be the optical flow vector of the event. , , These are the weighting coefficients for each item. The weight vector corresponds to two different event points.

[0012] A further improvement in the above technical solution is that, in step 3, the local plane fitting method includes fitting a two-dimensional plane to each event pixel in the neighborhood, solving the parameters of the two-dimensional plane by the least squares method, and then the plane gradient obtained by the two-dimensional plane fitting can be directly output as the optical flow field component.

[0013] A further improvement in the above technical solution is that, in step 4, the rapid estimation of multi-target collision time (TTC) includes solving the radiation center of the dilatation focal optical flow field of each separate dynamic target region on the image by using a joint clustering cost function, thereby generating a global collision risk map representing the multi-target collision time (TTC).

[0014] The further improvement in the above technical solution is that, in step 5, the multi-target weighted scoring refers to collecting the feature information of the detected dynamic target. The feature information includes: independent TTC value, target hazard coefficient, relative flight altitude layer, motion trend, and distance from the preset no-fly zone constraint. After normalizing the feature information, it is input into the weighted scoring function to obtain the comprehensive risk score of each dynamic target.

[0015] A further improvement in the above technical solution is that the comprehensive risk score of each dynamic target is recorded as an equivalent collision time. ,according to The value is used to classify the flight safety risk of aircraft into four levels: >5 seconds, low risk: current route is safe; 2 seconds ≤5 seconds, medium risk: potential collision threat exists; 0.5 seconds ≤2 seconds, high risk: threat is imminent, immediate intervention is required; ≤0.5 seconds, extremely high risk: a collision is imminent.

[0016] A further improvement in the above technical solution is that, in step 5, executing the progressive obstacle avoidance strategy includes executing the corresponding progressive obstacle avoidance strategy based on the classified flight safety risk level of the aircraft: Low risk: Routine patrols, maintaining the current flight plan, with only continuous monitoring; Medium risk: Warning and deceleration, issuing a warning signal, and the aircraft actively reduces its speed; High risk: Fine-tuning the trajectory, planning and executing small, low-G course corrections to bypass threatening targets at a safe distance while ensuring flight smoothness and avoiding secondary risks; Extremely high risk: Emergency avoidance, triggering the highest priority safety protection logic, executing maximum thrust climb or rapid deceleration hovering to avoid collision at the fastest speed.

[0017] Compared with the prior art, the present invention has the following advantages: First, by constructing a nonlinear distortion field that incorporates the initial coordinates of the event, the reprojection error of the edge region is reduced from 12.7 pixels to 3.1 pixels. This solves the problem that traditional linear motion compensation ignores the initial position of the event pixels, resulting in large aggregation error and severe trailing of background events far from the center of the image when the aircraft rotates at high speed.

[0018] Secondly, by introducing a joint constraint cost function of event timestamp difference and optical flow vector, the clustering separation success rate is increased from 62% to 94%, which solves the problem that traditional DBSCAN clustering relies only on spatial distance and cannot separate spatially adjacent (minimum spacing < 5 pixels) dynamic targets with different motion patterns.

[0019] Third, by using the local plane fitting regularization method, the spatiotemporal plane is fitted within a small neighborhood to directly output the optical flow, reducing the median angle error from 24.7° to 8.3°. This solves the problems of spatiotemporal gradient failure and optical flow direction ambiguity in sparse texture or low signal-to-noise ratio regions in event optical flow estimation.

[0020] Fourth, by optimizing the solution of FOE using only the optical flow direction, the standard deviation of FOE coordinate jitter is reduced from 3.5 pixels to 1.3 pixels, and the inter-frame fluctuation of collision time TTC estimation is reduced from ±0.8 seconds to ±0.2 seconds. This solves the problem that traditional collision time TTC estimation relies on optical flow amplitude, which leads to large noise interference in dilatation focus positioning and severe inter-frame jitter.

[0021] Fifth, by using multi-target weighted scoring (comprehensive collision time TTC, risk factor, altitude layer, motion trend, and no-fly constraint) to classify four levels of risk and execute a progressive obstacle avoidance strategy, the single decision time is less than 1ms, and it can track 8 targets at the same time, which solves the problem that a single minimum TTC obstacle avoidance logic cannot cope with complex airspace scenarios with multiple targets (≥8). Attached Figure Description

[0022] Figure 1 This is a schematic diagram of a biomimetic perception and obstacle avoidance method for aircraft based on an event camera, provided by the present invention. Detailed Implementation

[0023] The present invention will be further described in detail below. The following embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit the scope of protection of the present invention.

[0024] like Figure 1 As shown, this invention provides a biomimetic perception and obstacle avoidance method for aircraft based on an event camera, comprising the following steps: Step 1: Perform nonlinear non-warped field rotation filtering on the initial event stream captured by the airborne event camera: Simultaneously acquire the asynchronous event stream output by the airborne event camera and the attitude data output by the airborne inertial measurement unit to construct a nonlinear motion compensation distortion field; Use the constructed nonlinear motion compensation distortion field to remap and filter the output asynchronous event stream to achieve accurate alignment and aggregation of the dynamic target event stream and the static background event stream; Step 2: Perform dynamic threshold segmentation and joint constraint clustering on the asynchronous event stream after filtering in Step 1: Construct event count images and timestamp images, normalize the event count images and timestamp images, and use dynamic thresholds to distinguish between static background event streams and dynamic target event streams; A joint clustering cost function is constructed, and the DBSCAN algorithm is used to perform target clustering and separation on the spatiotemporal image of the asynchronous event stream after dynamic threshold segmentation. Step 3: Construct the optical flow field for each dynamic target region: The asynchronous event flow within the dynamic target region is sliced ​​into active event surfaces according to time. The spatial gradient of the constructed active event surfaces is solved and optimized using the local plane fitting method to obtain a computable optical flow field. Step 4: Fast estimation of multi-target collision time (TTC): Using the computable optical flow field obtained in Step 3, the radiation center (FOE) of the optical flow field on the image is solved by optimizing the optical flow direction, and the multi-target collision time (TTC) is estimated quickly. Step 5: Construct airspace hierarchical obstacle avoidance decision based on multi-target collision time (TTC): Based on the results of the rapid estimation of multi-target collision time (TTC) in Step 4, the flight safety risk level of the aircraft is divided by multi-target weighted scoring and a progressive obstacle avoidance strategy is executed.

[0025] This embodiment will describe the details of each step in the order of the above method.

[0026] Step 1: This step aims to address the "blurring" and "tailing" phenomena of background events caused by motion during high-maneuvering flight (especially rapid rotation) of aircraft, achieving precise aggregation of background events, including: Step 1.1: Data Acquisition and Initialization. Real-time synchronous acquisition of asynchronous event streams output from the airborne event camera. And the attitude data output by the airborne inertial measurement unit (IMU). Among them, The pixel coordinates where the event occurred. The timestamp of the event is in microseconds. This indicates an increase or decrease in brightness.

[0027] Step 1.2: Constructing a nonlinear motion compensation torsion field. Unlike traditional methods that rely solely on IMU angular velocity for linear compensation, this invention proposes a nonlinear torsion field. The distorted field The core of its construction lies in incorporating the initial pixel coordinates of the event. To compensate for the nonlinear projection deviation caused by the rotational motion of pixels far from the center of the image, specifically, the construction of the nonlinear motion compensation distortion field in step 1 includes adding a nonlinear term of the initial pixel coordinates of the event to the nonlinear distortion field and performing polynomial fitting compensation for the centrifugal projection deviation.

[0028] The nonlinear torsion field The model can be represented as: assuming the event camera only orbits... The axis rotates, and the event occurs along the axis. Axis rotation, for time interval The event flow within, ,in It is a rotation matrix obtained by interpolating data from the inertial measurement unit. It is related to the initial pixel position Related nonlinear compensation terms, Represents the spatiotemporal coordinates of an event. Indicates that the event only follows The spatiotemporal coordinates of axis rotation. This design can effectively correct motion compensation deviations caused by lens distortion and centrifugal projection effects.

[0029] Step 1.3: Event Stream Remapping and Filtering. Utilizing the constructed nonlinear distortion field... This method projects the current event stream back onto the coordinate system of the reference time, achieving precise alignment and aggregation of background events. By statistically analyzing the density of aligned events in the spatiotemporal neighborhood, it can effectively filter out events caused by noise or incompletely compensated motion interference, outputting clean and sharpened motion-compensated event frames.

[0030] Step 2: This step aims to accurately and quickly separate multiple foreground dynamic targets from the background event stream processed in Step 1, and suppress noise, including: After motion compensation, a large number of background events are aggregated into a few pixels, while independently moving objects are not affected by the camera's own motion and continue to generate events scattered across the image plane. Since the timestamps of the distorted events remain unchanged, for a small group of events at a single pixel, the average timestamp of the events triggered by the moving object is different from the average timestamp of the background events, making it possible to segment and detect moving objects.

[0031] Specifically, event clusters belonging to independent moving objects are relatively dispersed in the compensated image, with fewer events per pixel, while the background is more concentrated, resulting in a higher event count per pixel. This difference is reflected in the average event timestamp of each pixel, providing a basis for isolating dynamic regions. Therefore, this principle is used to construct an event count image and a timestamp image.

[0032] Step 2.1: Construct the event count image and timestamp image. For a given spatiotemporal window, construct the event count image and timestamp image respectively. and event timestamp image ; Record falling pixels The number of events in the neighborhood, Record the timestamp of the most recent event in the neighborhood of this pixel.

[0033] Step 2.2: Dynamic threshold segmentation. To make the data representation more stable, dynamic threshold segmentation is performed on the data. and Normalization is performed, and a dynamically adjustable threshold is used. To distinguish between static backgrounds and dynamic targets, a threshold is used. It can adaptively adjust based on the scene's average event rate and noise level. (The following appears to be unrelated and possibly a separate instruction: "Higher than...") The pixel region is marked as the foreground (dynamic target) region.

[0034] Step 2.3: Joint Constraint Clustering. The DBSCAN algorithm is a density-based clustering method adopted due to its noise robustness and relatively low average computational complexity, which is particularly advantageous for event data with low signal-to-noise ratios and helps maintain real-time performance. For the foreground (dynamic target) region, this invention employs an improved DBSCAN clustering algorithm for target separation. The core of this algorithm lies in constructing a joint clustering cost function, Cost, which simultaneously integrates the spatial coordinates of the event, the temporal difference, and the initial optical flow vector pre-calculated in Step 3 (Event Optical Flow Calculation). An example definition of the joint cost function is as follows: in, and The event coordinates are the event coordinates. and For event timestamps, and Let be the optical flow vector of the event. , , These are the weighting coefficients for each item. For two different event points, there is a weight vector. Through this joint constraint, random noise events that do not conform to motion consistency can be effectively eliminated, and neighboring dynamic targets that are spatially close but have different motion patterns can be accurately separated.

[0035] Step 3: This step aims to provide a computationally reliable and dense optical flow field for each dynamic target region, providing a data foundation for subsequent TTC estimation, including: Step 3.1: Construct the Active Event Surface (SAE). The asynchronous event stream within the dynamic target region is sliced ​​into an active event surface. For each pixel location within the target, its surface value... Defined as the normalized timestamp of the most recent event at that location. Active Event Surface (SAE) encodes sparse spatiotemporal events as a continuous two-dimensional surface.

[0036] Step 3.2: Spatiotemporal gradient calculation. Solve for the spatial gradient of the constructed Active Event Surface (SAE). .

[0037] Step 3.3: Local Plane Fitting Regularization. The original spatiotemporal gradient may be inaccurate in regions with sparse texture or high noise. To address this issue, this invention employs a local plane fitting method for optimization. For each pixel, a local plane is fitted within its neighborhood (e.g., a 5x5 or 7x7 pixel region). The parameters of the plane are solved using the least squares method. Optimized optical flow components Planar gradients that can be directly obtained from fitting Give, that is This method utilizes statistical information from the local neighborhood to significantly improve the robustness of optical flow estimation and avoid the gradient failure problem.

[0038] Step 4: This step aims to quickly estimate the collision time of multiple targets using the optical flow field obtained in Step 3, including: Cost-optimized FOE (focal point of expansion, i.e., the center of radiation of the optical flow field in an image) estimation. Consider a drone traveling at a constant speed. To a fixed point in the 3D world Move. At this point, FOE corresponds to a point. The projection onto the image plane is denoted as a pixel. Pixels closer to the FOE exhibit smaller flow amplitudes, while pixels farther away exhibit larger amplitudes. The FOE can be determined solely from the radial divergence (i.e., direction) of the surrounding flow, meaning flow size can be ignored, thus enhancing the stability of the estimation. Specifically: Step 4.1: Dilatation Focus Localization. For each separated dynamic target region, based on its internal sparse optical flow field, the dilatation focus of the target is solved by cost function optimization. The dilatation focus is the radiation center of the optical flow field on the image, representing the direction of relative motion between the aircraft and the target. The objective function is: in, It is a pixel The optical flow vector at that location, These are the coordinates of the dilatation focus to be solved. To improve stability, this optimization process ignores the amplitude information of the optical flow and only uses the direction of the optical flow.

[0039] Step 4.2: Local TTC Calculation. Combining the pinhole camera model and the dilated focal point (FOE), the local TTC calculation logic is constructed. For each pixel p within the target region, its... It can be determined by its position on the image plane and optical flow calculate: or In practical calculations, various robust statistical methods, such as median filtering, can be used to obtain the overall TTC of the target.

[0040] Step 4.3: Generate a global collision risk map. By calculating the TTC (Tracking Time and Cost) of each pixel across the entire image, a corresponding collision risk map is obtained. This map is then projected onto the extracted dynamic target regions to identify the primary targets posing the highest threat. Steps 4.1 and 4.2 are repeated for all dynamic targets within the image to generate a global collision risk map representing the TTC of each region. This risk map visually demonstrates the urgency of each threatening target in the airspace without requiring the reconstruction of a global 3D map, resulting in extremely high computational efficiency.

[0041] Step 5: This step aims to estimate the TTC of multiple targets based on Step 4, and, in conjunction with flight status and airspace rules, execute intelligent and safe obstacle avoidance decisions, including: Step 5.1: Multi-target weighted scoring and normalization. For all detected dynamic targets, collect their feature information, including: independent TTC values. The data includes the target's hazard factor (e.g., target size, type), relative flight altitude, motion trend (determined through optical flow history), and distance from pre-defined no-fly zones (e.g., geofences). This information is normalized and then input into a weighted scoring function. In this process, a comprehensive risk score is obtained for each objective.

[0042] Step 5.2: Equivalent TTC Calculation and Risk Level Classification. The comprehensive scores of all targets are integrated into an equivalent collision time. (For example, the minimum TTC value or a weighted harmonic mean can be used). According to The value is used to classify multiple risk levels. For example: Low risk >5 seconds): The current route is safe.

[0043] Medium risk (2 seconds < ≤5 seconds): There is a potential collision threat.

[0044] High risk (0.5 seconds < ≤2 seconds): Threat is approaching, immediate intervention is required.

[0045] Extremely high risk ≤0.5 seconds): A collision is imminent.

[0046] Step 5.3: Execution of Progressive Obstacle Avoidance Strategy. Based on the identified risk levels, execute the corresponding progressive obstacle avoidance strategy: Routine patrol: Maintain the current flight plan and only conduct continuous monitoring.

[0047] Warning and deceleration: When a warning signal is issued, the aircraft actively reduces its speed.

[0048] Fine-tuning of the trajectory: planning and executing small, low-G course corrections to bypass threatening targets at a safe distance while ensuring flight smoothness and avoiding secondary risks.

[0049] Emergency avoidance: Triggers the highest priority safety protection logic, such as executing maximum thrust climb or rapid deceleration and hovering, to avoid a collision as quickly as possible.

[0050] This step addresses the scenario of parallel interference from multiple obstacles in the three-dimensional airspace for aircraft. It abandons the single minimum TTC (Traffic Troubleshooting) decision logic and adopts a multi-target TTC weighted comprehensive evaluation mechanism. This mechanism combines four dimensions—target type, flight altitude, relative motion trend, and no-fly zone constraints—to construct a refined, hierarchical obstacle avoidance decision logic, fully aligning with air traffic control rules. The decision-making process is as follows: First, independent Time To Call (TTC) values ​​are simultaneously calculated for dynamic aircraft, low-altitude obstacles, hovering obstacles, and restricted airspace boundaries within the airspace. Multi-target TTC normalization is then performed. A comprehensive hazard score is calculated by combining the target hazard coefficient (pedestrians and general aviation aircraft have the highest weight, followed by static obstacles, and then hovering debris) and the risk of altitude-level incursions. When the equivalent TTC after comprehensive evaluation is greater than 5 seconds, and there is no risk of altitude-level incursions, it is classified as extremely low risk. The module only monitors the trajectory deviation of each target in real time in the background, without triggering warnings or attitude interventions, maintaining the UAV's stable flight at altitude and speed.

[0051] When 3s < equivalent TTC ≤ 5s, it is judged as a moderate warning risk. The module triggers the airborne warning prompt, executes uniform deceleration + altitude fine adjustment dual control, simultaneously predicts the trajectory of multiple targets, reserves airspace obstacle avoidance margin in advance, and avoids interference from lateral approaching targets. When 1.5s < equivalent TTC ≤ 3s, it is upgraded to high-order active obstacle avoidance. The module combines three-dimensional airspace passage conditions to perform deceleration + altitude layer climb / descent + heading fine-tuning linkage control, prioritizing the avoidance of high-weight dangerous targets, while avoiding secondary collisions and intrusion into restricted areas. When the equivalent TTC is less than or equal to 1.5 seconds, the highest level of emergency avoidance is triggered. The module immediately performs power flow limiting, emergency braking and hovering (rotor aircraft) / gliding obstacle avoidance (fixed-wing aircraft), shields low-weight target interference, prioritizes core flight safety, and exits the emergency mode step by step after the risk is eliminated, resumes autonomous flight, and takes into account obstacle avoidance response speed and flight attitude stability throughout the process.

[0052] Experimental results show that the nonlinear motion compensation algorithm of this invention has a significantly higher pixel event density than the traditional linear compensation method, and its compensation accuracy is comparable to the optimal four-degree-of-freedom optimization algorithm, while reducing computation latency by more than 60%. It completes multi-dynamic target detection within 10 milliseconds and exhibits excellent robustness in high-noise, small-target, and high-speed motion scenarios. TTC estimation can accurately identify target threat levels, output a local risk map in real time, and the hierarchical obstacle avoidance strategy can effectively mitigate various collision risks, fully meeting the low-latency and high-robustness perception requirements of high-dynamic autonomous navigation for aircraft. Specific experimental results are as follows: 1. The nonlinear torsion field reduces the rotation compensation error to 1 / 3 of that of the traditional linear model. Traditional methods rely solely on the angular velocity of the onboard inertial measurement unit (IMU) for linear rotation compensation, resulting in residual pixel errors exceeding 10 pixels for pixels far from the image center (e.g., the four corners of the image). This invention incorporates the initial pixel coordinates of the event into the distortion field. The nonlinear term is used to compensate for the centrifugal projection deviation using polynomial fitting. Experimental results show that, under a rotation with an angular velocity of 150° / s, the event reprojection error after edge region aggregation decreases from 12.7 pixels to 3.1 pixels, and the background shadow length is reduced by approximately 75%.

[0053] 2. Dynamic threshold segmentation reduces foreground / background misclassification rate by 40%. By simultaneously constructing an event count image With event timestamp image After normalization, an adaptive threshold is applied. (Dynamically adjusted based on local event rate) instead of a fixed threshold. In high event rate scenarios (propeller rotation, rapid cloud changes), the probability of a static background being misidentified as a dynamic target decreased from 21% to 12.6%, and the false negative rate of dynamic targets decreased from 18% to 10.2%.

[0054] 3. The joint clustering cost function can separate neighboring targets with a minimum spacing of less than 5 pixels. Traditional DBSCAN cannot separate closely spaced moving targets based solely on spatial distance. However, by introducing constraints based on temporal difference and optical flow vector norms, two targets with a center-to-center distance of only 4 pixels and opposite directions of motion can still be correctly separated. In the specific implementation... The clustering success rate increased from 62% to 94%.

[0055] 4. Local plane fitting optical flow improves the accuracy of gradient failure regions by 3 times. In areas with repetitive textures (such as sky or ocean) or low illumination, SAE surface gradients are prone to directional blurring. This invention fits a plane within a 7×7 neighborhood. With plane parameters Direct output of optical flow. Compared with the traditional Lucas Kanade method, in a simulation image with a signal-to-noise ratio of 3dB, the median optical flow angle error decreased from 24.7° to 8.3°, and the relative magnitude error decreased from 58% to 19%.

[0056] 5. TTC optimization ignores optical flow amplitude, reducing the FOE positioning standard deviation by 2.2 pixels. When solving for the dilatation focus, the cost function only uses the cross product of the optical flow direction vector and the pixel position, without considering the amplitude. This avoids the pull of amplitude noise (usually caused by inconsistent event polarity) on the center. In actual rotation and translation scenarios, the standard deviation of FOE coordinate jitter decreased from 3.5 pixels to 1.3 pixels, and the inter-frame fluctuation of TTC estimates decreased from ±0.8 seconds to ±0.2 seconds.

[0057] 6. Multi-objective weighted scoring enables the obstacle avoidance logic to handle ≥8 dynamic objectives simultaneously. Calculate the independent TTC, hazard factor, altitude level, motion trend, and no-fly zone constraint score for each target, and obtain the equivalent score after weighted summation. The decision module is based on The algorithm categorizes targets into four levels (>5s, 2-5s, 0.5-2s, <0.5s) and outputs four corresponding actions. A single decision takes less than 1ms (ARM Cortex-A76) and can track up to eight targets simultaneously, avoiding the computational explosion problem of traditional potential field methods with mixed targets.

[0058] 7. The algorithm requires no GPU and can achieve a processing frequency of 200Hz on a Jetson Xavier NX. The entire process (filtering—segmentation—optical flow—TTC—decision) does not involve convolutional neural networks or dense matrix inversion; it utilizes only lightweight algebraic and statistical operations. On a 640×480 resolution event stream (average event rate 1.2 Meps), the measured single-frame processing time is 4.9 ms, corresponding to an output rate of approximately 200 Hz, significantly higher than the 30 Hz of traditional frame-based cameras. CPU utilization is only 32%, leaving ample margin for other flight control tasks.

[0059] 8. Neither TTC estimation nor obstacle avoidance decision-making requires the reconstruction of 3D maps or depth information. By relying on the geometric relationship of the pinhole model and dilated focus, TTC is calculated directly on the image plane, avoiding the additional hardware costs and latency of binocular stereo matching, depth estimation networks, or LiDAR. The total system latency (event triggering → obstacle avoidance command output) is <15ms, which is more than 5 times faster than the frame-based image reconstruction-optical flow-SFM scheme (typically >80ms).

Claims

1. A biomimetic perception and obstacle avoidance method for aircraft based on event cameras, characterized in that, Includes the following steps: Step 1: Perform nonlinear non-warped field rotation filtering on the initial event stream captured by the airborne event camera: Simultaneously acquire the asynchronous event stream output by the airborne event camera and the attitude data output by the airborne inertial measurement unit to construct a nonlinear motion compensation distortion field; Use the constructed nonlinear motion compensation distortion field to remap and filter the output asynchronous event stream to achieve accurate alignment and aggregation of the dynamic target event stream and the static background event stream; Step 2: Perform dynamic threshold segmentation and joint constraint clustering on the asynchronous event stream after filtering in Step 1: Construct event count images and timestamp images, normalize the event count images and timestamp images, and use dynamic thresholds to distinguish between static background event streams and dynamic target event streams; A joint clustering cost function is constructed, and the DBSCAN algorithm is used to perform target clustering and separation on the spatiotemporal image of the asynchronous event stream after dynamic threshold segmentation. Step 3: Construct the optical flow field for each dynamic target region: The asynchronous event flow within the dynamic target region is sliced ​​into active event surfaces according to time. The spatial gradient of the constructed active event surfaces is solved and optimized using the local plane fitting method to obtain a computable optical flow field. Step 4: Fast estimation of multi-target collision time TTC: Using the computable optical flow field obtained in Step 3, the radiation center FOE of the optical flow field on the timestamp image is solved by optimizing the optical flow direction, and the multi-target collision time TTC is estimated quickly. Step 5: Constructing a hierarchical obstacle avoidance decision based on multi-target collision time (TTC): Based on the results of the rapid estimation of multi-target collision time (TTC) in Step 4, the flight safety risk level of the aircraft is divided by multi-target weighted scoring and a progressive obstacle avoidance strategy is executed. The construction of the nonlinear motion compensation distortion field in step 1 includes adding a nonlinear term to the event initial pixel coordinates in the nonlinear distortion field and performing polynomial fitting compensation for the centrifugal projection deviation. The nonlinear torsion field The model can be represented as: assuming the event camera only orbits... The axis rotates, and the event occurs along the axis. Axis rotation, for time interval The event flow within, ,in It is a rotation matrix obtained by interpolating data from the inertial measurement unit. It is related to the initial pixel position Related nonlinear compensation terms, Represents the spatiotemporal coordinates of an event. Indicates that the event only follows The spacetime coordinates of the axis of rotation.

2. The biomimetic perception and obstacle avoidance method for aircraft based on an event camera according to claim 1, characterized in that, The joint clustering cost function constructed in step 2 includes the spatial coordinates, time difference, and optical flow vector of the event. Through the joint clustering cost function, random noise event streams that do not conform to motion consistency are eliminated, and neighboring dynamic target event streams that are spatially close but have different motion patterns are separated.

3. The biomimetic perception and obstacle avoidance method for aircraft based on an event camera according to claim 2, characterized in that, The joint clustering cost function expression is as follows: in, and The event coordinates are the event coordinates. and For event timestamps, and Let be the optical flow vector of the event. , , These are the weighting coefficients for each item. The weight vector corresponds to two different event points.

4. The biomimetic perception and obstacle avoidance method for aircraft based on an event camera according to claim 1, characterized in that, In step 3, the local plane fitting method includes fitting a two-dimensional plane to each event pixel in the neighborhood, solving the parameters of the two-dimensional plane by the least squares method, and then the plane gradient obtained by the two-dimensional plane fitting can be directly output as the optical flow field.

5. The biomimetic perception and obstacle avoidance method for aircraft based on an event camera according to claim 1, characterized in that, In step 4, the rapid estimation of multi-target collision time (TTC) includes solving for the radiation center of the dilatation focal optical flow field of each separated dynamic target region on the timestamp image by using a joint clustering cost function, thereby generating a global collision risk map representing the multi-target collision time (TTC).

6. The biomimetic perception and obstacle avoidance method for aircraft based on an event camera according to claim 1, characterized in that, In step 5, multi-target weighted scoring refers to collecting data on the detected dynamic target regions. Its characteristic information includes: independent TTC value, target hazard coefficient, relative flight altitude layer, motion trend, and distance from preset airspace no-fly constraints. After normalizing the characteristic information, it is input into the weighted scoring function to obtain the comprehensive risk score of each dynamic target area.

7. The biomimetic perception and obstacle avoidance method for aircraft based on an event camera according to claim 6, characterized in that, The comprehensive risk score for each dynamic target region is recorded as an equivalent collision time. ,according to The value is used to classify the flight safety risk of aircraft into four levels: >5 seconds, low risk: current route is safe; 2 seconds ≤5 seconds, medium risk: potential collision threat exists; 0.5 seconds ≤2 seconds, high risk: threat is imminent, immediate intervention is required; ≤0.5 seconds, extremely high risk: a collision is imminent.

8. The biomimetic perception and obstacle avoidance method for aircraft based on an event camera according to claim 7, characterized in that, In step 5, implementing the progressive obstacle avoidance strategy includes executing the corresponding progressive obstacle avoidance strategy based on the classified flight safety risk level of the aircraft: Low risk: Routine patrols, maintaining the current flight plan, with only continuous monitoring; Medium risk: Warning and deceleration, issuing a warning signal, and the aircraft actively reduces its speed; High risk: Fine-tuning the trajectory, planning and executing small, low-G course corrections to bypass threatening targets at a safe distance while ensuring flight smoothness and avoiding secondary risks; Extremely high risk: Emergency avoidance, triggering the highest priority safety protection logic, executing maximum thrust climb or rapid deceleration hovering to avoid collision at the fastest speed.

Images