Event camera based ultra-low latency visual positioning system and method thereof

Abstract

The application relates to an event camera-based ultra-low latency visual positioning system and method, which comprises an event sensing module, an inertial measurement unit (IMU) and a mixed signal processing unit. The event sensing module has a microsecond-level response characteristic and is integrated with dynamic bias adjustment and a tunable optical filter; the IMU is rigidly connected to the event sensing module, and the physical center alignment error is less than 0.5 DEG; the mixed processing unit adopts a FPGA-ARM cooperative architecture to realize event filtering, time synchronization, motion compensation and tight coupling optimization. The method combines event brightness residuals and IMU pre-integration residuals to construct a joint cost function, dynamically adjusts a sliding window based on event entropy, and realizes loop closure correction through a mutual information criterion. The application can realize 6-degree-of-freedom pose estimation up to 500 Hz, has the advantages of low latency, high precision and high robustness, and is suitable for high-speed scenes such as unmanned aerial vehicles, AR navigation and industrial robots.

Description

Technical Field

[0001] This invention relates to the fields of robot perception and computer vision technology, and in particular to an ultra-low latency visual positioning system and method based on an event camera. It is suitable for achieving high-precision, low-latency pose estimation in high-speed motion or high-dynamic environments, and belongs to the related technical field of event visual positioning and visual inertial navigation systems. Background Technology

[0002] With the increasing demands for high-precision, low-latency positioning in applications such as autonomous driving, unmanned aerial vehicles, augmented reality, and robotics, visual-inertial (VIO) joint positioning systems have become one of the mainstream technical solutions in current research and engineering practice. Traditional VIO systems mainly rely on frame-based cameras to acquire image sequences and combine them with angular velocity and acceleration information provided by inertial measurement units (IMUs) to estimate the six-degree-of-freedom (6-DOF) pose of the device. However, in complex environments such as high dynamics, high contrast, and extreme lighting, traditional frame-based vision systems have exposed several key bottlenecks, severely restricting their stability and accuracy in practical applications.

[0003] First, traditional frame-based cameras, which capture images at a fixed frame rate (e.g., 30FPS or 60FPS), are prone to significant motion blur when the device is moving at high speeds (e.g., >5m / s). Image blurring compromises the clarity and stability of key features, thereby affecting feature extraction, matching, and tracking at the visual front end, leading to a significant decrease in positioning accuracy. Related experimental results show that in high-speed motion scenes, the image blur rate can reach over 60%, severely limiting the usability of traditional vision systems in high-dynamic tasks.

[0004] Secondly, in traditional VIO systems, the IMU and camera modules generally rely on software-level or externally triggered synchronization mechanisms for time synchronization, which makes it difficult to avoid time alignment errors caused by non-ideal factors such as sensor sampling clock drift and data transmission delay. Especially in systems with high requirements for low latency and high-frequency updates, if the time synchronization error exceeds 100μs, it can easily cause a shift in the spatial correspondence between the image and IMU data, which in turn leads to a cumulative drift in pose estimation, with accuracy errors potentially reaching 0.1° / m or higher.

[0005] Furthermore, in high dynamic range (HDR) scenarios, such as backlighting, tunnel entrances and exits, and areas with strong contrast lighting, traditional frame-based cameras generally use a fixed exposure time mode, making it difficult to simultaneously capture details in dark areas and information in bright areas. This leads to image saturation or underexposure at dynamic ranges exceeding 120dB, resulting in the loss of a large amount of effective visual information. Consequently, the system may experience positioning failures or even crashes in environments with drastic changes in lighting.

[0006] In summary, current mainstream visual-inertial positioning technologies still face many challenges in handling high-speed motion, asynchronous sensor data fusion, and complex lighting environments. Innovative improvements are urgently needed in terms of sensor hardware responsiveness, spatiotemporal alignment accuracy, and information robustness to meet the higher requirements for real-time performance, accuracy, and reliability in future complex application scenarios. Summary of the Invention

[0007] To overcome the problems of high latency, unstable accuracy, and poor adaptability to weak textures in existing visual positioning systems under high dynamic scenes, this invention proposes an ultra-low latency visual positioning system and method based on an event camera. This system integrates a microsecond-level response event visual sensor with a tightly coupled inertial fusion algorithm, achieving high-frequency and robust pose estimation in complex environments through hardware and software co-design.

[0008] The core innovation of this invention lies in: introducing an event-aware module with dynamic response characteristics to achieve asynchronous data acquisition; constructing a tightly coupled event-IMU joint optimization framework, which improves positioning accuracy and stability by driving sliding window adjustment through event entropy and event-inertial constraint modeling; and combining a multi-scale mutual information closed-loop detection strategy to enhance the global consistency of the system.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] In one possible implementation, an ultra-low latency visual positioning system based on an event camera is provided, comprising:

[0011] The event sensing module includes a photoelectric sensor array with dynamic response characteristics, wherein the response delay of a single pixel in the sensor array is no greater than 10μs;

[0012] An inertial measurement unit (IMU) is rigidly connected to the event sensing module, with the physical center distance between the two not exceeding 8 mm and the optical axis alignment error less than 0.5°.

[0013] The mixed-signal processing unit, integrating an FPGA chip and an ARM processor, is configured to perform the following operations:

[0014] a) Preprocess the event stream, including filtering out noisy events based on spiking neural networks;

[0015] b) Perform motion compensation based on IMU data;

[0016] c) Align event data with IMU data using a timestamp synchronization module to ensure that the time synchronization error is no greater than 20μs.

[0017] Furthermore, the event sensing module further includes: a tunable optical filter whose transmission wavelength is programmably adjustable in the range of 400-1000nm; and a dynamic bias circuit that automatically adjusts the pixel bias voltage in response to changes in ambient light intensity, with the adjustment range being ±15% of the reference voltage.

[0018] Furthermore, the control logic of the dynamic bias circuit includes: when the event trigger rate exceeds 1×10⁻⁶ per unit time, 6 When the event rate is less than 1×10 events / s, reduce the bias voltage by 10%-15%; when the event trigger rate is less than 1×10 events / s for 5 consecutive ms. 4 Increase the bias voltage by 5%-8% for each event / s.

[0019] Furthermore, the mixed signal processing unit further includes: a hardware-accelerated Bundle Adjustment (BA) module, optimized using a truncated Newton method, configured as follows:

[0020] a) Maintain a dynamically adjustable sliding window, where the window size N satisfies the following relationship: ;

[0021] in This is the current event entropy value (a dynamic indicator representing the amount of information in the event stream, reflecting the complexity of the scene). The baseline entropy value (the average event entropy reference value during the system calibration phase), k∈[0.5,1.2];

[0022] In this invention, the adjustment coefficient k is not a fixed empirical quantity, but an optimized variable determined collaboratively through dynamic calibration and an adaptive operating condition strategy. This method significantly differs from existing visual-inertial estimation schemes based on fixed window parameters, and possesses stronger adaptability and precision control capabilities.

[0023] b) When an angular velocity greater than 5 rad / s is detected, N is forcibly limited to the range of 5-8 frames.

[0024] In another possible implementation, the present invention also provides a visual positioning method based on the above-described system, comprising the following steps:

[0025] S1. Asynchronously receive event stream data output by the event sensing module and IMU data output by the inertial measurement unit;

[0026] S2. Perform spatiotemporal consistency filtering on the event stream, the filtering including: constructing an event cube in xyt space, removing outlier events, and performing pulse coding processing based on the Leaky Integrate-and-Fire neuron model;

[0027] S3. Perform event-IMU tight coupling optimization, including:

[0028] A joint cost function is constructed, comprising an event brightness residual term, an IMU pre-integration residual term, and an event-inertial cooperative residual term. The optimization objective is to minimize the following joint cost function: ;

[0029] in:

[0030] The brightness residual corresponding to the i-th event frame.

[0031] For the j-th IMU pre-integration residual,

[0032] For event-IMU constrained residuals,

[0033] For residual collaborative weights;

[0034] The optimization employs the First Estimate Jacobians (FEJ) method to maintain observation consistency;

[0035] S4. Output the 6-DOF pose estimation result at the current time, with an update frequency of no less than 500Hz.

[0036] Preferably, step S1 further includes: synchronously receiving image frame data and event stream data, and performing inter-frame differential processing on the image frames to assist in normalizing the event trigger intensity; the sampling interval of the auxiliary image frames is 20ms±1ms to improve the positioning stability of weak texture areas.

[0037] Preferably, the rules for determining the spatiotemporal consistency of event e(x,y,t,p) in step S2 include:

[0038] In the spatiotemporal neighborhood N(e) of event e, if there exists an event e'(x',y',t',p') that satisfies the following condition, then event e is considered a valid event:

[0039] ,in ;

[0040] Spatial distance satisfies ,in 1 pixel.

[0041] Furthermore, the joint cost function in step S3 also includes an event-inertia constraint term, the weight ω of which is adaptively adjusted according to the current acceleration measurement value, satisfying the following relationship: ;

[0042] in This represents the current acceleration measurement, where g is the gravitational acceleration constant.

[0043] Preferably, step S3 further includes:

[0044] The event intensity entropy value H is calculated based on the event flow and used to dynamically adjust the sliding window size. The event entropy H is defined as follows: ;

[0045] Where f(p) represents the normalized frequency distribution within the event intensity interval p; P is the set of all discretized intensity intervals;

[0046] The event entropy H is used to drive the adaptive adjustment of the sliding window size N to adapt to scenarios with different motion complexities.

[0047] Furthermore, the above method also includes the following steps:

[0048] Local keyframe maps are constructed based on event streams, and loop closure detection is performed by combining IMU state estimation.

[0049] When the event mutual information metric between two keyframes exceeds a preset threshold, pose closure correction is performed to improve global consistency.

[0050] The formula for calculating the mutual information metric is as follows: ;

[0051] in:

[0052] This represents the joint probability distribution of event intensity in keyframes A and B;

[0053] and are the marginal probability distributions of event intensity in keyframes A and B, respectively;

[0054] Mutual information values ​​are used to measure keyframe similarity and serve as the basis for loopback triggering.

[0055] This invention demonstrates significant performance improvements in real-world high-dynamic testing environments by constructing an event entropy-driven sliding window adaptive mechanism and combining it with an optimization strategy that dynamically adjusts the coefficient k. For example, in a scenario with a sudden change in strong light from 500 lux to 2000 lux, the system can still maintain a tracking offset of less than 0.3 m, reducing the positioning error by an average of approximately 57% compared to a fixed window mechanism. In a scenario with an angular velocity of 6 rad / s, the attitude calculation accuracy is improved by more than 66%, verifying the robustness and stability of the system under complex working conditions.

[0056] Based on the above technical solutions, the ultra-low latency visual positioning system based on an event camera of the present invention achieves high-precision and robust real-time positioning performance by constructing a tightly coupled architecture integrating an event perception module, an inertial measurement unit, and a hybrid signal processing unit. This system features significant innovations in hardware, algorithms, and system-level co-design, and its overall performance is significantly improved compared to existing visual-inertial positioning systems.

[0057] At the hardware level, the event sensing module adopts a photoelectric sensor array with dynamic response characteristics, with a single-pixel response delay as low as 10μs, which is faster than the response speed of mainstream event cameras (such as the 15μs of DVS346) and can more sensitively capture changes in optical flow in high-speed scenes; it integrates a tunable optical filter that supports wavelength programmable adjustment in the range of 400-1000nm to adapt to complex lighting conditions; the matching dynamic bias circuit automatically adjusts the bias voltage through environmental perception to maintain a stable event trigger rate, effectively suppress saturation and overexposure noise, and provide a high signal-to-noise ratio input for subsequent processing.

[0058] At the algorithm level, the system's built-in spatiotemporal consistency filtering algorithm performs neighborhood analysis on the three-dimensional event cube, and combined with the LIF neuron coding model, it realizes sparse compression and saliency enhancement of event data, which greatly improves data utilization efficiency and feature fidelity. The joint optimization module uses the event brightness residual and the IMU pre-integration residual to construct a joint cost function, and combined with the dynamic weight adjustment mechanism and the FEJ observation consistency method, it realizes efficient fusion of event-inertial information. The event entropy-driven sliding window BA optimization strategy ensures that the system can still output high-quality pose estimation stably under violent motion (such as angular velocity > 5 rad / s), preventing error accumulation and trajectory drift.

[0059] At the system level, the event camera and IMU employ a rigid mounting structure with a physical center-to-center distance of no more than 8mm and an optical axis alignment error of less than 0.5°. A hardware timestamp alignment module controls data synchronization errors to within 20μs, significantly reducing the impact of asynchronous sampling on the fusion algorithm. Furthermore, the system supports synchronous processing of image frames and event streams, and constructs local keyframe maps based on the event stream to achieve loop closure detection and loop closure optimization, effectively enhancing global consistency and localization robustness.

[0060] In comprehensive testing, the system demonstrated excellent performance on both the CARLA simulation platform and the physical drone platform: it achieved an end-to-end latency of 2.1ms in high-speed drone scenarios and a pose estimation frequency of 500Hz, which is far superior to traditional solutions such as VINS-Fusion; it maintained sub-meter level error and continuous working capability in complex environments (such as dark tunnels, high dynamic range scenarios, and sharp turns), demonstrating strong adaptability and high stability.

[0061] Therefore, this invention not only effectively overcomes the technical bottlenecks of traditional visual positioning systems, such as image blurring, large timing errors between the IMU and camera, and the inability of fixed exposure to adapt to high dynamic environments, but also constructs a highly robust visual-inertial tightly coupled positioning system for complex, high-dynamic scenarios. This system is particularly suitable for applications requiring extremely high positioning accuracy and system response time, such as high-speed flying drones, minimally invasive surgical navigation, and autonomous driving under conditions of sudden changes in strong light, demonstrating broad engineering application value and promising prospects for widespread adoption.

[0062] The technical solution of this invention has been verified through multiple sets of comparative experiments, demonstrating a significantly superior dynamic adaptability compared to traditional fixed-parameter mechanisms. Under conditions of rapid fluctuations in event entropy Et and high angular velocity input, the adaptive adjustment strategy can effectively coordinate the sampling frequency, voltage regulation amplitude, and thermal compensation response, maintaining system stability and accuracy. Experimental results show that, in typical indoor high-speed navigation tasks, the solution of this invention can control the average root mean square error (RMSE) of positioning to 0.12m (standard deviation σ=0.03), and improve the accuracy of loop closure recognition driven by mutual information to 92.3%, fully verifying its wide applicability in various high-dynamic scenarios such as robot navigation, industrial palletizing, and surgical-assisted positioning. Attached Figure Description

[0063] Figure 1 System overall structure diagram;

[0064] Figure 2 Schematic diagram of dynamic bias circuit control;

[0065] Figure 3 Event-IMU Joint Optimization Flowchart;

[0066] Figure 4 Event filtering structure diagram;

[0067] Figure 5 The logic diagram is dynamically adjusted via a sliding window.

[0068] Figure 6 : Graph of residual terms of joint cost function;

[0069] Figure 7 : Schematic diagram of closed-loop detection trigger logic;

[0070] Figure 8 Flowchart of dynamic parameter adaptive mechanism;

[0071] Figure 9 Schematic diagram of the k-value calibration experimental platform. Detailed Implementation

[0072] This invention proposes an ultra-low latency visual positioning system and method based on an event camera. The system integrates high-response-rate event-aware hardware with a tightly coupled multimodal optimization algorithm, achieving microsecond-level latency and high-frequency, robust positioning capabilities for pose estimation in high-speed dynamic scenes. To support these performance indicators, this invention performs collaborative optimizations in system structure design, spatiotemporal data synchronization, event filtering modeling, and inertial fusion calculation.

[0073] To facilitate understanding of this invention, the following explanations are provided for some of the terms used in the specification:

[0074] 1. Event Entropy H

[0075] Event entropy H refers to the normalized entropy value calculated after constructing a grayscale histogram based on the pixel intensity changes output by the event camera within a given time window. This metric reflects the complexity and information density of event triggering in the current scene and is typically used to dynamically adjust perception processing parameters. Its unit is dimensionless, with a typical value range of 0 to 4. This calculation is based on Shannon's entropy definition formula and is normalized to adapt to different resolution scenes.

[0076] 2. Sliding window size N

[0077] The sliding window size N represents the number of keyframes retained by the system during joint optimization, used to construct the state variables of the optimization problem. The size of N directly affects the convergence speed, computational load, and tracking stability of the optimizer. To adapt to dynamic scene changes, this invention proposes an event entropy-driven window adaptive mechanism, which dynamically adjusts the value of N under different entropy values, generally ranging from 5 to 20 frames.

[0078] 3. Adjustment coefficient k

[0079] The adjustment coefficient k is a sensitive parameter used to control the sliding window size, sampling frequency, and bias voltage adjustment range, typically set to [0.5, 1.2]. The value of k is determined by the current environmental event entropy Et, the reference value E0, and the dynamic states of the IMU, such as angular velocity and temperature. This invention establishes the mapping relationship between k and multiple factors through systematic calibration experiments, realizing a multidimensional adaptive adjustment strategy.

[0080] 4. Event Camera

[0081] An event camera is a visual sensor based on the principle of event-based imaging. Its output data is asynchronously triggered in the form of pixel-level brightness change events, featuring microsecond-level temporal resolution, low latency, and high dynamic range. Unlike traditional frame-based image acquisition, event cameras only output data when scene changes occur, thus significantly reducing redundancy and improving response speed.

[0082] 5. Mutual Information

[0083] Mutual information is a metric that measures the statistical correlation between two signals or datasets, and in this invention, it is used to determine the degree of matching between the current keyframe and historical frames. This metric is constructed based on event density distribution or brightness histograms, and triggers the loop closure detection module when a set threshold is reached. The mutual information calculation results can effectively guide the graph optimization module in performing loop closure correction.

[0084] The following will provide a detailed explanation of the various implementation details of this invention, based on the system structure, core algorithm processing flow, and linkage mechanism between functional modules, combined with the actual technical implementation path.

[0085] (I) System Hardware Integration

[0086] like Figure 1 The diagram shows a schematic of an ultra-low latency visual positioning system based on an event camera according to the present invention. The system mainly includes an event sensing module, an inertial measurement unit (IMU), and a hybrid signal processing unit, which are rigidly integrated to collaboratively achieve high-precision, low-latency pose estimation. The event sensing module uses an iniVation DVXplorer event camera, whose sensor array consists of 128×128 pixels, with each pixel embedding a dynamic bias circuit. Figure 2 The diagram shows the control principle of the dynamic bias circuit. The circuit comprises an event trigger rate monitoring module, a control logic unit, and a voltage regulator chip, forming a closed-loop bias adjustment structure. This structure automatically adjusts the pixel bias voltage when the event trigger rate changes, improving photosensitivity robustness. The bias circuit uses a TI TPS7A4700 voltage regulator chip to achieve ±15% voltage adjustment, automatically responding to changes in ambient light. Specifically, when the event trigger rate exceeds 1×10⁻⁶... 6 At events / s, the FPGA control circuit reduces the bias voltage by 12% within 200 nanoseconds to adapt to environments with high event trigger rates. In low-light environments, the wavelength is adjusted using a tunable filter with a MEMS-Fabry-Perot interference structure to achieve continuous adjustment from 400nm to 1000nm, optimizing image quality. Especially in low-light conditions, the system can automatically switch to 400nm mode to enhance image visibility.

[0087] The IMU module uses a BMI088 six-axis inertial sensor, rigidly connected to the event sensing module via a 7075 aluminum alloy clamp. Measurements using a laser tracker show that the physical center-to-center distance between the sensor modules is 7.5mm ± 0.1mm. The optical axis error was precisely calibrated using a Thorlabs PAX1000 polarizer, ensuring an error of less than 0.3°. The system also integrates a u-blox ZED-F9P GPS clock source, achieving high-precision synchronization between the event stream and IMU data through an FPGA hardware timer, with a time synchronization error controlled within 18μs.

[0088] (II) Data Processing and Optimization

[0089] After system startup, data preprocessing is performed first. Event stream data is processed through a 3D event cube construction module. Each event (e(x,y,t,p)) undergoes validity assessment within a 1ms time window and a 3-pixel spatial neighborhood. In e(x,y,t,p), x and y are pixel coordinates (unit: pixels), t is the timestamp (unit: μs), and p is the event polarity (+1 / -1 indicates brightness increase / decrease). Figure 4 The diagram shows the three-layer structure of the event filtering system. Event data first enters the 3D event cube construction module, and then undergoes spatiotemporal consistency judgment and pulse coding processing via a three-layer neuron structure. This structure employs the Leaky Integrate-and-Fire (LIF) model, effectively compressing data and improving the signal-to-noise ratio. Membrane potential accumulation and pulse output logic jointly achieve event validity screening. Pulse coding is performed using the Leaky Integrate-and-Fire neuron model, with the membrane potential leakage coefficient set to 0.85. When the accumulated event intensity reaches the pulse threshold of 0.6, the system triggers pulse output. Experimental results show that this parameter configuration can effectively filter out 92.3% of motion blur pseudo-events, especially under 1000 lux illumination, where the scheme performs particularly well.

[0090] After the event data and IMU data have been preprocessed and time-aligned, the system enters the tight-coupling optimization phase. For example... Figure 3 The diagram illustrates the processing flow of the event-IMU joint optimization module described in this invention. The diagram sequentially shows the steps of preprocessing event data and IMU data, residual construction, joint optimization, and pose output, serving to illustrate the implementation path of multimodal data fusion and high-frequency pose estimation. The tightly coupled algorithm used in this invention is executed in the mixed signal processing unit, integrating a hardware-accelerated Bundle Adjustment (BA) module. It employs the truncated Newton method for minimization, with the optimization objective being the joint cost function. The optimization objective is to minimize the following joint cost function: ;

[0091] like Figure 6 The diagram shown illustrates the residual term structure in the joint cost function of this invention. This function contains three types of residuals: event brightness residuals. IMU pre-integration residuals Event-inertial co-residual Furthermore, a collaborative weighting factor is introduced to achieve multimodal data fusion.

[0092] in, This represents the pose state of the i-th frame; For the event brightness residual, Let its covariance matrix be ; For IMU pre-integration residuals, Let its covariance matrix be ; For event-inertia constraint residuals, adaptive weights The calculation is adaptively performed based on the current magnitude of acceleration. The specific calculation formula is as follows: ;in As the initial weighting factor, The linear acceleration measured by the IMU at the current moment. is the gravitational acceleration constant. This adaptive weight design is used to enhance the influence of state consistency constraints between events and IMUs in highly dynamic environments, and its ability to suppress trajectory drift has been verified in high-speed AGV load tests.

[0093] To enhance optimization robustness, the system also introduces event luminance entropy. This serves as the basis for dynamic adjustment of the sliding window. Event brightness entropy is represented as the statistical divergence of the event intensity distribution per unit time (quantifying the disorder of the event intensity distribution), and its calculation method is as follows: ;

[0094] Where f(p) represents the normalized frequency distribution of the event intensity range; the event entropy H is used to dynamically adjust the size of the BA sliding window to adapt to different motion complexity scenarios. p∈[0,1]. The event intensity range is discretized into 16 levels, and the histogram is calculated every 50ms to determine the entropy value. The system dynamically adjusts and optimizes the sliding window size based on the following relationship. :

[0095] ;

[0096] Where N is the size of the current sliding window. The entropy value of the current event. As the baseline window frame number, As the empirical entropy baseline value, The adjustment coefficient (0.5 ≤ k ≤ 1.2) was determined through calibration experiments to control the sensitivity of entropy to window size. For example... Figure 5 As shown, the dynamic adjustment logic of the sliding window size N is driven by the current event entropy H. The system monitors the divergence of the event intensity distribution per unit time, calculates the entropy value in real time, and adjusts and optimizes the number of window frames according to a preset functional relationship to adapt to different dynamic complexity scenarios. The dynamic adjustment process of the sliding window size N uses event entropy as the driving variable and activates the window limiting mechanism when the angular velocity exceeds the threshold. When the IMU angular velocity exceeds 5 rad / s, the system triggers the limiting strategy, reducing the window size N by 100%. The frame rate is fixed between 5 and 8 frames, and some parameter dimensions (such as translation) can be disabled to improve convergence speed. In actual testing, when the angular velocity reaches 8 rad / s, this mechanism can reduce trajectory error by 58%.

[0097] Experimental results show that the adaptive sliding window mechanism can effectively avoid numerical instability during the optimization process under conditions of rapid rotation or drastic acceleration, thereby improving the global consistency and real-time performance of the system.

[0098] Example: Selection and calibration method of adjustment coefficient k

[0099] To improve the adaptability of the sliding window adjustment strategy to different motion scenarios, this invention provides a systematic calibration method for the adjustment coefficient k. This method includes the following steps:

[0100] 1. Dynamic Scene Stimulation Experiment

[0101] During system initialization or when application conditions change, a standard motion platform (such as a six-degree-of-freedom robotic arm) is built to perform sinusoidal frequency sweep excitation, with angular velocities covering the range of 0.5–10 rad / s, and the event entropy Et and the true pose error δ are recorded.

[0102] Example conditions:

[0103] Trajectory type: a combined pitch and yaw motion with an amplitude of ±30°, and a frequency step change of 0.1–5 Hz.

[0104] Light intensity: approximately 1000 lux, event trigger rate stable at 2×10 5 events / s

[0105] 2. k-value grid search and error assessment

[0106] Keeping Nbase=5, we perform window adjustment and pose estimation by enumerating k∈[0.5,1.2] (step size 0.1), and calculate the root mean square error (RMSE) for each k: ;

[0107] in, To estimate pose, For actual reference trajectory.

[0108] 3. kopt fitting and constraint rules

[0109] Select the value kopt with the smallest error from the search results, and further constrain it based on the dynamic nature of the actual scenario:

[0110] If the average Et / E0 > 3, then kopt ∈ [0.9, 1.2].

[0111] If Et / E0 < 1.5, then kopt ∈ [0.5, 0.8];

[0112] Example: In a high-speed turning experiment of a drone, the peak angular velocity was about 8 rad / s. The calibration results showed that the error was the smallest when k=1.05 (RMSE=0.12 m).

[0113] Application Scenario Description

[0114] Surgical navigation:

[0115] Due to the low event trigger rate in soft tissue endoscopy scenarios, the modified formula is recommended: ;

[0116] in The reference entropy value for the motion of medical devices (such as during electrosurgical operation). ).

[0117] Industrial robot verification:

[0118] The results of the test on the ABB IRB6700 robot are as follows:

[0119] ;

[0120] (III) Dynamic Parameter Adaptation Mechanism

[0121] The dynamic parameter adaptive mechanism of this system is implemented through a three-level collaborative architecture. For example... Figure 8 As shown, environmental information and motion status are first acquired in real time by an event camera and an IMU sensor. The raw data, after preprocessing, is input into an adaptive decision-making unit built on an ARM processor. This unit calculates the current environmental characteristic quantity E based on an event entropy dynamic analysis algorithm. t It is then compared with the reference value E0 to generate an initial adjustment signal.

[0122] For adjusting the parameters of the event camera, the system uses an adjustable bias circuit built with an AD5391 digital potentiometer. When a sudden change in ambient light intensity is detected (E... tWhen / E0>1.5), the adjustment engine increases the bias voltage in 0.05V steps, up to a maximum of 3.6V, and this process is completed within 200μs. At the same time, the state machine inside the FPGA monitors the angular velocity ω and linear velocity v output by the IMU. When ω exceeds 5rad / s, it automatically switches to a 1kHz sampling rate mode, while in a static environment (ω<0.2rad / s), it downclocks to 100Hz to reduce power consumption.

[0123] The temperature compensation module monitors the chip temperature in real time using a DS18B20 sensor. When the temperature exceeds 45°C, a protection strategy is activated: firstly, the voltage regulation range is limited to within ±10% of the default value; secondly, a speed adjustment request is sent to the heat dissipation unit. All adjustment parameters are smoothed using a Kalman filter to avoid system oscillations caused by sudden parameter changes.

[0124] In the specific implementation process, during the system initialization phase, a 10ms environmental baseline event stream is first collected to calculate E0, and a preset k=0.8 is loaded as the initial value. After entering the working state, the following adaptive adjustment process is executed every 20ms: read the current event entropy E. t Substituting the IMU motion state vector [ω,v,T] into the dynamic adjustment equation k=0.7+0.5tanh((E) t The new parameters are calculated using -E0) / E0)×(1+0.2 ω / 10)×(1-0.01|T-25|), and the final k value is constrained within the range of [0.5,1.2] and distributed to each execution unit.

[0125] Experiments show that this mechanism can reduce tracking offset from complete loss to within 0.3m under sudden intense light (500→2000 lux) scenarios; and reduce attitude calculation error by 66% under rapid rotation (6 rad / s) conditions. For resource-constrained embedded devices, a simplified adjustment strategy can be adopted: when E t When / E0>1.5, k is fixed at 1.1, E t When / E0 < 0.7, take k = 0.6; within the linear interval, take k = 0.8 + 0.4(E t Calculate using / E0-0.7).

[0126] To further support the implementation effect and engineering feasibility of the above dynamic adaptive mechanism in actual systems, the following sections (3.1-3.4) will provide a detailed explanation of the underlying implementation and performance parameters of the key components of the system from four aspects: mechanical installation tolerance control, circuit transient response characteristics, numerical optimization solution strategy and mutual information calculation accuracy.

[0127] 3.1 Mechanical Installation Tolerances and Dynamic Response Characteristics

[0128] During the implementation of the system's mechanical structure, the mounting brackets for the event camera and IMU sensor are made of 7075 aerospace-grade aluminum alloy and precision-machined using five-axis CNC machining. During installation, the reference flatness is first measured using a laser interferometer to ensure the flatness error of the mounting surface is controlled within 5μm. Subsequently, an electronic level and autocollimator are used for axis alignment, ensuring that the actual angle θ between the optical axis and the IMU sensing axis satisfies tanθ≤0.00087 after cubic polynomial fitting. Within the operating temperature range of -20℃ to 60℃, the deformation compensation capability of the mounting structure is verified through finite element thermodynamic analysis. The compensation force F of the preload spring exhibits a linear relationship with the temperature change ΔT: F=0.1+0.002|ΔT| (unit: N), effectively suppressing relative displacement caused by thermal deformation.

[0129] 3.2 Transient Response Characteristics of Dynamic Bias Circuit

[0130] The dynamic bias circuit employs a three-stage cascaded architecture: the first stage is a reference voltage source, using an ADR4525 to provide an initial reference of 2.5V ± 0.01%; the second stage is a programmable gain amplifier, using an AD8251 to achieve voltage regulation from 1.8V to 3.3V; the third stage is a noise suppression module, using a π-type LC filter (L = 2.2μH, C = 10μF × 3) to control the output ripple below 15μVrms. When the system detects a sudden change in ambient light intensity, the regulation circuit stabilizes the voltage within 180μs. Its transient response exhibits typical characteristics of an underdamped second-order system, with an experimentally measured damping ratio ζ = 0.72 ± 0.03 and a natural frequency ωn = 3.5 x 10⁻⁶. 3 rad / s. In practical applications, the dynamic adjustment of the bias voltage follows V bias The exponential law of 1.8 + 1.5[1 - exp(-t / 60μs)] ensures the stable operation of the event camera under different lighting conditions.

[0131] 3.3 Numerical Implementation of Joint Optimization Solution

[0132] The vision-inertial joint optimization solver adopts a modular design in its implementation. First, a sparse Hessian matrix containing 156 state variables is constructed, where the diagonal blocks Hvv are stored in compressed columnar format, and the off-diagonal blocks Hvi are stored in CSR format. The solution process consists of two stages: the first stage processes the vision-related block matrix using the SimpleLLT decomposer provided by the Eigen library; the second stage applies the preconditional conjugate gradient (PCG) method to the inertial-related blocks, with the preconditioning matrix being a block diagonal approximation of Hii. During the PCG iteration, the norm of the residual vector is calculated using the Kahan summation algorithm to improve numerical stability. When the relative residual norm ‖r k || / ||r0|| reaches 10 -4The calculation terminates when the number of iterations exceeds 50. Experimental results show that the solver takes an average of 1.83ms on a 1.8GHz ARM Cortex-A72 processor, meeting real-time requirements.

[0133] 3.4 Calibration method of adjustment coefficient k

[0134] To further enhance the environmental adaptability of the sliding window adjustment mechanism, the system needs to accurately calibrate the adjustment coefficient k. The following section will introduce a systematic calibration method for the k value and verify its practical effects.

[0135] In the process of adaptive adjustment of dynamic parameters, an optimized calibration method for the adjustment coefficient k was established through systematic experiments. This calibration process employs the following... Figure 9 The six-degree-of-freedom motion simulation platform (Moog FCS-Electric C500) shown is complete, possessing a translational accuracy of ±0.01mm and a rotational positioning accuracy of ±0.005°. The experimental environment is configured with a Thorlabs MLS203-1 multispectral light source system, capable of precisely adjusting the illumination intensity from 100 to 10,000 lux within 50ms. This embodiment's six-degree-of-freedom platform can be implemented using electrically parallel mechanisms known in the art, including but not limited to the Moog FCS-Electric C500 and PI H-840 models, which possess equivalent precision in motion simulation. During the experiment, it is necessary to ensure that the platform's closed-loop control bandwidth is ≥50Hz to meet the real-time requirements of dynamic parameter adjustment.

[0136] During calibration, IMU benchmark tests were first conducted in a constant-temperature (23±1℃) cleanroom environment. Static IMU data was continuously collected for 2 hours, and Allan variance analysis was used to obtain the angular velocity random walk coefficient as 0.00015 rad / √Hz. Based on this, an exponentially decaying integral term was used to preprocess the raw angular velocity data, with a time constant set to 1.2 seconds. This preprocessing effectively suppressed measurement noise introduced by high-frequency vibrations.

[0137] During the dynamic calibration phase, the motion platform is controlled to run along a preset trajectory, which includes angular velocity variations of 0.5–10 rad / s and various motion modes such as sinusoidal frequency sweep and step excitation. Simultaneously, the illumination system adjusts its intensity in 10 levels according to a logarithmic law, with each test combination repeated three times to eliminate random errors. The event camera's output data undergoes entropy calculation every 50 ms, and the intensity distribution is statistically analyzed using a 16-level grayscale histogram. The normalized event entropy Et is then obtained through normalization.

[0138] Parameter optimization using a grid search method revealed that the system exhibited optimal performance under all test conditions when k=1.05. Specifically, the positioning RMSE reached 0.12m (σ=0.03), a 57% improvement in accuracy compared to the traditional fixed-window method. A significance test (p=0.0023) confirmed the statistical significance of this optimization. Particularly under the condition of a sudden change in angular velocity from 5 to 8 rad / s, the trajectory offset was reduced to 42% of that of the unoptimized system.

[0139] Based on extensive experimental data, the inventors established a dynamic adjustment strategy for the k-value. Under conditions where the angular velocity exceeds 5 rad / s, linear compensation is achieved using the ratio of the event entropy Et to the baseline value E0; under normal conditions, a hyperbolic tangent function is used to achieve a smooth transition. This strategy was further optimized for surgical navigation applications. A dedicated adjustment formula was developed for the 1.8 entropy baseline specific to electrosurgical operations, enabling a static tissue measurement accuracy of 0.15 mm and maintaining a positioning accuracy of 0.8 mm even under rapid cutting conditions.

[0140] Validated on an ABB IRB6700 industrial robot, this calibration method ensures stable repeatability of positioning accuracy within ±0.6mm during palletizing operations, and ±0.3mm under high-speed assembly conditions. Experimental data demonstrates that this k-value calibration scheme effectively adapts to different lighting conditions and motion states, providing reliable technical support for the adaptive algorithm of this invention.

[0141] 3.5 Quantitative Compensation Method for Mutual Information Calculation

[0142] The hardware implementation of the mutual information calculation module is based on the programmable logic section of a Xilinx Zynq FPGA. In the timing design, the 1ms time window is divided into 1024 clock cycles, with each cycle completing event counting for a 4×4 pixel area. The quantization compensation unit uses 32-bit fixed-point arithmetic, and its compensation coefficient is dynamically adjusted according to the actual number of sampling points Ns: when Ns < 5000, the compensation coefficient is 0.85 / Ns; when Ns ≥ 5000, it is 0.72 / Ns. To prevent data overflow, the accumulator is designed with a 48-bit width, and normalization is performed after every 256 accumulations. Practical testing has verified that this compensation scheme can reduce the mutual information calculation error from 7.2% to less than 0.8% in typical operating scenarios, significantly improving the accuracy of multi-sensor data fusion.

[0143] (iv) Closed-loop testing and system characteristic verification

[0144] During system operation, to improve global consistency and suppress drift errors, a keyframe map based on event mutual information metric is continuously constructed. Specifically, the system adopts a spatiotemporal pyramid matching strategy, segmenting the event stream according to time scales such as 1ms, 5ms, and 10ms to construct a multi-scale event intensity distribution, and calculating the mutual information value between keyframes as a similarity metric.

[0145] The formula for calculating mutual information is as follows: ;

[0146] in:

[0147] This represents the joint probability distribution of event intensity in keyframes A and B;

[0148] These are the marginal probability distributions of event intensity in keyframes A and B, respectively.

[0149] Mutual information value Used to measure keyframe similarity and as a basis for loopback triggering.

[0150] like Figure 7 The diagram shows the closed-loop detection trigger logic. The system compares the mutual information values ​​of events between keyframes with the displacement difference estimated by the IMU to determine whether the closed-loop condition is met. If the cumulative mutual information value exceeds a threshold and the displacement difference is below a set range, the pose graph optimization process is triggered to achieve loop closure correction.

[0151] To integrate structural similarity at different time granularities, the system introduces a weighted multi-scale cumulative mutual information calculation method: ;

[0152] in, The time decay factor is recommended to be [value missing]. , , These represent the event distribution of frame pairs at a time scale of t.

[0153] When the cumulative mutual information value When the displacement difference calculated by the IMU pre-integration exceeds the set threshold (e.g., 0.75) and is less than 0.2m, the system will automatically trigger the closed-loop detection module to perform pose graph optimization and loop closure correction, thereby significantly suppressing the spread of cumulative error.

[0154] In actual testing, such as in a warehouse AGV scenario, the system successfully identified the loop closure on the third loop (t=215s). Through this mechanism, the cumulative error was reduced from 2.1m to 0.4m, verifying its global consistency correction capability.

[0155] In addition, the stability and real-time performance of this system were comprehensively tested through three typical application scenarios:

[0156] 1. High-speed obstacle avoidance for UAVs: The system flies at a speed of 12m / s in dense obstacle environments, achieving a 97.5% obstacle avoidance success rate thanks to a 500Hz pose update rate and a highly robust event filtering strategy.

[0157] 2. Surgical navigation application: By locking the filter in the wavelength range near the hemoglobin absorption peak (approximately 550nm), the system maintains a positioning accuracy of 0.3mm even on soft tissue surfaces with a reflectivity of less than 5%.

[0158] 3. Underwater SLAM Experiment: In a water environment with a turbidity of 15 NTU, the event intensity normalization model was adjusted to... Combined with a blue channel filter, it effectively suppresses reflection noise, reducing the error by 63% compared to traditional image SLAM solutions.

[0159] The following are several preferred or improved embodiments to further illustrate the technical implementation details and advantages of the present invention. These embodiments do not constitute a limitation on the scope of protection of the present invention.

[0160] Example 1: Image Frame-Assisted Event Normalization Mechanism

[0161] To enhance the system's positioning stability in areas with weak texture or excessively flat surfaces, this invention introduces an image frame difference-assisted event intensity normalization strategy. While receiving the event stream, the system simultaneously acquires grayscale image frames and constructs a pixel change image ΔI(x,y) through inter-frame difference. The event trigger intensity p is normalized according to the following relationship:

[0162] ;

[0163] Where ε is the regularization factor. This processing significantly improves the stability of event triggering in sparse texture regions and effectively controls false triggering. Experiments show that in pathological image navigation scenarios, this mechanism improves the accuracy of event validity judgment by 21.4% and reduces the false triggering rate of pulses by 31.7%.

[0164] Example 2: Event Cube Construction and Consistency Determination Mechanism

[0165] The system constructs an xyt spatial cube centered on each event e(x,y,t,p), with a time window of 1ms and a spatial window of ±3 pixels. If there are events with consistent polarity in the neighborhood that satisfy the following constraints:

[0166] ;

[0167] The event is then considered a valid event. This strategy significantly improves the accuracy of spatiotemporal consistency screening, eliminating an average of 87% of isolated spurious events in experiments, thus providing high signal-to-noise ratio input for subsequent neuron modeling.

[0168] Example 3: Neural coding mechanism based on LIF model

[0169] The event filtering module employs a LIF (Leaky Integrate-and-Fire) neural model to sparsely encode valid events. This model maintains the membrane potential Vm as iteratively decays over time and outputs a pulse when the accumulated intensity exceeds a threshold θ.

[0170] ;

[0171] If Vm(t+1)≥θ, trigger the pulse and reset.

[0172] Preferred parameters:

[0173] Leakage coefficient λ = 0.85;

[0174] Trigger threshold θ = 0.6;

[0175] This model has a high compression capability for large-scale event streams, retaining more than 97% of high-value events in tests, while reducing data redundancy by more than 80%.

[0176] Example 4: Event Entropy-Driven Adaptive Adjustment of Sliding Window

[0177] like Figure 5 As shown, this invention proposes a sliding window adjustment method based on event intensity entropy. After event intensity normalization, the system calculates the intensity histogram every 50ms, calculates the entropy value H, and dynamically adjusts the optimization window size N according to the following function: ;in:

[0178] N0 is the baseline frame number;

[0179] H0 is the reference value for the system calibration entropy;

[0180] k∈[0.5,1.2] is the adjustment coefficient.

[0181] When the IMU angular velocity is higher than 5 rad / s, the frame limit N∈[5,8] is enforced, and the optimization variable dimension is simplified to accelerate convergence. Under high-speed rotation conditions, the sliding window mechanism can reduce the RMSE error from 0.36m to 0.15m, an improvement of 58%.

[0182] Example 5: Event-Inertia Cooperative Residual Modeling

[0183] In the joint cost function, besides the traditional event brightness residual r e With IMU pre-integration residual r I In addition, event-inertia constraint residuals r are further introduced. c And construct the adaptive weighting factor ω: ;

[0184] Where a is the current acceleration, g is the gravitational acceleration constant, and ω0 is the initial weight. This modeling enhances the consistency constraints of event-IMU fusion under high dynamic conditions. In the acceleration range above 3.5 m / s², r is added. c The system positioning error decreased by 42% after the project.

[0185] Example 6: Event Mutual Information-Driven Loopback Detection Mechanism

[0186] The system constructs a multi-scale event intensity distribution map and calculates the cumulative mutual information value I between keyframes. accum : ;

[0187] Where λt is the time decay weight, and t represents the time scale (e.g., 1ms, 5ms, 10ms). When the cumulative mutual information > 0.75 and the IMU estimated displacement difference < 0.2m, the system determines that a loop closure has occurred and triggers the global graph optimization process. In AGV testing, the success rate of loop closure triggering increased from 52% to 86%.

[0188] Example 7: Dynamic bias circuit and filter coordination mechanism

[0189] like Figure 2 As shown, the event sensing module integrates a dynamic bias circuit and a MEMS tunable filter. The bias circuit adjusts the pixel bias voltage according to the event trigger rate, with a variation range of ±15%, controlling the response time to <200ns. The filter wavelength can be adjusted between 400-1000nm, automatically switching to the blue light band (e.g., 400-500nm) in low-light scenes to enhance visibility.

[0190] In a 1000 lux test scenario, this mechanism can reduce the proportion of pseudo-events from 17.6% to 4.1%; in soft tissue navigation tasks, the positioning error is controlled within 0.3 mm, which is significantly better than the existing DVS standard configuration system.

[0191] Example 8: Two-layer time synchronization compensation mechanism

[0192] To overcome the timing offset between the IMU and the event stream under complex operating conditions, this invention adopts a combined strategy of "hardware timing + software residual compensation":

[0193] The hardware layer uses ZED-F9P GPS timing + FPGA interrupt sampling, with the time difference controlled within <20μs;

[0194] Software layer construction of aligned residual term r syncr The sampling offset is corrected by minimizing the error.

[0195] This mechanism is particularly suitable for edge-deployed or high-temperature drift systems, ensuring data fusion with sub-millisecond accuracy. The overall fusion error is reduced by approximately 19.7% compared to software alignment.

[0196] The aforementioned enhancement modules can be used independently or in combination in different application scenarios, such as surgical navigation, underwater SLAM, autonomous driving, and drone obstacle avoidance. They significantly improve the accuracy, response speed, and robustness of the system and constitute the key technical innovation points and creative supporting elements of this invention.

[0197] Example 9: Application Validation Data and Performance Comparison

[0198] To further verify the application advantages of this invention, the trajectory accuracy, event perception performance, sliding window adjustment response capability, and closed-loop recognition effect of this invention and traditional solutions were compared in typical dynamic scenarios. The following experiments were built on a unified event visual localization platform and compared using publicly available datasets and real-world scene data.

[0199] Table 1: Comparison of RMSE under different scenarios (unit: m)

[0200]

[0201] Table 2: Event Triggering Performance and Pseudo-Event Ratio

[0202]

[0203] Table 3: Event Entropy vs. Results of Sliding Window Resizing

[0204]

[0205] Table 4: Loopback Detection Success Rate and Loop Closure Accuracy

[0206]

[0207] In summary, this invention achieves high-precision, low-latency visual positioning in complex environments such as high speed, strong dynamics, and weak texture by introducing a highly responsive event-aware mechanism, a precisely calibrated IMU synchronization scheme, and a hardware-accelerated tightly coupled optimization module. The system design maintains a compact structure while also possessing algorithmic adaptability and scalability, making it suitable for various scenarios such as UAV navigation, augmented reality, industrial automation, and intelligent transportation.

[0208] It should be noted that the embodiments described in this invention are merely preferred examples and do not constitute a limitation on the scope of protection of this invention. Those skilled in the art can make equivalent substitutions or adjustments to the structure of each module, parameter configuration, and algorithm flow without departing from the core idea of ​​this invention, and all such substitutions or adjustments should be covered within the scope of protection of this invention.

Claims

1. An ultra-low latency visual positioning method based on an event camera, characterized in that, Includes the following steps: S1. Through a hybrid signal processing unit integrating an FPGA chip and an ARM processor, the event stream data output by the event sensing module and the IMU data output by the inertial measurement unit are received asynchronously; The event sensing module includes a photoelectric sensor array with dynamic response characteristics, wherein the response delay of a single pixel in the sensor array is ≤10μs; The inertial measurement unit (IMU) is rigidly connected to the event sensing module, with a physical center distance of ≤8mm and an optical axis alignment error of <0.5°. The mixed signal processing unit performs the following operations: a) Preprocess the event stream, including filtering out noisy events based on spiking neural networks; b) Perform motion compensation based on IMU data; c) Align event data with IMU data using a timestamp synchronization module to ensure a time synchronization error of ≤20μs; S2. Perform spatiotemporal consistency filtering on the event stream, the filtering including: constructing an event cube in xyt space, removing outlier events, and performing pulse coding processing based on the Leaky Integrate-and-Fire neuron model; S3. Perform event-IMU tight coupling optimization, including: A joint cost function is constructed, comprising an event brightness residual term, an IMU pre-integration residual term, and an event-inertial cooperative residual term. The optimization objective is to minimize the following joint cost function: ; in: The brightness residual corresponding to the i-th event frame. For the j-th IMU pre-integration residual, For event-IMU constrained residuals, For residual collaborative weights; The optimization employs the First Estimate Jacobians (FEJ) method to maintain observation consistency; S4. Output the 6-DOF pose estimation result at the current time, with an update frequency of no less than 500Hz; The optimization employs a sliding window mechanism, with the window size dynamically adjusted based on event entropy.

2. The method according to claim 1, characterized in that, Step S1 further includes: It synchronously receives image frame data and event stream data, and performs inter-frame differential processing on the image frames to assist in the normalization correction of event trigger intensity; The sampling interval of the image frames is 20ms ± 1ms to enhance the stability of event response and positioning accuracy in low-texture areas.

3. The method according to claim 1, characterized in that, The rules for determining the spatiotemporal consistency of event e(x,y,t,p) in step S2 include: In the spatiotemporal neighborhood N(e) of event e, if there exists an event e'(x',y',t',p') that satisfies the following condition, then event e is considered a valid event: ,in ; Spatial distance satisfies ,in =3 pixels.

4. The method according to claim 1, characterized in that, The joint cost function in step S3 also includes an event-inertia constraint term, the weight ω of which is adaptively adjusted according to the current acceleration measurement value, satisfying the following relationship: ; in This represents the current acceleration measurement, where g is the gravitational acceleration constant.

5. The method according to claim 1, characterized in that, Step S3 further includes: The event intensity entropy value H is calculated based on the event flow and used to dynamically adjust the sliding window size. The event entropy H is defined as follows: ; Where f(p) represents the normalized frequency distribution within the event intensity interval p; P is the set of all discretized intensity intervals; The event entropy H is used to drive the adaptive adjustment of the sliding window size N to adapt to scenarios with different motion complexities.

6. The method according to claim 1, characterized in that, It also includes the following steps: Local keyframe maps are constructed based on event streams, and loop closure detection is performed by combining IMU state estimation. When the event mutual information metric between two keyframes exceeds a preset threshold, pose closure correction is performed to improve global consistency. The formula for calculating the mutual information metric is as follows: ; in: This represents the joint probability distribution of event intensity in keyframes A and B; These are the marginal probability distributions of event intensity in keyframes A and B, respectively. Mutual information value Used to measure keyframe similarity and as a basis for loopback triggering.

7. The method according to claim 1, characterized in that, The event sensing module includes: Tunable optical filters whose transmission wavelength can be programmably adjusted in the range of 400-1000nm; The dynamic bias circuit automatically adjusts the pixel bias voltage in response to changes in ambient light intensity, with an adjustment range of ±15% of the reference voltage.

8. The method according to claim 7, characterized in that, The control logic of the dynamic bias circuit includes: When the event trigger rate exceeds 1×10⁻⁶ per unit time, 6 When the events / s are constant, the bias voltage will be reduced by 10%-15%; When the event trigger rate is less than 1×10 within 5ms consecutively 4 Increase the bias voltage by 5%-8% for each event / s.

9. The method according to claim 1, characterized in that, The hybrid signal processing unit includes: The hardware-accelerated Bundle Adjustment (BA) module is optimized using the truncated Newton method and is configured as follows: a) Maintain a dynamically adjustable sliding window, where the window size N satisfies the following relationship: ； in The entropy value of the current event. Let k be the baseline entropy value, k∈[0.5,1.2]; b) When an angular velocity greater than 5 rad / s is detected, N is forcibly limited to the range of 5-8 frames.

Images