Ultra-low delay visual positioning system based on event camera and method thereof

Abstract

The invention relates to an ultralow-delay visual positioning system and method based on an event camera. The system comprises an event sensing module, an inertial measurement unit (IMU) and a mixed signal processing unit. The event sensing module has a microsecond-level response characteristic and is integrated with a dynamic bias adjustment and tunable optical filter; the IMU is rigidly connected with the IMU, and the physical center alignment error is less than 0.5 degree; and the hybrid processing unit adopts an FPGA (Field Programmable Gate Array) and ARM (Advanced RISC Machines) collaborative architecture to realize event filtering, time synchronization, motion compensation and tight coupling optimization. According to the method, an event brightness residual error and an IMU pre-integration residual error are combined to construct a joint cost function, a sliding window is dynamically adjusted based on event entropy, and loop-back and closed-loop correction is realized through mutual information criteria. The method can realize six-degree-of-freedom pose estimation as high as 500Hz, has the advantages of low delay, high precision, high robustness and the like, and is suitable for high-speed scenes such as unmanned aerial vehicles, AR navigation, industrial robots and the like.

Description

Technical Field

[0001] The present invention relates to the fields of robot perception and computer vision technology, and in particular to an ultra-low latency visual positioning system and method based on an event camera. The system is suitable for achieving high-precision, low-latency pose estimation in high-speed motion or high-dynamic environments, and belongs to the technical scope related to event visual positioning and visual inertial navigation systems. Background Art

[0002] With the increasing demand for high-precision, low-latency positioning in applications such as autonomous driving, unmanned aerial vehicles, augmented reality, and robotics, visual-inertial odometry (VIO) positioning systems have become one of the mainstream technical solutions in current research and engineering practice. Traditional VIO systems primarily rely on frame-based cameras to capture image sequences, combined with angular velocity and acceleration information provided by inertial measurement units (IMUs) to estimate the device's six-degree-of-freedom (6-DOF) pose. However, in complex environments such as high dynamics, high contrast, and extreme lighting, traditional frame-based vision systems expose multiple key bottlenecks, severely restricting their stability and accuracy in practical applications.

[0003] First, because traditional frame cameras capture images at a fixed frame rate (such as 30FPS or 60FPS), they are prone to significant motion blur when the device moves at high speeds (e.g., >5m / s). This image blur can disrupt the clarity and stability of key features, affecting feature extraction, matching, and tracking in the visual front-end, leading to a significant decrease in positioning accuracy. Field measurements have shown that in high-speed motion scenes, image blur rates can reach over 60%, severely limiting the usability of traditional vision systems for highly dynamic tasks.

[0004] Secondly, in traditional VIO systems, the IMU and camera modules generally rely on software-level or externally triggered synchronization mechanisms for time synchronization. This makes it difficult to avoid time alignment errors caused by non-ideal factors such as sensor sampling clock drift and data transmission delays. Especially in systems with high requirements for low latency and high-frequency updates, time synchronization errors exceeding 100μs can easily cause the spatial correspondence between image and IMU data to shift, leading to cumulative drift in pose estimation, and accuracy errors of up to 0.1° / m.

[0005] Thirdly, in high dynamic range (HDR) scenarios, such as backlighting, entering and exiting tunnels, and areas with strong contrast, traditional frame cameras generally use a fixed exposure time mode, making it difficult to simultaneously capture both dark details and bright information. This results in saturated or underexposed images in dynamic ranges exceeding 120dB, leading to the loss of a significant amount of effective visual information. This can cause positioning failures or even system crashes in environments with drastic lighting changes.

[0006] In summary, current mainstream visual-inertial positioning technology still faces many challenges when dealing with high-speed motion, asynchronous sensor data fusion, and complex lighting environments. It is urgent to propose innovative improvements in sensor hardware responsiveness, spatiotemporal alignment accuracy, and information robustness to meet the higher requirements for real-time, accuracy, and reliability in future complex application scenarios. Summary of the Invention

[0007] To overcome the problems of existing visual positioning systems in highly dynamic scenes, such as high latency, unstable accuracy, and poor adaptability to weak textures, this paper proposes an ultra-low-latency visual positioning system and method based on an event camera. This system combines an event-based vision sensor with a microsecond response time and a tightly coupled inertial fusion algorithm. Through hardware and software collaborative design, it achieves high-frequency and robust pose estimation in complex environments.

[0008] The core innovation of this invention lies in: introducing an event perception module with dynamic response characteristics to realize asynchronous data acquisition; building a tightly coupled event-IMU joint optimization framework to improve positioning accuracy and stability through event entropy-driven sliding window adjustment and event-inertia constraint modeling; and combining a multi-scale mutual information closed-loop detection strategy to enhance the global consistency of the system.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] In one possible implementation, an ultra-low latency visual positioning system based on an event camera is provided, including:

[0011] An event sensing module comprising a photosensor array with dynamic response characteristics, wherein the response delay of a single pixel in the sensor array is no greater than 10 μs;

[0012] An inertial measurement unit (IMU) is rigidly connected to the event sensing module, with the physical center distance between the two being no greater than 8 mm and the optical axis alignment error being less than 0.5°;

[0013] The mixed signal processing unit integrates an FPGA chip and an ARM processor and is configured to perform the following operations:

[0014] a) Preprocessing the event stream, including noise event filtering based on spiking neural networks;

[0015] b) Perform motion compensation based on IMU data;

[0016] c) The event data and IMU data are aligned through the timestamp synchronization module so that the time synchronization error is no more than 20μs.

[0017] Furthermore, the event sensing module further includes: a tunable optical filter, whose transmission wavelength can be programmably adjusted within the range of 400-1000nm; a dynamic bias circuit, which automatically adjusts the pixel bias voltage in response to changes in ambient light intensity, with an adjustment range of ±15% of the reference voltage.

[0018] Furthermore, the control logic of the dynamic bias circuit includes: when it is detected that the event trigger rate per unit time exceeds 1×10 6 events / s, reduce the bias voltage by 10%-15%; when the event trigger rate is less than 1×10 4 events / s, increase the bias voltage by 5%-8%.

[0019] Furthermore, the mixed signal processing unit further includes: a hardware-accelerated bundle adjustment (BA) module, which uses the truncated Newton method for optimization and is configured as follows:

[0020] a) Maintain a dynamically adjusted sliding window with a window size N that satisfies the following relationship:

[0021]

[0022] Among them E t is the current event entropy value (a dynamic indicator that characterizes the amount of information in the event flow and reflects the complexity of the scene), E0 is the benchmark entropy value (the average event entropy reference value in the system calibration phase), k∈[0.5,1.2];

[0023] The adjustment coefficient k in this invention is not a fixed empirical variable, but an optimization variable determined through dynamic calibration and adaptive working condition strategies. This method significantly differs from existing visual-inertial estimation schemes based on fixed window parameters and has greater adaptability and precision control capabilities.

[0024] b) When the angular velocity is detected to be greater than 5 rad / s, N is forced to be limited to the range of 5-8 frames.

[0025] In another possible implementation, the present invention further provides a visual positioning method based on the above system, comprising the following steps:

[0026] S1. Asynchronously receive event stream data output by the event perception module and IMU data output by the inertial measurement unit;

[0027] S2. Performing spatiotemporal consistency filtering on the event stream, the filtering comprising: constructing an event cube in xyt space, removing outlier events, and performing pulse coding processing based on a Leaky Integrate-and-Fire neuron model;

[0028] S3. Perform event-IMU tight coupling optimization, including:

[0029] A joint cost function is constructed, which includes the event brightness residual term, the IMU pre-integration residual term, and the event-inertia cooperation residual term. The optimization goal is to minimize the following joint cost function:

[0030]

[0031] in:

[0032] is the brightness residual corresponding to the i-th event frame,

[0033] is the j-th IMU pre-integration residual,

[0034] r c is the event-IMU constraint residual,

[0035] ω is the residual collaborative weight;

[0036] The First Estimate Jacobians (FEJ) method was used in the optimization to maintain observation consistency;

[0037] S4. Output the 6-DOF pose estimation result at the current moment, with an update frequency of no less than 500 Hz.

[0038] Preferably, step S1 further includes: synchronously receiving image frame data and event stream data, and performing inter-frame difference processing on the image frame to assist in normalizing the event trigger intensity; the sampling interval of the auxiliary image frame is 20ms±1ms to improve the positioning stability of the weak texture area.

[0039] Preferably, the rules for determining the spatiotemporal consistency of the event e(x, y, t, p) in step S2 include:

[0040] In the spatiotemporal neighborhood N(e) of event e, if there exists an event e'(x', y', t', p') that satisfies the following conditions, then event e is considered a valid event:

[0041] |t-t'|<Δt, where Δt=1ms;

[0042] The spatial distance satisfies d((x,y),(x',y'))<Δd, where Δd=3 pixels.

[0043] Furthermore, the joint cost function of step S3 also includes an event-inertia constraint term, whose weight ω is adaptively adjusted according to the current acceleration measurement value to satisfy the following relationship:

[0044]

[0045] where a t is the current acceleration measurement value, and g is the gravitational acceleration constant.

[0046] Preferably, step S3 further comprises:

[0047] The event intensity entropy value H is calculated based on the event stream and used to dynamically adjust the sliding window size, where the event entropy H is defined as follows:

[0048]

[0049] Where f(p) represents the normalized frequency distribution within the event intensity interval p; P is the set of all discretized intensity intervals;

[0050] The event entropy H is used to drive the adaptive adjustment of the sliding window size N to adapt to scenes with different motion complexities.

[0051] Furthermore, the above method further comprises the following steps:

[0052] Construct a local keyframe graph based on the event stream and perform closed-loop detection in combination with IMU state estimation;

[0053] When the event mutual information measure between two keyframes exceeds a preset threshold, pose loop correction is performed to improve global consistency;

[0054] The mutual information metric calculation formula is:

[0055]

[0056] in:

[0057] P(p A ,p B ) represents the joint probability distribution of event intensities in key frames A and B;

[0058] P(p A )、P(p B ) are the marginal probability distributions of event intensity in key frames A and B respectively;

[0059] The mutual information value I(A,B) is used to measure the similarity of key frames and serves as the basis for loop triggering.

[0060] By constructing an event entropy-driven sliding window adaptive mechanism, combined with an optimization strategy for the dynamic adjustment coefficient k, this invention demonstrates significant performance improvements in real-world high-dynamic test environments. For example, in a sudden change in strong light intensity from 500 lux to 2000 lux, the system can maintain a tracking offset of less than 0.3 meters, reducing positioning error by an average of approximately 57% compared to a fixed window mechanism. At an angular velocity of 6 rad / s, attitude solution accuracy improves by over 66%, demonstrating the system's robustness and stability under complex operating conditions.

[0061] Based on the above technical solution, the present invention's ultra-low-latency event-camera-based visual positioning system achieves high-precision, robust real-time positioning performance by building a tightly coupled architecture integrating an event perception module, an inertial measurement unit, and a hybrid signal processing unit. This system features significant innovations in hardware, algorithms, and system-level collaborative design, significantly improving overall performance compared to existing visual-inertial positioning systems.

[0062] At the hardware level, the event perception module uses a photoelectric sensor array with dynamic response characteristics. The single-pixel response delay is as low as 10μs, which is faster than the response speed of mainstream event cameras (such as the 15μs of DVS346) and can more sensitively capture optical flow changes in high-speed scenes. The integrated tunable optical filter supports wavelength programming adjustment in the range of 400-1000nm to adapt to complex lighting conditions. The supporting dynamic bias circuit automatically adjusts the bias voltage through environmental perception to maintain a stable event trigger rate, effectively suppressing saturation and overexposure noise, and providing high signal-to-noise ratio input for subsequent processing.

[0063] At the algorithm level, the system's built-in spatiotemporal consistency filtering algorithm performs neighborhood analysis on the three-dimensional event cube, and combined with the LIF neuron coding model to achieve sparse compression and saliency enhancement of event data, greatly improving data utilization efficiency and feature fidelity; the joint optimization module uses event brightness residuals and IMU pre-integration residuals to construct a joint cost function, and combines the dynamic weight adjustment mechanism with the FEJ observation consistency method to achieve efficient fusion of event-inertial information; the event entropy-driven sliding window BA optimization strategy ensures that the system can still stably output high-quality pose estimates under intense motion (such as angular velocity > 5rad / s), preventing error accumulation and trajectory drift.

[0064] At the system level, the event camera and IMU utilize a rigid mounting structure with a physical center-to-center spacing of no more than 8mm and an optical axis alignment error of less than 0.5°. A hardware timestamp alignment module controls data synchronization errors to within 20μs, significantly reducing the impact of asynchronous sampling on the fusion algorithm. Furthermore, the system supports synchronous processing of image frames and event streams, and constructs local keyframe graphs based on the event stream, enabling closed-loop detection and loop optimization, effectively enhancing global consistency and positioning robustness.

[0065] In comprehensive tests, the system demonstrated excellent performance on both the CARLA simulation platform and the physical drone platform: it achieved an end-to-end latency of 2.1ms in high-speed drone scenarios and a pose estimation frequency of 500Hz, far superior to traditional solutions such as VINS-Fusion; it maintained sub-meter error and continuous working capabilities in complex environments (such as dark tunnels, high dynamic range scenes, sharp turns, etc.), with strong adaptability and high stability.

[0066] Therefore, this invention not only effectively overcomes the technical bottlenecks of traditional visual positioning systems, such as image blur, large timing errors between the IMU and camera, and the incompatibility of fixed exposure with high-dynamic environments, but also establishes a highly robust visual-inertial tightly coupled positioning system for complex and high-dynamic scenarios. This system is particularly suitable for applications requiring extremely high positioning accuracy and system response time, such as high-speed drones, minimally invasive surgical navigation, and autonomous driving under strong light conditions. It has broad engineering application value and widespread potential.

[0067] The technical solution of the present invention has been verified through multiple groups of control experiments, showing a dynamic adaptability that is significantly superior to the traditional fixed parameter mechanism. Under the conditions of rapid fluctuations in event entropy Et and high angular velocity input, the adaptive adjustment strategy can effectively coordinate the sampling frequency, voltage regulation amplitude and thermal compensation response to maintain system stability and accuracy. The measured results show that in typical indoor high-speed navigation tasks, the solution of the present invention can control the average positioning root mean square error (RMSE) to 0.12m (standard deviation σ = 0.03), and improve the mutual information-driven loop recognition accuracy to 92.3%, fully verifying its wide applicability in various high-dynamic scenarios such as robot navigation, industrial palletizing and surgical assisted positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 : Schematic diagram of the overall system structure;

[0069] Figure 2 : Dynamic bias circuit control schematic diagram;

[0070] Figure 3 : Event-IMU joint optimization flow chart;

[0071] Figure 4 : Event filtering structure diagram;

[0072] Figure 5 : Sliding window dynamically adjusts the logic graph;

[0073] Figure 6 : The residual term of the joint cost function constitutes a graph;

[0074] Figure 7 : Schematic diagram of closed-loop detection trigger logic;

[0075] Figure 8 : Flowchart of dynamic parameter adaptation mechanism;

[0076] Figure 9 : Schematic diagram of the k-value calibration experimental platform structure. DETAILED DESCRIPTION

[0077] This paper proposes an ultra-low-latency visual positioning system and method based on an event camera. This system integrates high-response event-sensing hardware with a tightly coupled multimodal optimization algorithm. This system achieves high-frequency, robust positioning with microsecond-level latency for pose estimation in high-speed dynamic scenes. To support these performance targets, the present invention integrates coordinated optimizations in system architecture design, spatiotemporal data synchronization, event filtering modeling, and inertial fusion computation.

[0078] To facilitate understanding of the present invention, some of the terms used in the specification are explained below:

[0079] 1. Event entropy H

[0080] Event entropy, H, is the normalized entropy value calculated by constructing a grayscale histogram based on pixel intensity changes output by the event camera within a given time window. This metric reflects the complexity and information density of event triggering in the current scene and is often used to dynamically adjust perception processing parameters. Its units are dimensionless, with a typical range of 0 to 4. This calculation is based on the Shannon entropy definition formula and is normalized to adapt to scenes of varying resolutions.

[0081] 2. Sliding window size N

[0082] The sliding window size N represents the number of keyframes retained by the system during the joint optimization process and is used to construct the state variables of the optimization problem. The size of N directly affects the optimizer's convergence speed, computational load, and tracking stability. To adapt to dynamic scene changes, this paper proposes an event-entropy-driven window adaptation mechanism. This mechanism dynamically adjusts the value of N to varying entropy values, typically within a range of 5 to 20 frames.

[0083] 3. Adjustment coefficient k

[0084] The adjustment coefficient k is a sensitivity parameter used to control the sliding window size, sampling frequency, and bias voltage adjustment range, typically ranging from 0.5 to 1.2. The value of k is determined by the current environmental event entropy Et and the baseline value E0, as well as the IMU's dynamic state, such as angular velocity and temperature. This paper establishes a mapping relationship between k and multiple factors through systematic calibration experiments, implementing a multidimensional adaptive adjustment strategy.

[0085] 4. Event Camera

[0086] An event camera is a visual sensor based on the principle of event-based imaging. Its output data is asynchronously triggered by pixel-level brightness change events, offering microsecond temporal resolution, low latency, and high dynamic range. Unlike traditional frame-based image acquisition, an event camera only outputs data when a scene changes, significantly reducing redundancy and improving response speed.

[0087] 5. Mutual Information

[0088] Mutual information (MI) is a metric that measures the statistical correlation between two signals or datasets. In this paper, it is used to determine the degree of match between the current keyframe and the historical frames. This metric is constructed based on the event density distribution or brightness histogram, and triggers the loop detection module when a set threshold is reached. The calculated mutual information effectively guides the graph optimization module in loop correction.

[0089] The following will describe in detail the various implementation details of the present invention according to the system structure, core algorithm processing flow and the linkage mechanism between functional modules, combined with the actual technical implementation path.

[0090] (1) System hardware integration

[0091] like Figure 1 As shown in the figure, it is a structural diagram of an ultra-low latency visual positioning system based on an event camera of the present invention. The system mainly includes an event perception module, an inertial measurement unit (IMU) and a mixed signal processing unit. The three form a rigid integration and work together to achieve high-precision, low-latency pose estimation. The event perception module uses the iniVation DVXplorer event camera, whose sensor array consists of 128×128 pixels, and each pixel has a dynamic bias circuit embedded in it. Figure 2 The figure shows the control principle diagram of the dynamic bias circuit. The circuit consists of three parts: an event trigger rate monitoring module, a control logic unit, and a voltage regulator chip. This forms a closed-loop bias regulation structure that can automatically adjust the pixel bias voltage when the event trigger rate changes, thereby improving the photosensitivity robustness. The bias circuit uses the TITPS7A4700 voltage regulator chip to achieve ±15% voltage regulation, which can automatically respond to changes in ambient light. Specifically, when the event trigger rate exceeds 1×10 6 When the trigger rate reaches 10 events / s, the FPGA control circuitry reduces the bias voltage by 12% within 200 nanoseconds to adapt to environmental conditions with high event trigger rates. In low-light environments, the tunable filter with a MEMS-Fabry-Perot interferometer structure adjusts the wavelength to achieve continuous adjustment from 400nm to 1000nm, optimizing image quality. In particularly low-light conditions, the system automatically switches to 400nm mode to enhance image visibility.

[0092] The IMU module uses a BMI088 six-axis inertial sensor, rigidly connected to the event sensing module via a 7075 aluminum alloy fixture. Laser tracker measurements show the physical center-to-center spacing of the sensor modules is 7.5 mm ± 0.1 mm. The optical axis error is precisely calibrated using a Thorlabs PAX1000 polarimeter, ensuring an error of less than 0.3°. The system also integrates a u-blox ZED-F9P GPS clock source, enabling high-precision synchronization of the event stream and IMU data via an FPGA hardware timer, maintaining a time synchronization error of less than 18 μs.

[0093] (2) Data processing and optimization

[0094] After the system is started, data preprocessing is performed first. Event stream data is processed by the 3D event cube building module. Each event (e(x,y,t,p)) is judged for validity within a 1ms time window and a 3-pixel spatial neighborhood. In e(x,y,t,p), x and y are pixel coordinates (unit: pixel), t is the timestamp (unit: μs), and p is the event polarity (+1 / -1 indicates brightness increase / decrease). Figure 4 The figure shows a schematic diagram of the three-layer structure of event filtering. The event data first enters the three-dimensional event cube construction module, and then passes through the three-layer neuron structure for spatiotemporal consistency judgment and pulse coding processing. This structure adopts the Leaky Integrate-and-Fire (LIF) model to effectively compress data and improve the signal-to-noise ratio. The membrane potential accumulation and pulse output logic jointly realize event validity screening. The Leaky Integrate-and-Fire neuron model is used for pulse coding, in which the leakage coefficient of the membrane potential is set to 0.85. When the event intensity accumulation reaches the pulse threshold = 0.6, the system triggers the pulse output. Experimental results show that this parameter configuration can effectively filter out 92.3% of motion blur pseudo events, especially under an illumination of 1000 lux, where the performance of this solution is particularly outstanding.

[0095] After the event data and IMU data are pre-processed and time-aligned, the system enters the tightly coupled optimization phase. Figure 3 As shown in the figure, it is a schematic diagram of the processing flow of the event-IMU joint optimization module of the present invention. The figure shows the steps of preprocessing of event data and IMU data, residual construction, joint optimization and pose output in sequence, which is used to assist in explaining the implementation path of multimodal data fusion and high-frequency pose estimation. The tightly coupled algorithm adopted in the present invention is executed in a mixed signal processing unit, and a hardware-accelerated bundle adjustment (BA) module is integrated. The truncated Newton method is used for minimization solution, and the optimization target is a joint cost function. Its optimization goal is to minimize the following joint cost function:

[0096]

[0097] like Figure 6 The figure shows the residual term composition diagram in the joint cost function of the present invention. The function contains three types of residuals: event brightness residual r e , IMU pre-integration residual r I and event-inertia co-residual r c , and introduce collaborative weight factors to achieve multimodal data fusion.

[0098] Among them, T i is the posture state of the i-th frame; is the event brightness residual, Σ e is its covariance matrix; is the IMU pre-integration residual, Σ I is its covariance matrix; r c is the event-inertia constraint residual term, and the adaptive weight ω is adaptively calculated according to the current acceleration. The specific calculation formula is:

[0099]

[0100] Where ω0 is the initial weight factor, a t is the linear acceleration measured by the IMU at the current moment, and g is the gravitational acceleration constant. This adaptive weight design is used to enhance the influence of the state consistency constraint between events and the IMU in highly dynamic environments. Its ability to suppress trajectory drift has been verified in AGV high-speed load testing.

[0101] To enhance optimization robustness, the system also introduces event brightness entropy H as a basis for dynamic adjustment of the sliding window. Event brightness entropy is expressed as the statistical divergence of the event intensity distribution per unit time (quantifying the degree of disorder of the event intensity distribution), and its calculation method is as follows:

[0102]

[0103] Where f(p) represents the normalized frequency distribution of the event intensity interval; the event entropy H is used to dynamically adjust the size of the BA sliding window to adapt to different motion complexity scenarios. p∈[0,1]. The event intensity range is discretized into 16 levels, and the entropy value E is calculated by counting the histogram every 50ms. t , the system dynamically adjusts and optimizes the sliding window size N according to the following relationship:

[0104]

[0105] Among them, N is the size of the current sliding window, E t is the entropy value of the current event, N baseis the number of frames in the benchmark window, E0 is the empirical entropy benchmark value, and k is the adjustment coefficient, which is determined by calibration experiments (0.5≤k≤1.2) to control the sensitivity of entropy to window size. Figure 5 As shown in the figure, the dynamic adjustment logic of the sliding window size N is driven by the current event entropy H. The system calculates the entropy value in real time by monitoring the divergence of the event intensity distribution per unit time, and adjusts and optimizes the window frame number according to the preset functional relationship to adapt to different dynamic complexity scenarios. The dynamic adjustment process of the sliding window size N uses event entropy as the driving variable, and activates the window limitation mechanism when the angular velocity exceeds the threshold. When the IMU angular velocity is detected to exceed 5rad / s, the system triggers the limitation strategy, fixing N to between 5 and 8 frames, and can close some parameter dimensions (such as translation) to improve the convergence speed. In actual tests, when the angular velocity reaches 8rad / s, this mechanism can reduce the trajectory error by 58%.

[0106] Experimental results show that the adaptive sliding window mechanism can effectively avoid numerical instability in the optimization process under conditions of rapid rotation or severe acceleration, and improve the global consistency and real-time performance of the system.

[0107] Example: Selection and calibration method of adjustment coefficient k

[0108] To improve the adaptability of the sliding window adjustment strategy to different motion scenes, the present invention provides a systematic calibration method for the adjustment coefficient k. The method includes the following steps:

[0109] 1. Dynamic scene incentive experiment

[0110] During the system initialization phase or when the application conditions change, a standard motion platform (such as a six-degree-of-freedom robotic arm) is built to perform sinusoidal frequency sweep excitation, with the angular velocity covering the range of 0.5–10 rad / s, and the event entropy Et and the true posture error δ are recorded.

[0111] Example conditions:

[0112] Trajectory form: ±30° pitch and yaw compound motion, frequency step change 0.1–5Hz

[0113] Light intensity: about 1000 lux, event trigger rate is stable at 2×10 5 events / s

[0114] 2. k-value grid search and error evaluation

[0115] Keep Nbase = 5, perform window adjustment and pose estimation by enumerating k∈[0.5,1.2] (step size 0.1), and calculate the root mean square error RMSE corresponding to each k:

[0116]

[0117] Among them, T est To estimate the pose, T gt is the true reference trajectory.

[0118] 3.kopt fitting and constraint rules

[0119] Select the value kopt corresponding to the minimum error from the search results, and further constrain it according to the dynamics of the actual scene:

[0120] If the average Et / E0>3, then kopt∈[0.9,1.2]

[0121] If Et / E0<1.5, then kopt∈[0.5,0.8]

[0122] Case example: In a high-speed turning experiment of a UAV, the peak angular velocity is about 8 rad / s. The calibration results show that the error is minimized (RMSE = 0.12m) when k = 1.05.

[0123] Application scenario description

[0124] Surgical navigation:

[0125] Due to the low event trigger rate in soft tissue endoscopy scenarios, it is recommended to use the correction formula:

[0126]

[0127] Among them E med is the baseline entropy value of medical device motion (such as E med ≈1.8).

[0128] Industrial Robot Verification:

[0129] Tested on an ABB IRB6700 robot, the results are as follows:

[0130] Working conditions k value trajectory error Routine handling 0.8 0.8mm / m High-speed assembly 1.1 0.5mm / m

[0131] (3) Dynamic parameter adaptation mechanism

[0132] The dynamic parameter adaptation mechanism of this system is realized through a three-level collaborative architecture. Figure 8 As shown in the figure, the event camera and IMU sensor first collect environmental information and motion status in real time, and the raw data is pre-processed and input into the adaptive decision unit built by the ARM processor. This unit calculates the current environmental feature value E based on the event entropy dynamic analysis algorithm. t and compares it with the reference value E0 to generate a preliminary adjustment signal.

[0133] For the parameter adjustment of the event camera, the system is implemented through an adjustable bias circuit built with the AD5391 digital potentiometer. t / E0>1.5), the regulation engine increases the bias voltage in 0.05V steps up to a maximum of 3.6V, a process that completes within 200μs. Simultaneously, the state machine within the FPGA monitors the angular velocity ω and linear velocity v output by the IMU. When ω exceeds 5 rad / s, it automatically switches to a 1kHz sampling rate. In static conditions (ω<0.2 rad / s), the sampling rate is reduced to 100Hz to reduce power consumption.

[0134] The temperature compensation module monitors the chip temperature in real time using a DS18B20 sensor. When the temperature exceeds 45°C, a protection strategy is activated: it limits the voltage adjustment range to within ±10% of the default value and sends a speed adjustment request to the cooling unit. All adjustment parameters are smoothed using a Kalman filter to avoid system oscillations caused by sudden parameter changes.

[0135] In the specific implementation process, the system initialization phase first collects 10ms environmental benchmark event stream to calculate E0, and loads the preset k=0.8 as the initial value. After entering the working state, the following adaptive adjustment process is executed every 20ms period: read the current event entropy E t and the IMU motion state vector [ω, v, T], and substitute into the dynamic adjustment equation k = 0.7 + 0.5tanh ((E t -E0) / E0)×(1+0.2ω / 10)×(1-0.01|T-25|) to calculate the new parameters, and finally constrain the k value to be in the range of [0.5,1.2] and send it to each execution unit.

[0136] Experiments show that this mechanism can reduce the tracking offset from complete loss to less than 0.3m in sudden strong light (500→2000 lux) scenes; and reduce the attitude solution error by 66% in fast rotation (6rad / s) conditions. For resource-constrained embedded devices, a simplified adjustment strategy can be adopted: when E t / E0>1.5, fixed k=1.1, E t / E0<0.7, take k=0.6, and in the linear range, take k=0.8+0.4(E t / E0-0.7) calculation.

[0137] To further support the implementation and engineering feasibility of the above-mentioned dynamic adaptive mechanism in actual systems, the following subsections (3.1-3.4) will provide detailed descriptions of the underlying implementation and performance parameters of key system components from four aspects: mechanical installation tolerance control, circuit transient response characteristics, numerical optimization solution strategy, and mutual information calculation accuracy.

[0138] 3.1 Mechanical installation tolerance and dynamic response characteristics

[0139] During the implementation of the system's mechanical structure, the mounting bracket for the event camera and IMU sensor is constructed from 7075 aviation aluminum alloy and precision-machined using five-axis CNC machining. During installation, the reference flatness is first measured using a laser interferometer to ensure that the mounting surface flatness error is within 5μm. An electronic level and autocollimator are then used to align the axes, ensuring that the actual angle θ between the optical axis and the IMU's sensitive axis satisfies tanθ ≤ 0.00087 after fitting a cubic polynomial. Finite element thermodynamic analysis verified the deformation compensation capability of the mounting structure within an operating temperature range of -20°C to 60°C. The compensation force F of the preload spring exhibits a linear relationship with the temperature change ΔT: F = 0.1 + 0.002|ΔT| (unit: N), effectively suppressing relative displacement caused by thermal deformation.

[0140] 3.2 Transient Response Characteristics of Dynamic Bias Circuit

[0141] The dynamic bias circuit is implemented using a three-stage cascade architecture: the first stage is a voltage reference, using the ADR4525 to provide an initial reference of 2.5V ± 0.01%. The second stage is a programmable gain amplifier, using the AD8251 to adjust the voltage from 1.8V to 3.3V. The third stage is a noise suppression module, using a π-type LC filter (L = 2.2μH, C = 10μF × 3) to control the output ripple to below 15μVrms. When the system detects a sudden change in ambient light intensity, the regulation circuit completes voltage stabilization within 180μs. Its transient response exhibits typical underdamped second-order system characteristics. Experimental measurements show a damping ratio of ζ = 0.72 ± 0.03 and a natural frequency of ωn = 3.5x10 3 rad / s. In practical applications, the dynamic adjustment of bias voltage follows V bias =1.8+1.5[1-exp(-t / 60μs)], which ensures the stable operation of the event camera under different lighting conditions.

[0142] 3.3 Numerical implementation of joint optimization solution

[0143] The visual-inertial joint optimization solver adopts a modular design in its specific implementation. First, a sparse Hessian matrix containing 156 state variables is constructed, where the diagonal block Hvv is stored in compressed column format and the non-diagonal block Hvi is stored in CSR format. The solution process is divided into two stages: the first stage processes the visual-related block matrix through the SimplicialLLT decomposer provided by the Eigen library, and the second stage applies the preconditioned conjugate gradient method (PCG) to the inertial-related block, and the preconditioned matrix is ​​taken as the block diagonal approximation of Hii. During the PCG iteration process, the norm of the residual vector is calculated using the Kahan summation algorithm to improve numerical stability. When the relative residual norm || r k || / ||r0|| reaches 10 -4 Or the calculation is terminated when the number of iterations exceeds 50. Actual measurements show that the solver takes an average of 1.83ms on a 1.8GHz ARM Cortex-A72 processor, meeting real-time requirements.

[0144] 3.4 Calibration implementation method of adjustment coefficient k

[0145] To further enhance the sliding window adjustment mechanism's environmental adaptability, the system requires precise calibration of the adjustment coefficient k. The following describes a systematic calibration method for the k value and demonstrates its effectiveness.

[0146] In the process of dynamic parameter adaptive adjustment, the optimization calibration method of the adjustment coefficient k was established through systematic experiments. Figure 9 The six-degree-of-freedom motion simulation platform shown in the figure (Moog FCS-Electric C500 model) is completed. The platform has a translation accuracy of ±0.01mm and a rotation positioning accuracy of ±0.005°. The experimental environment is equipped with a Thorlabs MLS203-1 multi-spectral light source system, which can achieve precise adjustment of the light intensity of 100-10,000 lux within 50ms. The six-degree-of-freedom platform of this embodiment can be implemented using an electric parallel mechanism well known in the art, including but not limited to Moog FCS-Electric C500 model, PI H-840 model and other motion simulation equipment with equivalent accuracy. During the experiment, it is necessary to ensure that the closed-loop control bandwidth of the platform is ≥50Hz to meet the real-time requirements of dynamic parameter adjustment.

[0147] During calibration, the IMU benchmark test was first performed in a dust-free laboratory environment at a constant temperature (23±1°C). By continuously collecting static IMU data for two hours, Allan variance analysis was used to obtain an angular velocity random walk coefficient of 0.00015 rad / √Hz. In the noise compensation model established based on this, an exponential decay integral term was used to preprocess the raw angular velocity data, with a time constant set to 1.2 seconds. This process effectively suppressed measurement noise introduced by high-frequency vibrations.

[0148] During the dynamic calibration phase, the motion platform was controlled to follow a pre-set trajectory, encompassing angular velocity variations of 0.5–10 rad / s, as well as various motion patterns such as sinusoidal sweeps and step excitation. Simultaneously, the illumination system was adjusted in 10 levels using a logarithmic pattern, with each test combination repeated three times to eliminate random errors. The entropy of the event camera's output data was calculated every 50 ms, using a 16-level grayscale histogram to calculate the intensity distribution. The normalized event entropy, Et, was then obtained through normalization.

[0149] A grid search method was used to optimize the k value, finding that when k = 1.05, the system achieved optimal performance under all test conditions. Specifically, the positioning RMSE reached 0.12m (σ = 0.03), a 57% improvement in accuracy compared to the traditional fixed-window method. A significance test (p = 0.0023) confirmed the statistical significance of this optimization. In particular, under the angular velocity jump from 5 to 8 rad / s, the trajectory deviation was reduced to 42% of the unoptimized system.

[0150] Based on extensive experimental data, the inventors established a dynamic adjustment strategy for the k value. Under conditions with angular velocities exceeding 5 rad / s, linear compensation is performed using the ratio of the event entropy Et to the baseline value E0. Under normal conditions, a hyperbolic tangent function is used to achieve a smooth transition. This strategy was further optimized for surgical navigation applications, with a dedicated adjustment formula developed for the 1.8 entropy baseline unique to electrosurgical procedures. This achieves a static tissue measurement accuracy of 0.15 mm, while maintaining a positioning accuracy of 0.8 mm even under rapid cutting conditions.

[0151] Verified on an ABB IRB6700 industrial robot, this calibration method can achieve a stable repeatability accuracy of ±0.6mm in palletizing operations and ±0.3mm in high-speed assembly. Experimental data demonstrates that this k-value calibration scheme effectively adapts to varying lighting conditions and motion states, providing reliable technical support for the adaptive algorithm described in claim 4.

[0152] 3.5 Quantitative Compensation Method for Mutual Information Calculation

[0153] The hardware implementation of the mutual information calculation module is based on the programmable logic of a Xilinx Zynq FPGA. In the timing design, a 1ms time window is divided into 1024 clock cycles, with each cycle counting events in a 4×4 pixel area. The quantization compensation unit uses 32-bit fixed-point arithmetic, and its compensation coefficient is dynamically adjusted based on the actual number of sampling points, Ns: when Ns < 5000, the compensation coefficient is 0.85 / Ns; when Ns ≥ 5000, the compensation coefficient is 0.72 / Ns. To prevent data overflow, the accumulator is designed with a 48-bit width and normalization is performed after every 256 accumulations. Field measurements have verified that this compensation scheme can reduce the mutual information calculation error from 7.2% to within 0.8% under typical operating scenarios, significantly improving the accuracy of multi-sensor data fusion.

[0154] (4) Closed-loop detection and system characteristics verification

[0155] During system operation, to improve global consistency and suppress drift errors, a keyframe map based on event mutual information is continuously constructed. Specifically, the system uses a spatiotemporal pyramid matching strategy to segment the event stream into time scales of 1ms, 5ms, and 10ms, constructing a multi-scale event intensity distribution and calculating the mutual information between keyframes as a similarity measure.

[0156] The calculation formula of mutual information is as follows:

[0157]

[0158] in:

[0159] P(p A ,p B ) represents the joint probability distribution of event intensities in key frames A and B;

[0160] P(p A )、P(p B ) are the marginal probability distributions of event intensity in key frames A and B respectively;

[0161] The mutual information value I(A,B) is used to measure the similarity of key frames and serves as the basis for loop triggering.

[0162] like Figure 7 Figure 2 shows the logic diagram for triggering loop closure detection. The system determines whether loop closure conditions are met by comparing the mutual information between keyframe events with the IMU-estimated displacement difference. If the accumulated mutual information exceeds a threshold and the displacement difference falls below a set range, the pose graph optimization process is triggered, achieving loop correction.

[0163] In order to integrate the structural similarity at different time granularities, the system introduces a weighted multi-scale cumulative mutual information calculation method:

[0164] I total =∑(λ t ·I(A t ,B t ))

[0165] Among them, λ t is the time attenuation coefficient, the recommended value is λ t =0.9 t , A t ,B t They represent the event distribution of frames with time scale t respectively.

[0166] When the cumulative mutual information value I total When the displacement difference calculated by IMU pre-integration exceeds the set threshold (such as 0.75) and is less than 0.2m, the system will automatically trigger the closed-loop detection module to perform pose graph optimization and loop correction, thereby significantly suppressing the spread of the accumulated error.

[0167] In actual tests, such as in a warehouse AGV scenario, the system successfully identified a loop in the third lap (t=215s). Through this mechanism, the cumulative error was converged from 2.1m to 0.4m, verifying its global consistency correction capability.

[0168] In addition, the stability and real-time performance of the system were comprehensively tested through three typical application scenarios:

[0169] 1. High-speed obstacle avoidance for drones: The system flies at a speed of 12m / s in a dense obstacle environment, achieving a 97.5% obstacle avoidance success rate with a 500Hz pose update rate and a highly robust event filtering strategy.

[0170] 2. Surgical navigation applications: By locking the filter near the hemoglobin absorption peak (approximately 550nm), the system maintains 0.3mm positioning accuracy on soft tissue surfaces with a reflectivity of less than 5%;

[0171] 3. Underwater SLAM experiment: In a water environment with a turbidity of 15 NTU, the event intensity normalization model was adjusted to Combined with the blue light channel filter, it effectively suppresses reflection noise and reduces the error by 63% compared to traditional image SLAM solutions.

[0172] The following provides several preferred or improved embodiments to further illustrate the technical implementation details and effect advantages of the present invention. These embodiments do not limit the scope of protection of the present invention.

[0173] Example 1: Image frame-assisted event normalization mechanism

[0174] To enhance the system's positioning stability in weakly textured or excessively flat areas, this paper introduces an event intensity normalization strategy assisted by image frame differencing. While receiving the event stream, the system simultaneously captures grayscale image frames and constructs a pixel change image ΔI(x,y) through inter-frame differencing. The event trigger intensity p is normalized according to the following relationship:

[0175] p'=p / (ΔI(x,y)+ε)

[0176] Where ε is the regularization factor. This processing significantly improves the stability of event triggering in sparsely textured areas and effectively controls false triggering. Experiments show that in pathological image navigation scenarios, this mechanism improves the accuracy of event validity judgment by 21.4% and reduces the pulse false trigger rate by 31.7%.

[0177] Example 2: Event Cube Construction and Consistency Judgment Mechanism

[0178] For each event e(x,y,t,p), the system constructs an xyt spatial cube centered on it, with a time window of 1ms and a spatial window of ±3 pixels. If there are events in the neighborhood with the same polarity and satisfying the following constraints:

[0179] |t-t'|<1ms, and √((x-x') 2 +(y-y') 2 )<3

[0180] This strategy significantly improves the accuracy of spatiotemporal consistency screening, eliminating an average of 87% of isolated pseudo-events in the experiment, providing high signal-to-noise ratio input for subsequent neuron modeling.

[0181] Example 3: Neural Coding Mechanism Based on the LIF Model

[0182] The event filter module uses the Leaky Integrate-and-Fire (LIF) neural model to sparsely encode valid events. This model maintains the membrane potential Vm attenuated over time and outputs a pulse when the accumulated intensity exceeds a threshold θ:

[0183] Vm(t+1)=λ·Vm(t)+w·p

[0184] If Vm(t+1)≥θ, trigger pulse and reset

[0185] Optimized parameters:

[0186] Leakage coefficient λ = 0.85;

[0187] Trigger threshold θ = 0.6;

[0188] This model has a high compression capability for large-scale event streams. In tests, it can retain more than 97% of high-value events while reducing data redundancy by more than 80%.

[0189] Example 4: Event entropy-driven sliding window adaptive adjustment

[0190] like Figure 5 As shown in the figure, the present invention proposes a sliding window adjustment method based on event intensity entropy. After the event intensity is normalized, the system counts the intensity histogram every 50ms, calculates the entropy value H, and dynamically adjusts the optimized window size N according to the following function:

[0191] N=N0·(H / H0) k

[0192] in:

[0193] N0 is the reference frame number;

[0194] H0 is the reference value of the system calibration entropy;

[0195] k∈[0.5,1.2] is the adjustment coefficient.

[0196] When the IMU angular velocity exceeds 5 rad / s, the number of frames is limited to N∈[5,8], and the optimization variable dimensions are simplified to accelerate convergence. Under high-speed rotation conditions, the sliding window mechanism can reduce the RMSE error from 0.36m to 0.15m, an improvement of 58%.

[0197] Example 5: Event-Inertia Co-Residual Modeling

[0198] In the joint cost function, the traditional event brightness residual r e and IMU pre-integration residual r I In addition, we further introduce the event-inertia constraint residual r c , and construct the adaptive weight factor ω:

[0199] ω=ω0·(a / g) 2

[0200] Where a is the current acceleration, g is the gravity acceleration constant, and ω0 is the initial weight. This modeling can enhance the consistency constraint of event-IMU fusion under high dynamic conditions. 2 For the acceleration segment above, add r c After the project, the system positioning error decreased by 42%.

[0201] Example 6: Loop Detection Mechanism Driven by Event Mutual Information

[0202] The system constructs a multi-scale event intensity distribution map and calculates the cumulative mutual information value I between key frames accum :

[0203] I accum =∑λt·It(A,B)

[0204] Where λt is the time decay weight, and t represents the time scale (e.g., 1ms, 5ms, 10ms). When the cumulative mutual information is greater than 0.75 and the IMU-estimated displacement difference is less than 0.2m, the system determines that a loop has been established and triggers the global graph optimization process. In AGV testing, the loop trigger success rate increased from 52% to 86%.

[0205] Example 7: Dynamic bias circuit and filter coordination mechanism

[0206] like Figure 2 As shown, the event sensing module integrates a dynamic bias circuit and a MEMS tunable filter. The bias circuit adjusts the pixel bias voltage based on the event trigger rate, with a variation range of ±15%, and a control response time of <200ns. The filter band is adjustable between 400-1000nm, and in low-light scenes, it automatically switches to the blue light band (e.g., 400-500nm) to enhance visibility.

[0207] In the 1000 lux test scenario, this mechanism can reduce the proportion of false events from 17.6% to 4.1%; in soft tissue navigation tasks, the positioning error is controlled within 0.3mm, which is significantly better than the existing DVS standard configuration system.

[0208] Example 8: Dual-layer time synchronization compensation mechanism

[0209] To overcome the timing offset between IMU and event stream in complex working conditions, the present invention adopts a combination strategy of "hardware timing + software residual compensation":

[0210] The hardware layer uses ZED-F9P GPS timing + FPGA interrupt sampling, and the time difference is controlled within <20μs;

[0211] The software layer constructs the alignment residual term r syncr , the sampling offset is inversely corrected by minimizing the error.

[0212] This mechanism is particularly suitable for edge deployments or high-temperature drift systems, ensuring data fusion with sub-millisecond accuracy. The overall fusion error is reduced by approximately 19.7% compared to software alignment.

[0213] The above-mentioned enhancement modules can be used independently or in combination in different application scenarios, such as surgical navigation, underwater SLAM, autonomous driving, drone obstacle avoidance, etc., which have a significant effect on improving the accuracy, response speed and robustness of the system, and constitute the key technical innovation points and creative supporting elements of the present invention.

[0214] Example 9: Application Verification Data and Performance Comparison

[0215] To further demonstrate the advantages of this invention, we compared the trajectory accuracy, event perception performance, sliding window adjustment responsiveness, and closed-loop recognition performance of this invention with traditional solutions in typical dynamic scenarios. The following experiments, built on the Unified Event Visual Localization Platform, employed public datasets and real-world scene acquisition for comparative testing.

[0216] Table 1: RMSE comparison in different scenarios (unit: m)

[0217] Application Scenario Traditional VIO (fixed sliding window) The present invention (entropy driven sliding window) Accuracy improvement ratio Indoor AGV navigation 0.41 0.18 ↓56.1％ surgical navigation robot 0.23 0.10 ↓56.5％ Drone high-speed cruising 0.58 0.26 ↓55.2％

[0218] Table 2: Event triggering performance and pseudo-event ratio

[0219]

[0220] Table 3: Event entropy value vs sliding window size adjustment results

[0221]

[0222] Table 4: Loop detection success rate and loop closure accuracy

[0223] Scenario Type DVS-SLAM success rate The success rate of this invention Error after closed loop (m) Warehouse AGV 52％ 86％ 0.38 Simulated mine inspection 48％ 82％ 0.41 Office area drones 56％ 88％ 0.33

[0224] In summary, this invention achieves high-precision, low-latency visual positioning in complex environments, such as high speed, strong dynamics, and weak textures, by introducing a highly responsive event-sensing mechanism, a precisely calibrated IMU synchronization scheme, and a hardware-accelerated tightly coupled optimization module. This system design maintains a compact structure while also embracing algorithmic adaptability and scalability, making it suitable for a variety of scenarios, including drone navigation, augmented reality, industrial automation, and intelligent transportation.

[0225] It should be noted that the embodiments described in the present invention are merely preferred examples and do not limit the scope of protection of the present invention. Those skilled in the art may make equivalent substitutions or adjustments to the module structures, parameter configurations, and algorithm flows without departing from the core concept of the present invention, and all such modifications shall be within the scope of protection of the present invention.

Claims

1. An ultra-low latency visual positioning system based on event camera, characterized in that: include: An event sensing module comprising a photosensor array with dynamic response characteristics, wherein the response delay of a single pixel in the sensor array is ≤10 μs; An inertial measurement unit (IMU) is rigidly connected to the event sensing module, with the physical center distance between the two being ≤8mm and the optical axis alignment error being <0.5°; The mixed signal processing unit integrates an FPGA chip and an ARM processor and is configured to perform the following operations: a) Preprocessing the event stream, including noise event filtering based on spiking neural networks; b) Perform motion compensation based on IMU data; c) The event data and IMU data are aligned through the timestamp synchronization module to ensure that the time synchronization error is ≤ 20μs.

2. The system according to claim 1, wherein: The event perception module includes: Tunable optical filter, whose transmission wavelength can be adjusted programmably in the range of 400-1000nm; Dynamic bias circuitry automatically adjusts pixel bias voltage in response to changes in ambient light intensity, with an adjustment range of ±15% of the reference voltage.

3. The system according to claim 2, characterized in that The control logic of the dynamic bias circuit includes: When the event trigger rate per unit time exceeds 1×10 6 events / s, reduce the bias voltage by 10%-15%; When the event trigger rate is less than 1×10 4 events / s, increase the bias voltage by 5%-8%.

4. The system according to claim 1, wherein: The mixed signal processing unit comprises: The hardware-accelerated Bundle Adjustment (BA) module uses the truncated Newton method for optimization and is configured as follows: a) Maintain a dynamically adjusted sliding window with a window size N that satisfies the following relationship: Among them E t is the current event entropy value, E0 is the baseline entropy value, k∈[0.5,1.2]; b) When the angular velocity is detected to be greater than 5 rad / s, N is forced to be limited to the range of 5-8 frames.

5. A visual positioning method based on the system according to any one of claims 1 to 4, characterized in that: The following steps are involved: S1. Asynchronously receive event stream data output by the event perception module and IMU data output by the inertial measurement unit; S2. Performing spatiotemporal consistency filtering on the event stream, the filtering comprising: constructing an event cube in xyt space, removing outlier events, and performing pulse coding processing based on a Leaky Integrate-and-Fire neuron model; S3. Perform event-IMU tight coupling optimization, including: A joint cost function is constructed, which includes the event brightness residual term, the IMU pre-integration residual term, and the event-inertia cooperation residual term. The optimization goal is to minimize the following joint cost function: in: is the brightness residual corresponding to the i-th event frame, is the j-th IMU pre-integration residual, r c is the event-IMU constraint residual, ω is the residual collaborative weight; The First Estimate Jacobians (FEJ) method was used in the optimization to maintain observation consistency; S4. Output the 6-DOF pose estimation result at the current moment, with an update frequency of no less than 500 Hz; The optimization adopts a sliding window mechanism, and the window size is dynamically adjusted according to event entropy, as described in claim 4.

6. The method according to claim 5, characterized in that The step S1 further includes: Synchronously receive image frame data and event stream data, and perform inter-frame difference processing on the image frames to assist in normalization correction of event trigger intensity; The sampling interval of the auxiliary image frame is 20ms±1ms to enhance event response stability and positioning accuracy in low-texture areas.

7. The method according to claim 5, characterized in that The rules for determining the spatiotemporal consistency of the event e(x, y, t, p) in step S2 include: In the spatiotemporal neighborhood N(e) of event e, if there exists an event e'(x', y', t', p') that satisfies the following conditions, then event e is considered a valid event: |t-t'|<Δt, where Δt=1ms; The spatial distance satisfies d((x,y),(x',y'))<Δd, where Δd=3 pixels.

8. The method according to claim 5, characterized in that The joint cost function of step S3 also includes an event-inertia constraint term, whose weight ω is adaptively adjusted according to the current acceleration measurement value to satisfy the following relationship: where a t is the current acceleration measurement value, and g is the gravitational acceleration constant.

9. The method according to claim 5, characterized in that The step S3 further comprises: The event intensity entropy value H is calculated based on the event stream and used to dynamically adjust the sliding window size, where the event entropy H is defined as follows: Where f(p) represents the normalized frequency distribution within the event intensity interval p; P is the set of all discretized intensity intervals; The event entropy H is used to drive the adaptive adjustment of the sliding window size N to adapt to scenes with different motion complexities.

10. The method according to claim 5, characterized in that The following steps are also included: Construct a local keyframe graph based on the event stream and perform closed-loop detection in combination with IMU state estimation; When the event mutual information measure between two keyframes exceeds a preset threshold, pose loop correction is performed to improve global consistency; The mutual information metric calculation formula is: in: P(p A ,p B ) represents the joint probability distribution of event intensities in key frames A and B; P(p A )、P(p B ) are the marginal probability distributions of event intensity in key frames A and B respectively; The mutual information value I(A,B) is used to measure the similarity of key frames and serves as the basis for loop triggering.

Images