An event camera based method for measuring velocity of a group of fragments

Abstract

This invention discloses a fragment swarm velocity measurement method based on an event camera. The method includes the following steps: camera calibration and time synchronization; extraction of the time-weighted centroid and two-dimensional motion features of fragments at the monocular level; establishment of a multi-dimensional feature matching cost function; and completion of globally unique binding of fragment trajectories in left and right views through frame-level voting and a global trajectory scoring strategy; three-dimensional triangulation reconstruction of the successfully matched binocular trajectory point sequences; calculation of the total spatial length and time span of the trajectory to determine the three-dimensional flight velocity of the fragments; filtering out abnormal scale trajectories; and outputting a multi-dimensional mapping relationship containing fragment spatial coordinates, flight velocity, and identification number. This invention effectively overcomes the defects of light saturation and motion blur, significantly reduces the mismatch rate of dense tracking, and provides high-confidence data support for warhead power parameter evaluation, demonstrating high practical value.

Description

Technical Field

[0001] This invention belongs to the field of damage assessment technology, specifically relating to a method for measuring fragment velocity based on an event camera. Background Technology

[0002] The assessment of the destructive effectiveness of fragmentation warheads is a crucial step in weapon system development, directly determining their combat effectiveness. The warhead's power parameters are the core basis for evaluating its destructive effectiveness. However, traditional mainstream non-contact measurement methods, namely high-speed photography, face insurmountable bottlenecks—limited dynamic range and the "light saturation" effect: the intense flash of the explosion causes overexposure of the detector, obscuring fragment details. Furthermore, its frame-based exposure principle results in unavoidable motion blur and inter-frame information loss, limiting the analytical accuracy for microsecond-level ultra-high-speed phenomena.

[0003] To overcome the aforementioned limitations, visual measurement technology based on event cameras offers a highly promising innovative solution for high spatiotemporal resolution observation of warhead explosion processes. The event camera employs a biomimetic working principle, abandoning the fixed frame rate exposure mode of traditional cameras. By asynchronously outputting pixel-level brightness change event streams, it possesses core advantages such as microsecond-level high temporal resolution, extremely high dynamic range (>120dB), and low data redundancy. Theoretically, it can effectively overcome the oversaturation problem of intense explosion light, clearly capture subtle brightness changes caused by high-speed fragments, and record the entire dynamic evolution of the fragment field in the form of a continuous asynchronous stream, avoiding the motion blur problem of traditional cameras.

[0004] In the extreme environment of warhead explosion, fragmentation groups exhibit ultra-high speed and high density dispersion. How to accurately and reliably obtain the continuous three-dimensional trajectory and velocity parameters of the target under strong interference scenarios is a major challenge currently facing the field of power assessment. Summary of the Invention

[0005] The purpose of this invention is to provide a high-precision method for measuring fragment swarm velocity based on an event camera, primarily addressing the problem of obtaining high-precision fragment swarm velocity under extreme conditions of intense light interference and dense, high-speed fragment dispersion in explosion scenarios. This method enables binocular time synchronization and highly robust trajectory matching based on global spatiotemporal features, optimizing the 3D trajectory reconstruction and velocity calculation performance of a binocular event camera visual measurement system in extremely dynamic, multi-target overlapping scenarios.

[0006] The technical solution for achieving the objective of this invention is: a method for measuring fragment velocity based on an event camera, comprising the following steps:

[0007] Step (1): Camera calibration and time synchronization: Based on Zhang's calibration method, high-frequency flashing checkerboard is used to calibrate the spatial geometric parameters of the binocular event camera, and microsecond-level global automatic time alignment is performed based on the collected asynchronous event stream data;

[0008] Step (2): Binocular trajectory feature matching: First, extract the time-weighted centroid and two-dimensional motion features of the fragment at the monocular level, establish a multi-dimensional feature matching cost function, and complete the global unique binding of the fragment trajectory in the left and right views through frame-level voting and global trajectory scoring strategy.

[0009] Step (3): 3D reconstruction and velocity calculation: Perform 3D triangulation reconstruction on the successfully matched binocular trajectory point sequence, calculate the total spatial length and time span of the trajectory to calculate the 3D flight velocity of the fragment, filter out abnormal scale trajectories, and output a multi-dimensional mapping relationship containing the spatial coordinates, flight velocity and identification number of the fragment.

[0010] Compared with the prior art, the significant advantages of this invention are:

[0011] (1) This invention effectively overcomes strong light overexposure and motion blur: Based on the characteristics of the event camera's ultra-high dynamic range (>120dB) and microsecond-level time resolution, it solves the "light saturation" problem of traditional photoelectric measurement under strong light of explosion, avoids motion blur when capturing ultra-high speed fragments, and ensures the quality of perception data in extreme scenarios.

[0012] (2) The present invention achieves microsecond-level binocular time synchronization: global time alignment is performed by coarse and fine double-layer histogram correlation algorithm, which makes up for the small delay of the hardware triggering of the binocular event camera and reduces the spatial error of 3D reconstruction caused by time misalignment;

[0013] (3) This invention improves the accuracy of centroid positioning and motion feature extraction: the introduction of time exponential weighted calculation of fragment centroid can more accurately reflect the real-time physical position of the target, and thus calculate the high-precision two-dimensional instantaneous velocity, providing a reliable kinematic basis for subsequent binocular matching;

[0014] (4) This invention improves the robustness of dense multi-target matching: it breaks through the single geometric epipolar constraint, introduces multi-dimensional features into the matching cost calculation, and combines frame-level voting and global relevance scoring to effectively suppress identity (ID) jumps and mismatches caused by dense fragment cross-occlusion.

[0015] (5) This invention improves the reliability of three-dimensional velocity measurement data: the average velocity is calculated by dividing the total length of the three-dimensional continuous trajectory by the time span, and a physical scale filter is introduced to remove noise and false trajectories caused by splashes, thus ensuring the accuracy of the final velocity measurement results;

[0016] (6) Achieve intuitive mapping of multidimensional physical parameters: Construct a three-dimensional display model that can simultaneously output the absolute three-dimensional coordinates, unique identifier and flight speed of the target fragment, which greatly improves the analysis efficiency of data processing and power assessment. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the overall process of the fragment group velocity measurement method based on an event camera according to the present invention.

[0018] Figure 2 This is a block diagram of the microsecond-level time synchronization logic in this invention.

[0019] Figure 3 This is a block diagram of the binocular trajectory matching and binding logic in this invention.

[0020] Figure 4 This is a schematic diagram illustrating the principle of event cluster centroid extraction based on time weight in this invention.

[0021] Figure 5 This is a schematic diagram showing the test scene, imaging effect, target recognition, and 3D reconstruction model in this invention; where (a) is the test scene of a projectile impacting a target plate; (b) is the imaging effect of a fragment group; (c) is the target recognition effect of the fragment group; and (d) is the 3D reconstruction effect of the fragment group. Detailed Implementation

[0022] The present invention will now be described in further detail with reference to the accompanying drawings.

[0023] This application provides a method for measuring fragment group velocity based on an event camera, the method flow is as follows: Figure 1 As shown, the method includes the following steps:

[0024] Step (1): Camera calibration and time synchronization: Based on Zhang's calibration method, high-frequency flashing checkerboard is used to calibrate the spatial geometric parameters of the binocular event camera, and microsecond-level global automatic time alignment is performed based on the collected asynchronous event stream data;

[0025] Step (2): Binocular trajectory feature matching: First, extract the time-weighted centroid and two-dimensional motion features of the fragment at the monocular level, establish a multi-dimensional feature matching cost function, and complete the global unique binding of the fragment trajectory in the left and right views through frame-level voting and global trajectory scoring strategy, so as to realize binocular trajectory feature matching.

[0026] Step (3): 3D reconstruction and velocity calculation: Perform 3D triangulation reconstruction on the successfully matched binocular trajectory point sequence to obtain the spatial coordinates of the fragment; calculate the total spatial length and time span of the trajectory to calculate the 3D flight velocity of the fragment; after filtering out abnormal scale trajectories, output a multi-dimensional mapping relationship containing the spatial coordinates, flight velocity and identification number of the fragment.

[0027] Furthermore, step (1) specifically includes the following steps:

[0028] Step (11): Camera calibration. Place a high-frequency flashing black and white checkerboard pattern with timestamps within the field of view of the binocular event camera. Use the event camera to capture the dense event stream generated by the flashing of the black and white checkerboard pattern. Extract the corner pixel coordinates based on Zhang's calibration method to calculate the intrinsic parameter matrix of the binocular camera (including the intrinsic parameter matrix of the left camera). and the right camera intrinsic parameter matrix ) and extrinsic parameter matrices (including rotation matrix R and translation vector L);

[0029] Step (12): Microsecond-level time alignment. For the high-dynamic data generated at the moment of the explosion, the coarse step-size histogram (time-event density) of the binocular events is extracted and cross-correlation is performed to obtain the coarse alignment offset. Then, the explosion mutation point is located by calculating the first derivative of the left eye histogram. Within a small time window (1000us~8000us) at this point, a subdivided histogram is generated at a high resolution of 1us and cross-correlation is performed again. Finally, the microsecond-level global time synchronization compensation is obtained by superimposing the results.

[0030] Furthermore, step (2) specifically includes the following steps:

[0031] Step (21): Time-weighted centroid and 2D motion feature extraction. The aligned monocular event stream is sliced ​​at microsecond steps, and the DBSCAN algorithm is used for density clustering to extract fragment clusters. In high-speed motion scenarios, the targets collected by the system exhibit trailing, and traditional centroid calculation cannot accurately reflect the physical location of high-speed moving fragments. Therefore, a time-weighted rule is introduced, which applies time weighting to any event within a cluster. timestamp Normalization yields:

[0032]

[0033] And assign weight to exponential growth. :

[0034]

[0035] Where 2.0 is the time decay coefficient.

[0036]

[0037]

[0038] To obtain a time-weighted center of mass that better aligns with the forefront of the target motion Based on the intersection-to-union ratio (IOU) to correlate preceding and following frames, the instantaneous velocity characteristics of the fragment in the two-dimensional image plane are deduced by calculating the pixel displacements of adjacent centroids on the image plane. , It should be noted that this two-dimensional instantaneous velocity is only used as a local feature to characterize the movement trend of the fragment in the focal plane, and is used for subsequent multi-dimensional feature matching cost gating (filtering out mismatches). It does not represent the final three-dimensional physical flight velocity of the fragment.

[0039] Step (22): Establish a multi-dimensional feature matching cost function, traverse the left and right eye targets within the synchronized frame, break the limitation of the traditional single spatial epipolar line, and comprehensively calculate the epipolar line distance cost. Cost of size and area difference Cost of difference in the number of events and the cost of two-dimensional instantaneous velocity vector difference The total local matching cost is obtained by summing the results according to the set weights. And based on the cost threshold, candidate pairs with too large a difference are eliminated to obtain preliminary candidate targets.

[0040]

[0041] Step (23): Frame-level voting is linked to the global score. A frame-level Top-K (e.g., Top-3) weighted vote is performed on the candidate targets that passed Step (22), filtering out targets whose co-visibility (targets can be observed in both left and right eyes) vote rate is lower than a set threshold (e.g., 0.4). For the remaining valid candidate trajectory combinations, the epipolar root mean square error of the overlapping frames of the left and right eye trajectories, the consistency of motion shape (Pearson correlation coefficient; the closer the absolute value is to 1, the more consistent the motion shape and trend of the left and right eye trajectories in space), and the temporal overlap rate (i.e., the ratio of the number of frames where the left and right eye trajectories coexist and are successfully matched to the total number of surviving frames of the left eye baseline trajectory, used to measure the synchronization consistency of the left and right eye candidate trajectories on the time axis) are calculated, generating a global comprehensive score. The calculation formula is as follows:

[0042] Score = 0.6 × (RMSE / 10.0) + 0.3 ×(1 - corr) + 0.1 × (1 - overlap)

[0043] RMSE: Root Mean Square Error of the Epipolar Line; corr: Pearson Correlation Coefficient; overlap: Trajectory Temporal Overlap Rate. The aligned trajectory with the lowest score (less than 1.0) is selected as the final matching result. A greedy algorithm is then used to achieve robust one-to-one binding of the stereo trajectories, and the successfully matched left and right eye 2D pixel centroid homogeneous coordinate sequences are finally output. =( , , 1)T and =( , , 1) T

[0044] Step (3) specifically includes the following steps:

[0045] Step (31) Three-dimensional reconstruction, using the matching centroid coordinate sequence output in step (23) and and the parameter matrix calibrated in step (11) , R and T are the inputs. The left camera projection matrix is ​​constructed with the optical center of the left camera as the origin of the world coordinate system. = [I|0] (where I is the identity matrix and 0 is the zero vector), and the right camera projection matrix. = [R|T]. Let the homogeneous coordinates of the three-dimensional physical space corresponding to the fragment be... = (X, Y, Z, 1) T Utilizing the projection relationship of spatial points on the image plane ×( ) = 0 and ×( To eliminate the scale factor, we construct the epipolar geometric space triangulation constraint by setting the value of 0, and establish the following system of linear equations:

[0046] A = 0

[0047] The coefficient matrix A is constructed by combining the row vectors of the projection matrices of the left and right cameras with the corresponding two-dimensional pixel coordinates.

[0048] The linear equations are solved using singular value decomposition (SVD) to obtain a continuous set of three-dimensional coordinate points of the fragment in the three-dimensional physical coordinate system. , , (where i = 1, 2, ..., n).

[0049] Step (32) Velocity calculation: using the sequence of three-dimensional coordinate points output in step (31) , , ) and its corresponding microsecond-level aligned timestamp sequence As input. First, by accumulating the Euclidean distances between adjacent three-dimensional coordinate points, the total physical length of the fragment's complete trajectory is obtained. The calculation formula is:

[0050] =

[0051] By combining the timestamps of the start and end frames, calculate the precise time span ΔT during which the trajectory occurred. - Finally, dividing the total spatial length by the time span yields the true three-dimensional average physical velocity of the fragment. :

[0052] =

[0053] Step (33) Scale filtering: Based on the set total trajectory length and the diagonal span threshold of the spatial bounding box, false trajectories are eliminated;

[0054] Step (34) Data output: The effective trajectory is rendered as a three-dimensional model. When a fragment node is selected in the three-dimensional view, the system outputs the physical data of the target in real time: absolute three-dimensional coordinates, unique trajectory identifier (TrackID), and calculated three-dimensional average flight speed.

[0055] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for measuring velocity of a group of fragments based on an event camera, characterized in that, Includes the following steps: Step (1): Camera calibration and time synchronization: Based on Zhang's calibration method, high-frequency flashing checkerboard is used to calibrate the spatial geometric parameters of the binocular event camera, and microsecond-level global automatic time alignment is performed based on the collected asynchronous event stream data; Step (2): Binocular trajectory feature matching: First, extract the time-weighted centroid and two-dimensional motion features of the fragment at the monocular level, establish a multi-dimensional feature matching cost function, and complete the global unique binding of the fragment trajectory in the left and right views through frame-level voting and global trajectory scoring strategy. Step (3): 3D reconstruction and velocity calculation: Perform 3D triangulation reconstruction on the successfully matched binocular trajectory point sequence, calculate the total spatial length and time span of the trajectory to calculate the 3D flight velocity of the fragment, filter out abnormal scale trajectories, and output a multi-dimensional mapping relationship containing the spatial coordinates, flight velocity and identification number of the fragment.

2. The fragment group velocity measurement method based on an event camera according to claim 1, characterized in that, Step (1) specifically includes the following steps: Step (11): Place a high-frequency flashing black and white checkerboard with timestamp display within the field of view of the binocular event camera, use the event camera to capture the dense event stream generated by the black and white flashing of the checkerboard, and extract the corner pixel coordinates based on Zhang's calibration method to solve the intrinsic and extrinsic parameter matrices of the binocular camera. Step (12): For the high dynamic data generated at the moment of the explosion, extract the coarse step size histogram of the binocular event and perform cross-correlation calculation to obtain the coarse alignment offset; then locate the explosion mutation point by taking the first derivative of the left eye histogram, and generate a subdivided histogram with a high resolution of 1us within a window of 1000us~8000us at this point and perform cross-correlation calculation again, and finally superimpose to obtain the microsecond-level global time synchronization compensation.

3. The fragment group velocity measurement method based on an event camera according to claim 2, characterized in that, The intrinsic parameter matrix of a stereo camera includes the intrinsic parameter matrix of the left camera. and the right camera intrinsic parameter matrix The extrinsic parameters of the binocular camera include the rotation matrix R and the translation vector L.

4. The fragment group velocity measurement method based on an event camera according to claim 3, characterized in that, Step (2) specifically includes the following steps: Step (21): Monocular independent tracking and 2D motion feature extraction: The aligned monocular event stream is sliced ​​at microsecond-level steps, and the DBSCAN algorithm is used to perform density clustering to extract fragment clusters. The event weights within the clusters are calculated using the exponential decay formula, and the weighted average of the spatial coordinates is used to obtain the time-weighted centroid biased towards the latest occurrence time. Based on the cross-union ratio (CUB) correlation between preceding and following frames, the instantaneous velocity characteristics of the fragment in the two-dimensional image plane are deduced by calculating the pixel displacements of adjacent centroids on the image plane. , ); Step (22): Establish a multi-dimensional feature matching cost function, traverse the left and right eye targets within the synchronized frame, and comprehensively calculate the epipolar distance cost. Cost of size and area difference Cost of difference in the number of events and the cost of two-dimensional instantaneous velocity vector difference The total local matching cost is obtained by summing the results according to the set weights. And based on the cost threshold, candidate pairs with excessive differences are eliminated to obtain preliminary candidate targets; Step (23): Perform frame-level Top-K weighted voting on the candidate targets that passed step (22) to filter out targets with a common-view vote rate lower than a set threshold; for the remaining valid candidate trajectory combinations, calculate the epipolar root mean square error, motion shape consistency, and temporal overlap rate of the overlapping frames of the left and right eye trajectories to generate a global comprehensive score, and use a greedy algorithm to select the combination with the best score to complete one-to-one trajectory binding, and output the matching left and right eye two-dimensional pixel centroid homogeneous coordinate sequence. =( , , 1) T and =( , , 1) T .

5. The fragment group velocity measurement method based on an event camera according to claim 4, characterized in that, Time-weighted centroid in step (21) The calculation is as follows: For any event within the cluster timestamp Normalization yields: ， And assign weight to exponential growth. : ， Where 2.0 is the time decay coefficient; To obtain a time-weighted centroid that better aligns with the forefront of the target motion: ， 。 6. The fragment group velocity measurement method based on an event camera according to claim 5, characterized in that, Local total matching cost in step (22) The calculation formula is as follows: 。 7. The fragment group velocity measurement method based on an event camera according to claim 6, characterized in that, The formula for calculating the overall score in step (23) is as follows: Score = 0.6 × (RMSE / 10.0) + 0.3 × (1 - corr) + 0.1 × (1 - overlap), In the formula, RMSE is the root mean square error of the epipolar line; corr is the Pearson correlation coefficient; and overlap is the trajectory time overlap rate.

8. The fragment group velocity measurement method based on an event camera according to claim 7, characterized in that, Step (3) specifically includes the following steps: Step (31): 3D reconstruction: Triangulation of the matched binocular trajectory points is performed using the Direct Linear Transform (DLT) algorithm to generate a 3D coordinate sequence; Step (32): Velocity calculation: Using the three-dimensional coordinate sequence output in step (31) and its corresponding microsecond-level aligned timestamp sequence as input, the total physical length of the complete trajectory of the fragment is obtained by accumulating the Euclidean distance between adjacent three-dimensional coordinate points. Combined with the timestamps of the start and end frames, the precise time span of the trajectory is calculated. The total spatial length is divided by the time span to obtain the true three-dimensional average flight physical velocity of the fragment. Step (33): Scale filtering: Based on the set total trajectory length and the diagonal span threshold of the spatial bounding box, false trajectories are eliminated; Step (34) Data output: The effective trajectory is rendered as a three-dimensional model. When a fragment node is selected in the three-dimensional view, the system outputs the physical data of the target: absolute three-dimensional coordinates, trajectory unique identifier, and calculated three-dimensional average flight speed.

9. The fragment group velocity measurement method based on an event camera according to claim 8, characterized in that, Step (31) is as follows: The sequence of matching centroid coordinates output in step (23) and and the parameter matrix calibrated in step (11) , R and T are the inputs; the left camera projection matrix is ​​constructed with the optical center of the left camera as the origin of the world coordinate system. = [I|0], where I is the identity matrix, 0 is the zero vector, and the right camera projection matrix. = [R|T]; Let the homogeneous three-dimensional physical space coordinates corresponding to the fragment be... = (X, Y, Z, 1) T Utilizing the projection relationship of spatial points on the image plane ×( ) = 0 and ×( =0 eliminates the scale factor. Construct triangulation constraints in the epipolar geometric space to establish the following system of linear equations: A = 0， The coefficient matrix A is constructed by combining the row vectors of the projection matrices of the left and right cameras with the corresponding two-dimensional pixel coordinates. The linear equations were solved using Singular Value Decomposition (SVD) to obtain a continuous set of three-dimensional coordinate points of the fragment in the three-dimensional physical coordinate system. , , ), where i = 1, 2, ..., n.

10. The fragment group velocity measurement method based on an event camera according to claim 9, characterized in that, Step (32) is as follows: The sequence of three-dimensional coordinate points output in step (31) , , ) and its corresponding microsecond-level aligned timestamp sequence As input; by accumulating the Euclidean distances between adjacent three-dimensional coordinate points, the total physical length of the fragment's complete trajectory is obtained. The calculation formula is: = ， By combining the timestamps of the start and end frames, calculate the precise time span ΔT during which the trajectory occurred. - ; Dividing the total spatial length by the time span yields the true three-dimensional average physical velocity of the fragment. : = 。

Images