A method and device for post-target damage assessment based on binocular event camera

Abstract

The application discloses a target post damage evaluation method and device based on a binocular event camera. First, a binocular event camera shooting device is constructed by designing an installation registration mode; event stream data of a target post fragment field is acquired based on the binocular event camera, and time integration is performed to form a fragment field image; feature matching of a fragment light spot is performed according to a time identifier and a space registration feature of the event stream data, and a light spot centroid is extracted; three-dimensional coordinates of the fragment in a world coordinate system are constructed based on a parallax method, and further time difference is obtained to obtain a fragment velocity. Through implementation of the application scheme, the problem that a detector is saturated and cannot be imaged due to strong light caused by explosion when a high-speed camera is used to shoot a target post fragment field can be solved, reconstruction of a target post explosion fragment field is effectively realized, and technical support is provided for kinetic energy damage evaluation. Compared with high-speed photographic equipment, the cost of a computing platform and economic cost are reduced.

Description

Technical Field

[0001] This invention relates to the field of post-target damage assessment technology, specifically, to a method and apparatus for post-target damage assessment based on a binocular event camera. Background Technology

[0002] Damage assessment is a key focus of defense research, with countries worldwide actively developing new warhead technologies based on various damage mechanisms to enhance the high-efficiency damage capabilities of munitions and missiles. Fragmentation parameter testing is a crucial aspect of damage assessment. Fragmentation velocity is one of the most typical fragmentation parameters, reflecting not only the projectile's impact on the target but also assessing whether the warhead structure and loading coefficient meet requirements. It is an important parameter for verifying weapon performance. Currently, domestic and international research on fragmentation parameter measurement based on high-definition image acquisition and processing technology mainly focuses on high-speed imaging and light curtain target testing.

[0003] With the continuous development of digital high-speed cameras, high-speed imaging methods have provided technical support for range testing applications. When measuring the velocity of high-speed moving fragments, using a high-speed camera instead of a regular camera can achieve tracking of high-speed fragments. However, high-speed imaging methods have disadvantages such as the high cost of high-speed cameras, local saturation caused by experimental explosion light, high power consumption, and large data volume that is difficult to process. The light curtain target testing method measures the time difference of fragments passing through multiple light curtain targets that are relatively close together, indirectly calculating the velocity, but it has disadvantages such as low data acquisition rate and susceptibility to damage.

[0004] This invention uses an event camera, a novel bio-inspired sensor, to calculate three-dimensional spatial coordinates and velocity in the world coordinate system based on a binocular camera. Its cost is far lower than that of a high-speed camera. Because its high dynamic range will not cause local saturation due to explosion light, the event camera's low power consumption, small data volume, and microsecond-level data acquisition rate solve the shortcomings of high-speed imaging methods and light curtain target testing methods. Summary of the Invention

[0005] The purpose of this invention is to provide a method and apparatus for post-target damage assessment based on a binocular event camera. This invention aims to utilize the low power consumption, high dynamic range, and high temporal resolution characteristics of the event camera to solve the problem of detector saturation and inability to image due to intense explosion light, effectively reconstructing the fragmentation field after the explosion and providing technical support for kinetic damage assessment. Furthermore, compared to high-speed photography equipment, it reduces the cost of the computing platform and overall economic costs.

[0006] The technical solution to achieve the purpose of this invention is: a method and apparatus for post-target damage assessment based on a binocular event camera, comprising the following steps:

[0007] (1) Using a binocular event camera acquisition structure device, calculate the parameters that match the structure;

[0008] (2) Obtain the original event streams of the target fragments captured by the binocular event cameras from the left and right eyes respectively from the two host computers, and remove the noise in the target fragment event streams by setting a density threshold.

[0009] (3) Accumulate the target fragment event stream after noise reduction in step (2) according to the dynamic integration time T. Initially, the dynamic integration time T is the initial integration time τ. Record all events in the integrated time slice as the sample set D.

[0010] Perform DBSCAN spatial clustering on the event frames integrated in step (3) according to the matching parameters. Given the neighborhood parameters (∈, MinPts), for x j ∈D, its ∈-neighborhood is N ∈ (x j )={x i ∈D|dist(x i ,x j If x ≤ ∈}; j The ∈-neighborhood contains at least MinPts samples, i.e., |N ∈ (x j If |≥MinPts, then x j It is a core object. Based on the given neighborhood parameters ∈ and MinPts, all core points are determined. For each core point, an unprocessed core point is selected, and clusters are generated from samples whose density is reachable from it. This process is repeated until all core points are processed. The centroid depth information of the stereo event frames is parameterized using the nearest neighbor matching algorithm, and stereo matching of the fragments is performed to achieve matching of the centroids of the fragments corresponding to the stereo event frames.

[0011] (4) The depth information of the centroid of the binocular event frame is parameterized by the nearest neighbor matching algorithm, and the fragment is matched in stereo to achieve the matching of the centroid of the fragment corresponding to the binocular event frame;

[0012] (5) Use the parallax method to reconstruct the three-dimensional coordinates of the corresponding centroids matched in step (4) in the world coordinate system, and perform time difference to obtain the velocity of the fragment.

[0013] A target damage assessment device based on a binocular event camera, comprising the steps of a target damage assessment method based on a binocular event camera.

[0014] Compared with the prior art, the present invention has the following significant advantages: (1) The purpose of the present invention is to provide a method and device for post-target damage assessment based on a binocular event camera. The present invention aims to utilize the low power consumption, high dynamic range and high temporal resolution of the event camera to solve the problem of detector saturation and inability to image due to the strong light of the explosion, and effectively realize the reconstruction of the post-target explosion fragmentation field. (2) The present invention utilizes the characteristics of the event camera output event stream with smaller data volume and microsecond-level data acquisition rate. The parallax method is used to reconstruct the three-dimensional coordinates of the corresponding centroids obtained by matching the output event stream in the world coordinate system, and the velocity of the fragments is obtained by time difference. Compared with the traditional method, the power consumption is lower, the cost is lower and the space utilization is higher. It provides a new idea for post-target fragmentation field damage assessment. Attached Figure Description

[0015] Figure 1 This is a flowchart of a target damage assessment method based on a binocular event camera according to the present invention.

[0016] Figure 2 This is a structural diagram of the binocular acquisition device of the present invention.

[0017] Figure 3 This is an assembly diagram of the binocular acquisition device of the present invention.

[0018] Figure 4 A comparison of the results before and after density threshold denoising for the collected fragment field data stream.

[0019] Figure 5 The clustering algorithm of this invention identifies the circular bulletproof material at the center of the target plate that was knocked away. Detailed Implementation

[0020] The present invention will now be further described with reference to the accompanying drawings.

[0021] This invention provides a method for post-target damage assessment based on a binocular event camera, comprising the following steps:

[0022] (1) Using a binocular event camera acquisition structure device, calculate the parameters that match the structure;

[0023] (2) Obtain the original event streams of the target fragments captured by the binocular event cameras from the left and right eyes respectively from the two host computers, and remove the noise in the target fragment event streams by setting a density threshold.

[0024] (3) Accumulate the target fragment event stream after noise reduction in step (2) according to the dynamic integration time T. Initially, the dynamic integration time T is the initial integration time τ. Record all events in the integrated time slice as the sample set D.

[0025] (4) Perform DBSCAN spatial clustering on the event frames integrated in step (3) according to the matching parameters. Given the neighborhood parameters (∈, MinPts), for x j ∈D, its ∈-neighborhood is N ∈ (x j )={x i ∈D|dist(x i ,x j If x ≤ ∈}; j The ∈-neighborhood contains at least MinPts samples, i.e., |N ∈ (x j If |≥MinPts, then x j It is a core object. Based on the given neighborhood parameters ∈ and MinPts, all core points are determined. For each core point, an unprocessed core point is selected, and samples with reachable density are found to generate a cluster. The above process is repeated until all core points are processed. The depth information of the centroid of the stereo event frame is parameterized by the nearest neighbor matching algorithm, and the fragment is stereo matched to achieve the matching of the centroid of the fragment corresponding to the stereo event frame.

[0026] (5) Use the parallax method to reconstruct the three-dimensional coordinates of the corresponding centroids matched in step (4) in the world coordinate system, and perform time difference to obtain the velocity of the fragment.

[0027] Furthermore, the structure diagram of the binocular event camera acquisition device in step (1) is as follows. Figure 2 As shown: The bottom has a level tripod for fixing and keeping the entire acquisition device level. It is connected to the top horizontal slide rail via the main rod. There is a slot in the middle of the slide rail, in which two modules for fixing the event cameras are installed. This ensures that the left and right event cameras are parallel and adjusts the baseline distance between the two event cameras.

[0028] The parameter calculation for matching the structure in step (1) is characterized by configuring appropriate shooting parameters based on the horizontal field of view, vertical field of view, and viewing distance of the event camera:

[0029]

[0030] In the formula, B represents the baseline of the binocular event camera; α represents the horizontal field of view of the event camera; and L represents the vertical distance between the focal plane of the binocular event camera system and the field of view corner.

[0031] Furthermore, in step (2), the event stream data of the left eye camera and the event stream data of the right eye camera captured by the binocular event camera are obtained from the two host computers respectively.

[0032] In step (2), the event stream typically uses address events to represent readings:

[0033] e = (x, y, t)

[0034] In the formula, x is the number of columns; y is the number of rows; and t is the timestamp.

[0035] Valid events are generated by the motion of objects or changes in light intensity. A valid event typically does not occur alone but activates pixels in its surrounding area. Therefore, the density of valid events is usually greater than that of invalid events. The density of the input event e(x,y,t) within the spatiotemporal neighborhood U(x,y,t) is calculated using the following formula:

[0036]

[0037] In the formula, C represents the set threshold; N valid Represents the target event; N noise Indicates noise;

[0038] If the event density is less than C, the event is considered noise and is filtered out; if the event density is greater than or equal to C, it is considered the target event and is retained.

[0039] Furthermore, in step (3), the denoised event stream data from step (2) is accumulated according to the dynamic integration time T, where the initial dynamic integration time T is the initial integration time τ.

[0040] After adaptive time integration, the noise-removed event stream data is integrated into image frames. All event points in the image frames to be processed are placed into the sample set D = {x1, x2, ..., x...} m}middle.

[0041] Furthermore, in step (4), the density-based DBSCAN spatial clustering method is used to identify the target fragments:

[0042] Given the neighborhood parameters (∈, MinPts), where ∈ represents the minimum neighborhood radius and MinPts is the minimum number of samples.

[0043] For x j ∈D{x1,x2,...,x m}, whose ∈-neighborhood contains samples in the sample set D that are the same as x. j The distance to samples whose values ​​are not greater than those of ∈, i.e., N ∈ (x j )={x i ∈D|dist(x i ,x j )≤∈}

[0044] Where, N ∈ (x j ) represents point x j∈-neighborhood; dist(x) i ,x j ) represents x i and x j The distance between two points.

[0045] x j The ∈-neighborhood contains at least MinPts samples, i.e., the core object x. j Must satisfy |N ∈ (x j )|≥MinPts where,|N ∈ (x j | represents x j The number of event points in the ∈-neighborhood.

[0046] Furthermore, the cluster is solved:

[0047] Initialize the core point set Initialize the number of clusters k = 0, then partition the clusters.

[0048] For i = 1, 2, ..., n, find the sample x using a distance metric. i The ∈-neighborhood sample set N ∈ (x i )

[0049] If x i Satisfy | N ∈ (x j If |≥MinPts, then the sample x i Add the core set: Ω=Ω∪{x i}

[0050] In the core point set Ω, randomly select a core point o and initialize the current cluster core point queue Ω. cur ={o}, initialize the category index k = k + 1, initialize the current cluster sample set C k ={o}, update the core point set Ω = Ω - C K Update the cluster partition C = {C1, C2, ..., C} K},

[0051] The centroid depth information of the stereo event frames is parameterized using the nearest neighbor matching algorithm, and the corresponding clusters C in the image frames of the left and right eyes at the same timestamp are identified. i and C j The matching process is performed to match the fragments corresponding to the stereo event frames.

[0052] Furthermore, in step (5), the planar coordinates of the centroid of the target fragment cluster after matching in step (4) in the pixel coordinate system are calculated, as follows:

[0053]

[0054] In the formula, x c y c The x and y coordinates represent the centroid; N represents the total number of events in the cluster; x i y i This represents the cluster C obtained in step (4). i The x-coordinate and y-coordinate of each event.

[0055] Combining the planar coordinates of the centroids of the target fragments in the pixel coordinate system of the left and right event cameras, the initial calibration relative extrinsic matrix, and the initial calibration intrinsic matrix, the three-dimensional spatial coordinates of the target fragments in the world coordinate system are calculated according to a preset three-dimensional stereo imaging formula:

[0056]

[0057] In the formula, X, Y, and Z represent the three-dimensional spatial coordinates of the centroid of the target fragment in the world coordinate system, B represents the baseline distance, f represents the ratio of focal length to pixel width, and x... l x r These represent the x-coordinates of the centroid of the target fragment in the pixel coordinate systems of the left and right event cameras, respectively.

[0058] The three-dimensional spatial coordinates of the centroids of the target fragments obtained from the above calculations in the world coordinate system are used to calculate the displacement of the same point within a set time interval according to the Euclidean distance calculation formula; the Euclidean distance calculation formula is expressed as:

[0059]

[0060] In the formula S dist The Euclidean distance between the same target point in two frames is selected by the user; x1, y1, z1 and x2, y2, z2 represent the three-dimensional spatial coordinates of the centroid of the target fragment calculated twice within a certain time interval in the world coordinate system.

[0061] Calculate the velocity between two points using the velocity calculation formula:

[0062] V = S dist / (T i+j -T i )

[0063] In the formula, V represents the measured fragment velocity behind the target; S dist The Euclidean distance between the same target point in two manually selected frames; T i+j and T i These represent the time interval between the two selected images.

[0064] Figure 4 A comparison chart showing the results of collecting fragment field data streams before and after density threshold denoising.

[0065] Figure 5 The clustering algorithm of this invention identifies the circular bulletproof material at the center of the target plate that was knocked away.

[0066] Table 1 shows the reconstruction test verification of the method of the present invention.

[0067] Table 1

[0068]

[0069] The above are preferred embodiments of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. However, the embodiments of the present invention are not limited to the above content. For those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention should be considered as equivalent substitutions and are included within the protection scope of the present invention.

Claims

1. A target damage assessment method based on a binocular event camera, characterized in that, Includes the following steps: (1) Using a binocular event camera acquisition structure device, calculate the parameters that match the structure; (2) Obtain the original event streams of the target fragments captured by the binocular event cameras from the left and right eyes respectively from the two host computers, and remove the noise in the target fragment event streams by setting a density threshold. (3) Accumulate the target fragment event stream after noise reduction in step (2) according to the dynamic integration time T. Initially, the dynamic integration time T is the initial integration time τ. Record all events in the integrated time slice as the sample set D. (4) Perform DBSCAN spatial clustering on the event frames integrated in step (3) according to the matching parameters, and extract the centroid of the selected target fragment in the event frames; parameterize the depth information of the centroid of the binocular event frames through the nearest neighbor matching algorithm, and perform stereo matching on the fragments to achieve matching of the centroid of the fragments corresponding to the binocular event frames. (5) Use the parallax method to reconstruct the three-dimensional coordinates of the corresponding centroids matched in step (4) in the world coordinate system, and perform time difference to obtain the velocity of the fragment.

2. The target damage assessment method based on a binocular event camera according to claim 1, characterized in that: The bottom of the binocular event camera acquisition structure is a level tripod, which is used to fix and keep the entire acquisition device horizontal. It is connected to the top horizontal slide rail through the main rod. There is a slot in the middle of the slide rail, in which two modules for fixing the event cameras are installed. This is to ensure that the left and right event cameras are parallel and to adjust the baseline distance between the two event cameras.

3. The target damage assessment method based on a binocular event camera according to claim 1, characterized in that: In step (1), the parameters matching the structure are calculated, and appropriate shooting parameters are configured according to the horizontal field of view, vertical field of view, and viewing distance of the event camera. The formula is as follows: In the formula, B represents the baseline of the binocular event camera, α represents the horizontal field of view of the event camera, and L represents the vertical distance between the focal plane of the binocular event camera system and the field of view corner.

4. The target damage assessment method based on a binocular event camera according to claim 1, characterized in that: In step (2), noise in the fragment event stream after the target is removed by setting a density threshold. The event stream typically uses address events to represent readings, as shown below: e = (x, y, t) In the formula, x is the number of columns; y is the number of rows; and t is the timestamp. A valid event activates pixels in its surrounding area; its density value is greater than that of an invalid event; the density calculation formula for the input event e(x,y,t) within the spatiotemporal neighborhood U(x,y,t) is as follows: In the formula, C represents the set threshold; Nvalid represents the target event; Nnoise represents noise; if the event density is less than C, the event is invalid and filtered out; otherwise, it is valid and retained.

5. The target damage assessment method based on a binocular event camera according to claim 1, characterized in that: In step (3), all events in the integrated time slice are denoted as sample set D. After adaptive time integration, the noise-removed event stream data is integrated into image frames. All event points in the image frames to be processed are placed into the sample set D = {x1, x2, ..., x...} m }middle.

6. The target damage assessment method based on a binocular event camera according to claim 1, characterized in that: The DBSCAN clustering method in step (4) is as follows: Given the neighborhood parameters (∈, MinPts), for x j ∈D, its ∈-neighborhood contains samples in the sample set D that are the same as x. j The distance to samples whose values ​​are not greater than those of ∈, i.e., N ∈ (x j )={x i ∈D|dist(x i ,x j If x ≤ ∈}; j The ∈-neighborhood contains at least MinPts samples, i.e., |N ∈ (x j If |≥MinPts, then x j It is a core object; based on the given neighborhood parameters ∈ and MinPts, all core points are determined. For each core point, an unprocessed core point is selected, and clusters are generated from samples whose density is reachable. The above process is repeated until all core points are processed. By parameterizing the centroid depth information of the binocular event frame using the nearest neighbor matching algorithm, the fragments are stereo matched to achieve the matching of the centroids of the fragments corresponding to the binocular event frames.

7. A target damage assessment device based on a binocular event camera, characterized in that: The steps of the target damage assessment method based on a binocular event camera as described in any one of claims 1-6 are adopted.

Images