Event Camera Network Optical Measurement Method for High-Dynamic High-Speed Target Motion Parameters

Abstract

The present application relates to an optical measurement method for high-dynamic high-speed target motion parameters using an event camera network. The method includes: arranging a multi-view event camera synchronous observation system in the area to be observed and performing parameter calibration, observing a high-dynamic high-speed target to obtain observation event stream data; constructing a two-dimensional map of event-starting point distance versus time for each view according to the observation event stream data, and obtaining a target head feature event set from the two-dimensional map of event-starting point distance versus time; using the target head feature event sets in multiple views to intersect the target motion straight-line trajectory in three-dimensional space; retrieving the target motion straight-line trajectory according to the three-dimensional straight-line point homologous event retrieval method based on reprojection error, establishing the correspondence between three-dimensional space position and time, and using a fitting method to achieve motion parameter measurement calculation and reconstruct a three-dimensional motion field. Using this method can achieve high time resolution and high dynamic range.

Description

Technical Field

[0001] This application relates to the technical field of target motion optical measurement, and particularly to an optical measurement method for event camera network of high-dynamic high-speed target motion parameters. Background Art

[0002] Measurement methods for high-dynamic high-speed targets mainly include contact target method, light screen target method, acoustic measurement method, radar method, optical measurement method, etc. Among them, the contact target method realizes the measurement of dispersion by arranging target plates on the predicted scattering path of the target for recovery; the light screen target method avoids contact with the target and uses a light screen to measure the speed; the acoustic measurement method measures the spatial position and motion parameters of the target by using acoustic wave reflection; the radar method analyzes the echo according to the Doppler effect to measure the motion parameters; the optical measurement method uses high-speed cameras, X-rays and other high-frame-rate optical measurement devices to record the high-speed motion process of the target, and uses image processing technology to measure the motion parameters.

[0003] However, the above-mentioned many methods have their own advantages and disadvantages. The contact target method has a simple principle, low test cost, and intuitive dispersion, but the workload of data recovery is large, and it interferes with the target motion and cannot continuously track the motion process of the same target; the light screen target method avoids contact with the target and has high speed measurement accuracy, but the equipment cost is high and it is not easy to protect. The above two methods for observing high-dynamic high-speed targets have very limited measurement points and are usually restricted by the problem of range estimation, so the flexibility is poor. The acoustic measurement method has a low cost, but due to the low speed of sound, the speed measurement and positioning accuracy are low; the radar method has a high cost, and the echo signal is relatively complex and difficult to process; the optical imaging method has a high resolution and can continuously record the high-speed motion process of multiple targets at the same time, but the high-speed camera has the problem of being difficult to balance high time resolution and high dynamic range, is prone to overexposure, and has the hidden danger of equipment overheating; equipment such as X-rays is extremely expensive and has extremely large storage overhead, and can only continuously shoot very few frames. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide an optical measurement method for event camera network of high-dynamic high-speed target motion parameters, which has high time resolution, high dynamic range, and low cost, can realize microsecond-level measurement of motion parameters such as three-dimensional position, scattering speed, and scattering trajectory of high-dynamic high-speed moving targets, and can realize the reconstruction and visualization of the high-dynamic high-speed motion process of the target.

[0005] An optical measurement method for event camera network of high-dynamic high-speed target motion parameters, the method includes:

[0006] Arrange a multi-view event camera synchronous observation system in the area to be observed and perform parameter calibration, and observe the high-dynamic high-speed target to obtain observation event stream data;

[0007] Construct a two-dimensional graph of event-starting point distance over time for each view based on the observed event stream data, and obtain the target head feature event set from the two-dimensional graph of event-starting point distance over time; utilize the target head feature event sets under multiple views to intersect the target motion straight-line trajectory in three-dimensional space;

[0008] Retrieve the target motion straight-line trajectory according to the three-dimensional straight-line point homonymous event retrieval method based on reprojection error, and establish the corresponding relationship between three-dimensional space position and time;

[0009] Based on the corresponding relationship between three-dimensional space position and time, adopt a fitting method to achieve the measurement and solution of motion parameters and reconstruct the three-dimensional motion field.

[0010] The above optical measurement method of an event camera network for high-dynamic high-speed target motion parameters utilizes the advantages of high dynamic range, high time resolution, and low cost of event cameras to perform optical measurement on high-dynamic high-speed targets. First, according to the spatio-temporal law of event distribution, accurately extract the target head trigger event set and intersect to obtain the straight-line motion trajectory in three-dimensional space; secondly, use the three-dimensional straight-line point homonymous event retrieval method based on reprojection error to match the event stream obtained by each event camera observation in the event camera network with its homonymous point on the three-dimensional straight-line motion trajectory, and establish the connection between three-dimensional coordinates and time; finally, measure the motion field of the target based on the world system coordinates and timestamps of the three-dimensional retrieved points. This method has high time resolution and high dynamic range, avoids the problem of difficult matching of homonymous points in the measurement of fragments and projectiles by multi-view event cameras, solves the problem of few optical measurement points for high-dynamic high-speed targets, can utilize the high time resolution of the event stream to achieve multi-view and multi-measurement point joint solution, and can be applied to various high-dynamic and high-speed scenarios such as range tests and particle image velocimetry. Brief Description of the Drawings

[0011] Figure 1 It is a schematic flowchart of the optical measurement method of an event camera network for high-dynamic high-speed target motion parameters in an embodiment;

[0012] Figure 2 It is a flowchart of the optical measurement method of an event camera network for high-dynamic high-speed target motion parameters in an embodiment;

[0013] Figure 3 It is a flowchart of the target head feature event set extraction algorithm in an embodiment;

[0014] Figure 4 It is a schematic flowchart of the three-dimensional straight-line point homonymous event retrieval algorithm (unidirectional) based on reprojection error in another embodiment. Detailed Description of the Invention

[0015] To make the objectives, technical solutions and advantages of this application more clear and understandable, the following further details this application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely used to explain this application and are not used to limit this application.

[0016] In one embodiment, as Figure 1 and Figure 2 shown, an optical measurement method for an event camera network of high-dynamic high-speed target motion parameters is provided, including the following steps:

[0017] Step 102, arrange a multi-view event camera synchronous observation system in the area to be observed and perform parameter calibration, and observe the high-dynamic high-speed target to obtain observation event stream data.

[0018] Event cameras have advantages such as high dynamic range, high time resolution, and low cost. Since they do not need to output grayscale information and the amount of output data is significantly reduced compared to traditional image frame cameras, they can achieve continuous observation for a long time. These advantages are very suitable for measuring the motion parameters of high-dynamic high-speed targets. Therefore, this application selects multiple appropriate models of event cameras to perform video measurement on high-dynamic high-speed targets: build a multi-view event camera network observation system in the test site and calibrate it. After collecting the observation event stream data, design a complete set of high-dynamic high-speed target motion parameter measurement processes based on the event stream data.

[0019] Step 104, construct a two-dimensional graph of event-starting point distance versus time for each view according to the observation event stream data, and obtain the target head feature event set from the two-dimensional graph of event-starting point distance versus time; use the target head feature event sets of multiple views to intersect the target motion straight line trajectory in three-dimensional space.

[0020] In the event stream data, the high-dynamic and high-speed target motion has a relatively significant "trailing" effect, which will cause great interference to the final measurement result. Therefore, it is necessary to eliminate the influence caused by the trailing effect, extract the event of the target head feature, and complete the intersection of the three-dimensional linear motion trajectory in space based on this to obtain the three-dimensional motion trajectory. First, given the initial endpoints and the end endpoints of all target two-dimensional trajectory segments under each camera view, and retrieve all event points located near the two-dimensional line segment formed by the initial endpoint and the end endpoint, so as to form an event set for each high-speed target under each view; then calculate the Euclidean distance between all events and the starting point for each event set, and construct a two-dimensional graph of the event-starting point distance versus time. By processing the event stream data in this way, the information about time and space in the event data can be further mined, enabling a more refined analysis of the target motion in the time dimension and contributing to achieving high-time-resolution measurement. At the same time, the high dynamic range characteristic of the event camera enables it to accurately capture the changes of the target under different lighting conditions and can effectively respond to scenes from low brightness to high brightness, which ensures that the system has a high dynamic range.

[0021] Furthermore, starting from the event with the smallest timestamp in the event set, traverse all events in the order of increasing timestamp. If the distance between the current event and the starting point is greater than the reference event, and there is no step in the explosion center distance and timestamp with respect to the reference event (less than a pre-set threshold), then the current event is taken as the target head feature event, and at the same time the current event is updated as the reference event. The initial reference event is set as the starting event. Finally, after traversing all events, the target head feature event set is obtained. The algorithm flow of this part can be as Figure 2 shown. In this way, the events triggered by the head of the high-dynamic and high-speed target can be accurately extracted from the observation data of the event camera quickly and conveniently. These feature event sets can be used as an important basis for the subsequent homonymous matching of three-dimensional space points and events. Since the extracted are the target head feature events, the interference of irrelevant events is reduced, making the homonymous matching more targeted and accurate.

[0022] Then, use the plane straight line equation to fit all the target head feature events extracted in the pixel coordinate system to obtain the slope and intercept of the fitted straight line, and based on the camera system parameters obtained by calibration, use the epipolar constraint to obtain a homonymous point on the straight line under multiple views; finally, take the homonymous point and the slope as inputs, and use the straight line and straight line stereo intersection method to obtain the target motion straight line trajectory in three-dimensional space. It can effectively eliminate the influence of the "trailing" effect on the subsequent measurement and can effectively obtain the straight line motion trajectory of the target in three-dimensional space.

[0023] Step 106, retrieve the target motion straight line trajectory according to the three-dimensional straight line point homonymous event retrieval method based on the reprojection error, and establish the corresponding relationship between the three-dimensional space position and time.

[0024] In order to further measure the motion speed and the change of motion position of a high-dynamic high-speed target, after obtaining its three-dimensional space motion trajectory, it is also necessary to obtain the three-dimensional space position and its corresponding timestamp. The present invention adopts a three-dimensional straight-line point homonymous event retrieval method based on reprojection error: First, select a three-dimensional point on the three-dimensional motion trajectory obtained by the above intersection as the retrieval starting point, and perform forward and backward searches respectively according to the direction vector of the straight line; then, according to the parameters obtained by calibration, project each retrieval point onto the pixel coordinate system of each camera respectively, and calculate the Euclidean distance between the projected point and the event in each view; subsequently, when all the projected points are retrieved, select the one with the minimum Euclidean distance as the three-dimensional matching homonymous point for each event in multiple views (the Euclidean distance needs to be less than 1 pixel), and assign the timestamp of the event to the timestamp of the three-dimensional retrieval point, thereby establishing a model of the evolution of position over time on the fragment target trajectory. The single-view one-way search process of this algorithm is shown as Figure 3 shown. Among them is the straight-line direction vector, is the retrieval starting point, is the retrieval step size, is the projection matrix, is the three-dimensional coordinate of the retrieval point, is the timestamp of the retrieval point, is the two-dimensional coordinate of the event, is the timestamp of the event, is to obtain the number of elements in the set. Using the three-dimensional straight-line point homonymous event retrieval method based on reprojection error, match the event stream obtained by each event camera observation in the event camera network with its homonymous point on the three-dimensional straight-line motion trajectory. The three-dimensional straight-line point homonymous event retrieval method based on reprojection error can match the event stream obtained by each event camera observation in the event camera network with its homonymous point on the three-dimensional straight-line motion trajectory, establish the connection between the three-dimensional coordinate and time according to the timestamp provided by the event, and form an event optical measurement point set; it can avoid the problem that multi-view events are difficult to match, and can provide more optical measurement points than traditional multi-view intersection methods,

[0025] Step 108, based on the correspondence between the three-dimensional space position and time, use the fitting method to realize the motion parameter measurement and calculation and reconstruct the three-dimensional motion field.

[0026] Using the above method, a corresponding relationship between the three-dimensional spatial position and time is established for each high-dynamic high-speed target. In this application, the three-dimensional spatial position of the target at the earliest timestamp is used as a reference point, and the relative motion distance corresponding to all timestamps is calculated in space according to the three-dimensional coordinates, so as to obtain a scatter plot of the motion distance changing with time. By using a polynomial or fitting according to the velocity decay empirical formula of the target, the variation function of the motion distance with time is obtained. Further differentiating the function can obtain the motion speed at each timestamp. At the same time, according to the three-dimensional spatial position and timestamp information, the scattering process of the target can be reproduced in the three-dimensional space, and the measurement of its three-dimensional motion field can be realized. Using the parameter measurement solution and reconstruction of the motion field method, fitting is carried out by using a polynomial or a velocity decay empirical formula, etc., to realize the measurement of motion parameters such as the three-dimensional position, motion speed, and motion trajectory of high-dynamic high-speed targets with microsecond-level and multiple optical measurement points, and to realize the reconstruction and visualization of the target motion process.

[0027] In the above optical measurement method of an event camera network for high-dynamic high-speed target motion parameters, the advantages of high dynamic range, high time resolution, and low cost of the event camera are utilized to perform optical measurement on high-dynamic high-speed targets. First, according to the spatio-temporal law of event distribution, the accurate extraction of the target head trigger event set is carried out, and the linear motion trajectory in three-dimensional space is obtained by intersection; secondly, the three-dimensional straight line point homologous event retrieval method based on reprojection error is used to match the event stream obtained by each event camera observation in the event camera network with its homologous points on the three-dimensional linear motion trajectory, and the connection between three-dimensional coordinates and time is established; finally, the motion field of the target is measured based on the world coordinate system coordinates and timestamps of the three-dimensional retrieval points. This method has high time resolution and high dynamic range, avoiding the problem of difficult matching of homologous points in the measurement of fragments and projectiles by multi-camera event cameras, solving the problem of few optical measurement points for high-dynamic high-speed targets, and being able to realize multi-camera and multi-measurement point joint solution by using the high time resolution of the event stream, and can be applied to various high-dynamic and high-speed scenarios such as range tests and particle image velocimetry.

[0028] In one embodiment, constructing a two-dimensional map of event-starting point distance versus time according to the observed event stream data, including:

[0029] Given the initial endpoints and end endpoints of all target two-dimensional trajectory segments under each camera view, and retrieving all event points located near the two-dimensional line segment formed by the initial endpoint and the end endpoint, forming an event set for each high-speed target under each view;

[0030] Calculating the Euclidean distance between all events and the starting point for each event set, and constructing a two-dimensional map of event-starting point distance versus time.

[0031] In one embodiment, obtaining the target head feature event set from the two-dimensional map of event-starting point distance versus time, including:

[0032] Starting from the event with the smallest timestamp in the event set of the two-dimensional graph of the distance from the event to the starting point over time, traverse all events in ascending order of the timestamp. If the distance from the current event to the starting point is greater than the reference event, and both the distance from the blast center and the timestamp of the current event are less than the preset thresholds compared to the reference event, then the current event is taken as the target head feature event, and at the same time, the current event is updated as the reference event. The initial reference event is set as the starting event, and finally, the target head feature event set is obtained after traversing all events.

[0033] In one embodiment, the intersection of the target head feature event sets in multiple views is used to obtain the target motion straight-line trajectory in three-dimensional space, including:

[0034] All the target head feature events extracted are fitted using the plane straight-line equation in the pixel coordinate system to obtain the slope and intercept of the fitted straight line, and based on the camera system parameters obtained by calibration, a corresponding point on the straight line in multiple views is obtained using the epipolar constraint.

[0035] Taking the corresponding point and the slope as inputs, the target motion straight-line trajectory in three-dimensional space is obtained using the method of straight-line to straight-line stereo intersection.

[0036] In one embodiment, the target motion straight-line trajectory is retrieved according to the three-dimensional straight-line point corresponding event retrieval method based on the reprojection error, and the corresponding relationship between the three-dimensional space position and time is established, including:

[0037] A three-dimensional point on the target motion straight-line trajectory is selected as the retrieval starting point, and forward and backward searches are respectively carried out along the direction vector of the straight line.

[0038] According to the parameters obtained by calibration, each retrieval point is respectively projected into the pixel coordinate system of each camera, and the Euclidean distance between the projected point and the event in each view is calculated. When all the projected points are retrieved, for each event in multiple views, the one with the smallest Euclidean distance and less than the preset acceptance interval is selected as the three-dimensional matching corresponding point, and the timestamp of the event is assigned to the timestamp of the three-dimensional retrieval point, establishing the corresponding relationship between the three-dimensional space position and time and forming the event optical measurement point set.

[0039] In one embodiment, based on the corresponding relationship between the three-dimensional space position and time, a fitting method is used to realize the motion parameter measurement and calculation and reconstruct the three-dimensional motion field, including:

[0040] Based on the corresponding relationship between the three-dimensional space position and time, taking the target three-dimensional space position corresponding to the event with the earliest timestamp as the reference point, and calculating the relative motion distance corresponding to the timestamp of all event optical measurement points according to the three-dimensional coordinates in space, obtaining a scatter plot of the motion distance changing with time.

[0041] The scatter plot of the moving distance varying with time is fitted by using polynomials or according to the empirical formula of the speed decay of the target, so as to obtain the variation function of the moving distance with time;

[0042] The derivative of the variation function of the moving distance with time is calculated to obtain the moving speed at each time stamp;

[0043] According to the three-dimensional spatial position and time stamp information, the scattering process of the target is reproduced in the three-dimensional space to realize the reconstruction of its three-dimensional motion field.

[0044] It should be understood that although Figure 1 each step in the flowchart of Figure 1 is shown in sequence according to the indication of the arrow, these steps do not necessarily need to be executed in sequence according to the order indicated by the arrow. Unless otherwise clearly stated in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover,

[0045] at least a part of the steps in

[0046] may include multiple sub-steps or multiple stages. These sub-steps or stages do not necessarily need to be executed at the same moment, but can be executed at different moments. The execution order of these sub-steps or stages does not necessarily need to be in sequence either, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps. The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0046] The above-described embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several deformations and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. An event camera network optical measurement method for high-dynamic and high-speed target motion parameters, characterized in that: The method comprises: Arrange a multi-camera synchronous observation system in the area to be observed and calibrate the parameters, observe high-dynamic and high-speed targets to obtain observation event stream data; Constructing a two-dimensional graph of event-starting point distance over time for each view according to the observed event stream data, obtaining a target head feature event set from the two-dimensional graph of event-starting point distance over time; using the target head feature event set under multiple views to intersect the target motion straight line trajectory in three-dimensional space; Retrieving the target motion straight line trajectory according to a three-dimensional straight line point homonymous event retrieval method based on reprojection error, and establishing a corresponding relationship between the three-dimensional spatial position and time; Based on the correspondence between the three-dimensional spatial position and time, a fitting method is used to measure and calculate motion parameters and reconstruct a three-dimensional motion field.

2. The method according to claim 1, characterized in that Constructing a two-dimensional graph of event-starting point distance over time according to the observed event stream data, including: Given the initial endpoint and the final endpoint of all target two-dimensional trajectory segments under each camera view, all event points located near the two-dimensional line segment formed by the initial endpoint and the final endpoint are retrieved to form an event set for each high-speed target under each view; For each event set, the Euclidean distances between all events and the starting point are calculated, and a two-dimensional graph of event-starting point distances over time is constructed.

3. The method according to claim 1, characterized in that Obtaining a target head feature event set from the event-starting point distance versus time two-dimensional graph, including: Starting from the event with the smallest timestamp in the event set of the event-starting point distance versus time two-dimensional graph, all events are traversed in the order of increasing timestamps. If the distance between the current event and the starting point is greater than the reference event, and the distance from the center of gravity and the timestamp to the reference event are both less than a preset threshold, the current event is taken as the target head feature event, and the current event is updated to the reference event, and the initial reference event is set as the starting event. Finally, after traversing all events, the target head feature event set is obtained.

4. The method according to any one of claims 1 to 3, characterized in that: Using the target head feature event set under multiple views to intersect the target motion straight line trajectory in three-dimensional space, including: All the extracted target head feature events are fitted in the pixel coordinate system using the plane line equation to obtain the slope and intercept of the fitted line. Based on the camera system parameters obtained by calibration, an epipolar constraint is used to obtain a point of the same name on the line under multiple viewing angles. Taking the same-name points and slope as input, the straight-line trajectory of the target motion in three-dimensional space is obtained by using the straight-line stereo intersection method.

5. The method according to claim 1, characterized in that: The target motion straight line trajectory is retrieved according to a three-dimensional straight line point same-name event retrieval method based on reprojection error, and a corresponding relationship between a three-dimensional spatial position and time is established, including: Select a 3D point on the target motion straight line trajectory as the retrieval starting point, and conduct positive and negative searches respectively according to the direction vector of the straight line; According to the parameters obtained by calibration, each retrieval point is projected to the pixel coordinate system of each camera, and the Euclidean distance between the projection point and the event in each view is calculated. When all projection points are retrieved, the one with the smallest Euclidean distance and less than the pre-set acceptance interval is selected as the three-dimensional matching homonymous point for each event in multiple views, and the timestamp of the event is assigned to the timestamp of the three-dimensional retrieval point, so as to establish the correspondence between the three-dimensional spatial position and time and form the event light measurement point set.

6. The method according to claim 1, characterized in that Based on the correspondence between the three-dimensional spatial position and time, a fitting method is used to measure and calculate motion parameters and reconstruct a three-dimensional motion field, including: Based on the correspondence between the three-dimensional spatial position and time, the target three-dimensional spatial position corresponding to the earliest timestamp event is used as a reference point, and the relative movement distance corresponding to the timestamps of all event light measurement points is calculated in space according to the three-dimensional coordinates to obtain a scatter plot of the movement distance changing with time; Fitting the scatter plot of the movement distance over time using a polynomial or an empirical formula based on the speed attenuation of the target to obtain a function of the movement distance over time; Derivate the function of the movement distance changing with time to obtain the movement speed at each timestamp; According to the three-dimensional spatial position and timestamp information, the target's scattering process is reproduced in the three-dimensional space to achieve reconstruction of its three-dimensional motion field.

Images