Event camera network optical measurement method for motion parameters of high-dynamic high-speed target

Abstract

The invention relates to an event camera network optical measurement method for motion parameters of a high-dynamic high-speed target. The method comprises the following steps: arranging a multi-view event camera synchronous observation system in a to-be-observed area, carrying out parameter calibration, and observing a high-dynamic high-speed target to obtain observation event stream data; constructing an event-starting point distance two-dimensional diagram along with time for each view according to the observation event stream data, and obtaining a target head feature event set from the event-starting point distance two-dimensional diagram along with time; utilizing the target head feature event sets under the plurality of views to intersect a target motion linear track in a three-dimensional space; according to a three-dimensional straight line point homonymy event retrieval method based on a re-projection error, a target motion straight line track is retrieved, a corresponding relation between a three-dimensional space position and time is established, a fitting method is adopted to realize motion parameter measurement and calculation, and a three-dimensional motion field is reconstructed. By adopting the method, high time resolution and high dynamic range can be achieved.

Description

Technical Field

[0001] The present application relates to the technical field of target motion optical measurement, and in particular to an event camera network optical measurement method for high-dynamic and high-speed target motion parameters. Background Art

[0002] The measurement methods for high-dynamic and high-speed targets mainly include contact target method, light curtain target method, acoustic measurement method, radar method, optical measurement method, etc. Among them, the contact target method realizes the measurement of dispersion by arranging target plates on the predicted dispersion path of the target for recovery; the light curtain target method avoids contact with the target and uses the light curtain to measure the speed; the acoustic measurement method uses the reflection of sound waves to measure the spatial position and motion parameters of the target; the radar method analyzes the echo according to the Doppler effect to measure the motion parameters; the optical measurement method uses high-speed cameras, X-rays and other high-frame rate optical measurement equipment to record the high-speed motion process of the target, and uses image processing technology to measure the motion parameters.

[0003] However, the above methods have their own advantages and disadvantages. The contact target method has a simple principle, low test cost, and intuitive dispersion, but the workload of data recovery is large, and it interferes with the movement of the target and cannot continuously track the movement process of the same target; the light curtain target method avoids contact with the target and has high speed measurement accuracy, but the equipment cost is high and it is not easy to protect. The above two methods observe high-dynamic high-speed targets, the measurement points are very limited, and there are usually problems with range estimation, so the flexibility is poor. The acoustic measurement method has a low cost, but because the speed of sound is not high, the speed measurement and positioning accuracy is low; the radar method has a high cost, and the echo signal is more complex and difficult to process; the optical imaging method has a high resolution and can continuously record the high-speed movement process of multiple targets at the same time, but high-speed cameras have the problem of difficulty in balancing high temporal resolution and high dynamic range, which is prone to overexposure and the risk of equipment overheating; X-ray and other equipment are expensive and have huge storage overhead, and can only continuously shoot very few frames. Summary of the invention

[0004] Based on this, it is necessary to provide an event camera network optical measurement method for high dynamic and high speed target motion parameters with high temporal resolution, high dynamic range, and low cost, which can realize microsecond measurement of motion parameters such as three-dimensional position, flying speed, and flying trajectory of high dynamic and high speed moving targets, and can realize reconstruction and visualization of high dynamic and high speed motion process of targets.

[0005] An event camera network optical measurement method for high-dynamic and high-speed target motion parameters, the method comprising: Arrange a multi-camera synchronous observation system in the area to be observed and calibrate the parameters, observe high-dynamic and high-speed targets to obtain observation event stream data; According to the observed event stream data, a two-dimensional graph of event-starting point distance over time is constructed for each view, and a target head feature event set is obtained from the two-dimensional graph of event-starting point distance over time; the target head feature event sets under multiple views are used to intersect the target motion straight line trajectory in the three-dimensional space; The target motion straight line trajectory is retrieved according to the 3D straight line point homonymous event retrieval method based on reprojection error, and the corresponding relationship between 3D spatial position and time is established; Based on the correspondence between three-dimensional spatial position and time, a fitting method is used to measure and calculate motion parameters and reconstruct the three-dimensional motion field.

[0006] The above-mentioned event camera network optical measurement method for the motion parameters of high-dynamic and high-speed targets uses the advantages of event cameras such as high dynamic range, high temporal resolution, and low cost to perform optical measurement of high-dynamic and high-speed targets. First, according to the spatiotemporal law of event distribution, the target head trigger event set is accurately extracted, and the linear motion trajectory in three-dimensional space is obtained by intersection; secondly, the three-dimensional straight line point homonymous event retrieval method based on reprojection error is used to match the homonymous points on the three-dimensional linear motion trajectory for each event camera observation in the event camera network, and establish the connection between three-dimensional coordinates and time; finally, the target's motion field is measured based on the world system coordinates and timestamps of the three-dimensional retrieval points. This method has high temporal resolution and high dynamic range, avoids the problem of difficulty in matching homonymous points in the measurement of fragments and projectiles by multi-eye event cameras, solves the problem of few optical measurement points for high-dynamic and high-speed targets, and can use the high temporal resolution of the event stream to realize multi-eye and multi-measurement point joint solution. It can be applied to many types of high-dynamic and high-speed scenes such as range tests and particle image velocimetry. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 A schematic diagram of a flow chart of an event camera network optical measurement method for high-dynamic and high-speed target motion parameters in one embodiment; Figure 2 A flowchart of an event camera network optical measurement method for high-dynamic and high-speed target motion parameters in one embodiment; Figure 3 It is a flow chart of a target head feature event set extraction algorithm in one embodiment; Figure 4 The figure is a flowchart of a three-dimensional straight line point same-name event retrieval algorithm (one-way) based on reprojection error in another embodiment. DETAILED DESCRIPTION

[0008] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0009] In one embodiment, Figure 1 and Figure 2 As shown, a method for event camera network optical measurement of high-dynamic and high-speed target motion parameters is provided, comprising the following steps: Step 102: deploy a multi-camera synchronous observation system in the area to be observed and perform parameter calibration, observe high-dynamic and high-speed targets to obtain observation event stream data.

[0010] Event cameras have the advantages of high dynamic range, high temporal resolution, and low cost. Since there is no need to output grayscale information, the amount of output data is significantly reduced compared to traditional image frame cameras, so continuous observation for a longer period of time can be achieved. These advantages are very suitable for the motion parameter measurement of high-dynamic and high-speed targets. Therefore, this application selects multiple event cameras of suitable models to perform video measurement of high-dynamic and high-speed targets: a multi-eye event camera network observation system is built and calibrated at the test site, and after collecting and observing event stream data, a complete set of high-dynamic and high-speed target motion parameter measurement processes are designed based on the event stream data.

[0011] Step 104, construct a two-dimensional graph of event-starting point distance over time for each view based on the observed event stream data, obtain the target head feature event set from the two-dimensional graph of event-starting point distance over time; and use the target head feature event sets of multiple views to intersect the target motion straight line trajectory in the three-dimensional space.

[0012] Since the high-dynamic and high-speed target motion has a significant "tailing" effect in the event stream data, it will cause great interference to the final measurement results. Therefore, it is necessary to eliminate the influence of the tailing effect, extract the target head feature event, and complete the intersection of the three-dimensional linear motion trajectory in space based on this, and obtain the three-dimensional motion trajectory. First, given the initial endpoints and the final endpoints of all target two-dimensional trajectory segments under each camera perspective, all event points located near the two-dimensional line segment formed by the initial endpoint and the final endpoint are retrieved, so as to form an event set for each high-speed target under each perspective; then, for each event set, the Euclidean distance between all events and the starting point is calculated, and a two-dimensional graph of the event-starting point distance over time is constructed. By processing the event stream data in this way, the information about time and space in the event data can be further mined, so that the target motion can be analyzed more finely in the time dimension, which is helpful to achieve high time resolution measurement. At the same time, the high dynamic range characteristics of the event camera enable it to accurately capture the changes of the target under different lighting conditions, and can effectively respond to scenes from low brightness to high brightness, which ensures that the system has a high dynamic range.

[0013] Then, starting from the event with the smallest timestamp in the event set, all events are traversed in the order of increasing timestamps. If the distance between the current event and the starting point is greater than the reference event, and there is no step (less than the preset threshold) in the distance from the center of the explosion and the timestamp with the reference event, the current event is taken as the target head feature event, and the current event is updated to the reference event, and the initial reference event is set as the starting event. Finally, after traversing all events, the target head feature event set is obtained. The algorithm flow of this part can be as follows: Figure 2 As shown in the figure. In this way, events triggered by high-dynamic and high-speed target heads can be quickly and easily extracted from event camera observation data. These feature event sets can serve as an important basis for subsequent matching of three-dimensional space points with events. Since the target head feature events are extracted, the interference of irrelevant events is reduced, making the matching of the same name more targeted and accurate.

[0014] Then, the plane line equation is used to fit all the extracted target head feature events in the pixel coordinate system to obtain the slope and intercept of the fitting line, and based on the camera system parameters obtained by calibration, the epipolar constraint is used to obtain a same-name point on the line under multiple viewing angles; finally, the same-name point and slope are used as input, and the straight line trajectory of the target motion in three-dimensional space is obtained by using the straight line and straight line stereo intersection method. It can effectively eliminate the influence of the "tailing" effect on subsequent measurements and can effectively obtain the straight line motion trajectory of the target in three-dimensional space.

[0015] Step 106, searching the target motion straight line trajectory according to the three-dimensional straight line point same-name event retrieval method based on reprojection error, and establishing a corresponding relationship between the three-dimensional spatial position and time.

[0016] In order to further measure the movement speed and movement position changes of high-dynamic high-speed targets, after obtaining its three-dimensional space motion trajectory, it is also necessary to obtain the three-dimensional space position and its corresponding timestamp. The present invention adopts a three-dimensional straight line point homonymous event retrieval method based on reprojection error: first, select a three-dimensional point from the three-dimensional motion trajectory obtained by the above intersection as the retrieval starting point, and carry out positive and negative searches according to the direction vector of the straight line; then, according to the parameters obtained by calibration, project each retrieval point to the pixel coordinate system of each camera, and calculate the Euclidean distance between the projection point and the event in each view; then, after all projection points are retrieved, select the one with the smallest Euclidean distance for each event in multiple views as the three-dimensional matching homonymous point (the Euclidean distance must be less than 1 pixel), and assign the timestamp of the event to the timestamp of the three-dimensional retrieval point, thereby establishing a position evolution model over time on the fragment target trajectory. The single-view one-way search process of the algorithm is shown as follows Figure 3 As shown. is the line direction vector, is the starting point for the search. is the search step length, is the projection matrix, is the three-dimensional coordinate of the retrieval point, is the timestamp of the retrieval point, is the two-dimensional coordinate of the event, is the timestamp of the event, To find the number of elements in the set. The 3D straight line point homonymous event retrieval method based on reprojection error is used to match the homonymous points on the 3D straight line motion trajectory for the event stream obtained by each event camera observation in the event camera network. The 3D straight line point homonymous event retrieval method based on reprojection error can match the homonymous points on the 3D straight line motion trajectory for the event stream obtained by each event camera observation in the event camera network, establish the connection between the 3D coordinates and time according to the timestamp provided by the event, and form an event light measurement point set; it can avoid the problem of multi-view event matching difficulties, and can provide more light measurement points compared to the traditional multi-view intersection method. Step 108: Based on the correspondence between the three-dimensional spatial position and time, a fitting method is used to measure and calculate the motion parameters and reconstruct the three-dimensional motion field.

[0017] The above method establishes a correspondence between the three-dimensional spatial position and time for each high-dynamic high-speed target. The present application uses the target three-dimensional spatial position of the earliest timestamp as the reference point, and calculates the relative motion distance corresponding to all timestamps in space according to the three-dimensional coordinates, so as to obtain a scatter plot of the motion distance over time. By fitting with a polynomial or according to the empirical formula of the target's speed attenuation, a function of the motion distance over time is obtained. Further derivative of the function, the motion speed of each timestamp can be obtained. At the same time, according to the three-dimensional spatial position and timestamp information, the target's scattering process can be reproduced in three-dimensional space to achieve the measurement of its three-dimensional motion field. By using parameter measurement to solve and reconstruct the motion field method, polynomials or speed attenuation empirical formulas are used for fitting, and the measurement of motion parameters such as the three-dimensional position, motion speed, and motion trajectory of high-dynamic high-speed targets at the microsecond level and multiple light measurement points is achieved, and the reconstruction and visualization of the target motion process is achieved.

[0018] In the above-mentioned event camera network optical measurement method for high-dynamic and high-speed target motion parameters, the advantages of event cameras such as high dynamic range, high temporal resolution, and low cost are used to perform optical measurement of high-dynamic and high-speed targets. First, according to the spatiotemporal law of event distribution, the target head trigger event set is accurately extracted, and the linear motion trajectory in three-dimensional space is obtained by intersection; secondly, the three-dimensional straight line point homonymous event retrieval method based on reprojection error is used to match the homonymous points on the three-dimensional linear motion trajectory for each event camera observation in the event camera network, and establish the connection between three-dimensional coordinates and time; finally, the target's motion field is measured based on the world system coordinates and timestamps of the three-dimensional retrieval points. This method has high temporal resolution and high dynamic range, avoids the problem of difficulty in matching homonymous points in the measurement of fragments and projectiles by multi-eye event cameras, solves the problem of few optical measurement points for high-dynamic and high-speed targets, and can use the high temporal resolution of the event stream to realize multi-eye and multi-measurement point joint solution. It can be applied to many types of high-dynamic and high-speed scenes such as range tests and particle image velocimetry.

[0019] In one embodiment, constructing a two-dimensional graph of event-starting point distance over time based on observed event stream data includes: Given the initial endpoint and the final endpoint of all target two-dimensional trajectory segments under each camera view, all event points located near the two-dimensional line segment formed by the initial endpoint and the final endpoint are retrieved to form an event set for each high-speed target under each view; For each event set, the Euclidean distances between all events and the starting point are calculated, and a two-dimensional graph of event-starting point distances over time is constructed.

[0020] In one embodiment, obtaining a target head feature event set from a two-dimensional graph of event-starting point distance over time includes: Starting from the event with the smallest timestamp in the event set of the two-dimensional graph of event-starting point distance over time, all events are traversed in the order of increasing timestamps. If the distance between the current event and the starting point is greater than the reference event, and the distance from the center of gravity and the timestamp of the reference event are both less than the preset threshold, the current event is taken as the target head feature event, and the current event is updated to the reference event, and the initial reference event is set as the starting event. Finally, after traversing all events, the target head feature event set is obtained.

[0021] In one embodiment, using multiple view target head feature event sets to intersect the target motion straight line trajectory in three-dimensional space includes: All the extracted target head feature events are fitted in the pixel coordinate system using the plane line equation to obtain the slope and intercept of the fitted line. Based on the camera system parameters obtained by calibration, an epipolar constraint is used to obtain a point of the same name on the line under multiple viewing angles. Taking the same-name points and slope as input, the straight-line trajectory of the target motion in three-dimensional space is obtained by using the straight-line stereo intersection method.

[0022] In one embodiment, a target motion straight line trajectory is retrieved according to a three-dimensional straight line point same-name event retrieval method based on reprojection error, and a corresponding relationship between a three-dimensional spatial position and time is established, including: Select a 3D point on the target motion straight line trajectory as the retrieval starting point, and conduct positive and negative searches respectively according to the direction vector of the straight line; According to the parameters obtained by calibration, each retrieval point is projected to the pixel coordinate system of each camera, and the Euclidean distance between the projection point and the event in each view is calculated. When all projection points are retrieved, the one with the smallest Euclidean distance and less than the pre-set acceptance interval is selected as the three-dimensional matching homonymous point for each event in multiple views, and the timestamp of the event is assigned to the timestamp of the three-dimensional retrieval point, so as to establish the correspondence between the three-dimensional spatial position and time and form the event light measurement point set.

[0023] In one embodiment, based on the correspondence between the three-dimensional spatial position and time, a fitting method is used to implement motion parameter measurement and calculation and reconstruct the three-dimensional motion field, including: Based on the correspondence between three-dimensional spatial position and time, the three-dimensional spatial position of the target corresponding to the earliest timestamp event is used as the reference point, and the relative movement distance corresponding to the timestamps of all event light measurement points is calculated in space according to the three-dimensional coordinates to obtain a scatter plot of the movement distance changing with time; Fitting the scatter plot of the movement distance over time using a polynomial or an empirical formula based on the target's speed attenuation to obtain a function of the movement distance over time; Derivate the function of the movement distance changing with time to obtain the movement speed at each timestamp; According to the three-dimensional spatial position and timestamp information, the target's scattering process is reproduced in the three-dimensional space to achieve reconstruction of its three-dimensional motion field.

[0024] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0025] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0026] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An event camera network optical measurement method for high-dynamic and high-speed target motion parameters, characterized in that: The method comprises: Arrange a multi-camera synchronous observation system in the area to be observed and calibrate the parameters, observe high-dynamic and high-speed targets to obtain observation event stream data; Constructing a two-dimensional graph of event-starting point distance over time for each view according to the observed event stream data, obtaining a target head feature event set from the two-dimensional graph of event-starting point distance over time; using the target head feature event set under multiple views to intersect the target motion straight line trajectory in three-dimensional space; Retrieving the target motion straight line trajectory according to a three-dimensional straight line point homonymous event retrieval method based on reprojection error, and establishing a corresponding relationship between the three-dimensional spatial position and time; Based on the correspondence between the three-dimensional spatial position and time, a fitting method is used to measure and calculate motion parameters and reconstruct a three-dimensional motion field.

2. The method according to claim 1, characterized in that Constructing a two-dimensional graph of event-starting point distance over time according to the observed event stream data, including: Given the initial endpoint and the final endpoint of all target two-dimensional trajectory segments under each camera view, all event points located near the two-dimensional line segment formed by the initial endpoint and the final endpoint are retrieved to form an event set for each high-speed target under each view; For each event set, the Euclidean distances between all events and the starting point are calculated, and a two-dimensional graph of event-starting point distances over time is constructed.

3. The method according to claim 1, characterized in that: Obtaining a target head feature event set from the event-starting point distance versus time two-dimensional graph, including: Starting from the event with the smallest timestamp in the event set of the event-starting point distance versus time two-dimensional graph, all events are traversed in the order of increasing timestamps. If the distance between the current event and the starting point is greater than the reference event, and the distance from the center of gravity and the timestamp to the reference event are both less than a preset threshold, the current event is taken as the target head feature event, and the current event is updated to the reference event, and the initial reference event is set as the starting event. Finally, after traversing all events, the target head feature event set is obtained.

4. The method according to any one of claims 1 to 3, characterized in that: Using the target head feature event set under multiple views to intersect the target motion straight line trajectory in three-dimensional space, including: All the extracted target head feature events are fitted in the pixel coordinate system using the plane line equation to obtain the slope and intercept of the fitted line. Based on the camera system parameters obtained by calibration, an epipolar constraint is used to obtain a point of the same name on the line under multiple viewing angles. Taking the same-name points and slope as input, the straight-line trajectory of the target motion in three-dimensional space is obtained by using the straight-line stereo intersection method.

5. The method according to claim 1, characterized in that The target motion straight line trajectory is retrieved according to a three-dimensional straight line point same-name event retrieval method based on reprojection error, and a corresponding relationship between a three-dimensional spatial position and time is established, including: Select a 3D point on the target motion straight line trajectory as the retrieval starting point, and conduct positive and negative searches respectively according to the direction vector of the straight line; According to the parameters obtained by calibration, each retrieval point is projected to the pixel coordinate system of each camera, and the Euclidean distance between the projection point and the event in each view is calculated. When all projection points are retrieved, the one with the smallest Euclidean distance and less than the pre-set acceptance interval is selected as the three-dimensional matching homonymous point for each event in multiple views, and the timestamp of the event is assigned to the timestamp of the three-dimensional retrieval point, so as to establish the correspondence between the three-dimensional spatial position and time and form the event light measurement point set.

6. The method according to claim 1, characterized in that Based on the correspondence between the three-dimensional spatial position and time, a fitting method is used to measure and calculate motion parameters and reconstruct a three-dimensional motion field, including: Based on the correspondence between the three-dimensional spatial position and time, the target three-dimensional spatial position corresponding to the earliest timestamp event is used as a reference point, and the relative movement distance corresponding to the timestamps of all event light measurement points is calculated in space according to the three-dimensional coordinates to obtain a scatter plot of the movement distance changing with time; Fitting the scatter plot of the movement distance over time using a polynomial or an empirical formula based on the speed attenuation of the target to obtain a function of the movement distance over time; Derivate the function of the movement distance changing with time to obtain the movement speed at each timestamp; According to the three-dimensional spatial position and timestamp information, the target's scattering process is reproduced in the three-dimensional space to achieve reconstruction of its three-dimensional motion field.

Images