A method for reconstructing a particle field motion trajectory based on an event camera

Abstract

The present application relates to a kind of methods for reconstructing the trajectory of particle field motion based on event camera, laser beam is passed through moving particle fluid domain, and the motion of particle field is recorded using event camera.Event stream data photographed by event camera is imported into event stream data reconstruction system, and the relationship between relevant event points is obtained and reconstructed using trained artificial intelligence algorithm.Local feature recognition algorithm is used to the original event stream data for local feature recognition, and the event set generated by the motion of the object is obtained in accordance with the motion characteristics of specific object.Thereby, the local area original event stream point set constituting the whole particle field event stream data and the trajectory line event set generated in accordance with the particle motion characteristics are obtained.The present application reconstructs the trajectory of particle field using autocorrelation or cross-correlation technology to obtain flow velocity field.In addition, particle field detection also has wide application in industrial detection, and the moving particle field can be obtained by detecting the trajectory of particle motion using the method shown in the present application.

Description

Technical Field

[0001] This invention belongs to the field of flow measurement technology and relates to a method for reconstructing particle field motion trajectories based on an event camera. Background Technology

[0002] In experimental fluid mechanics, particle imaging velocimetry (PIV) or particle tracking velocimetry (PTV) techniques are commonly used to quantitatively measure changes in flow field velocity. Both techniques require capturing images of particle field motion, which is a prerequisite for PIV or PTV analysis to obtain the flow velocity field. Traditional PIV or PTV techniques typically use a CMOS camera to acquire image data of a laser-illuminated particle field, then import the captured particle image sequence into analysis software to calculate the flow velocity field. These two techniques offer advantages such as the ability to perform transient measurements, non-contact and undisturbed flow fields, and high measurement accuracy, and have been widely used in the field of flow velocity field measurement. Willert & Klinner of the German Aerospace Center (DLR), in an unaccepted paper, proposed event-based imaging velocimetry—An Assessment of Event-based Cameras for the Measurement of FluidFlows—using event cameras to capture moving particle fields.

[0003] Image-based particle velocimetry (or tracking) technology, whose data acquisition system mainly consists of a CMOS camera, a high-power laser, and a synchronization control device, has the following technical drawbacks:

[0004] 1. Measuring particle fields requires high-speed recording of large amounts of image data, demanding high bandwidth and storage capacity from the system. The image data contains excessive redundant information, leading to low efficiency in processing large volumes of image data by analysis software.

[0005] 2. High-speed recording of particle field motion is achieved by increasing the shutter speed of the CMOS camera and reducing the exposure time, which requires high illumination from the light source. Generally, PIV or PTV systems need to be equipped with high-power pulsed lasers to achieve high-speed particle field motion data acquisition.

[0006] 3. Synchronous shooting with CMOS camera and high-power pulsed laser requires high precision. A synchronization device is needed to control the CMOS camera and laser to achieve synchronous data acquisition.

[0007] 4. Image-based particle velocimetry (or tracking) technology requires high frame rate cameras and high-power lasers, which are expensive.

[0008] 5、Event camera event stream data has certain background noise, and the contrast-based particle field reconstruction technique used by Willert & Klinner is not efficient. SUMMARY

[0009] TECHNICAL PROBLEMS TO BE SOLVED

[0010] In order to avoid the shortcomings of the prior art, the present application provides a method for reconstructing a particle field motion trajectory based on an event camera, which uses an event camera to replace a traditional CMOS high-speed camera to shoot a moving particle field, and reconstructs a particle field motion trajectory, so as to solve the problems of high cost, large image data storage, low efficiency of analyzing and reconstructing a particle field motion, and high requirements for illumination and synchronization in the above-mentioned image-based particle velocity measurement (or tracking) method, while still having the advantages of transient measurement, non-contact undisturbed flow field, and high measurement accuracy.

[0011] TECHNICAL SCHEME

[0012] A method for reconstructing a particle field motion trajectory based on an event camera, characterized in that a laser beam is passed through a moving particle fluid domain, and an event camera is used to record a moving particle field, and the reconstruction steps are as follows:

[0013] Step 1: In t time, an event camera is used to shoot a moving particle field to obtain original event stream data;

[0014] The original event stream data includes a determined spatial coordinate and a time coordinate of each event position point in the particle field, which constitutes original event stream, wherein the spatial coordinate is the spatial position of the event occurrence, and the time coordinate is the time of the event occurrence;

[0015] Step 2: The original event stream data shot by the event camera is divided into local area original event stream data to obtain a local area original event stream point set to be reconstructed, and each point in the set represents an event or a noise point;

[0016] When dividing, the time scale size of the local reconstruction area is selected as:

[0017]

[0018] Wherein, L is the characteristic size of the particle field shooting area, and U is the flow velocity of the particle field

[0019] The size of the local reconstruction space scale is selected as:

[0020] l' = 5 × U × t'

[0021] Step 3: using a local feature recognition algorithm, such as local feature recognition on the original event stream data, reconstructing the relationship between related event points, obtaining the object motion feature, i.e. the trajectory line, and the event set generated by the object motion, i.e. the trajectory line event set;

[0022] Step 4: repeating step 3 to obtain the trajectory line event set generated by the particle motion feature in the local area original event stream point set constituting the full particle field event stream data;

[0023] Step 5: using a full-field particle trajectory splicing algorithm to splice the full particle field event stream data to obtain the particle motion trajectory; traversing each trajectory line in the trajectory line event set data set that meets the particle motion feature, and finally obtaining the full-field particle trajectory.

[0024] The step 5 uses a full-field particle trajectory splicing algorithm to splice the full particle field event stream data: first, put the trajectory line event set that meets the particle motion feature in the local area obtained in steps 3 to 4 into a data set, then traverse the numerous trajectory lines in the data set, and randomly select one; search for trajectory lines near and adjacent to the head and tail of the trajectory line 5, and the trajectory lines that meet the requirements form a set A; then, judge whether all the trajectory lines in the set A meet the randomly selected trajectory line, and the judgment standard is that for each trajectory line in the set A, the distance d and the acceleration Acc between the trajectory line and the randomly selected trajectory line are calculated respectively, and it is determined that when d and Acc are less than a certain threshold, it is an effective trajectory that meets the randomly selected trajectory line, and if it is greater than or equal to the characteristic threshold, it does not meet.

[0025] The local feature recognition algorithm of step 3 uses singular value decomposition (SVD) or random sample consensus (RANSAC).

[0026] Beneficial effects

[0027] The method for reconstructing the particle field motion trajectory based on the event camera proposed in the application passes a laser beam through a moving particle fluid domain and uses an event camera to record the moving particle field. The event stream data photographed by the event camera is imported into an event stream data reconstruction system, and a trained artificial intelligence algorithm is used to obtain and reconstruct the relationship between related event points. A local feature recognition algorithm is used to perform local feature recognition on the original event stream data to obtain an event set that meets a specific object motion feature and is generated by the object motion. Thus, a local area original event stream point set constituting full particle field event stream data and a trajectory line event set generated by the particle motion feature are obtained.

[0028] The particle field trajectory reconstruction is the basis for obtaining the flow velocity field by using the autocorrelation or cross-correlation technology in the flow measurement field. In addition, the particle field detection has wide application in industrial detection, and the particle field in motion can be obtained by detecting the particle motion trajectory by using the method shown in the application.

[0029] The beneficial effects are as follows:

[0030] (1) The image-based particle velocity measurement (or tracking) method records a large amount of pixel data without particle information, resulting in excessive redundant information, high image data transmission rate requirement and large storage space requirement. Excessive redundant information can lead to low analysis efficiency. The event camera only records event information generated when the particle moves, and does not record spatial invariant information and information irrelevant to the motion, thereby greatly reducing the data transmission and storage amount.

[0031] (2) The particle velocity measurement (or tracking) method based on the recording of images by the traditional high-speed camera needs a high-brightness light source due to the short exposure time of each frame of image, and only high-power lasers can meet the light source brightness requirement. Such lasers are high in price and have strict safety use conditions. The event camera only captures light intensity changes, can work under different lighting conditions, has low light source brightness requirement, can use various types of continuous light sources, is low in price and safer to use.

[0032] (3) The image-based particle velocity measurement method needs a high-frame-rate high-speed camera to avoid the large image space difference caused by the long distance of particle motion in adjacent time sequences when the particle motion speed is high, and the high-frame-rate high-speed camera is expensive. The event camera adopts an asynchronous event recording mode, and can realize high particle motion recording speed at a low cost.

[0033] (4) The random fitting method is proposed to directly identify the particle field. The local area random fitting method is used to identify the particles meeting the motion characteristics based on the original event stream data, and the particle identification efficiency is greatly improved compared with the particle field reconstruction technology based on the contrast of the event stream data. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 : Schematic diagram of particle field trajectory analysis system composition

[0035] Figure 2 : Synchronous acquisition of particle field motion by event camera and CMOS camera

[0036] Figure 3 : Local point set of original event stream data

[0037] Figure 4 : Full-field particle field trajectory identification

[0038] Figure 5(a) single particle and (b) full field particle trajectory DETAILED DESCRIPTION

[0039] The present application will be further described in conjunction with the embodiments, drawings:

[0040] The event camera-based particle field trajectory analysis system in the embodiment mainly consists of two parts: a data acquisition system and an event stream data reconstruction system, wherein the data acquisition system mainly includes an event camera and an illumination light source (such as a semiconductor continuous laser and a light sheet mirror). The light source illuminates the part of interest in the flow field, and a laser can emit a circular dot laser beam. The laser beam becomes a light sheet with uniform light density and good linearity after passing through the light sheet mirror, which is shown in FIG. 1 as a light sheet with a certain sector angle and thickness on the plane. The light sheet is used to illuminate the particle field area to be measured, and the event camera is perpendicular to the light sheet and focuses on the event stream data acquisition of the light illuminated area. Figure 1

[0041] The core of the event stream data reconstruction system is a random fitting method, including a local trajectory line identification algorithm and a full field particle trajectory splicing algorithm. First, the local feature identification algorithm is used to identify the characteristic events in the local area, and then the full field particle trajectory splicing algorithm is used to reconstruct the motion trajectory of the particle field.

[0042] Application case:

[0043] 1. Comparison of data storage capacity

[0044] The event camera and the COMS camera synchronously and at the same frame rate shoot the same area of the moving particle field (FIG. 2), and the shooting duration is 2 seconds. The event camera stores 59 Mb of event stream data, and the CMOS camera stores 927 Mb of image data. The event camera solves the problem of too large image data of the CMOS camera when shooting the particle field. Figure 2

[0045] 2. Particle field event stream data processing steps

[0046] Step 1: In t time, the event camera data acquisition system shown in FIG. 1 is used to shoot the moving particle field to obtain the event stream data. Each event position point in the local particle field has a certain spatial coordinate and time coordinate, which constitutes the original event stream. The spatial coordinate is the spatial position of the event occurrence, and the time coordinate is the time of the event occurrence; Figure 1

[0047] Step 2: The event stream data shot by the event camera is imported into the event stream data reconstruction system, and a trained artificial intelligence algorithm is used to obtain and reconstruct the relationship between the related event points. Figure 3 ​​​Each hollow circle in the figure represents an event or a noise point, and the original event stream is composed of these points. The time scale size of the local reconstruction is selected as:

[0048]

[0049] where L is the characteristic size of the particle field imaging area, and U is the flow velocity of the particle field.

[0050] The spatial scale size of the local reconstruction is selected as:

[0051] l' = 5 x U x t' (2)

[0052] Step 3: Local feature recognition is performed on the original event stream data using a local feature recognition algorithm to obtain an event set that meets the motion characteristics of a specific object and is generated by the motion of the object. For example, Figure 3 The black line in the figure shows the trajectory line event set that meets the linear motion characteristics, that is, the events connected by the black line within a certain tolerance range. Hereinafter, the black line represents the trajectory line event set with specific motion characteristics.

[0053] Step 4: Based on the time scale selection (formula (1)) and spatial scale selection (formula (2)) criteria of the local area reconstruction of the event stream point set, steps 2 and 3 are repeated to obtain the local area original event stream point set constituting the full particle field event stream data and the trajectory line event set meeting the particle motion characteristics.

[0054] Step 5: The particle motion trajectory is obtained using a full-field particle trajectory splicing algorithm. First, the trajectory line event set meeting the particle motion characteristics obtained in steps 2 to 4 is placed in a data set, and then a plurality of trajectory lines in the data set are traversed, and one is randomly selected, for example, Figure 4 Trajectory line 5 shown in the figure. The trajectory lines that meet the requirements form a set A including trajectory lines 1-4 and 6-9.

[0055] Then, all the trajectory lines in set A are judged whether they meet the randomly selected trajectory line, and the judgment criteria are that for each trajectory line in set A, the distance d and the acceleration Acc between the randomly selected trajectory line are calculated, and it is determined that when d and Acc are less than a certain threshold, it is an effective trajectory that meets the randomly selected trajectory line, and if it is greater than or equal to the characteristic threshold, it does not meet. For example, the effective trajectories that meet Figure 4 Trajectory line 5 in the figure are Figure 5 1 and 8 shown in a in the figure.

[0056] The above process is repeated to traverse each trajectory in the trajectory event set data set that meets the particle motion characteristics, and finally the full-field particle trajectory shown in Figure b of Figure 5 is obtained.

[0057] The present application is in the field of flow measurement, and particle field trajectory reconstruction is the basis for obtaining a flow velocity field using autocorrelation or cross-correlation techniques. In addition, particle field detection has wide application in industrial detection, and the method shown in the present application can be used to obtain a moving particle field by detecting particle motion trajectories.

Claims

1. A method for reconstructing particle field motion trajectories based on an event camera, characterized in that: The laser beam is passed through the moving particle fluid domain, and the moving particle field is recorded using an event camera, and the reconstruction steps are as follows: Step 1: In t time, the moving particle field is photographed by using the event camera to obtain the original event stream data; The original event stream data includes a determined spatial coordinate and time coordinate for each event position point in the particle field, forming an original event stream, wherein the spatial coordinate is the spatial position of the event occurrence, and the time coordinate is the time of the event occurrence; Step 2: The original event stream data photographed by the event camera is divided into local area original event stream data to obtain a set of local area original event stream points to be reconstructed, and each point in the set represents an event or a noise point; When dividing, the time scale size of the local reconstruction area is selected as: Wherein, L is the characteristic size of the particle field shooting area, and U is the flow velocity of the particle field The local reconstruction space scale is selected as: l′=5×U×t′ Step 3: A local feature recognition algorithm is used to recognize the local features of the original event stream data, reconstruct the relationship between the related event points, obtain the object motion characteristics, i.e., the trajectory line, and the event set generated by the object motion, i.e., the trajectory line event set; Step 4: Step 3 is repeated to obtain the trajectory line event set generated by the particle motion characteristics in the local area original event stream point set constituting the full particle field event stream data; Step 5: A full-field particle trajectory splicing algorithm is used to splice the full particle field event stream data to obtain the particle motion trajectory; each trajectory line in the trajectory line event set data set that meets the particle motion characteristics is traversed to finally obtain the full-field particle trajectory.

2. The method of claim 1, wherein: The step 5 uses a full-field particle trajectory splicing algorithm to splice the full particle field event stream data: first, the local area trajectory line event set that meets the particle motion characteristics obtained in steps 3 to 4 is placed in a data set, then a plurality of trajectory lines in the data set are traversed, and one is randomly selected; search for trajectory lines similar and adjacent to the head and tail of the trajectory line 5, and the trajectory lines that meet the requirements form a set A; then, all the trajectory lines in the set A are judged whether they meet the randomly selected trajectory line, and the judgment standard is that for each trajectory line in the set A, the distance d and the acceleration Acc between the trajectory line and the randomly selected trajectory line are calculated respectively, and it is determined that when d and Acc are less than a threshold value, it is an effective trajectory that meets the randomly selected trajectory line, and if greater than or equal to the threshold value, it does not meet.

3. The method of claim 1, wherein: The local feature recognition algorithm of step 3 uses singular value decomposition, i.e., singular value decomposition, SVD or random sample consensus, i.e., random sample consensus, RANSAC.

Images