Particle trajectory velocity measurement method based on multi-plane calibration and line-of-sight constraint
The method addresses image distortion in 4D PTV by using multi-plane calibration and line-of-sight constraints, improving accuracy and efficiency in fluid measurement.
Patent Information
- Application Number
- CN202310641036.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2043-05-31
AI Technical Summary
The severe distortion and distortion of particle images caused by the air-plexiglass interface-fluid refraction interface in fluid measurement hinders the large-scale application of 4D PTV technology, leading to high calibration errors and low calculation efficiency.
A particle trajectory velocity measurement method based on multi-plane calibration and line-of-sight constraint, involving camera calibration using multiple planes, line-of-sight storage, and refractive index compensation, with parallel processing and triangulation reconstruction to improve measurement accuracy and efficiency.
Enhances measurement accuracy and calculation efficiency by compensating for refraction interface distortions and optimizing data processing, facilitating the wide application of 4D PTV technology in flow field reconstruction.
Smart Images

Figure CN119064626A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of particle image velocimetry, and relates to a four-dimensional Lagrangian particle trajectory velocimetry method, in particular to a particle trajectory velocimetry method based on multi-plane calibration and line-of-sight constraints. Background Art
[0002] 4D PTV (4D Lagrangian Particle Tracking Velocimetry) is a non-contact and non-interference quantitative measurement method for spatial flow fields. 4D PTV uses evenly distributed tracer particles of small size to follow the movement of the fluid, and records the movement of the tracer particles by continuous "lighting" and "photographing". Based on algorithms such as three-dimensional particle matching, iterative particle reconstruction of images, trajectory initialization, Wiener filter prediction, and trajectory connection, the motion trajectory information of particles in the time series (the fourth dimension) is extracted from the continuous tracer particle images to obtain an approximation of the motion velocity of the fluid space point. The four-dimensional Lagrangian particle trajectory velocimetry combines PTV technology with a triangular reconstruction algorithm, uses multiple perspectives (at least three cameras) to simultaneously capture and record the illuminated particle field, and reconstructs the three-dimensional spatial position of the tracer particles through three-dimensional particle matching and triangular reconstruction algorithms; then the tracer particles are reprojected onto the image for iterative particle reconstruction to improve the three-dimensional spatial position accuracy of the particles and the image particle utilization rate, so that particle images with high particle concentrations can be processed; then the trajectory of the initial four-frame particle three-dimensional spatial position is initialized to obtain the initial four-dimensional Lagrangian particle trajectory; then, based on the initial four-dimensional Lagrangian particle trajectory, the possible positions of all particle trajectories in the next frame are predicted based on the Wiener filter, and iterative particle reconstruction is performed with the captured particle image to improve the three-dimensional spatial position accuracy of the particles and the image particle utilization rate; then the particle trajectory is optimally fitted to obtain a particle trajectory with higher accuracy; finally, the first-order differential and second-order differential of the time dimension are performed on each spatial dimension of the particle trajectory to obtain the velocity and acceleration of the particle at the three-dimensional spatial position.
[0003] Compared with traditional two-dimensional PIV technology, three-dimensional stereo PIV technology and three-dimensional tomographic PIV technology, 4D PTV technology can provide higher temporal resolution and spatial resolution, realize efficient instantaneous three-dimensional flow field velocity and acceleration quantitative measurement, and realize full-field quantitative reconstruction of three-dimensional flow field. Four-dimensional Lagrangian particle trajectory velocimetry technology has the advantages of simple operation, very high measurement accuracy and more information; however, due to the existence of air-plexiglass interface-fluid refraction interface in the actual measurement process, the image captured by the camera has serious distortion and distortion, resulting in high calibration error and very low calculation efficiency. Therefore, a higher-precision refraction compensation algorithm and a more demanding camera shooting angle are required.
[0004] In recent years, the research on 4D Lagrangian particle trajectory velocimetry at home and abroad has focused on improving the reconstruction accuracy and image utilization rate of 4D PTV technology. Many high-precision reconstruction algorithms, including triangulation, iterative particle reconstruction (IPR), shake the box (STB) and Wiener filter prediction, have been proposed to improve the quality of 3D particle spatial position reconstruction and the accuracy of 4D particle trajectory. These algorithms perform well when tested with synthetic particle images under ideal conditions, but when applied to actual measurement processes, the camera must be completely facing the fluid-glass-air interface to avoid serious distortion and distortion of the image caused by the air-plexiglass interface-fluid refraction interface. However, the geometric layout of the measurement object in the actual fluid measurement process is relatively complex, and it is almost difficult to meet the strict requirements of the existing 4D PTV technology; secondly, these algorithms run on a single CPU core, and the efficiency of 4D particle trajectory reconstruction for longer time series is low. Scholars at home and abroad have conducted relevant research such as multi-camera calibration compensation to address the severe image distortion caused by the air-plexiglass interface-fluid refraction interface. However, the camera simulations obtained in these studies are more complex and difficult to integrate and apply in 4DPTV technology.
[0005] In summary, the severe distortion and distortion of particle images caused by the air-plexiglass interface-fluid refraction interface in fluid measurement seriously hinders the large-scale application of 4D PTV technology. Summary of the invention
[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a particle trajectory velocity measurement method based on multi-plane calibration and line of sight constraint, which effectively improves the measurement accuracy and calculation efficiency through multi-plane calibration, line of sight storage and refractive index compensation.
[0007] The purpose of the present invention can be achieved by the following technical solutions:
[0008] A particle trajectory velocity measurement method based on multi-plane calibration and line-of-sight constraint comprises the following steps:
[0009] 1) Calibrate multiple cameras based on multi-plane calibration images to obtain camera calibration function relationships;
[0010] 2) calculating the three-dimensional spatial line of sight corresponding to the image pixel coordinates of each camera-captured image according to the camera calibration function relationship, and storing the three-dimensional line of sight as a line of sight library;
[0011] 3) obtaining a particle image, calculating the three-dimensional spatial line of sight corresponding to the particle pixel coordinates in the particle image based on the camera calibration function relationship, and performing particle matching and triangulation reconstruction on each particle in parallel to obtain a preliminary three-dimensional spatial position of the particle;
[0012] 4) performing nearest neighbor matching of the preliminary three-dimensional spatial position of each particle with all three-dimensional spatial sight lines in the sight line library in parallel, and performing sight line interpolation and sight line jittering to obtain the optimal reprojected particle image, thereby generating the precise three-dimensional spatial position of the particle;
[0013] 5) Construct a four-dimensional particle trajectory based on the precise three-dimensional spatial position of each particle, and calculate the velocity vector and acceleration vector of each particle at each moment;
[0014] 6) The velocity vector and acceleration vector of each particle are interpolated onto the orthogonal Euler grid to obtain the three-dimensional flow field.
[0015] Furthermore, in step 1), each camera photographs a plurality of calibration planes at different distances to obtain the multi-plane calibration image.
[0016] Further, in step 2), the sightline library is obtained by the following steps:
[0017] Based on the camera calibration function relationship, the image pixel coordinates of each camera-captured image are mapped to the world coordinates of multiple calibration planes, and multiple intersection points of a single pixel and multiple planes are obtained. The multiple intersection points form a three-dimensional line of sight, and the direction unit vector of the three-dimensional line of sight is calculated. The image pixel coordinates and the direction unit vector of the corresponding line of sight and the intersection points with each calibration plane are stored to form the line of sight library.
[0018] Furthermore, in step 3) and step 4), parallel processing of each particle is implemented based on OpenMP.
[0019] Further, in step 3), the pixel coordinates of the particles in the particle image are determined by:
[0020] The particle pixel peak value is searched for the particle image actually taken and the particle image position center Gaussian fitting is performed to obtain the particle pixel coordinates.
[0021] Furthermore, in step 3), the particle matching and triangulation reconstruction specifically include:
[0022] 301) based on the three-dimensional space sight line and direction unit vector of each camera corresponding to each particle, obtain the particle image pixel position projected on each camera by the three-dimensional space position of the particle;
[0023] 302) mapping each of the particle image pixel positions to the world coordinates of a plurality of calibration planes, respectively, obtaining a plurality of intersection points between the particle image pixel position corresponding sight lines and the plurality of calibration planes, and connecting the plurality of intersection points to obtain a plurality of three-dimensional space sight lines;
[0024] 303) Perform triangulation reconstruction based on each three-dimensional spatial line of sight to obtain the preliminary three-dimensional spatial position of the particle.
[0025] Further, in step 4), the sight line interpolation and sight line jitter specifically include:
[0026] 401) performing three-dimensional space interpolation based on a plurality of sight lines obtained by the nearest neighbor matching to obtain a particle pixel coordinate, generating a particle pixel distribution on a new blank image based on the particle pixel coordinate as the center point of the particle image, and obtaining a reprojected particle image;
[0027] 402) obtaining the three-dimensional space sight line of each camera corresponding to the particle pixel coordinates;
[0028] 403) on the reprojected particle image, the particle pixel coordinates are spatially translated and jittered corresponding to the three-dimensional spatial line of sight of each camera, and the reprojected particle image generated by each jitter is subtracted from the original particle image to obtain a residual particle image distribution, and the reprojected particle image with the smallest total particle brightness on the residual particle image is taken as the best reprojected particle image;
[0029] 404) extracting the particle pixel coordinates on the best reprojected particle image, and calculating the particle pixel coordinates according to the camera calibration function relationship to perform line of sight triangulation reconstruction corresponding to the three-dimensional line of sight of each camera to obtain the precise three-dimensional space position of the particle.
[0030] Further, in step 5), the construction of the four-dimensional particle trajectory specifically includes:
[0031] 501) based on the accurate three-dimensional spatial positions of particles obtained in the previous frames, starting from the accurate three-dimensional spatial positions of all particles in the first frame, all possible particle trajectories of the next frame are sequentially connected to form a particle trajectory tree network;
[0032] 502) traversing each branch of the particle trajectory tree network, and finding the branch with the smallest acceleration change as the optimal particle initial trajectory;
[0033] 503) Based on the initial trajectory of the particle in the previous frames, an N-order polynomial fitting is used to predict the possible position of the particle in the next frame;
[0034] 504) performing nearest neighbor matching between the possible positions and all three-dimensional spatial sight lines in the sight line library, and performing sight line interpolation and sight line reprojection to obtain an optimal reprojected particle image;
[0035] 505) Based on the camera calibration function relationship, the three-dimensional spatial line of sight corresponding to the particle pixel coordinates in the best reprojected particle image is calculated, and triangulated reconstruction is performed to obtain the accurate three-dimensional spatial position of the particle in the next frame;
[0036] 506) Connecting the accurate three-dimensional spatial position of the next frame of particles into the initial trajectory of the particles;
[0037] 507) Repeat steps 503)-506) until all particles at all times are connected.
[0038] Further, in step 5), the calculation of the velocity vector and acceleration vector of each particle at each moment specifically includes:
[0039] According to the four-dimensional particle trajectory, the velocity vector of the particle at the nth moment is calculated based on the three-dimensional spatial positions of every three adjacent particles on the time series trajectory;
[0040]
[0041] in, is the velocity vector of the particle at the nth moment, X(n-1) and X(n+1) are the three-dimensional spatial positions of the particle at the n-1th and n+1th moments respectively, and Δt is the time interval between every two moments;
[0042] Based on the velocity vector of every two adjacent particles on the time series trajectory, the acceleration of the previous particle is calculated;
[0043]
[0044] in, and are the velocity vectors of the particle at the nth and n+1th moments, respectively. The acceleration vector of the particle at the nth moment.
[0045] Further, in step 6), obtaining the three-dimensional flow field specifically includes:
[0046] Based on scattered point interpolation, the velocity vector and acceleration vector of each particle at each moment are interpolated to the orthogonal Euler grid nodes to obtain the velocity vector of the grid nodes and construct the Euler velocity field;
[0047] The error vector correction is performed on the Euler velocity field to obtain the three-dimensional flow field.
[0048] The present invention can be applied to particle reconstruction and velocity field reconstruction of four-dimensional Lagrangian particle trajectory velocimetry (4D PIV) technology or equipment, provides a basis for the wide application of 4D PTV technology in flow field reconstruction, and is of great significance for accelerated experimental fluid research.
[0049] Compared with the prior art, the present invention has the following beneficial effects:
[0050] 1. The present invention replaces the camera calibration of the traditional camera model with multi-plane calibration and stores the line of sight of each image pixel, which can compensate for the refraction interface existing in the actual measurement process and greatly improve the measurement accuracy.
[0051] 2. The present invention decomposes the time-consuming processes such as 3D particle matching, triangulation reconstruction and jittering sight lines into multi-threaded parallel calculations, which greatly improves the calculation efficiency;
[0052] 3. The present invention establishes a four-dimensional particle trajectory by first constructing the initial trajectory of the particles of the previous frames, and then predicting, comparing and accurately locating the possible positions of the particles in the next frame, thereby effectively improving the overall calculation accuracy while ensuring the calculation efficiency;
[0053] 4. The present invention constructs a three-dimensional flow field of particles through operations such as particle matching, triangulation reconstruction, and line of sight jitter, which can greatly improve the measurement accuracy of 4D PTV when there is an air-plexiglass interface-fluid refraction interface. At the same time, it greatly improves the calculation efficiency of the 4DPTV technology, reduces the tedious data processing work of users when using hardware equipment related to the 4D PTV technology, saves a lot of time, and speeds up the research of experimental fluid mechanics. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 It is a flow chart of the particle trajectory velocity measurement method of the present invention;
[0055] Figure 2 A flow chart of stereo matching and triangulation reconstruction of the present invention;
[0056] Figure 3 A flow chart of jittering sight lines and triangulation reconstruction according to the present invention;
[0057] Figure 4 A flow chart for the initial trajectory connection of the present invention;
[0058] Figure 5 : are calibration images in the embodiment of the present invention, wherein (a) is a calibration image at Z=+15 mm, and (b) is a calibration image at Z=-15 mm;
[0059] Figure 6 Schematic diagram of the sight line corresponding to each pixel in the calibration process in an embodiment of the present invention;
[0060] Figure 7 Schematic diagram of particle image and distribution Gaussian fitting in an embodiment of the present invention;
[0061] Figure 8 Schematic diagram of a stereo matching process in an embodiment of the present invention;
[0062] Fig. 9 is the particle image stereo matching result in an embodiment of the present invention;
[0063] Fig.10 Schematic diagram of the line of sight triangulation reconstruction process in an embodiment of the present invention;
[0064] Fig.11 This is a schematic diagram of line of sight interpolation in an embodiment of the present invention;
[0065] Fig.12 Schematic diagram of line of sight translation jitter in an embodiment of the present invention;
[0066] Fig.13 This is a schematic diagram of trajectory initialization in an embodiment of the present invention;
[0067] Fig.14 It is a schematic diagram of long trajectory and speed calculation in an embodiment of the present invention;
[0068] Fig.15 A schematic diagram of particle point interpolation to an orthogonal Euler grid in an embodiment of the present invention;
[0069] Fig.16 It is the three-dimensional velocity field result in the embodiment of the present invention. DETAILED DESCRIPTION
[0070] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.
[0071] Reference Figure 1 As shown, this embodiment provides a particle trajectory velocity measurement method based on multi-plane calibration and line of sight constraint, which is used to implement four-dimensional Lagrangian particle trajectory velocity measurement, including the following steps:
[0072] S1. Calibrate multiple cameras based on multi-plane calibration images to obtain camera calibration function relationships.
[0073] 11) For the standard chessboard calibration plane in the multi-camera three-dimensional complex flow field measurement system, each camera shoots the calibration plane at multiple different distances to obtain a multi-plane calibration image.
[0074] In this embodiment, for the standard chessboard calibration plane in the multi-camera three-dimensional complex flow field measurement system, each camera shoots the calibration plane at two different distances to obtain a dual-plane calibration image, such as Figure 5 Shown are two calibration images taken by camera 1 at Z=+15 mm and Z=-15 mm, respectively. The distances between the dots are ΔX=5 mm and ΔY=5 mm.
[0075] 12) Perform camera calibration according to the multi-plane calibration image obtained in step 11), and obtain the calibration function relationship between the calibration image or particle image obtained by the camera and the plane physical three-dimensional space. The calibration function polynomial is:
[0076] u=a 0 +a 1 X+a 2 X 2 +a 3 X 3 +a 4 Y+a 5 Y 2 +a 6 Y 3 +a 7 XY+a 8 X 2 Y+a 9 XY 2
[0077] v=b 0 +b 1 X+b 2 X 2 +b 3 X 3 +b 4 Y+b 5 Y 2 +b 6 Y 3 +b 7 XY+b 8 X 2 Y+b 9 XY 2
[0078] X=c 0 +c 1 u+c 2 u 2 +c 3 u 3 +c 4v+c 5 v 2 +c 6 v 3 +c 7 uv+c 8 u 2 v+c 9 uv 2
[0079] Y=d 0 +d 1 u+d 2 u 2 +d 3 u 3 +d 4 v+d 5 v 2 +d 6 v 3 +d 7 uv+d 8 u 2 v+d 9 uv 2
[0080] Where u and v are the pixel coordinates of the calibration image or particle image in the width and height directions, respectively; X and Y are the spatial coordinates in the X and Y axis directions on the calibration plane, respectively; a 0 -a 9 is the coefficient of the physical space coordinate pair calibration image or particle image in the width direction, b 0 -b 9 is the coefficient of the physical space coordinate pair calibration image or particle image in the height direction, c 0 -c 9 is the coefficient of the pixel coordinates of the calibration image or particle image to the physical space coordinates in the X direction, d 0 -d 9 It is the coefficient of the pixel coordinates of the calibration image or particle image to the Y direction of the physical space coordinates.
[0081] In this embodiment, there are 4 cameras and 2 calibration planes, and a total of 320 parameters are obtained.
[0082] S2. According to the camera calibration function relationship obtained in step 1), the three-dimensional space line of sight corresponding to the image pixel coordinates of each camera-captured image is calculated and stored as a line of sight library.
[0083] 21) Mapping the image pixel coordinates of each image captured by the camera to the world coordinates of multiple calibration planes through the camera calibration function relationship obtained in step 12), obtaining multiple intersection points between a single pixel and multiple planes, and connecting these intersection points to form a three-dimensional space line of sight.
[0084] In this embodiment, for the standard chessboard calibration plane in the multi-camera three-dimensional complex flow field measurement system, each camera shoots the calibration plane at two different distances to obtain a dual-plane calibration image, such as Figure 6 The figure shows a schematic diagram of the line of sight corresponding to each pixel in the calibration process of this embodiment. The line of sight of each pixel will be bent when passing through the refraction interface. The intersection of the line of sight of each pixel and the two calibration planes is calculated through the camera calibration function relationship obtained in step 12) to form a certain line of sight.
[0085] 22) Calculate the direction unit vector of the sight line obtained in step 21), and store all image pixel coordinates and their corresponding direction unit vectors of the sight line and their intersections with the two calibration planes to form the sight line library. The unit vector is calculated as follows:
[0086]
[0087] in, is the sight unit vector corresponding to j pixels, j is a natural number 1, 2, 3…, P 1,j and P 2,j are the intersection points of the line of sight corresponding to j pixels and any two calibration planes in the multiplane, ||P 1,j -P 2,j || 2 It is expressed as the geometric distance between the line of sight and the intersection of any two calibration planes in the multiplane.
[0088] S3. Obtain a particle image, calculate the three-dimensional spatial line of sight corresponding to the particle pixel coordinates in the particle image based on the camera calibration function relationship, use a stereo matching algorithm to perform particle matching and triangulation reconstruction on each particle in parallel, and obtain the preliminary three-dimensional spatial position of the particle (i.e., the three-dimensional spatial point closest to the line of sight).
[0089] In this embodiment, the parallel processing of particles is implemented based on OpenMP parallel programming, and the stereo matching and triangulation reconstruction of each particle is completed by one OpenMP thread.
[0090] Take the four cameras of this embodiment as an example. Figure 2 As shown, step S3 specifically includes the following steps:
[0091] 31) Perform particle pixel peak search and particle image position center Gaussian fitting on the particle image actually captured.
[0092] like Figure 7 The image distribution of particles actually captured in this embodiment is shown, and Gaussian fitting is performed on the distribution to obtain the particle pixel coordinates.
[0093] 32) The particle pixel coordinates of all camera particle images are calibrated according to the camera calibration function relationship obtained in step 1), and the three-dimensional space line of sight of each camera corresponding to the particle pixel coordinates is calculated and stored.
[0094] 33) Based on the line of sight and direction unit vector of the first camera corresponding to the single particle p1 obtained in step 32), three-dimensional stereo matching is performed, the line of sight of particle p1 corresponding to camera 1 is projected onto camera 2, the epipolar line L12 of camera 1 to camera 2 is obtained, and the particle image pixel position p12 in a certain area near the epipolar line L12 is found and stored.
[0095] 34) Project the line of sight of camera 1 corresponding to the single particle p1 onto camera 3 to obtain the epipolar line L13 of camera 1 to camera 3, then project the line of sight of camera 2 corresponding to the particle p12 near the epipolar line L12 in step 33) onto camera 3 to obtain the epipolar line L23 of camera 2 to camera 3, and find the particle image pixel position p123 in a certain area near the intersection of these two epipolar lines.
[0096] In this embodiment, Figure 8 The figure shows a detailed schematic diagram of the particle matching process. Based on the line of sight of camera 1 corresponding to the single particle p1 obtained in step 32), it is projected onto the second camera to obtain the epipolar line L12 of camera 1 to camera 2, and the particle image pixel position p12 in a certain area near the epipolar line L12 is searched and stored; then the line of sight of camera 1 corresponding to particle p1 is projected onto camera 3 to obtain the epipolar line L13 of camera 1 to camera 3, and then the line of sight of camera 2 corresponding to the particle in the particle image pixel position p12 is projected onto camera 3 to obtain the epipolar line L23 of camera 2 to camera 3, and the particle image pixel position p123 in a certain area near the intersection of the two epipolar lines is searched.
[0097] 35) Project the line of sight of camera 1 corresponding to the single particle p1 onto camera 4 to obtain the epipolar line L14 of camera 1 to camera 4, then project the line of sight of camera 2 corresponding to the particle p12 near the epipolar line in step 33) onto camera 4 to obtain the epipolar line L24 of camera 2 to camera 4; finally, project the line of sight of camera 3 corresponding to the particle p123 near the intersection of the epipolar lines in step 34) onto camera 4 to obtain the epipolar line L34 of camera 2 to camera 4; find the particle image pixel position in a certain area near the intersection of these three epipolar lines, and the particle image pixel position p1234 closest to the intersection is the particle image pixel position projected onto this image by the particle's three-dimensional spatial position. If there are more cameras, the same logic applies.
[0098] In this embodiment, the line of sight of camera 1 corresponding to particle p1 is projected onto camera 4 to obtain the epipolar line L14 of camera 1 to camera 4, and then the line of sight of camera 2 corresponding to the particle in the particle image pixel position p12 is projected onto camera 4 to obtain the epipolar line L24 of camera 2 to camera 4, and then the line of sight of camera 3 corresponding to the particle in the particle image pixel position p123 is projected onto camera 4 to obtain the epipolar line L34 of camera 3 to camera 4, and the nearest particle image pixel position p1234 near the intersection of these three epipolar lines is found. Fig. 9 Shown is a schematic diagram of the epipolar lines on the particle image during the stereo matching process in this embodiment.
[0099] 36) The particle pixel positions p1, p12, p123, p1234 on each camera particle image obtained in step 35) are first mapped to the world coordinates of multiple calibration planes through the camera calibration function relationship obtained in step 12) to obtain multiple intersections of the particle pixel position corresponding to the line of sight and the multiple planes, and these intersections are connected to form four three-dimensional space lines of sight.
[0100] The direction unit vectors of the four lines of sight are calculated, and the direction unit vectors of the four lines of sight and the intersection points of the four lines of sight with multiple calibration planes are stored. The unit vector of each line of sight is calculated as follows:
[0101]
[0102] in, is the sight unit vector of a particle corresponding to the i-th camera, i is a natural number, and its value is 1, 2, 3, 4, P 1,i and P 2,i are the intersection points of the line of sight of the i-th camera corresponding to a particle and any two calibration planes in the multiplane, ||P 1,i -P 2,i || 2 It is expressed as the geometric distance between the line of sight of the particle corresponding to the i-th camera and the intersection of any two calibration planes in the multiplane.
[0103] Based on the four sight lines, triangulation reconstruction is performed. Due to the errors in the measurement process, triangulation reconstruction does not necessarily intersect at one point. The three-dimensional space intersection point closest to the four sight lines is obtained, and the three-dimensional space position of the particle is preliminarily obtained (that is, the three-dimensional space point closest to all sight lines). The calculation process of the three-dimensional space intersection point closest to the four sight lines is as follows:
[0104]
[0105] Among them, X is the three-dimensional spatial position of the particle, I is the third-order unit matrix, and T represents the transpose of the vector.
[0106] In this embodiment, the particle pixel positions p1, p12, p123, p1234 are adjusted according to Fig.10 As shown, the triangulated reconstruction of the four lines of sight is performed. Due to the errors in the measurement process, the triangulated reconstruction does not necessarily intersect at one point. The three-dimensional spatial intersection point closest to the four lines of sight is obtained, and the three-dimensional spatial position of the particle is preliminarily obtained.
[0107] S4. In parallel, the preliminary three-dimensional spatial position of each particle obtained in step S3 is nearest-neighbor matched with all three-dimensional spatial sight lines in the sight line library in step S2, and sight line interpolation and sight line jitter are performed to obtain the optimal reprojected particle image, thereby generating the precise three-dimensional spatial position of the particle.
[0108] In this embodiment, the parallel processing of particles is implemented based on OpenMP parallel programming, and the line of sight jitter of each particle is completed by an OpenMP thread.
[0109] like Figure 3 As shown, step S4 specifically includes the following steps:
[0110] 41) Perform nearest neighbor matching on the three-dimensional spatial position of the particle obtained in step S3 and all the lines of sight corresponding to the pixel coordinates of all the calibration images or particle images obtained and stored in step S2, and obtain the four lines of sight closest to the particle space coordinates (lines of sight corresponding to integer pixel coordinates), perform three-dimensional spatial interpolation of the lines of sight based on the four closest lines of sight, and then obtain the particle pixel coordinates of the four interpolated integer pixel coordinates of the lines of sight, and generate the particle pixel distribution on a new blank image based on the interpolated particle pixel coordinates as the center point of the particle image, and obtain the reprojected particle image, and then obtain the line of sight and direction unit vector corresponding to this particle pixel coordinate for each camera.
[0111] like Fig.11 The figure shows a schematic diagram of line of sight interpolation in this embodiment, in which the four lines of sight closest to the three-dimensional spatial position of the particle (lines of sight corresponding to integer pixel coordinates) are obtained, and the three-dimensional spatial interpolation of the lines of sight is performed based on the four lines of sight, thereby obtaining the particle pixel coordinates corresponding to the interpolation of four integer pixel coordinates on the reprojected particle image.
[0112] 42) The particle pixel coordinates on the reprojected particle image obtained in step 41) are spatially translated and jittered corresponding to the sight lines of each camera. The jittering process is an iterative process. The sight lines are first translated greatly in three spatial dimensions, and then translated slightly in three spatial dimensions. During the jittering process, the particle pixel coordinates of the reprojected particle image will also change. The residual particle image distribution is obtained by subtracting the reprojected particle image from the original particle image. The best reprojected particle image (i.e., the image closest to the original particle image) is obtained when the sum of the particle brightness on the residual particle image is minimized.
[0113] like Fig.12 The figure shows a schematic diagram of the line of sight translation jitter in this embodiment. The particle pixel coordinates will also change during the jittering process. The residual image distribution is obtained by subtracting the reprojected particle image from the original particle image. The best reprojected particle image (i.e., the image closest to the original particle image) is obtained when the sum of the particle brightness on the residual particle image is minimized.
[0114] 43) Based on step 42), the particle pixel coordinates are obtained based on the particle pixel peak search and the particle image position center Gaussian fitting according to the optimal reprojected particle image, and the particle pixel coordinates are used to calculate the particle pixel coordinates according to the camera calibration function relationship obtained by calibration in step 12), and the three-dimensional space line of sight of each camera is calculated to perform line of sight triangulation reconstruction to obtain the accurate three-dimensional space position of the particle (i.e., the three-dimensional space intersection of each line of sight).
[0115] S5, repeat steps S3 and S4 four times, obtain the three-dimensional spatial position of the particles in the first four frames, connect the initial particle trajectories, and find the trajectory with the smallest acceleration change among all possible trajectories as the optimal initial trajectory, such as Figure 4 As shown, specifically:
[0116] 51) Based on the three-dimensional spatial positions of the particles in the first four frames, starting from the three-dimensional spatial positions of all particles in the first frame, all possible particle trajectories of the next frame are sequentially connected to form a particle trajectory tree network.
[0117] 52) Traverse each branch of the particle trajectory tree network and calculate three velocity vectors based on the two adjacent particle points of each branch. The specific process is to calculate the velocity vector of the particle at the nth moment based on the three-dimensional spatial position of each two adjacent particles (nth moment, n+1th moment) on the initial trajectory:
[0118]
[0119] in, is the velocity vector of the particle at the nth moment, X(n+1) is the three-dimensional spatial position where the particle may appear at the n+1th moment, and Δt is the time interval between every two moments.
[0120] According to the three speed values, an acceleration value is calculated for each two adjacent speeds:
[0121]
[0122] in, and are the nth and n+1th velocity vectors respectively, The acceleration vector of the particle at the nth moment.
[0123] 53) Traverse each branch of the particle trajectory tree network and find the branch with the smallest acceleration change as the optimal particle initial trajectory.
[0124] In this embodiment, based on the three-dimensional spatial positions of the particles in the first four frames, starting from the three-dimensional spatial position p1 of the particles in the first frame, the optimal initial particle trajectory with the smallest acceleration change is as follows: Fig.13 The first four particles are shown.
[0125] S6. According to the optimal initial particle trajectory obtained in the first four frames, the possible position of the particle in the next frame is predicted based on the N-order polynomial fitting, and the position is projected onto the particle image of each camera to obtain the fifth frame of reprojected image. Compared with the fifth frame of measured image, the three-dimensional spatial position of the particle is jittered corresponding to the line of sight of each camera to obtain the optimal reprojected image. The specific steps include:
[0126] 61) According to the obtained optimal initial trajectory of the first four frames, the three-dimensional spatial position of the particle that may appear in the next frame is predicted based on N-order polynomial fitting.
[0127] In this embodiment, based on the particle trajectory track1 obtained by connecting the three-dimensional spatial positions of the particles in the first four frames, the three-dimensional spatial position p5 of the particle that may appear in the next frame is predicted based on the N-order polynomial fitting, such as Fig.13 As shown by the fifth particle, the predicted particles do not necessarily coincide with the actually measured particles.
[0128] 62) Perform nearest neighbor matching on the three-dimensional spatial position of the particle obtained in step 61) and the corresponding sight lines of all image pixel coordinates obtained in step 2), and perform sight line interpolation and reprojection according to the direction unit vector of the sight line corresponding to the nearest neighbor pixel coordinate, and project it onto the particle image of each camera to obtain the fifth frame of reprojected image, and use the original image to subtract the reprojected image to obtain the residual image distribution, and jitter the sight line so that the total brightness of the particles on the residual image is minimized, and finally obtain the best reprojected image (i.e., the image closest to the original image).
[0129] In this embodiment, the predicted particles are Fig.12 The figure shows a schematic diagram of the line of sight translation jitter in this embodiment. The particle pixel coordinates will also change during the jittering process. The residual image distribution is obtained by subtracting the reprojected image from the original image. The best reprojected image (i.e., the image closest to the original image) is obtained when the total brightness of the particles on the residual image is minimized.
[0130] 63) Based on step 62), the particle pixel coordinates are obtained based on the particle pixel peak search and the Gaussian fitting of the particle image position center according to the optimal reprojection image, and the particle pixel coordinates are used to calculate the three-dimensional space line of sight of each camera corresponding to the particle pixel coordinates according to the camera calibration function relationship obtained by calibration in step 1) to perform line of sight triangulation reconstruction to obtain the accurate three-dimensional space position of the particle (i.e., the three-dimensional space point closest to each of the four lines of sight).
[0131] 64) Connect the precise three-dimensional spatial position of the particle obtained in step 63) to the existing initial trajectory to form a long-time series of four-dimensional Lagrangian particle trajectories. If the captured particle images have more moments, repeat step S6 until the particles at all moments are connected to the trajectory to construct a four-dimensional particle trajectory.
[0132] S7. According to the four-dimensional particle trajectory, the velocity and acceleration of each particle point at each moment on the time series trajectory are calculated, specifically including:
[0133] 71) According to the four-dimensional Lagrangian particle trajectory obtained in step 64), based on the three-dimensional spatial positions of every three adjacent particles (n-1th moment, nth moment, n+1th moment) on the time series trajectory, the velocity of the particle at the nth moment is calculated.
[0134] Fig.14 This is a schematic diagram of the calculation of the medium trajectory and velocity in this embodiment. According to the four-dimensional Lagrangian particle trajectory, based on the three-dimensional spatial positions of the three particles on the time series trajectory (the 49th moment, the 50th moment, and the 51st moment), the velocity of the particle at the 50th moment is calculated. Based on the three-dimensional spatial positions of the three particles (the 50th moment, the 51st moment, and the 52nd moment) on the time series trajectory, the velocity of the particle at the 51st moment is
[0135] 72) According to the particle point velocity obtained in step 71), based on the velocity of each two adjacent particles on the time series trajectory, the acceleration of the intermediate particle is calculated.
[0136] The acceleration of the particle at the 50th moment in this example is
[0137] S8. According to the velocity and acceleration of each particle, interpolate it onto the orthogonal Euler grid to obtain the three-dimensional flow field, including:
[0138] 81) Based on scattered point interpolation, the velocity vector of the three-dimensional scattered point particles obtained in step 71) and step 72) is converted into and the acceleration vector Interpolate to the orthogonal Euler grid nodes to obtain the velocity vector of the grid nodes Note: The subscript p represents three-dimensional scattered particles, and the subscript m represents orthogonal grid nodes.
[0139] Fig.15 The velocity vector of the particle point in the embodiment of the present invention is shown as follows: and the acceleration vector Interpolation to an orthogonal Euler grid diagram, using the nearest neighbor interpolation algorithm to obtain the velocity vector on the orthogonal Euler grid node
[0140] 82) Correcting the error vector of the Euler velocity field obtained in step 81).
[0141] In this embodiment, each velocity vector is traversed and compared with surrounding velocity vectors. If there is a large difference, the median of the surrounding velocity vectors is used for replacement and correction.
[0142] 83) Obtain the three-dimensional flow field, output and save the calculated three-dimensional particle field and three-dimensional velocity field results, and end the calculation.
[0143] The four-dimensional particle trajectory and the three-dimensional velocity field results at a certain moment in this embodiment are as follows: Fig.16 shown.
[0144] The performance improvement of using CPU single core and OpenMP multi-core parallel in this embodiment is shown in Table 1.
[0145] Table 1 Performance comparison using CPU single core and OpenMP multi-core parallel
[0146]
[0147] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0148] In another embodiment, an electronic device is provided, comprising one or more processors, a memory and one or more programs stored in the memory, wherein the one or more programs include instructions for executing the particle trajectory velocimetry method based on multi-plane calibration and line of sight constraints as described above.
[0149] The present invention also provides a corresponding Figure 1 The invention discloses a particle trajectory speed measuring device with multi-plane calibration and sight-line constraint, comprising a calibration module, a sight-line library generation module, a preliminary position acquisition module, a precise position acquisition module, a particle trajectory acquisition module and a three-dimensional flow field construction module, wherein the calibration module is used to calibrate multiple cameras based on multi-plane calibration images to obtain a camera calibration function relationship; the sight-line library generation module is used to calculate the three-dimensional space sight line corresponding to the image pixel coordinates of each camera-taken image according to the camera calibration function relationship, and store it as a sight-line library; the preliminary position acquisition module is used to obtain a particle image, and based on the camera calibration function relationship, calculate the three-dimensional space sight line corresponding to the particle pixel coordinates in the particle image, and parallelly calculate the particle trajectory speed measuring device with multi-plane calibration and sight-line constraint, comprising a calibration module, a sight-line library generation module, a preliminary position acquisition module, a preliminary position acquisition module, a precise position acquisition module, a particle trajectory acquisition module and a three-dimensional flow field construction module, wherein the calibration module is used to calibrate multiple cameras based on a multi-plane calibration image to obtain a camera calibration function relationship; the sight-line library generation module is used to calculate the three-dimensional space sight line corresponding to the image pixel coordinates of each camera-taken image according to the camera calibration function relationship, and store it as a sight-line library; the preliminary position acquisition module is used to obtain a particle image, and calculate the three-dimensional space sight line corresponding to the particle pixel coordinates in the particle image based on the camera calibration function relationship, and parallelly calculate the particle trajectory speed measuring device with multi-plane calibration and sight-line constraint, and ... Particle matching and triangulation reconstruction are performed on each particle to obtain the preliminary three-dimensional spatial position of the particle; the precise position acquisition module is used to perform nearest neighbor matching of the preliminary three-dimensional spatial position of each particle with all three-dimensional spatial sight lines in the sight library in parallel, and perform sight line interpolation and sight line jitter to obtain the optimal reprojected particle image, and then generate the precise three-dimensional spatial position of the particle; the particle trajectory acquisition module is used to construct a four-dimensional particle trajectory based on the precise three-dimensional spatial position of each particle, and calculate the velocity vector and acceleration vector of each particle at each moment; the three-dimensional flow field construction module is used to interpolate the velocity vector and acceleration vector of each particle onto the orthogonal Euler grid to obtain a three-dimensional flow field.
[0150] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.
Claims
1. A particle trajectory velocity measurement method based on multi-plane calibration and line of sight constraint, characterized in that: The following steps are involved: 1) Calibrate multiple cameras based on multi-plane calibration images to obtain camera calibration function relationships; 2) calculating the three-dimensional spatial line of sight corresponding to the image pixel coordinates of each camera-captured image according to the camera calibration function relationship, and storing the three-dimensional line of sight as a line of sight library; 3) obtaining a particle image, calculating the three-dimensional spatial line of sight corresponding to the particle pixel coordinates in the particle image based on the camera calibration function relationship, and performing particle matching and triangulation reconstruction on each particle in parallel to obtain a preliminary three-dimensional spatial position of the particle; 4) performing nearest neighbor matching of the preliminary three-dimensional spatial position of each particle with all three-dimensional spatial sight lines in the sight line library in parallel, and performing sight line interpolation and sight line jittering to obtain the optimal reprojected particle image, thereby generating the precise three-dimensional spatial position of the particle; 5) Construct a four-dimensional particle trajectory based on the precise three-dimensional spatial position of each particle, and calculate the velocity vector and acceleration vector of each particle at each moment; 6) The velocity vector and acceleration vector of each particle are interpolated onto the orthogonal Euler grid to obtain the three-dimensional flow field.
2. The particle trajectory velocity measurement method based on multi-plane calibration and line of sight constraint according to claim 1 is characterized in that: In step 1), each camera photographs a plurality of calibration planes at different distances to obtain the multi-plane calibration image.
3. The particle trajectory velocity measurement method based on multi-plane calibration and line of sight constraint according to claim 1 is characterized in that: In step 2), the sightline library is obtained by the following steps: Based on the camera calibration function relationship, the image pixel coordinates of each camera-captured image are mapped to the world coordinates of multiple calibration planes, and multiple intersection points of a single pixel and multiple planes are obtained. The multiple intersection points form a three-dimensional line of sight, and the direction unit vector of the three-dimensional line of sight is calculated. The image pixel coordinates and the direction unit vector of the corresponding line of sight and the intersection points with each calibration plane are stored to form the line of sight library.
4. The particle trajectory velocity measurement method based on multi-plane calibration and line of sight constraint according to claim 1, characterized in that: In step 3) and step 4), parallel processing of each particle is implemented based on OpenMP.
5. The particle trajectory velocity measurement method based on multi-plane calibration and line of sight constraint according to claim 1, characterized in that: In step 3), the pixel coordinates of the particles in the particle image are determined by: The particle pixel peak value is searched for the particle image actually taken and the particle image position center Gaussian fitting is performed to obtain the particle pixel coordinates.
6. The particle trajectory velocity measurement method based on multi-plane calibration and line of sight constraint according to claim 1, characterized in that: In step 3), the particle matching and triangulation reconstruction specifically include: 301) based on the three-dimensional space sight line and direction unit vector of each camera corresponding to each particle, obtain the particle image pixel position projected on each camera by the three-dimensional space position of the particle; 302) mapping each of the particle image pixel positions to the world coordinates of a plurality of calibration planes, respectively, obtaining a plurality of intersection points between the particle image pixel position corresponding sight lines and the plurality of calibration planes, and connecting the plurality of intersection points to obtain a plurality of three-dimensional space sight lines; 303) Perform triangulation reconstruction based on each three-dimensional spatial line of sight to obtain the preliminary three-dimensional spatial position of the particle.
7. The particle trajectory velocity measurement method based on multi-plane calibration and line of sight constraint according to claim 1 is characterized in that: In step 4), the sight line interpolation and sight line jitter specifically include: 401) performing three-dimensional space interpolation based on a plurality of sight lines obtained by the nearest neighbor matching to obtain a particle pixel coordinate, generating a particle pixel distribution on a new blank image based on the particle pixel coordinate as the center point of the particle image, and obtaining a reprojected particle image; 402) obtaining the three-dimensional space sight line of each camera corresponding to the particle pixel coordinates; 403) on the reprojected particle image, the particle pixel coordinates are spatially translated and jittered corresponding to the three-dimensional spatial line of sight of each camera, and the reprojected particle image generated by each jitter is subtracted from the original particle image to obtain a residual particle image distribution, and the reprojected particle image with the smallest total particle brightness on the residual particle image is taken as the best reprojected particle image; 404) extracting the particle pixel coordinates on the best reprojected particle image, and calculating the particle pixel coordinates according to the camera calibration function relationship to perform line of sight triangulation reconstruction corresponding to the three-dimensional line of sight of each camera to obtain the precise three-dimensional space position of the particle.
8. The particle trajectory velocity measurement method based on multi-plane calibration and line of sight constraint according to claim 1, characterized in that: In step 5), the construction of the four-dimensional particle trajectory specifically includes: 501) based on the accurate three-dimensional spatial positions of particles obtained in the previous frames, starting from the accurate three-dimensional spatial positions of all particles in the first frame, all possible particle trajectories of the next frame are sequentially connected to form a particle trajectory tree network; 502) traversing each branch of the particle trajectory tree network, and finding the branch with the smallest acceleration change as the optimal particle initial trajectory; 503) Based on the initial trajectory of the particle in the previous frames, an N-order polynomial fitting is used to predict the possible position of the particle in the next frame; 504) performing nearest neighbor matching between the possible positions and all three-dimensional spatial sight lines in the sight line library, and performing sight line interpolation and sight line reprojection to obtain an optimal reprojected particle image; 505) Based on the camera calibration function relationship, the three-dimensional spatial line of sight corresponding to the particle pixel coordinates in the best reprojected particle image is calculated, and triangulated reconstruction is performed to obtain the accurate three-dimensional spatial position of the particle in the next frame; 506) Connecting the accurate three-dimensional spatial position of the next frame of particles into the initial trajectory of the particles; 507) Repeat steps 503)-506) until all particles at all times are connected.
9. The particle trajectory velocity measurement method based on multi-plane calibration and line of sight constraint according to claim 1, characterized in that: In step 5), the calculation of the velocity vector and acceleration vector of each particle at each moment specifically includes: According to the four-dimensional particle trajectory, the velocity vector of the particle at the nth moment is calculated based on the three-dimensional spatial positions of every three adjacent particles on the time series trajectory; in, is the velocity vector of the particle at the nth moment, X(n-1) and X(n+1) are the three-dimensional spatial positions of the particle at the n-1th and n+1th moments respectively, and Δt is the time interval between every two moments; Based on the velocity vector of every two adjacent particles on the time series trajectory, the acceleration of the previous particle is calculated; in, and are the velocity vectors of the particle at the nth and n+1th moments, respectively. The acceleration vector of the particle at the nth moment.
10. The particle trajectory velocity measurement method based on multi-plane calibration and line of sight constraint according to claim 1, characterized in that: In step 6), the obtaining of the three-dimensional flow field specifically includes: Based on scattered point interpolation, the velocity vector and acceleration vector of each particle at each moment are interpolated to the orthogonal Euler grid nodes to obtain the velocity vector of the grid nodes and construct the Euler velocity field; The error vector correction is performed on the Euler velocity field to obtain the three-dimensional flow field.
Citation Information
Patent Citations
Therapy planning
CN106170317A
Multi-spacecraft four-dimensional cooperative trajectory determination method
CN111552317A
Subject three-dimensional region estimation method and program
JP2013058132A
Detection of the position of a moving object and treatment method
US20090163799A1
System and Method for Three Dimensional Particle Imaging Velocimetry and Particle Tracking Velocimetry
US20210241466A1
