A three-dimensional particle tracking method based on three-color mask three-dimensional imaging

Abstract

The application belongs to the technical field of three-dimensional particle tracking, and provides a three-dimensional particle tracking method based on three-color mask three-dimensional imaging, which comprises the following steps: image acquisition device construction, image shooting, three-primary-color color channel separation, de-mosaicking, hole interpolation, color crosstalk correction, trajectory fitting, position prediction, jitter prediction and projection, local range residual calculation, minimum residual screening, un-fitted trajectory particle extraction, trajectory fitting, position prediction, and predicted position iteration. The application realizes a shooting device with smaller volume, lower cost and simpler structure through a three-color mask and a single-color camera, and improves the spatial resolution of particle tracking through the use of PTV technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional particle tracking, in particular to a three-dimensional particle tracking method based on three-color mask three-dimensional imaging. BACKGROUND

[0002] By studying the motion law of tracer particles, the complex flow situation can be effectively understood. The particle image velocimetry, referred to as PIV, uses the particle image to reconstruct the particle three-dimensional voxel intensity distribution through the MLOS-SMART method, and then uses the cross-correlation calculation to calculate the displacement of two frames of voxel fields, so as to obtain the motion of the particles. The particle tracking velocimetry, referred to as PTV, tracks a specific particle in the Lagrangian coordinate system, and obtains the motion trajectory of the particle within a certain time through the velocity and position of the particle. Compared with the PIV technology, the PTV technology can obtain more accurate three-dimensional position of a specific particle and the specific three-dimensional velocity of the particle. An important factor affecting the tracking effect of the PTV technology is whether the particles overlap. The particle overlap will cause the particle concentration used by the PTV technology to be lower than that of the PIV technology, and thus the motion field resolution is reduced. Researchers have proposed a "shake box" method, which improves the particle tracking accuracy by time-constraining particle tracking; and a method of predicting the position of a particle with an existing trajectory first, and then reconstructing the three-dimensional image of the remaining particles, which improves the calculation speed. For the problem of low particle concentration of the PTV technology, researchers have proposed an iterative particle reconstruction method to delete overlapping particles through multiple iterations, which can obtain good tracking effect in a particle field with high concentration.

[0003] Due to the current PTV technology, three to four cameras are usually used to shoot with a certain parallax, which has problems such as high camera cost, complex camera system, and inability to adapt to narrow environments. The conventional PIV technology cannot directly calculate the three-dimensional position and velocity distribution of the particles, and the spatial resolution is low. SUMMARY

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present application is to provide a three-dimensional particle tracking method based on three-color mask three-dimensional imaging, which realizes a smaller volume, lower cost and simpler structure of the shooting device through a three-color mask and a single-color camera, and improves the spatial resolution of particle tracking by using the PTV technology.

[0005] To achieve the above purpose, the present application provides the following scheme:

[0006] A three-dimensional particle tracking method based on three-color mask three-dimensional imaging, comprising:

[0007] An image acquisition device is constructed, which comprises a laser generator, a three-color mask, and a single-color camera. The three-color mask comprises a red light transmission hole, a blue light transmission hole, a green light transmission hole, and a circular light blocking mask. The red light transmission hole, the blue light transmission hole, and the green light transmission hole are distributed in an equilateral triangle on the circular light blocking mask. The three-color mask is arranged in front of the lens of the single-color camera. The laser generator is used to irradiate tracer particles in a target flow field. The single-color camera is used to shoot three-primary-color images of the tracer particles in the target flow field by using the three-color mask.

[0008] The tracer particles in the target flow field are shot by using the image acquisition device to obtain an original shot image.

[0009] The original shot image is subjected to three-primary-color channel separation to obtain a channel separation image.

[0010] All the channel separation images are subjected to de-mosaicking hole interpolation and color crosstalk correction to obtain a preprocessed image.

[0011] The first four frames of the preprocessed images are subjected to three-dimensional reconstruction and trajectory fitting by using an IPR algorithm to obtain the first four frames of particle trajectories.

[0012] The particle position data of the nth-1 frame of the particle trajectories in the nth frame are predicted by using a Wiener filter to obtain the nth set of particle prediction position data.

[0013] The nth set of particle prediction position data is subjected to jitter prediction and projection to obtain the nth set of particle re-projection images.

[0014] The local range residual errors of the nth set of particle re-projection images and the nth frame of the preprocessed images are calculated.

[0015] The local range residual errors are screened by using a steepest descent method to obtain the minimum residual error, and the particle re-projection image corresponding to the minimum residual error is determined as the nth frame of prediction image.

[0016] The nth frame of prediction image and the nth frame of preprocessed image are subtracted by using an IPR algorithm to obtain the nth set of un-fitted trajectory particles.

[0017] The nth-3 set to the nth set of un-fitted trajectory particles are subjected to trajectory fitting to obtain the nth set of particle trajectories.

[0018] The nth set of particle trajectories are predicted by using a Wiener filter to obtain the nth+1 set of particle prediction position data.

[0019] The particle prediction position data are iterated to obtain particle position and velocity information.

[0020] Preferably, further comprising:

[0021] The particle trajectory is fitted and optimized in reverse time sequence, and the obtained optimization result is used to update the particle trajectory.

[0022] Preferably, the channel separation images are de-mosaicked, hole-filled and color crosstalk corrected to obtain a pre-processed image, comprising:

[0023] The tracer particles in the target flow field are photographed using the red, blue and green through-holes respectively to obtain single through-hole images;

[0024] Color crosstalk calibration coefficients are calculated according to the single through-hole images, and the channel separation images are corrected and ghost pixels are removed using the color crosstalk calibration coefficients to obtain the pre-processed image.

[0025] Preferably, an image acquisition device is constructed, comprising:

[0026] A planar calibration plate is placed at the center of the square area irradiated by the laser generator;

[0027] The planar calibration plate is moved up and down along the depth of field direction to obtain two-dimensional position information;

[0028] The conversion relationship between two-dimensional image coordinates and three-dimensional world coordinates is determined using the two-dimensional position information, and the parameters of the single-color camera are body calibrated.

[0029] Preferably, the radii of the circumscribed circles of the equilateral triangles formed by the red, blue and green through-holes are 32 mm, and the radii of the red, blue and green through-holes are 5 mm.

[0030] Preferably, the emission wavelengths of the laser generator include 450 nm, 532 nm and 650 nm.

[0031] Preferably, the de-mosaicking hole-filling method includes any one of double-tri-color interpolation, bilinear interpolation, pattern recognition interpolation and deep learning interpolation.

[0032] Preferably, the parameters of the single-color camera include magnification factor, rotation and translation matrix, off-axis offset condition and tangential and radial distortion.

[0033] The present application discloses the following technical effects:

[0034] The application provides a three-dimensional particle tracking method based on three-color mask three-dimensional imaging. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor under the premise of the drawings.

[0036] Figure 1 A three-dimensional particle tracking flowchart based on three-color mask three-dimensional imaging is provided for the embodiments of the present application.

[0037] Figure 2 A three-dimensional particle tracking flowchart is provided for the embodiments of the present application.

[0038] Figure 3 A STB flowchart is provided for the embodiments of the present application.

[0039] Figure 4 A calibration plate and a calibration method schematic diagram are provided for the embodiments of the present application.

[0040] Figure 5 A circular non-transparent mask schematic diagram is provided for the embodiments of the present application.

[0041] Figure 6 A three-color mask flow field measurement principle diagram is provided for the embodiments of the present application.

[0042] Figure 7 An IPR flowchart is provided for the embodiments of the present application.

[0043] Figure 8 A three-color mask three-dimensional imaging system schematic diagram is provided for the embodiments of the present application.

[0044] Figure 9 A cylindrical flow structure diagram is provided for the embodiments of the present application.

[0045] Figure 10 A color channel separation and demosaicing flowchart is provided for the embodiments of the present application.

[0046] Figure 11 A color crosstalk correction flowchart is provided for the embodiments of the present application.

[0047] Figure 12 R f With R g Result curve diagram.

[0048] Figure 13 D f Result curve diagram. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0050] The purpose of the present application is to provide a three-dimensional particle tracking method based on three-color mask three-dimensional imaging, which realizes a smaller volume, lower cost and simpler structure of the shooting device through a three-color mask and a single-color camera; and improves the spatial resolution of particle tracking by using PTV technology.

[0051] In order to make the above-mentioned purposes, characteristics and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0052] Figure 1 The three-dimensional particle tracking flowchart based on three-color mask three-dimensional imaging provided by the embodiments of the present application is shown in the figure, Figure 2 The three-dimensional particle tracking flowchart provided by the embodiments of the present application is shown in the figure, Figure 1 and Figure 2 The present application provides a three-dimensional particle tracking method based on three-color mask three-dimensional imaging, which comprises:

[0053] Step 100: constructing an image acquisition device; the image acquisition device comprises a laser generator, a three-color mask and a single-color camera; the three-color mask comprises a red light transmission hole, a blue light transmission hole, a green light transmission hole and a circular light shielding mask; the red light transmission hole, the blue light transmission hole and the green light transmission hole are distributed in an equilateral triangle on the circular light shielding mask; the three-color mask is arranged in front of the lens of the single-color camera; the laser generator is used for irradiating tracer particles in a target flow field; and the single-color camera is used for shooting three-primary-color images of the tracer particles in the target flow field by using the three-color mask;

[0054] Step 200: shooting the tracer particles in the target flow field by using the image acquisition device to obtain an original shooting image;

[0055] Step 300: performing three-primary color channel separation on the original shot image to obtain a channel separation image;

[0056] Step 400: performing demosaicing hole interpolation and color crosstalk correction on all channel separation images to obtain a preprocessed image;

[0057] Step 500: using an IPR algorithm to perform three-dimensional reconstruction and trajectory fitting on the first four frames of preprocessed images to obtain the first four frames of particle trajectories;

[0058] Step 600: using a Wiener filter to predict the particle position of the nth-1 frame of particle trajectories in the nth frame to obtain the nth set of particle prediction position data;

[0059] Step 700: performing jitter prediction and projection on the nth set of particle prediction position data to obtain the nth set of particle re-projection images;

[0060] Step 800: calculating the local range residual error of the nth set of particle re-projection images and the nth frame of preprocessed images;

[0061] Step 900: using the steepest descent method to screen the local range residual error to obtain the minimum residual error, and determining the particle re-projection image corresponding to the minimum residual error as the nth frame of prediction image;

[0062] Step 1000: using the IPR algorithm to subtract the nth frame of prediction image from the nth frame of preprocessed image to obtain the nth set of non-fitted trajectory particles;

[0063] Step 11000: performing trajectory fitting on the nth-3 set to the nth set of non-fitted trajectory particles to obtain the nth set of particle trajectories;

[0064] Step 1200: using a Wiener filter to predict the nth set of particle trajectories to obtain the nth+1 set of particle prediction position data;

[0065] Step 1300: iterating the particle prediction position data to obtain particle position and velocity information.

[0066] Reference Figure 3 , and further comprising:

[0067] Performing inverse time series fitting and optimization on the particle trajectories, and updating the particle trajectories using the obtained optimization results.

[0068] Specifically, performing demosaicing hole interpolation and color crosstalk correction on all channel separation images to obtain a preprocessed image, comprising:

[0069] Respectively and individually using a red light tunnel, a blue light tunnel, and a green light tunnel to shoot the tracer particles in the target flow field to obtain a single light tunnel image;

[0070] The color crosstalk calibration coefficient is calculated according to the single light hole image, and the color crosstalk calibration coefficient is used for correcting the channel separation image and removing ghost pixels to obtain a pretreated image.

[0071] Reference Figure 4 , the image acquisition device is constructed, comprising:

[0072] The planar calibration board is placed at the center of the square area irradiated by the laser generator;

[0073] The planar calibration board is moved up and down along the depth of field direction to obtain two-dimensional position information;

[0074] The conversion relationship between the two-dimensional image coordinates and the three-dimensional world coordinates is determined by using the two-dimensional position information, and the parameters of the single-color camera are body calibrated.

[0075] Reference Figure 5 , the radius of the outer circle of the equilateral triangle composed of the red light hole, the blue light hole and the green light hole is 32mm; the radius of the red light hole, the blue light hole and the green light hole is 5mm.

[0076] Specifically, the light-emitting wavelengths of the laser generator include 450nm, 532nm and 650nm respectively.

[0077] The method for mosaicking hole interpolation can include any one of bilinear interpolation, pattern recognition interpolation and deep learning interpolation.

[0078] Preferably, the parameters of the single-color camera include magnification coefficient, rotation and translation matrix, off-axis offset condition and tangential and radial distortion.

[0079] Specifically, the image of particle motion is obtained by using a three-color mask, and then a PTV related method is used to track the particle motion, including:

[0080] S1: In the measurement flow field of the scattered tracer particles, a high-energy white LED or halogen lamp body light source is used for illumination, and a three-color mask single-color camera is used to obtain a time sequence particle three-color mask color image of the tracer particles in the measured flow field (reference Figure 4 ).

[0081] S2: The time sequence particle three-color mask color image is subjected to three-view image separation processing, which includes three-view image separation of the red, green and blue color channels of the color image (reference Figure 6 ); the separated original image is subjected to mosaicking hole interpolation by using bilinear interpolation, pattern recognition interpolation or deep learning interpolation; color crosstalk correction is performed on the three-view image to remove the interference of other channel colors on the target channel.

[0082] S3: IPR algorithm is used for the first four frames of the processing time sequence to obtain the three-dimensional reconstruction position of the particles, and the method of velocity prediction is used to fit the trajectory of the first four frames. The content of the IPR algorithm is as follows: first, the two-dimensional pixel coordinates of the particles are identified in the two-dimensional particle image of each view; then, the distribution of the particles in the three-dimensional world coordinate system is determined at different camera views by using the triangulation method; subsequently, the known particle projection is deleted on the two-dimensional particle image, and the above process is repeated on the residual image after deleting the particle projection until the number of iterations is completed (reference Figure 7 ).

[0083] S4: Based on the existing trajectory prediction, the particle position of the next frame is predicted using the Wiener filter, and the predicted position is projected into the camera image and compared with the particle position of the camera image to adjust the predicted position. Finally, the IPR method is used again in the residual image after removing the predicted particles to reconstruct the three-dimensional position of the particles that have not been fitted into the trajectory, and a new trajectory is tried to be fitted (reference Figure 3 ).

[0084] S5: After completing the trajectory fitting of the forward time sequence, the trajectory is supplemented in the reverse time sequence and the correctness of the trajectory is verified. This step can extend the trajectory to the beginning of the time sequence. By this method, the motion of the particles in the flow field can be accurately tracked even with only one high-speed camera (reference Figure 3 ).

[0085] In S1, multiple views are obtained using a three-color mask. The angle of light of different colors passing through the filter changes with the depth, resulting in a certain deviation of the three colors projected on the camera. In S2, color crosstalk in the color channel needs to be removed. A single light hole picture that blocks two light holes is shot, and then a first-order fitting method is used to obtain the linear relationship between the light intensity of this light hole and other color channels, and the color crosstalk correction is realized. In S3, the three-dimensional particle position is obtained by triangulation, and the three-dimensional position of the predicted frame particles in S4 and S5 needs to be optimized by jitter. The specific process of jitter is as follows: given the three-dimensional coordinates of the particles and the two-dimensional particle image, the residual between the particle projection image after changing the X, Y, and Z three-dimensional world coordinates by ±1 and the original two-dimensional image is calculated, a residual and jitter displacement relationship curve is fitted, and finally the coordinate with the smallest residual in the relationship curve is obtained as the correct three-dimensional particle coordinate.

[0086] Reference Figure 3 , the STB process is as follows:

[0087] Initial stage (first four frames)

[0088] 1) For the first four frames of three-view two-dimensional particle images, use the IPR method to reconstruct the three-dimensional particle distribution of the four frames;

[0089] 2) Use the nearest neighbor, minimum acceleration, minimum jerk or velocity field interpolation method to confirm the trajectory of the first four frames;

[0090] Convergence phase (subsequent frames)

[0091] 3) Based on the existing trajectories of the first four frames, use the Wiener filter to predict the position of the next frame of these trajectories;

[0092] 4) Project the predicted position into three views, and calculate the residual with the captured particle image in a local range. By using the "dithering" method, fine-tune the three-dimensional position and intensity of the predicted particle, and confirm the three-dimensional position and intensity of the particle with the smallest residual as the particle in the next frame of the known trajectory;

[0093] 5) Project the predicted particle of the known trajectory into three views, and subtract the projected image from the captured image of the predicted frame to obtain the residual image, which is the particle image that has not completed trajectory fitting;

[0094] 6) Use the IPR method to reconstruct the three-dimensional particle, and try to fit the reconstructed three-dimensional particle with the particle of the previous frame to obtain a new trajectory;

[0095] 7) Supplement the new trajectory to the known trajectory for particle position prediction in the next frame, until all particle trajectory fitting in the time sequence is completed, and remove the incorrect trajectory through particle intensity, trajectory linear fitting check, etc.

[0096] Optionally, in reverse time sequence (for trajectories supplemented in the middle):

[0097] Predict the particle position of the previous frame of the supplemented trajectory through the Wiener filter;

[0098] Project into three views to calculate the residual between the projected image and the captured image of the previous frame in a local range, and use the "dithering" method to confirm the correct position of the predicted particle;

[0099] Iterate forward to achieve the length of the supplemented trajectory.

[0100] Reference Figure 7 , the IPR process is as follows:

[0101] S1: Input two-dimensional particle images of three views, and use peak detection method to confirm the two-dimensional position of the particle.

[0102] S2: using the triangulation method, through the first perspective of the target particle and the camera center line, the line is projected into the second perspective, the particles in a certain range of the projection line in the second perspective are regarded as the candidate particles of the target particle, and the same steps are used to confirm the line. The intersection of the two lines is confirmed by projecting the lines of the two perspectives into the third perspective, and the particles within a certain range of the intersection are regarded as the candidate particles of the target particle. The two-dimensional target particle coordinates and three-dimensional particle coordinates corresponding to the three perspectives are determined by the screening method.

[0103] S3: through the "shaking" method, the XYZ three-dimensional coordinates and intensity of the particle are fine-tuned respectively, and the fine-tuned position is projected into the original input image. The residual error of the projected particle image and the original input particle image is calculated in a local range, and then the steepest descent method is used to top the particle three-dimensional coordinates and intensity with the minimum residual error.

[0104] S4: by setting the particle intensity threshold, the three-dimensional particles below the threshold are regarded as ghost particles and deleted; the steps of S3 and S4 are repeated multiple times to complete the accurate reconstruction of part of the three-dimensional particles.

[0105] S5: the reconstructed three-dimensional particles are projected into three perspectives, the original input particle image is subtracted from the projected image of the completed reconstruction particles to obtain a residual image without reconstruction particles, and these residual images are input as the input image of the next outer loop.

[0106] Wherein, S1 to S5 is an outer loop; S3 and S4 are an inner loop; multiple execution of the outer loop of S1 to S5 ensures that the overlapping particle image completes the three-dimensional reconstruction.

[0107] Preferably, a three-color mask three-dimensional imaging system is constructed, and a cylindrical flow experiment device is built: the experiment uses a laser that can emit three wave bands at the same time, and the beam power of the laser is 3W, and the wavelengths are 450nm, 532nm and 650nm respectively. The customized three-color mask is installed in front of the ZEISS Milvus 2 / 100M lens and connected with the Revealer M120 color camera. The parameters of the three-color mask are: the diameter of a single mask hole, the magnification and the radius of the circumscribed circle of the mask hole (hole spacing), wherein the magnification is represented by the ratio of the image distance and the object distance. The three-dimensional imaging system is as shown in Figure 8 The red dotted line represents the circumscribed circle of the mask hole. The cylindrical flow experiment device is as shown in Figure 9As shown, the cylinder flow system is composed of an inner circle, an outer circle and a bottom surface, wherein the outer circle has an outer wall radius of 250 mm, the inner circle has an outer wall radius of 190 mm, the thickness of the outer circle and the inner circle is 5 mm, and the height is 100 mm. The bottom surface is a solid cylinder with a radius equal to the outer wall radius of the outer circle and a height of 10 mm. A cylinder with a diameter of 2 mm is placed in the middle of the water tank composed of the inner and outer circles as the flow cylinder. A stepper motor equipped with a planetary reducer is used as the power source for rotating the water tank, and the rotating speed is controlled to be 1.25 RPM. The laser is irradiated to the area at a certain distance behind the cylinder perpendicular to the outer wall of the outer circle, and the camera is placed under the bottom surface of the water tank to shoot the particle movement upward. Polyamide tracer particles with a diameter of 20 μm and a density of 1.03 g / cm3 are used, and the concentration of the particles is about 0.04 PPP.

[0108] Further, color channel separation and demosaicing processing: as shown Figure 10 The original image is separated by color channel to obtain three color-separated original images. Double-tri-color interpolation, bilinear interpolation, pattern recognition interpolation or deep learning interpolation method is used for demosaicing hole filling of the separated original image. From bottom to top: the first image is a color image taken by a single CMOS color camera. Since a single CMOS camera has only a single color sensor at each pixel, each pixel in the image has only one of the three colors red, green and blue, and RGB represents the corresponding color. The second row of images is based on color channel to separate the color image into three images. When there is a corresponding color sensor at the pixel position, the pixel position has a corresponding color, otherwise the pixel position is black. RGB is used to represent the position and color of the corresponding color, and black is used to represent the pixel position without corresponding color. These positions are also called demosaicing holes. The third row of images is the color intensity of the area with existing colors, and the demosaicing holes are assigned values. The last row is to combine the three color channel images after demosaicing, so that each pixel of the combined color image has red, green and blue colors.

[0109] Preferably, color crosstalk correction: take a single aperture image that blocks two apertures, and use the images of the three apertures as calibration images for color crosstalk correction. As shown Figure 11 After obtaining the calibration coefficient of color crosstalk from the calibration image, the separated single channel original image is corrected for color crosstalk to remove ghost pixels. In the image, the horizontal axis represents the particle concentration PPP, which means the number of particles per pixel; the vertical axis of the first image is percentage, and the vertical axis of the second image is distance in pixels. R f represents the proportion of real particles that can exist in a voxel range to all real particles; R gThis represents the proportion of reconstructed particles that do not contain real particles within a voxel range out of all reconstructed particles; D f It is calculated in R f The distances between the projected positions of the matched real and reconstructed particles at the three viewpoints are calculated, and the average distances of all matched particle pairs at each viewpoint are calculated using the following formula:

[0110]

[0111] Where N represents the number of matched particles, x iR x represents the projected position of the real particle at the i-th viewpoint. iS This represents the projection position of the reconstructed particle at the i-th viewpoint.

[0112] R f R represents the proportion of all real particles that can be reconstructed within a voxel range; g This represents the proportion of reconstructed particles that do not contain real particles within a voxel range out of all reconstructed particles.

[0113]

[0114] Where, N f N represents the actual number of particles that can exist within a voxel range; r N represents the total number of real particles; g N represents the number of reconstructed particles that do not contain real particles within a voxel range; c This represents the total number of reconstructed particles.

[0115] refer to Figure 11 RGB represents the three colors: red, green, and blue. This function is used to calculate the interference of blue light on the red channel when only blue light is used, and to calculate the light sensitivity of the red channel in an image illuminated only by blue light. Here, f represents the calibration function, used to quantify the intensity of color crosstalk between different color layers; I B Indicates the current intensity of blue light; Indicates intensity I B The number of pixels; Indicates intensity I B The light sensitivity of the red channel of pixel (i, j). Used to record the current intensity of blue light. By obtaining the current blue light intensity I... B The red channel corresponding to the current blue light intensity and green channel Intensity can be used to determine the first-order linear relationship between the intensity of color crosstalk generated in the red and green light channels and the intensity of blue light when only blue light is present. This can be extrapolated to other lighting conditions. Rread R represents the red channel separated image in the captured experimental image, cor G represents the green channel separated image in the captured experimental image, cor and B represents the blue channel separated image in the captured experimental image. cor R, G and B are the target values for color crosstalk correction, which are unknown, and are solved by known R, G and B data and color crosstalk linear relationship to remove color crosstalk of R, G and B. read read read cor cor cor

[0116] Specifically, body calibration: place the planar calibration board at the center of the laser irradiation square area, since the two-dimensional position information of the planar calibration board is known, move the planar calibration board up and down along the depth of field direction to determine the two-dimensional position information in different depth of field directions. By known two-dimensional position information and position in the depth of field direction, the calibrated three-dimensional position can be determined, and the conversion relationship between the two-dimensional image coordinates and the three-dimensional world coordinates can be calculated in combination with the known two-dimensional image coordinates. In this process, the magnification coefficient of the camera, the rotation and translation matrix of each view angle (used to determine the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates), the off-axis offset of the camera, the tangential and radial distortion of the camera and other parameters need to be determined, and the calibration board and the calibration process are shown in Figure 4 .

[0117] Preferably, the initial stage of particle tracking: for the first four frames of the captured particle images, after color channel separation, demosaicing and color crosstalk correction, the iterative particle reconstruction (IPR) method is used to preliminarily fit the particle trajectory, and the following parameters are selected: 5 external iterations, 8 internal iterations, a two-dimensional particle search range of 6 pixels, a ghost particle intensity threshold of 0.03, a two-dimensional line of sight search range of 0.15 voxels, and a three-dimensional line of sight search range of 0.4 voxels.

[0118] Further, the convergence stage of particle tracking: after completing the trajectory fitting in the initial stage, the subsequent frame particle images are also subjected to color channel separation, demosaicing and color crosstalk correction, and the "shake the box" (STB) method is used to predict the position of the existing trajectory through a Wiener filter, and the iterative particle reconstruction (IPR) method is used to reconstruct the three-dimensional position of the particles of the un-fitted trajectory and attempt to fit the particle trajectory. The following parameters are selected: a shaking range of 0.5 voxels, a maximum single-frame displacement of particles of 4 voxels, a particle intensity subtraction rate of 1.3 times, and a ghost particle intensity threshold of 0.001. In this way, the position and velocity information of the particles in the three-dimensional space can be obtained.

[0119] ​​​​​​Further, to verify the effect of the particle tracking velocimetry (PTV) technology of the three-color mask single-color camera, the embodiment uses the evaluation index commonly used in the PTV technology to explain the particle tracking effect of the synthetic flow field.

[0120] Optionally, the three-color mask camera used in the synthetic image has a resolution of 1280*1024 pixels 2 , the camera area used is 1240*984 pixels 2 , the pixel size is 0.004 mm, and the radius of the circle circumscribing the center of the three mask holes is 32 mm. The ray tracing technology is used to generate the volume calibration image, project the three-dimensional tracer particles onto the two-dimensional image plane, and obtain the three-view angle calibration information of the three-color mask. Then, a group of particles are randomly generated in the measurement volume, and the projection of each particle under each view angle is calculated through the camera calibration information to determine the image coordinates of the particle center. A Gaussian distribution is applied to the coordinates to generate a particle image with a diameter of 3 pixels. Finally, the particle field is moved frame by frame according to the set motion form. The particle concentration of the synthetic image of the embodiment is from 0.005 PPP (particles per pixel) to 0.05 PPP, the particles move linearly and uniformly along the y direction, and make sinusoidal motion according to the current y coordinate position in the z direction.

[0121] Further, to evaluate the effect of the PTV technology, first, the particle extraction efficiency R f is used, that is, whether there is a reconstructed particle within the range of one voxel of a real particle, if there is a reconstructed particle, it is considered that the real particle has been matched, R f represents the proportion of matched real particles to all real examples; then the proportion of ghost particles R g is used, that is, if there is no real particle within the range of one voxel of a reconstructed particle, it is considered that the reconstructed particle is a ghost particle, R f represents the proportion of ghost particles to all reconstructed particles; finally, the distance D f between the matched real particles and the reconstructed particles is used. Figure 12 、 13 The R f , R g and D f effects of the three-color mask single-color camera PTV are respectively explained:

[0122] Specifically, it can be seen from Figure 12 that the PTV using the three-color mask single-color camera can find more than 85% of the real particles at a particle concentration of 0.05 PPP, and the proportion of ghost particles is less than 15%. It can be seen from Figure 13 that the three-dimensional spatial distance between the matched real particles and the reconstructed particles is less than 0.3 voxels, which indicates that the accuracy of particle matching is high.

[0123] The beneficial effects of the present application are as follows:

[0124] The present application realizes a smaller volume, lower cost and simpler structure of the shooting device through the three-color mask and the single-color camera; and the spatial resolution of particle tracking is improved through the use of PTV technology.

[0125] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be mutually referred to.

[0126] The principles and implementation manners of the present application are described by using specific examples in this paper, and the above description of the embodiments is only used to help understand the method of the present application and its core idea; meanwhile, for the general technical personnel in the field, the specific implementation manners and application ranges will be changed according to the idea of the present application. In conclusion, the content of the specification should not be understood as the limitation of the present application.

Claims

1. A three-dimensional particle tracking method based on three-color mask three-dimensional imaging, characterized by, The application relates to a method for tracking particles in a target flow field. The image acquisition device comprises a laser generator, a three-color mask and a single-color camera; the three-color mask comprises a red light transmission hole, a blue light transmission hole, a green light transmission hole and a circular light shielding mask; the red light transmission hole, the blue light transmission hole and the green light transmission hole are distributed in an equilateral triangle on the circular light shielding mask; the three-color mask is arranged in front of a lens of the single-color camera; the laser generator is used for irradiating tracer particles in the target flow field; and the single-color camera is used for shooting three-primary-color images of the tracer particles in the target flow field by using the three-color mask. The tracer particles in the target flow field are shot by using the image acquisition device to obtain original shooting images. The original shooting images are subjected to three-primary-color channel separation to obtain channel separation images. All the channel separation images are subjected to de-mosaic hole interpolation and color crosstalk correction to obtain preprocessed images. Three-dimensional reconstruction and trajectory fitting are performed on the first four frames of the preprocessed images by using an IPR algorithm to obtain the first four frames of particle trajectories. The particle trajectories in the n-th frame are predicted by using a Wiener filter to obtain the n-th group of particle prediction position data. The n-th group of particle prediction position data is subjected to jitter prediction and projection to obtain the n-th group of particle re-projection images. Local range residuals of the n-th group of particle re-projection images and the n-th frame of preprocessed images are calculated. The local range residuals are screened by using a steepest descent method to obtain minimum residuals, and the particle re-projection image corresponding to the minimum residuals is determined as the n-th frame of prediction images. The n-th frame of prediction images and the n-th frame of preprocessed images are subtracted by using the IPR algorithm to obtain the n-th group of un-fitted trajectory particles. The n-th group of un-fitted trajectory particles is subjected to trajectory fitting to obtain the n-th group of particle trajectories. The n-th group of particle trajectories is predicted by using a Wiener filter to obtain the n+1-th group of particle prediction position data. The particle prediction position data is iterated to obtain particle position and velocity information. The particle trajectories are subjected to inverse time sequence fitting and optimization, and the particle trajectories are updated by using the obtained optimization results.

2. The three-dimensional particle tracking method based on three-color mask three-dimensional imaging according to claim 1, characterized in that, The method for tracking particles in a target flow field further comprises the following steps: The particle trajectories are subjected to inverse time sequence fitting and optimization, and the particle trajectories are updated by using the obtained optimization results.

3. The three-dimensional particle tracking method based on three-color mask three-dimensional imaging according to claim 1, wherein, The method for tracking particles in a target flow field further comprises the following steps: The target flow field is shot by using the red light transmission hole, the blue light transmission hole and the green light transmission hole respectively to obtain single light transmission hole images. Color crosstalk calibration coefficients are calculated according to the single light transmission hole images, and the channel separation images are corrected and ghost pixel removed by using the color crosstalk calibration coefficients to obtain the preprocessed images.

4. The three-dimensional particle tracking method based on three-color mask three-dimensional imaging according to claim 1, characterized in that, The image acquisition device comprises a laser generator, a three-color mask and a single-color camera; the three-color mask comprises a red light transmission hole, a blue light transmission hole, a green light transmission hole and a circular light shielding mask; the red light transmission hole, the blue light transmission hole and the green light transmission hole are distributed in an equilateral triangle on the circular light shielding mask; the three-color mask is arranged in front of a lens of the single-color camera; the laser generator is used for irradiating tracer particles in the target flow field; and the single-color camera is used for shooting three-primary-color images of the tracer particles in the target flow field by using the three-color mask. The method for tracking particles in a target flow field further comprises the following steps: The plane calibration plate is placed in the center of the square irradiation area of the laser generator; The plane calibration plate is moved up and down along the depth of field direction to obtain two-dimensional position information; The two-dimensional position information is used to determine a conversion relationship between two-dimensional image coordinates and three-dimensional world coordinates, and parameters of the single-color camera are body calibrated.

5. The three-dimensional particle tracking method based on three-color mask three-dimensional imaging according to claim 1, wherein, The red light hole, the blue light hole and the green light hole form an equilateral triangle, and the radius of the outer circle of the equilateral triangle is 32mm; the radius of the red light hole, the blue light hole and the green light hole is 5mm.

6. The three-dimensional particle tracking method based on three-color mask three-dimensional imaging according to claim 1, wherein, The light-emitting wavelengths of the laser generator include 450nm, 532nm and 650nm respectively.

7. The three-dimensional particle tracking method based on three-color mask three-dimensional imaging according to claim 1, wherein, The method for de-mosaicing hole interpolation includes any one of double-tri-color interpolation, double-linear interpolation, pattern recognition interpolation and deep learning interpolation.

8. The three-dimensional particle tracking method based on three-color mask three-dimensional imaging according to claim 4, wherein, The parameters of the single-color camera include magnification coefficient, rotation and translation matrix, off-axis offset condition and tangential and radial distortion.

Images