Three-dimensional particle tracking method based on three-color mask three-dimensional imaging

Abstract

The invention belongs to the technical field of three-dimensional particle tracking, and provides a three-dimensional particle tracking method based on three-color mask three-dimensional imaging. Comprising the steps of image acquisition device construction, image shooting, three-primary color channel separation, demosaicing hole interpolation and color crosstalk correction, trajectory fitting, position prediction, jitter prediction and projection, local range residual calculation, minimum residual screening, unfitted trajectory particle extraction, trajectory fitting, position prediction and predicted position iteration. Through the three-color mask and the single-color camera, the shooting device which is smaller in size, lower in cost and simpler in structure is realized; by using the PTV technology, the spatial resolution of particle tracking is improved.

Description

Technical Field

[0001] The invention relates to the technical field of three-dimensional particle tracking, and in particular to a three-dimensional particle tracking method based on three-color mask three-dimensional imaging. Background Art

[0002] By studying the motion law of tracer particles, we can effectively understand complex flow conditions. Particle image velocimetry, referred to as PIV, uses particle images to reconstruct the three-dimensional voxel intensity distribution of particles through the MLOS-SMART method, and then uses cross-correlation calculation to calculate the displacement of two frames of voxel fields to obtain the movement of particles. Particle tracking velocimetry, referred to as PTV, tracks specific particles in the Lagrangian coordinate system, and obtains the motion trajectory of particles in a certain period of time through the velocity and position of the particles. Compared with PIV technology, PTV technology can obtain a more accurate three-dimensional position of specific particles and the specific three-dimensional velocity of the particle. An important factor affecting the tracking effect of PTV technology is whether the particles overlap. Particle overlap will cause the particle concentration used in PTV technology to be lower than that of PIV technology, which will lead to a decrease in the resolution of the motion field. The researchers proposed a "shaking box" method, which improves the accuracy of particle tracking through time-constrained particle tracking; the method uses the prediction of the particle position of the existing trajectory first, and then the three-dimensional reconstruction of the remaining image particles to improve the calculation speed. For the problem of low particle concentration in PTV technology, the researchers proposed an iterative particle reconstruction method to delete overlapping particles through multiple iterations, which can obtain good tracking effects in particle fields with higher concentrations.

[0003] Due to the current PTV technology, three to four cameras are often used to shoot with a certain parallax, which will lead to problems such as high camera cost, complex camera system, and inability to adapt to small environments. Conventional PIV technology cannot directly calculate the three-dimensional position and velocity distribution of particles, and has low spatial resolution. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a three-dimensional particle tracking method based on three-color mask three-dimensional imaging. By using a three-color mask and a single-color camera, a smaller, lower-cost, and simpler-structured shooting device is realized; by using PTV technology, the spatial resolution of particle tracking is improved.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A three-dimensional particle tracking method based on three-color mask three-dimensional imaging, comprising:

[0007] An image acquisition device is constructed; the image acquisition device comprises: a laser generator, a three-color mask, and a single-color camera; the three-color mask comprises: a red light-through hole, a blue light-through hole, a green light-through hole, and a circular light-proof mask; the red light-through hole, the blue light-through hole, and the green light-through hole are distributed in an equilateral triangle on the circular light-proof mask; the three-color mask is arranged in front of the lens of the single-color camera; the laser generator is used to irradiate the tracer particles in the target flow field; the single-color camera is used to use the three-color mask to capture the three-primary-color image of the tracer particles in the target flow field;

[0008] Using the image acquisition device to photograph the tracer particles in the target flow field to obtain an original photographed image;

[0009] Performing three-primary color channel separation on the original captured image to obtain a channel-separated image;

[0010] Performing demosaicing, hole interpolation and color crosstalk correction on all the channel-separated images to obtain preprocessed images;

[0011] Use the IPR algorithm to perform three-dimensional reconstruction and trajectory fitting on the pre-processed images of the first four frames to obtain the particle trajectories of the first four frames;

[0012] Using the Wiener filter, the particle positions of the particle trajectories in the n-1th frame are predicted in the nth frame to obtain the nth group of particle prediction position data;

[0013] Performing jitter prediction and projection on the nth group of predicted particle position data to obtain the nth group of particle reprojection images;

[0014] Calculating the local range residual of the nth group of particle reprojection images and the nth frame of preprocessed image;

[0015] The local range residual is screened by using the steepest descent method to obtain the minimum residual, and the particle reprojection image corresponding to the minimum residual is determined as the n-th frame prediction image;

[0016] Subtracting the predicted image of the nth frame from the preprocessed image of the nth frame by using an IPR algorithm to obtain an nth group of particles without fitting trajectories;

[0017] Performing trajectory fitting on the particles without fitted trajectories from the n-3th group to the nth group to obtain the particle trajectories of the nth group;

[0018] Using a Wiener filter to predict the particle trajectories of the nth group, and obtaining predicted position data of the particles of the n+1th group;

[0019] The particle predicted position data is iterated to obtain particle position and velocity information.

[0020] Preferably, it also includes:

[0021] The particle trajectory is subjected to inverse time series fitting and optimization, and the particle trajectory is updated using the obtained optimization result.

[0022] Preferably, performing demosaicing, hole interpolation and color crosstalk correction on all the channel-separated images to obtain preprocessed images includes:

[0023] Using the red light through hole, the blue light through hole, and the green light through hole separately to photograph the tracer particles in the target flow field to obtain a single light through hole image;

[0024] A color crosstalk calibration coefficient is calculated according to the single light hole image, and the channel separation image is corrected and ghost pixels are removed using the color crosstalk calibration coefficient to obtain the preprocessed image.

[0025] Preferably, an image acquisition device is constructed, comprising:

[0026] Placing a flat calibration plate at the center of the square area illuminated by the laser generator;

[0027] The plane calibration plate is moved up and down along the depth direction to obtain two-dimensional position information;

[0028] The two-dimensional position information is used to determine the conversion relationship between the two-dimensional image coordinates and the three-dimensional world coordinates, and the parameters of the single color camera are volume calibrated.

[0029] Preferably, the radius of the outer circle of the equilateral triangle formed by the circles of the red light through hole, the blue light through hole and the green light through hole is 32 mm; the radius of the red light through hole, the blue light through hole and the green light through hole is 5 mm.

[0030] Preferably, the light emission wavelengths of the laser generator include: 450nm, 532nm and 650nm respectively.

[0031] Preferably, the de-mosaicing hole interpolation method includes any one of dual three-color interpolation, bilinear interpolation, pattern recognition interpolation and deep learning interpolation.

[0032] Preferably, the parameters of the single color camera include: magnification factor, rotation and translation matrix, off-axis offset, and tangential and radial distortion.

[0033] The present invention discloses the following technical effects:

[0034] The present invention provides a three-dimensional particle tracking method based on three-color mask three-dimensional imaging. By using a three-color mask and a single-color camera, the defect that conventional equipment requires multiple cameras is solved, and a shooting device with smaller volume, lower cost and simpler structure is realized. By using PTV technology, the defect of poor accuracy caused by the use of particle average value in conventional PIV technology is solved, and particle tracking with high spatial resolution is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 A schematic diagram of a three-dimensional particle tracking process based on three-color mask three-dimensional imaging provided by an embodiment of the present invention;

[0037] Figure 2 A three-dimensional particle tracking flow chart provided for an embodiment of the present invention;

[0038] Figure 3 A flow chart of a STB provided by an embodiment of the present invention;

[0039] Figure 4 A schematic diagram of a calibration plate and a calibration method provided in an embodiment of the present invention;

[0040] Figure 5 A schematic diagram of a circular light-proof mask provided in an embodiment of the present invention;

[0041] Figure 6 A schematic diagram of a three-color mask flow field measurement principle provided by an embodiment of the present invention;

[0042] Figure 7 An IPR flow chart provided for an embodiment of the present invention;

[0043] Figure 8 A schematic diagram of a three-color mask three-dimensional imaging system provided by an embodiment of the present invention;

[0044] Fig. 9 A diagram of a flow around a cylinder provided in an embodiment of the present invention;

[0045] Fig.10 A flow chart of color channel separation and demosaicing provided by an embodiment of the present invention;

[0046] Fig.11 A color crosstalk correction flow chart provided by an embodiment of the present invention;

[0047] Fig.12 The R provided in the embodiment of the present invention f With R g Result graph;

[0048] Fig.13 D provided in the embodiment of the present invention f Result graph. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] The purpose of the present invention is to provide a three-dimensional particle tracking method based on three-color mask three-dimensional imaging, and to realize a shooting device with smaller volume, lower cost and simpler structure by using a three-color mask and a single-color camera; and to improve the spatial resolution of particle tracking by using PTV technology.

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Figure 1 A schematic diagram of a three-dimensional particle tracking process based on three-color mask three-dimensional imaging provided by an embodiment of the present invention, Figure 2 A three-dimensional particle tracking flow chart provided by an embodiment of the present invention, such as Figure 1 and Figure 2 As shown, the present invention provides a three-dimensional particle tracking method based on three-color mask three-dimensional imaging, comprising:

[0053] Step 100: construct an image acquisition device; the image acquisition device comprises: a laser generator, a three-color mask, and a single-color camera; the three-color mask comprises: a red light hole, a blue light hole, a green light hole, and a circular light-proof mask; the red light hole, the blue light hole, and the green light hole are distributed in an equilateral triangle on the circular light-proof mask; the three-color mask is arranged in front of the lens of the single-color camera; the laser generator is used to irradiate the tracer particles in the target flow field; the single-color camera is used to use the three-color mask to capture the three-primary color image of the tracer particles in the target flow field;

[0054] Step 200: photographing the tracer particles in the target flow field using an image acquisition device to obtain an original photographed image;

[0055] Step 300: performing three-primary color channel separation on the original captured image to obtain a channel-separated image;

[0056] Step 400: performing demosaicing, hole interpolation and color crosstalk correction on all channel-separated images to obtain preprocessed images;

[0057] Step 500: Use the IPR algorithm to perform three-dimensional reconstruction and trajectory fitting on the first four frames of pre-processed images to obtain the first four frames of particle trajectories;

[0058] Step 600: using the Wiener filter to predict the particle position of the particle trajectory of the n-1th frame in the nth frame, and obtain the nth group of particle prediction position data;

[0059] Step 700: performing jitter prediction and projection on the nth group of particle predicted position data to obtain a reprojected image of the nth group of particles;

[0060] Step 800: Calculate the local range residual between the n-th group of particle reprojection images and the n-th frame preprocessed image;

[0061] Step 900: using the steepest descent method to screen the local range residuals to obtain the minimum residual, and determining the particle reprojection image corresponding to the minimum residual as the n-th frame prediction image;

[0062] Step 1000: using the IPR algorithm to subtract the n-th frame prediction image from the n-th frame preprocessed image to obtain the n-th group of unfitted trajectory particles;

[0063] Step 11000: performing trajectory fitting on the particles of the n-3th group to the nth group without fitting trajectories, and obtaining the trajectory of the nth group of particles;

[0064] Step 1200: using the Wiener filter to predict the trajectory of the nth group of particles, and obtaining the predicted position data of the n+1th group of particles;

[0065] Step 1300: Iterate the particle predicted position data to obtain particle position and velocity information.

[0066] refer to Figure 3 , also includes:

[0067] The particle trajectory is subjected to inverse time series fitting and optimization, and the particle trajectory is updated using the obtained optimization result.

[0068] Specifically, all channel-separated images are subjected to demosaicing, hole interpolation and color crosstalk correction to obtain preprocessed images, including:

[0069] The red light hole, the blue light hole and the green light hole are used separately to photograph the tracer particles in the target flow field to obtain a single light hole image;

[0070] The color crosstalk calibration coefficient is calculated according to the single aperture image, and the channel separation image is corrected and ghost pixels are removed by using the color crosstalk calibration coefficient to obtain a preprocessed image.

[0071] refer to Figure 4 , construct an image acquisition device, including:

[0072] Place the flat calibration plate at the center of the square area illuminated by the laser generator;

[0073] Move the plane calibration plate up and down along the depth of field direction to obtain two-dimensional position information;

[0074] The two-dimensional position information is used to determine the conversion relationship between the two-dimensional image coordinates and the three-dimensional world coordinates, and the parameters of the single color camera are volume calibrated.

[0075] refer to Figure 5 The radius of the outer circle of the equilateral triangle formed by the red light-through hole, the blue light-through hole and the green light-through hole is 32 mm; the radius of the red light-through hole, the blue light-through hole and the green light-through hole is 5 mm.

[0076] Specifically, the light emission wavelengths of the laser generator include: 450nm, 532nm and 650nm.

[0077] Preferably, the demosaic hole interpolation method includes any one of dual three-color interpolation, bilinear interpolation, pattern recognition interpolation and deep learning interpolation.

[0078] Preferably, the parameters of the single color camera include: magnification factor, rotation and translation matrix, off-axis offset, and tangential and radial distortion.

[0079] Specifically, a three-color mask is used to obtain an image of particle motion, and then a PTV-related method is used to track the particle motion, including:

[0080] S1: In the measurement flow field where tracer particles are scattered, illuminate with a high-energy white LED or halogen lamp light source, and use a three-color mask single color camera to obtain a time-series particle three-color mask color image of the tracer particles in the measured flow field (reference Figure 4 ).

[0081] S2: Perform three-view image separation on the time-series particle three-color mask color image. This process includes three-view image separation on the red, green and blue color channels of the color image (refer to Figure 6 ); use dual three-color interpolation, bilinear interpolation, pattern recognition interpolation or deep learning interpolation method to demosaic the separated original image and fill the void interpolation; perform color crosstalk correction on the three-view image to remove the interference of other channel colors on the target channel.

[0082] S3: Use the IPR algorithm to process the first four frames of the sequence to obtain the three-dimensional reconstructed position of the particles, and use the velocity prediction method to fit the first four frames of trajectory. The content of the IPR algorithm is: first, identify the two-dimensional pixel coordinates of the particles in the two-dimensional particle image of each view; then, use the triangulation method to determine the distribution of the particles in the three-dimensional world coordinate system at different camera viewpoints; then delete the known particle projection on the two-dimensional particle image, and continue to repeat the above process on the residual image with the particle projection deleted until all iterations are completed (refer to Figure 7 ).

[0083] S4: Based on the existing trajectory prediction, use the Wiener filter to predict the particle position of the next frame of the trajectory, and project the predicted position into the camera image for comparison with the particle position of the camera image, and adjust the predicted position by jitter. Finally, re-use the IPR method in the residual image without the predicted particles, reconstruct the particles without fitting the trajectory in three dimensions, and try to fit the new trajectory with the previous particles (refer to Figure 3 ).

[0084] S5: After completing the trajectory fitting of the forward time series, the reverse time series is used to supplement the trajectory and verify the correctness of the trajectory. This step can extend the trajectory to the beginning of the time series. In this way, the movement of particles in the flow field can be accurately tracked using only a high-speed camera (refer to Figure 3 ).

[0085] In S1, the use of a three-color mask to obtain multiple viewing angles is achieved by changing the depth, and the angles of light of different colors passing through the filter are different, resulting in a certain deviation in the three colors projected on the camera. In S2, to remove the color crosstalk of the color channel, it is necessary to take a single light hole picture that blocks two light holes, and then use the first-order fitting method to linearly relate the light intensity of this light hole to other color channels, thereby achieving color crosstalk correction. In S3, the three-dimensional particle position obtained by triangulation and the three-dimensional position of the predicted frame particles in S4 and S5 need to be optimized by dithering. The specific process of dithering is: the three-dimensional coordinates of the known particles and the two-dimensional particle image are changed by ±1 in the three-dimensional world coordinates of X, Y, and Z respectively, and the residual of the particle projection image after the coordinate change and the original two-dimensional image is obtained, and a relationship curve between the residual and the dithering displacement is fitted, and finally the coordinate with the smallest residual in the relationship curve is obtained, and this coordinate is regarded as the correct three-dimensional particle coordinate.

[0086] refer to Figure 3 , the STB process is as follows:

[0087] Initial stage (first four frames)

[0088] 1) For the first four frames of three-view two-dimensional particle images, the IPR method is used to reconstruct the three-dimensional particle distribution of the four frames;

[0089] 2) Confirm the trajectory of the first four frames using the nearest neighbor, minimum acceleration, minimum jerk, or velocity field interpolation method;

[0090] Convergence phase (subsequent frames)

[0091] 3) Based on the existing trajectories of the first four frames, use the Wiener filter to predict the positions of these trajectories in the next frame;

[0092] 4) Reproject the predicted position into three perspectives, calculate the residual with the captured particle image in the local range, and fine-tune the predicted 3D position and intensity of the particle through the "jitter" method. Confirm the 3D position and intensity of the particle with the smallest residual as the particle with the known trajectory in the next frame;

[0093] 5) Reproject the predicted particles with known trajectories into three perspectives, and use the captured images of the predicted frames to subtract the projected images to obtain residual images, which are the images of particles that have not completed trajectory fitting;

[0094] 6) The residual image is reconstructed using the IPR method, and the reconstructed three-dimensional particles are tried to fit with the particles in the previous frame to obtain a new trajectory;

[0095] 7) The new trajectory is added to the known trajectory and used for the prediction of the particle position of the next frame until the particle trajectory fitting of all time frames is completed along the time series, and the wrong trajectory is removed by particle intensity, trajectory linear fitting check, etc.

[0096] Optionally, reverse the time series (for mid-supplementation trajectories):

[0097] Through the Wiener filter, the particle position of the previous frame of the supplementary trajectory is predicted;

[0098] Projecting to three viewing angles, the residual between the projected image and the image captured in the previous frame is calculated locally, and the correct position of the predicted particles is confirmed using the "jittering" method;

[0099] Continue to iterate forward to supplement the length of the supplementary trajectory.

[0100] refer to Figure 7 , the IPR process is as follows:

[0101] S1: Input two-dimensional particle images from three viewing angles and use the peak detection method to confirm the two-dimensional position of the particles.

[0102] S2: Using the triangulation method, project the line between the target particle and the camera center in the first perspective to the second perspective. The particles within a certain range of the projection line in the second perspective are regarded as candidate particles of the target particles. Use the same steps to confirm the line, project the line between the two perspectives to the third perspective to confirm the intersection of the two lines, and regard the particles within a certain range of the intersection as candidate particles of the target particles. Determine the two-dimensional target particle coordinates and three-dimensional particle coordinates corresponding to the three perspectives through the screening method.

[0103] S3: Through the "jittering" method, the XYZ three-dimensional coordinates and intensity of the particles are fine-tuned respectively, and the fine-tuned positions are projected into the original input image. The residual between the projected particle image and the original input particle image is calculated in the local range, and then the steepest descent method is used to remove the three-dimensional coordinates and intensity of the particles with the smallest residual.

[0104] S4: By setting a particle intensity threshold, the three-dimensional particles below the threshold are regarded as ghost particles and deleted; S3 and S4 steps are repeated multiple times to complete the accurate reconstruction of some three-dimensional particles.

[0105] S5: Reproject the reconstructed three-dimensional particles into three perspectives, use the original input particle image to subtract the projection image of the reconstructed particles, obtain the residual image without the reconstructed particles, and use these residual images as the input images of the next outer loop.

[0106] Among them, S1 to S5 are an outer loop; S3 and S4 are an inner loop; the outer loop from S1 to S5 is executed multiple times to ensure that the overlapping particle images complete the three-dimensional reconstruction.

[0107] Preferably, a three-color mask three-dimensional imaging system is constructed, and a cylindrical flow experimental device is built: the experiment uses a laser that can emit three bands simultaneously, the laser beam power is 3W, and the wavelengths are 450nm, 532nm and 650nm respectively. The customized three-color mask is installed in front of the ZEISS Milvus 2 / 100M lens and connected to the RevealerM120 color camera. The parameters for constructing the three-color mask are: the diameter of a single mask hole, the magnification, and the radius of the circumscribed circle of the mask hole (hole spacing), where the magnification is expressed as the ratio of the image distance to the object distance. The three-dimensional imaging system is as follows: Figure 8 As shown in Figure 2, the red dotted line represents the circumscribed circle of the mask hole. Fig. 9As shown in the figure, the cylindrical flow system consists of an inner circle, an outer circle and a bottom surface, wherein the outer wall radius of the outer circle is 250 mm, the outer wall radius of the inner circle is 190 mm, the thickness of the outer circle and the inner circle are both 5 mm, and the height is 100 mm. The bottom surface is a solid cylinder with a radius equal to the outer wall radius of the outer circle and a height of 10 mm. A cylinder with a diameter of 2 mm is placed in the middle of the water tank composed of the inner and outer circles as a flow cylinder. A stepper motor equipped with a planetary reducer is used as the power source for the rotation of the water tank, and the speed is controlled to 1.25RPM. The laser is irradiated perpendicular to the outer wall of the outer circle to the area at a certain distance behind the cylinder, and the camera is placed under the bottom surface of the water tank to shoot the particle movement upward. Polyamide tracer particles are used with a diameter of 20μm and a density of 1.03g / cm3. The particle concentration is about 0.04PPP.

[0108] Further, color channel separation and demosaicing processing: Fig.10 As shown in the figure, the captured original image is separated by color channel to obtain three color-separated original images. Use dual three-color interpolation, bilinear interpolation, pattern recognition interpolation or deep learning interpolation method to perform de-mosaic hole interpolation on the separated original image. From bottom to top: The first picture is a color image captured by a single CMOS color camera. Since a single CMOS camera has only a single color sensor on each pixel, each pixel in the image has only one of the three colors of red, green and blue. RGB represents the color of the corresponding position. The second row of images decomposes the captured color image into three images based on the color channel. When there is a corresponding color sensor at the pixel position, the pixel position has the corresponding color, otherwise the pixel position is black. RGB is used to represent the position and color of the corresponding color, and black is used to represent the pixel position without the corresponding color. These positions are also called mosaic holes. The third row of images is the color intensity of the area with existing color, and the separated mosaic holes are assigned. The last row is to merge the images of the three color channels that have completed de-mosaicing, so that each pixel of the merged color image has three colors of red, green and blue.

[0109] Preferably, color crosstalk correction: take a single light hole image with two light holes blocked, and use the images of the three light holes as calibration images for color crosstalk correction. Fig.11 As shown in the figure, after obtaining the calibration coefficient of color crosstalk through the calibration image, the separated single-channel original image is corrected for color crosstalk to remove ghost pixels. The horizontal axis in the image represents the particle concentration PPP, which is particles per pixel, indicating the number of particles per pixel; the vertical axis of the first image is the percentage, and the vertical axis of the second image is the distance, in pixels. f It indicates the ratio of real particles that can be reconstructed within a voxel to all real particles; R gIt indicates the proportion of reconstructed particles that do not exist in a voxel range to all reconstructed particles; D f is calculated in R f The distances between the projection positions of the matched real particles and the reconstructed particles in the three perspectives are calculated, and the average distances of all matched particle pairs in each perspective are calculated. The formula is as follows:

[0110]

[0111] Where N represents the number of matching particles, x iR represents the projection position x of the real particle at the i-th viewing angle iS Represents the projection position of the reconstructed particle at the i-th viewing angle.

[0112] R f It indicates the ratio of real particles that can be reconstructed within a voxel to all real particles; R g Indicates the ratio of reconstructed particles that do not exist in a voxel range to all reconstructed particles

[0113]

[0114] Among them, N f N represents the number of real particles that can exist in a voxel range; r Represents the number of all real particles; N g Indicates the number of reconstructed particles within a voxel where no real particles exist; N c Represents the number of all reconstructed particles.

[0115] refer to Fig.11 , RGB represents the three colors red, green and blue respectively. It is used to calculate the interference of blue light on the red channel when there is only blue light, and calculate the photosensitivity of the red channel in the image illuminated by blue light only. Where f represents the calibration function, which is used to quantify the color crosstalk intensity between different color layers; I B Indicates the current intensity of blue light; Indicates intensity I B The number of pixels; Indicates intensity I B The red channel intensity of pixel (i, j). Used to record the current intensity of blue light. By obtaining the current intensity of blue light I B The red channel corresponding to the current blue light intensity and green channel Intensity can determine the first-order linear relationship between the intensity of color crosstalk generated on the red and green channels and the intensity of blue light when there is only blue light. This can be deduced for other lighting conditions. Rread Represents the red channel separated image in the captured experimental image, R cor , G cor and B cor are the target values ​​for color crosstalk correction. They are unknown numbers. read , G read and B read The linear relationship between data and color crosstalk is solved to remove the R of color crosstalk. cor , G cor and B cor .

[0116] Specifically, volume calibration: place the plane calibration plate at the center of the laser irradiated square area. Since the two-dimensional position information of the plane calibration plate is known, move the plane calibration plate up and down along the depth direction to determine the two-dimensional position information in different depth directions. The three-dimensional position of the calibration can be determined through the known two-dimensional position information and the position in the depth direction. Combined with the known two-dimensional image coordinates, the conversion relationship between the two-dimensional image coordinates and the three-dimensional world coordinates can be calculated. In this process, it is necessary to determine the camera's magnification factor, the rotation and translation matrix of each viewing angle (used to determine the conversion relationship between two-dimensional coordinates and three-dimensional coordinates), the camera's off-axis offset, the camera's tangential and radial distortion and other parameters. The situation of the calibration plate and the calibration process are as follows: Figure 4 shown.

[0117] Preferably, in the initial stage of particle tracking, for the first four frames of particle images captured, after color channel separation, demosaicing and color crosstalk correction, the particle trajectory is preliminarily fitted using an iterative particle reconstruction (IPR) method, and the parameters of 5 external iterations, 8 internal iterations, a two-dimensional particle search range of 6 pixels, a ghost particle intensity threshold of 0.03, a two-dimensional line of sight search range of 0.15 voxels, and a three-dimensional line of sight search range of 0.4 voxels are selected.

[0118] Furthermore, the convergence stage of particle tracking: after completing the trajectory fitting in the initial stage, the subsequent frame particle images are also separated by color channels, de-mosaiced and corrected for color crosstalk, and the "shaking box" (STB) method is used to predict the position of the existing trajectory through the Wiener filter, and the iterative particle reconstruction (IPR) method is continued to be used to reconstruct the three-dimensional position of the particles without fitting trajectories, and try to fit the particle trajectory. A jitter range of 0.5 voxels, a maximum single-frame displacement of 4 voxels, a particle intensity subtraction factor of 1.3 times, and a ghost particle intensity threshold of 0.001 are selected. In this way, the position and velocity information of the particles in three-dimensional space can be obtained.

[0119] Furthermore, in order to verify the effect of the particle tracking velocimetry (PTV) technology of the three-color mask single-color camera, this embodiment uses the commonly used evaluation index of the PTV technology to illustrate the particle tracking effect of the synthetic flow field.

[0120] Optionally, the three-color mask camera used for the composite image has a resolution of 1280*1024 pixels. 2 , the camera area used is 1240*984pixel 2 , the pixel size is 0.004mm, and the radius of the circumscribed circle of the center of the three mask holes is 32mm. The volume calibration image is generated by ray tracing technology, and the three-dimensional tracer particles are projected onto the two-dimensional image plane to obtain the three-view calibration information of the three-color mask. Subsequently, a group of particles are randomly generated in the measurement volume, and the projection of the particles at each viewing angle is calculated by the camera calibration information to determine the image coordinates of the center of the particle. Gaussian distribution is applied to these coordinates to generate a particle image with a diameter of 3 pixels. Finally, the particle field is moved frame by frame according to the set motion form. The particle concentration of the synthetic image of this embodiment ranges from 0.005PPP (particles per-pixel) to 0.05PPP, and the particles move at a uniform speed in a straight line along the y direction and perform sinusoidal motion in the z direction according to the position of the current y coordinate.

[0121] Furthermore, to evaluate the effect of PTV technology, the extraction efficiency of particles R f , is there a reconstructed particle within the range of one voxel of the real particle? If there is a reconstructed particle, it is considered that the real particle has been matched, then R f represents the proportion of matching real particles to all real examples; then the proportion of ghost particles R g If there is no real particle within a voxel of a reconstructed particle, the reconstructed particle is considered to be a ghost particle, R f represents the ratio of ghost particles to all reconstructed particles; finally, the distance D between the matched real particles and the reconstructed particles is used f , Fig.12 , 13 The R of the three-color mask single color camera PTV is respectively described as follows: f , R g and D f Effect:

[0122] Specifically, through Fig.12 It can be seen that PTV using a three-color mask and a single-color camera can find more than 85% of real particles at a particle concentration of 0.05PPP, and the proportion of ghost particles is less than 15%. Fig.13 It can be seen that the three-dimensional space distance between the matched real particles and the reconstructed particles is less than 0.3 voxels, which means that the accuracy of particle matching is high.

[0123] The beneficial effects of the present invention are as follows:

[0124] The present invention realizes a shooting device with smaller volume, lower cost and simpler structure by using a three-color mask and a single-color camera; and improves the spatial resolution of particle tracking by using the PTV technology.

[0125] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0126] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A three-dimensional particle tracking method based on three-color mask three-dimensional imaging, characterized in that: include: Constructing an image acquisition device; The image acquisition device comprises: a laser generator, a three-color mask, and a single-color camera; the three-color mask comprises: a red light hole, a blue light hole, a green light hole, and a circular light-proof mask; the red light hole, the blue light hole, and the green light hole are distributed in an equilateral triangle on the circular light-proof mask; the three-color mask is arranged in front of the lens of the single-color camera; the laser generator is used to irradiate the tracer particles in the target flow field; the single-color camera is used to use the three-color mask to capture the three-primary color image of the tracer particles in the target flow field; Using the image acquisition device to photograph the tracer particles in the target flow field to obtain an original photographed image; Performing three-primary color channel separation on the original captured image to obtain a channel-separated image; Performing demosaicing, hole interpolation and color crosstalk correction on all the channel-separated images to obtain preprocessed images; Use the IPR algorithm to perform three-dimensional reconstruction and trajectory fitting on the pre-processed images of the first four frames to obtain the particle trajectories of the first four frames; Using the Wiener filter, the particle positions of the particle trajectories in the n-1th frame are predicted in the nth frame to obtain the nth group of particle prediction position data; Performing jitter prediction and projection on the nth group of predicted particle position data to obtain the nth group of particle reprojection images; Calculating the local range residual of the nth group of particle reprojection images and the nth frame of preprocessed image; The local range residual is screened by using the steepest descent method to obtain the minimum residual, and the particle reprojection image corresponding to the minimum residual is determined as the n-th frame prediction image; Subtracting the predicted image of the nth frame from the preprocessed image of the nth frame by using an IPR algorithm to obtain an nth group of particles without fitting trajectories; Performing trajectory fitting on the particles without fitted trajectories from the n-3th group to the nth group to obtain the particle trajectories of the nth group; Using a Wiener filter to predict the particle trajectories of the nth group, and obtaining predicted position data of the particles of the n+1th group; The particle predicted position data is iterated to obtain particle position and velocity information.

2. The three-dimensional particle tracking method based on three-color mask three-dimensional imaging according to claim 1, characterized in that: Also includes: The particle trajectory is subjected to inverse time series fitting and optimization, and the particle trajectory is updated using the obtained optimization result.

3. The three-dimensional particle tracking method based on three-color mask three-dimensional imaging according to claim 1, characterized in that: De-mosaicing, hole interpolation and color crosstalk correction are performed on all the channel-separated images to obtain pre-processed images, including: Using the red light through hole, the blue light through hole, and the green light through hole separately to photograph the tracer particles in the target flow field to obtain a single light through hole image; A color crosstalk calibration coefficient is calculated according to the single light hole image, and the channel separation image is corrected and ghost pixels are removed using the color crosstalk calibration coefficient to obtain the preprocessed image.

4. The three-dimensional particle tracking method based on three-color mask three-dimensional imaging according to claim 1, characterized in that: Construct an image acquisition device, including: Placing a flat calibration plate at the center of the square area illuminated by the laser generator; The plane calibration plate is moved up and down along the depth direction to obtain two-dimensional position information; The two-dimensional position information is used to determine the conversion relationship between the two-dimensional image coordinates and the three-dimensional world coordinates, and the parameters of the single color camera are volume calibrated.

5. The three-dimensional particle tracking method based on three-color mask three-dimensional imaging according to claim 1, characterized in that: The radius of the outer circle of the equilateral triangle formed by the red light through hole, the blue light through hole and the green light through hole is 32 mm; the radius of the red light through hole, the blue light through hole and the green light through hole is 5 mm.

6. The three-dimensional particle tracking method based on three-color mask three-dimensional imaging according to claim 1, characterized in that: The light emission wavelengths of the laser generator include: 450nm, 532nm and 650nm.

7. The three-dimensional particle tracking method based on three-color mask three-dimensional imaging according to claim 1, characterized in that: The de-mosaicing hole interpolation method includes: any one of dual three-color interpolation, bilinear interpolation, pattern recognition interpolation and deep learning interpolation.

8. The three-dimensional particle tracking method based on three-color mask three-dimensional imaging according to claim 4, characterized in that: The parameters of the single color camera include: magnification factor, rotation and translation matrix, off-axis offset, and tangential and radial distortion.

Images