A continuous 3D particle field reconstruction method based on single-light field imaging and EM deconvolution.

Abstract

This invention relates to a continuous three-dimensional particle field reconstruction method based on single-light-field imaging using EM deconvolution, belonging to the field of optical imaging technology. This method employs a light field camera to capture light field images of a discretized three-dimensional flow field, acquiring the light intensity information of scattered light emitted by particles on each pixel of a CCD detector. Based on the light intensity information, a weight matrix for each voxel is calculated, and the entire weight matrix is ​​loaded only once. The nth frame of the light field image is loaded, and the non-zero pixels in the nth frame are identified. The non-zero pixel sequence formed by the scattered light from tracer particles is extracted. An EM deconvolution algorithm is used to iteratively calculate the non-zero pixel sequence, updating the intensity of each voxel, thereby reconstructing the three-dimensional particle field of a single frame. The above steps are repeated to complete the reconstruction of the three-dimensional particle field from multiple frames of high temporal resolution light field images, resulting in higher reconstruction accuracy and consistency, and less time consumption.

Description

Technical Field

[0001] This invention belongs to the field of optical imaging technology, specifically relating to a method for reconstructing continuous three-dimensional particle fields based on single-field imaging and EM deconvolution. Background Technology

[0002] Unsteady flows within confined spaces are prevalent in critical fields such as aerospace, automotive engineering, and energy and power, and their flow characteristics directly affect the efficiency, stability, and safety of equipment. The velocity field of unsteady flows within confined spaces is one of the important parameters characterizing various unsteady coherent flow topologies. Achieving high-precision, high-efficiency, non-invasive measurement and multi-dimensional quantitative characterization of the velocity field of unsteady flows within confined spaces can reveal the spatiotemporal evolution, scale distribution characteristics, and statistical properties of vortex structures within the flow field. It can also capture the evolution of complex flow phenomena such as turbulent fluctuations, separated flows, and vortex shedding over time, thus providing reliable experimental evidence and theoretical support for the design, optimization, and performance improvement of flow channel configurations, internal structures, and key components within confined spaces.

[0003] Tomographic particle image velocimetry (PTV) was first proposed by Elsinga GE in 2006, and further developed into Time-resolved Tomo-PIV (TR Tomo-PIV). This technique belongs to the three-dimensional three-directional velocity component (3D-3C) measurement method, capable of acquiring transient, high-precision, and high temporal resolution three-dimensional flow field velocity distributions. It is currently widely used in the measurement and analysis of three-dimensional velocity fields in complex flow fields. The basic principle of time-resolved tomo-PIV is as follows: Instantaneous images of tracer particles in the flow field are simultaneously captured by a multi-view high-speed camera; tomographic reconstruction algorithms are used to reconstruct the three-dimensional particle field from two-dimensional particle images at different angles; then, cross-correlation matching and velocity calculation are performed on the three-dimensional particle field at consecutive time points, ultimately obtaining a series of three-dimensional three-directional (3D-3C) transient velocity fields with high temporal resolution. This method can completely capture the spatiotemporal evolution of unsteady flow without disturbing the flow field. However, time-resolved tomo-PIV still has some inherent limitations. Its shortcomings are as follows: This technology requires multiple high-speed cameras, resulting in a large system cost and size. Multi-camera tomography PIV generally requires the flow field device under test to have at least two or more optical observation windows, or a sufficiently large area, in order to carry out flow measurement experiments. However, the optical window size for unsteady flow in a confined space is limited, which greatly increases the difficulty of setting up multiple cameras, making it difficult to carry out experimental measurements of unsteady flow. At the same time, sufficient installation space needs to be reserved around the flow field for arranging multiple cameras and other experimental equipment.

[0004] Unlike traditional cameras, light field cameras employ a microlens array positioned at a specific distance in front of the CCD detector. Object-side light rays converge onto the microlens array via the main lens and are then projected onto the CCD through secondary imaging, forming a series of macro-pixel images. Light field cameras can simultaneously acquire information on the light intensity, position, and propagation direction of tracer particles in a flow field within a single exposure. This feature allows a single light field camera to replace traditional multi-camera stereo imaging systems, enabling the acquisition of particle images of unsteady flows within confined spaces. This effectively solves the problems of multi-camera systems, such as difficulty in deployment in small spaces, limited field of view, and high synchronization accuracy requirements. Tomographic reconstruction of the three-dimensional particle field is a prerequisite for obtaining the three-dimensional velocity field. Commonly used three-dimensional particle field reconstruction algorithms based on light field imaging include the Multiplicative Algebraic Reconstruction Technique (MART), the Simultaneous Algebraic Reconstruction Technique (SART), and the Expectation Maximization (EM) algorithm. However, single-light-field cameras generally suffer from computational overhead and large data storage requirements in 3D particle field reconstruction, which limits the reconstruction efficiency of 3D velocity fields. Therefore, the pre-identification-based SART (PR-SART) algorithm, the dense ray tracing-based MART reconstruction method (DRT-MART), and the Multiplicative First Guess-based EM (MFG-EM) algorithm are further applied to 3D particle field reconstruction to improve the efficiency of tomographic reconstruction and reduce computer storage space. However, existing 3D particle field reconstruction methods still have significant shortcomings: fine analysis of unsteady flows typically requires high temporal resolution light field image sequences, with data volumes reaching thousands or even tens of thousands of frames. If the traditional reconstruction algorithms of existing light field tomography (PIV) are directly used to process such high temporal resolution data, the overall computation time often reaches tens of thousands of hours, which is not only extremely computationally expensive but also difficult to meet the real-time and engineering application requirements of actual flow measurements. Summary of the Invention

[0005] The purpose of this invention is to provide a method for reconstructing continuous three-dimensional particle fields based on single-field imaging using EM deconvolution, in order to solve the problem of low efficiency in reconstructing light field image sequences using existing EM deconvolution algorithms.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] This invention relates to a method for reconstructing a continuous three-dimensional particle field based on single-field imaging and EM deconvolution, comprising the following steps:

[0008] S1. A light field camera is used to capture light field images of a discretized three-dimensional flow field. A microlens array is installed between the CCD detector and the main lens of the light field camera to acquire the light intensity information of the scattered light emitted by the particles on each pixel of the CCD detector. The weight matrix of each voxel is calculated based on the light intensity information. Load the entire weight matrix exactly once. The entire weight matrix This is the weight matrix for all voxels;

[0009] S2. Load the light field image of the nth frame, with n initially set to 1;

[0010] S3. Find and remove the non-zero pixels in the nth frame of the light field image, and then extract the non-zero pixel sequence formed by the scattered light of the tracer particles;

[0011] S4. The EM deconvolution algorithm is used to iteratively calculate the non-zero pixel sequence, update the intensity of each pixel, and then reconstruct the three-dimensional particle field of a single frame, let n+1;

[0012] S5. Determine if the current frame rate has reached the preset total frame rate. If not, return to S2; if yes, end the reconstruction.

[0013] Preferably, in step S1, when the discretized three-dimensional flow field is captured, the main lens converges the object-side light rays of the tracer particles in the discretized three-dimensional flow field to the microlens array. After secondary imaging by the microlens array, the light rays are projected onto the CCD detector to form the original light field image. In each frame of the light field image, each microlens of the microlens array corresponds to a macro image. The macro image records the position information of the particles, the pixel position in the macro image records the particle orientation information, and the pixel grayscale records the particle light intensity information. That is, the light intensity information of the scattered light emitted by the particles on each pixel of the CCD detector is obtained.

[0014] Preferably, the formula for calculating the weight matrix of each voxel to each pixel of the light field camera based on light intensity information in step S1 is as follows:

[0015] (1),

[0016] in, The scattered light emitted by the particle at the 1st Intensity per pixel For the discretized three-dimensional flow field, the first The intensity of individual elements, For the first In a light field camera, the first individual pixel is used to represent the first individual pixel. The contribution of each pixel is the weight matrix.

[0017] Preferably, the expression for finding the non-zero pixels in the nth frame light field image in step S3 is:

[0018] (2),

[0019] in, For the first The initial value of the intensity of the individual element is calculated using the following formula:

[0020] (3).

[0021] Preferably, in step S4, the iterative calculation formula of the EM deconvolution algorithm is:

[0022] (4),

[0023] in, For the first Individual element Strength after the next iteration The scattered light emitted by the particle at the 1st Intensity per pixel For the first In a light field camera, the first individual pixel is used to represent the first individual pixel. The contribution of each pixel, i.e., the weight matrix. For the first Individual elements corresponding to 1 pixel.

[0024] Preferably, in step S4, the iterative process of the EM deconvolution algorithm is as follows:

[0025] S4.1. According to Calculate the expected grayscale value for each pixel, and then compare the expected grayscale value with the intensity of the scattered light emitted by the actual particles on the pixel. A comparison is made, and the light intensity values ​​of the effective voxels are updated and optimized based on the comparison results.

[0026] S4.2. Update and optimize the light intensity value of the effective voxel based on Formula 4;

[0027] S4.3. Determine whether the number of iterations has reached the set threshold. If not, return to S4.1. If it has, complete the reconstruction of the single-frame 3D particle field.

[0028] Compared with the prior art, the technical solution provided by this invention has the following advantages:

[0029] 1. The present invention relates to a method for continuous three-dimensional particle field reconstruction based on single-light-field imaging using EM deconvolution. This method proposes for the first time a strategy for continuous multi-frame EM deconvolution reconstruction of light field image sequences, breaking through the limitation of existing technologies that can only process single-frame images. It can efficiently complete the three-dimensional particle field reconstruction of thousands or even tens of thousands of high temporal resolution light field images. Through the design of single loading and multi-frame reuse of the weight matrix, the redundant operation of repeatedly loading the ultra-large sparse weight matrix for each frame of image processed by existing algorithms is completely avoided, saving tens of thousands of hours of time in the matrix loading stage alone. At the same time, by pre-identifying particles to remove zero voxels, the number of voxels actually participating in the iteration is reduced to 10% to 30% of the original voxel space, solving the core problem that the computation time for processing high temporal resolution data in existing technologies is as high as tens of thousands of hours.

[0030] 2. The present invention relates to an EM deconvolution continuous three-dimensional particle field reconstruction method based on single light field imaging. The method calculates the weight matrix of each voxel to each pixel of the light field camera based on light intensity information. The calculated weight matrix is ​​a large sparse matrix. The single-load design greatly reduces the repeated occupation of server memory and storage. It does not require reserving multiple times the matrix storage space for multi-frame calculation, and is more friendly to the hardware requirements of computing devices.

[0031] 3. The present invention relates to an EM deconvolution continuous three-dimensional particle field reconstruction method based on single-light field imaging, which relies on a single light field camera to complete all imaging work, eliminating the need for a combination of multiple high-speed cameras and high-power lasers. The system is small in size and has low requirements for optical observation windows, completely solving the problems of difficult camera deployment, limited field of view, and the need to reserve a large amount of equipment installation space around the flow field in traditional tomographic PIV technology. At the same time, the algorithm's high temporal resolution reconstruction capability can accurately capture strong transient and strong nonlinear flow phenomena such as turbulent fluctuations, separated flows, and vortex shedding, realizing a fine characterization of the spatiotemporal evolution law of vortex structure in confined space flow fields. It provides a highly adaptable measurement method for confined space flow field research in fields such as aviation, aerospace, and energy power.

[0032] 4. The present invention relates to an EM deconvolution continuous three-dimensional particle field reconstruction method based on single-light field imaging. The optimization of the EM deconvolution algorithm only targets the reduction of the calculation range and the optimization of the initial value. It does not change the core iterative logic of the algorithm and the mathematical model of light field propagation. Through particle pre-identification, it only removes the interference of invalid voxels without particles, without simplifying the calculation of the voxel intensity of effective particles, thus ensuring the high accuracy of three-dimensional particle field reconstruction. At the same time, it uses a uniformly loaded weight matrix to complete the calculation of all frames throughout the process, avoiding the systematic errors caused by multiple loading of the matrix, making the reconstruction accuracy of particle fields of continuous multiple frames more consistent, and providing reliable and accurate basic data for subsequent calculation of high temporal resolution three-dimensional velocity fields through cross-correlation matching. Attached Figure Description

[0033] Figure 1 This is a flowchart of a continuous three-dimensional particle field reconstruction method based on single-light field imaging using EM deconvolution;

[0034] Figure 2 This is a schematic diagram illustrating the principle of a light field camera acquiring a discretized three-dimensional flow field light field image. Detailed Implementation

[0035] To further understand the content of this invention, the invention will be described in detail with reference to the embodiments. The following embodiments are used to illustrate the invention, but are not intended to limit the scope of the invention.

[0036] Example:

[0037] See attached document Figure 1 As shown, this invention relates to a method for reconstructing a continuous three-dimensional particle field based on single-field imaging using EM deconvolution, which includes the following steps:

[0038] S1. A light field camera is used to capture a discrete three-dimensional flow field light field image. A microlens array is provided between the CCD detector and the main lens of the light field camera to acquire the light intensity information of the scattered light emitted by the particles on each pixel of the CCD detector, such as... Figure 2 As shown, the main lens converges the object-side rays of the tracer particles in the discretized three-dimensional flow field to a microlens array. After secondary imaging by the microlens array, the light is projected onto a CCD detector, forming the original light field image. In each frame of the light field image, each microlens in the microlens array corresponds to a macro image. The macro image records the particle's position information, the pixel position records the particle's orientation information, and the pixel grayscale records the particle's light intensity information. In other words, it acquires the light intensity information of the scattered light emitted by the particles on each pixel of the CCD detector. The weight matrix of each voxel is calculated based on the light intensity information. The formula for calculating the weight matrix is:

[0039] (1),

[0040] in, The scattered light emitted by the particle at the 1st Intensity per pixel For the discretized three-dimensional flow field, the first The intensity of individual elements, For the first In a light field camera, the first individual pixel is used to represent the first individual pixel. The contribution of each pixel is the weight matrix.

[0041] A voxel is a unit of measurement that discretizes the flow field being measured into a series of small cubes.

[0042] Using the load function in Matlab to load the sparse weight matrix The entire weight matrix is ​​loaded into the memory of the computing server all at once. The weight matrix for all voxels is loaded only once throughout the entire process, and the reconstruction calculations for all subsequent frames of light field images are based on this loaded weight matrix. There is no reloading operation of any kind, which completely avoids the time and resource consumption caused by reloading the weight matrix.

[0043] S2. Load the nth frame of the light field image, where n = 1, 2, 3, ..., h, h is generally greater than 1000, and the initial value of n is 1. That is, first use the load function to load the first frame of the light field image; the light field image sequence is a high temporal resolution image taken by a single all-optical light field camera, with a total number of frames h ≥ 1000, which meets the temporal resolution requirements for fine analysis of unsteady flow fields in confined spaces.

[0044] S3. Find the non-zero pixels in the nth frame of the light field image. In the iterative calculation of the EM deconvolution algorithm, only non-zero voxels (voxels with tracer particles) actually contribute to the pixel light intensity value. The light intensity value of zero voxels (voxels without tracer particles) (Ej=0) will significantly increase the computational load, reduce the iteration efficiency, and even introduce computational noise. Therefore, before the EM deconvolution iteration, particle pre-identification technology is used to quickly identify and accurately screen the non-zero voxels in the three-dimensional voxel space, eliminate the interference of zero voxels, reduce the computational scale, and then extract the non-zero pixel sequence formed by the scattered light of the tracer particles. The expression for non-zero pixels is:

[0045] (2),

[0046] in, For the first The initial value of the intensity of the individual element is calculated using the following formula:

[0047] (3).

[0048] S4. The EM deconvolution algorithm is used to iteratively calculate the non-zero pixel sequence, update the intensity of each pixel, and then reconstruct the three-dimensional particle field of a single frame. The iterative calculation formula of the EM deconvolution algorithm is as follows:

[0049] (4),

[0050] in, For the first Individual element Strength after the next iteration The scattered light emitted by the particle at the 1st Intensity per pixel For the first In a light field camera, the first individual pixel is used to represent the first individual pixel. The contribution of each pixel, i.e., the weight matrix. For the first Individual elements corresponding to 1 pixel.

[0051] The iterative process of the EM deconvolution algorithm is as follows:

[0052] S4.1. According to Calculate the expected grayscale value for each pixel, and compare the expected grayscale value with the actual grayscale value of the pixels in the acquired light field image. The grayscale value of the light field image can be represented by the intensity of the scattered light emitted by the particles on the pixel. This means that the comparison results will determine whether to update and optimize the light intensity value of the effective voxel. For example, if the difference between the two values ​​is greater than the set threshold, then an update and optimization are required.

[0053] S4.2. Update and optimize the light intensity value of the effective voxel based on Formula 4;

[0054] S4.3. Determine whether the number of iterations has reached the set threshold. If not, return to S4.1. If it has, complete the reconstruction of the single-frame 3D particle field.

[0055] After the update, let n+1;

[0056] S5. Determine if the current frame rate has reached the preset total frame rate. If not, return to S2; if yes, end the reconstruction.

[0057] Example of results:

[0058] To verify the actual computational efficiency of this algorithm, a real-world test was conducted on a server with a 32-core Intel(R) Xeon(R) CPU E5-2696V4@2.2GHz and 128GB RAW. The core test parameters were set as follows: the 3D flow field voxel space was 117 (X-axis) × 117 (Y-axis) × 117 (Z-axis), the tracer particle concentration was 0.5ppm, and the EM deconvolution algorithm was preset to 10 iterations. The measured time consumption data for the core steps of the algorithm are as follows:

[0059] (1) Complete weight matrix One-time loading time: 2194 seconds;

[0060] (2) Time taken for non-zero pixel extraction and non-zero voxel recognition in a single frame image: 56 seconds;

[0061] (3) Calculation time for the weight matrix corresponding to non-zero voxels: 155 seconds;

[0062] (4) Iterative calculation time of EM deconvolution algorithm for a single frame image: 17 seconds;

[0063] (5) Overall reconstruction time of a single frame image (excluding loading of the entire weight matrix): 228 seconds.

[0064] The measured data shows that the method involved in this invention avoids the 2194-second time consumption of repeatedly loading the matrix for each frame of image by using the core design of single loading of the weight matrix. When processing 1000 frames of light field images, it can save 2,194,000 seconds of redundant loading time, achieving an exponential improvement in reconstruction efficiency.

[0065] The present invention has been described in detail above with reference to the embodiments, but the content described is only a preferred embodiment of the present invention and should not be considered as limiting the scope of the present invention. All equivalent changes and improvements made in accordance with the scope of the present invention should still fall within the patent coverage of the present invention.

Claims

1. A method for reconstructing a continuous three-dimensional particle field based on single-field imaging and EM deconvolution, characterized in that, It includes the following steps: S1. A light field camera is used to capture light field images of a discretized three-dimensional flow field. A microlens array is installed between the CCD detector and the main lens of the light field camera to acquire the light intensity information of the scattered light emitted by the particles on each pixel of the CCD detector. The weight matrix of each voxel is calculated based on the light intensity information. Load the entire weight matrix exactly once. The entire weight matrix This is the weight matrix for all voxels; S2. Load the nth frame of the light field image, with n initially set to 1; where the light field image sequence consists of high temporal resolution images captured by a single all-optical light field camera, with a total number of frames h ≥ 1000; S3. Find and remove the non-zero pixels in the nth frame of the light field image, and then extract the sequence of non-zero pixels formed by the scattered light from the tracer particles. The expression for finding the non-zero pixels in the nth frame of the light field image is: (2)， in, For the first The initial value of the intensity of the individual element; S4. The EM deconvolution algorithm is used to iteratively calculate the non-zero pixel sequence, update the intensity of each voxel, and then reconstruct the three-dimensional particle field of a single frame, let n+1; S5. Determine if the current frame rate has reached the preset total frame rate. If not, return to S2; if yes, end the reconstruction.

2. The method for reconstructing continuous three-dimensional particle fields based on single-field imaging using EM deconvolution according to claim 1, characterized in that: In step S1, when the discretized three-dimensional flow field is captured, the main lens converges the object-side light rays of the tracer particles in the discretized three-dimensional flow field to the microlens array. After secondary imaging by the microlens array, the light rays are projected onto the CCD detector to form the original light field image. In each frame of the light field image, each microlens of the microlens array corresponds to a macro image. The macro image records the position information of the particles, the pixel position in the macro image records the particle orientation information, and the pixel grayscale records the particle light intensity information. That is, the light intensity information of the scattered light emitted by the particles on each pixel of the CCD detector is obtained.

3. The method for reconstructing continuous three-dimensional particle fields based on single-field imaging using EM deconvolution according to claim 1, characterized in that: The formula for calculating the weight matrix of each voxel based on light intensity information in S1 is as follows: (1)， in, The scattered light emitted by the particle at the 1st Intensity per pixel For the discretized three-dimensional flow field, the first The intensity of individual elements, For the first In a light field camera, the first individual pixel is used to represent the first individual pixel. The contribution of each pixel is the weight matrix.

4. The method for reconstructing continuous three-dimensional particle fields based on single-field imaging using EM deconvolution according to claim 1, characterized in that: In S3, the initial value of the voxel intensity is calculated using the following formula: (3)。 5. The method for reconstructing a continuous three-dimensional particle field based on single-field imaging using EM deconvolution according to claim 1, characterized in that: In S4, the iterative calculation formula of the EM deconvolution algorithm is: (4)， in, For the first Individual element Strength after the next iteration The scattered light emitted by the particle at the 1st Intensity per pixel For the first In a light field camera, the first individual pixel is used to represent the first individual pixel. The contribution of each pixel, i.e., the weight matrix. For the first Individual elements corresponding to 1 pixel.

6. The method for reconstructing continuous three-dimensional particle fields based on single-field imaging using EM deconvolution according to claim 5, characterized in that: In S4, the iterative process of the EM deconvolution algorithm is as follows: S4.

1. According to Calculate the expected grayscale value for each pixel, and then compare the expected grayscale value with the intensity of the scattered light emitted by the actual particles on the pixel. A comparison is made, and the light intensity values ​​of the effective voxels are updated and optimized based on the comparison results. S4.

2. Update and optimize the light intensity value of the effective voxel based on formula (4); S4.

3. Determine whether the number of iterations has reached the set threshold. If not, return to S4.

1. If it has, complete the single-frame 3D particle field reconstruction.

Images