Flame 4D chemiluminescence rapid reconstruction method

Abstract

The invention provides a flame 4D chemiluminescence rapid reconstruction method, and belongs to the technical field of combustion flame visual detection. Comprising the following steps: 1, calibrating a camera; 2, shooting chemiluminescence views of the flame at different angles; step 3, segmenting voxels of a target area and calculating a weight matrix; step 4, training a self-defined full-connection neural network FCN; 5, extracting a network parameter which is a flow field reconstruction value; and 6, evaluating the reconstruction precision. According to the method, the occupied memory is only 773MB, multiple frames can be reconstructed at the same time, the weight matrix is shared, and the memory occupied by simultaneous reconstruction of the multiple frames is almost the same as that of single-frame reconstruction. According to the method, the average reconstruction time of each frame is 0.063 s, and 7.8 s is needed for performing the same reconstruction through a common SART algorithm. The reconstruction precision of the method is basically equivalent to that of SART, but is more stable, and the average reconstruction precision is slightly high.

Description

Technical Field

[0001] The invention provides a flame 4D chemiluminescence rapid reconstruction method, belonging to the technical field of combustion flame visualization detection. Background Art

[0002] By monitoring the flow field in the combustion chamber, we can understand the details of the combustion process, optimize the fuel and air mixing ratio, improve combustion efficiency and thrust output, reduce fuel consumption, and reduce the release of emissions. The four-dimensional reconstruction of the flame (usually involving three-dimensional flame characteristics in time and space) is of great significance. It can help people study the morphology, structure and evolution of the flame and gain a deeper understanding of the combustion process; through accurate reconstruction, problems in the combustion process can be identified, thereby optimizing fuel utilization and reducing emissions; in fire safety research, understanding flame dynamics helps assess fire risks and develop preventive measures. These significances help promote the advancement of combustion science and engineering.

[0003] Flame combustion is a complex process with a small characteristic time scale and fast change rate. High temporal resolution and high spatial resolution reconstruction are required to restore the three-dimensional structure and change law of the flow field, so the efficiency of reconstruction is very important for the four-dimensional reconstruction of flames.

[0004] The three-dimensional structure of the flame is obtained using computed tomography (CT) technology, which is essentially an inverse problem. By measuring the path integral of the target flow field at multiple different angles, the discrete estimation of the chemiluminescence intensity of the flow field is obtained by inversion.

[0005] Existing technologies, such as parallel computing: 4D imaging of turbulent flames with improved multi-camera calibration method and processing speed, have the following disadvantages:

[0006] 1. High hardware requirements, requiring a large number of cores and memory. Using only the Python module to mobilize multiple processes to run simultaneously, the information between different processes is not shared, so the memory requirements for parallel computing are also doubled accordingly.

[0007] 2. The reconstruction speed is still not fast enough. Although GPU acceleration is available, the large memory requirements make it difficult to implement on civilian GPUs.

[0008] Another example is VT-Net based on deep learning: Limited-projection volumetric tomography for time-resolved turbulent combustion diagnostics via deep learning. This method has the following disadvantages:

[0009] 1. A large amount of valid data needs to be prepared for training. In 3D reconstruction, the accurate flow field information is unknown, and the reconstruction results of other algorithms (such as ART) can only be used as approximations for network training.

[0010] 2. Training is required in advance. Although a trained network can achieve a reconstruction speed of milliseconds, it takes a long time to train the network. The more complex the network is, the longer the training time is, and the more memory is required to reconstruct a complex flame.

[0011] 3. Poor generalization ability. VT-Net relies on training data and can only be used to reconstruct flames of the same type as the training data. For different flames, it is necessary to prepare training data again and retrain the network. Summary of the invention

[0012] The present invention provides a flame 4D chemiluminescence rapid reconstruction method, which solves the problems that the reconstruction occupies a large memory, takes a long time to reconstruct, and the 4D information of the flame cannot be obtained in real time.

[0013] The specific technical scheme of the present invention is:

[0014] A flame 4D chemiluminescence rapid reconstruction method comprises the following steps:

[0015] Step 1: Camera Calibration

[0016] Establish world coordinate system X w -Y w -Z w , camera coordinate system X c -Y c -Z c Based on the optical center of the lens, the pixel coordinates of the image plane are (u, v). The conversion relationship from the world coordinate system to the pixel coordinate system is:

[0017]

[0018] where f x 、f y is the focal length of the camera in the x and y directions; d x ,d y is the size of a single pixel; γ is the skew coefficient in the pixel coordinate system; K = [RT], is the rotation matrix, representing the direction of the camera, is the translation matrix, representing the position of the camera; X w , Y w , Z w are the coordinates in the world coordinate system; the matrices K and P are obtained using Zhang Zhengyou calibration method.

[0019] Step 2: Take pictures of the chemiluminescence of the flame from different angles.

[0020] The experimental device based on the image transmission bundle combined with a high-speed camera was used to reconstruct the Bunsen burner flame at a time resolution of 5kHz. It consists of a high-speed camera and a nine-in-one fiber bundle. The camera model is a PhtronSA5 high-speed camera with a resolution of 1024×1024, and the COMS sensor size is 20.48mm×20.48mm. Each input end of the nine-in-one fiber image transmission bundle contains 13,000 single-mode optical fibers, each with a core diameter of 13um, each input end size of 2mm×2mm, and the output end size of 6mm×6mm. The image transmission bundle lens is an 8mm short-focus lens. The camera is connected to a macro lens with a magnification of 1:1, and the resolution of each viewing angle is 100×100 (fiber bundle resolution). Nine split optical fibers are used to capture views of the flame from nine different viewing angles and display them on the other end face. Using a high-speed camera, high-time resolution views of the flame from different viewing angles can be obtained.

[0021] Step 3: Segment the target area voxels and calculate the weight matrix.

[0022] The pixel division of the image plane depends on the camera resolution, and its size is consistent with the camera sensor pixel size (5.5μm). The target area (32×32×100mm) is discretized into small cubes (voxels), and its side length is determined by the camera resolution and camera magnification;

[0023] According to the camera internal and external parameters obtained by calibration in step 1, the coordinates of the point in the world coordinate system on the image plane are calculated. The 8 vertices of the ith cube voxel are projected to the image plane, and the convex hull formed by these 8 points is used as the projection of the voxel on the image plane. The ratio of the area of ​​the intersection of the projection of the voxel on the image plane to the area of ​​the convex hull is used as the projection coefficient. The formula is as follows:

[0024]

[0025] Where S int is the projection area of ​​the j-th voxel on the i-th pixel, S con is the area of ​​the convex hull of the j-th voxel projection.

[0026] The weight matrix W is processed using the coo_matrix class provided by scipy.sparse, compressing the 32GB weight matrix to 0.4GB. coo_matrix is ​​the simplest sparse matrix storage method, which uses triples to store information about non-zero elements in the matrix.

[0027] Step 4: Train a custom fully connected neural network (FCN)

[0028] The flow field parameters are embedded into the neural network, and the reconstructed flow field is obtained through training, and multiple flow fields are reconstructed at the same time. The loss function is set to MSELoss, and the output reprojection W is obtained by inputting the weight matrix obtained in step 3. fre ,The training process is to minimize the root mean square error between the ,reprojection and the actual projection and obtain the target flow field prediction ,value.

[0029] The training process is divided into forward propagation and back propagation. Forward propagation is to reproject W according to the reconstructed flow field. fre , back propagation is the update of the reconstructed flow field, and the update formula is:

[0030]

[0031] Where α is the learning rate (set to 0.0001) and p is the column vector of projections.

[0032] By changing the number of output nodes and the number of network parameters, multiple flow fields can be reconstructed at the same time, s is the number of flow fields reconstructed simultaneously (set to 1000). It takes a total of 63.4 seconds to reconstruct 1000 frames of different flames at a time, an average of 0.0634 seconds per frame, and 773MB of memory is used.

[0033] Step 5: Extract network parameters as flow field reconstruction values

[0034] Read the parameters of the network trained in step 4 and rearrange them from 102400×1 to 32×32×100, which is the required three-dimensional structure of the flame.

[0035] Step 6: Reconstruction accuracy assessment

[0036] When reconstructing the actual flame, its true three-dimensional distribution is unknown, and its reconstruction accuracy cannot be directly evaluated. Eight of the nine views are randomly selected for reconstruction, and the other view is used to evaluate the reconstruction accuracy. The evaluation criterion is the correlation coefficient R:

[0037]

[0038] in To predict the flow field f re In the projection of the verification perspective, p tis the actual projection of the viewing angle, m t is the number of pixels of the viewing angle.

[0039] Compared with the prior art, the present invention has the following technical effects:

[0040] 1. When using this scheme to reconstruct, the memory occupied is only 773MB. With this scheme, multiple frames can be reconstructed at the same time, and the weight matrix is ​​shared. The memory occupied by multiple frames at the same time is not much different from that of single frame reconstruction.

[0041] 2. The reconstruction efficiency is greatly improved. The average reconstruction time per frame of this scheme is 0.063s, while the commonly used SART algorithm takes 7.8s to perform the same reconstruction.

[0042] 3. The reconstruction accuracy is basically the same as SART, but it is more stable and the average reconstruction accuracy is slightly higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A linear imaging model diagram of an embodiment

[0044] Figure 2 It is a schematic diagram of the experimental system of the embodiment;

[0045] Figure 3 It is a view of the flame of the embodiment from different perspectives;

[0046] Figure 4 A schematic diagram of a convex polygon formed by projecting voxels onto an image plane in an embodiment;

[0047] Figure 5 A diagram of a custom fully connected neural network structure of an embodiment;

[0048] Figure 6 This is the contour map of the reconstruction result. DETAILED DESCRIPTION

[0049] The specific technical solution of the present invention is described in conjunction with the accompanying drawings.

[0050] A flame 4D chemiluminescence rapid reconstruction method comprises the following steps:

[0051] Step 1: Camera calibration;

[0052] In visual measurement, the camera imaging model is divided into linear imaging model and nonlinear imaging model. The linear imaging model is also called the pinhole imaging model. Figure 1 As shown, the position change relationship between the position in the actual physical space and the image pixel is established through perspective projection transformation.

[0053] Establish world coordinate system X w -Y w -Z w, camera coordinate system X c -Y c -Z c Based on the optical center of the lens, the pixel coordinates of the image plane are (u, v). The conversion relationship from the world coordinate system to the pixel coordinate system is:

[0054]

[0055] where f x 、f y is the focal length of the camera in the x and y directions; d x d y is the size of a single pixel; γ is the skew coefficient in the pixel coordinate system; K = [RT], is the rotation matrix, representing the direction of the camera, is the translation matrix, representing the position of the camera; X w , Y w , Z w are the coordinates in the world coordinate system; the matrices K and P are obtained using Zhang Zhengyou calibration method.

[0056] Step 2: Take pictures of the chemiluminescence of the flame from different angles.

[0057] The flame of the Bunsen burner was reconstructed at a time resolution of 5 kHz using an experimental setup based on an imaging bundle combined with a high-speed camera. The experimental system consists of a high-speed camera and a nine-in-one fiber bundle, such as Figure 2 The camera model is a PhtronSA5 high-speed camera with a resolution of 1024×1024 and a COMS sensor size of 20.48mm×20.48mm. Each input end of the nine-in-one fiber optic imaging bundle contains 13,000 single-mode optical fibers, each with a core diameter of 13um. The size of each input end is 2mm×2mm, and the size of the output end is 6mm×6mm. The imaging bundle lens is an 8mm short-focus lens. The camera is connected to a macro lens with a magnification of 1:1, and the resolution of each viewing angle is 100×100 (fiber bundle resolution).

[0058] Figure 3 Nine views of the flame at 470ms from different directions are shown, with a total of 2687 frames acquired.

[0059] Step 3: Segment the target area voxels and calculate the weight matrix.

[0060] The pixel division of the image plane depends on the resolution of the camera, and its size is consistent with the pixel size of the camera sensor (5.5μm). The target area (32×32×100mm) is discretized into small cubes (voxels), and its side length is determined by the camera resolution and camera magnification, and is empirically determined to be 1mm here.

[0061] Based on the camera internal and external parameters obtained by calibration in step 1, the coordinates of the point in the world coordinate system on the image plane can be calculated. Project the 8 vertices of the i-th cube voxel to the image plane, as follows: Figure 4 , the convex hull formed by these 8 points is used as the projection of the voxel on the image plane, and the ratio of the area of ​​the intersection of the voxel projection on the image plane to the area of ​​the convex hull is used as the projection coefficient. The formula is as follows:

[0062]

[0063] Where S int is the projection area of ​​the j-th voxel on the i-th pixel, S con is the area of ​​the convex hull of the j-th voxel projection.

[0064] The weight matrix W is processed using the coo_matrix class provided in scipy.sparse, which can compress a 32GB weight matrix into 0.4GB. coo_matrix is ​​the simplest way to store sparse matrices, using a triple (row, col, data) to store information about non-zero elements in the matrix. In actual use, coo_matrix is ​​generally used to create matrices, because coo_matrix cannot add, delete, or modify the elements of a matrix; after successful creation, it can be converted into sparse matrices of other formats (such as csr_matrix, csc_matrix) for transposition, matrix multiplication, and other operations.

[0065] Step 4: Train a custom fully connected neural network (FCN)

[0066] Combining the pytorch deep learning framework, a method is proposed to embed the flow field parameters into the neural network, and the reconstructed flow field is obtained through training, and multiple flow fields can be reconstructed at the same time. The loss function is set to MSELoss, and the output reprojection Wf is obtained by inputting the weight matrix obtained in step 3. re ,The training process is to minimize the root mean square error between the re-projection and the actual ,projection (the flame views obtained in step 2 at different viewing angles) and obtain the target ,flow field prediction value, such as Figure 5 The training process is divided into forward propagation and back propagation. Forward propagation is to reproject Wf according to the reconstructed flow field. re , back propagation is the update of the reconstructed flow field, and the update formula is:

[0067]

[0068] Where α is the learning rate (set to 0.0001) and p is the column vector of projections.

[0069] By changing the number of output nodes and the number of network parameters, multiple flow fields can be reconstructed at the same time, s is the number of flow fields reconstructed simultaneously (set to 1000). It takes a total of 63.4 seconds to reconstruct 1000 frames of different flames at a time, an average of 0.0634 seconds per frame, and 773MB of memory is used.

[0070] Step 5: Extract network parameters as flow field reconstruction values

[0071] Read the parameters of the network trained in step 4 and rearrange them from 102400×1 to 32×32×100, which is the required three-dimensional structure of the flame.

[0072] Figure 6 The outline of the flame reconstruction result at 470 ms is shown.

[0073] Step 6: Reconstruction accuracy assessment

[0074] When reconstructing the actual flame, its true three-dimensional distribution is unknown, and its reconstruction accuracy cannot be directly evaluated. Eight of the nine views are randomly selected for reconstruction, and the other view is used to evaluate the reconstruction accuracy. The evaluation criterion is the correlation coefficient R:

[0075]

[0076] in To predict the flow field f re In the projection of the verification perspective, p t is the actual projection of the viewing angle, m t is the number of pixels of the viewing angle.

[0077] The scheme and SART algorithm were used to reconstruct the 2687 frames of flames captured, and the average value of the correlation coefficient R was taken:

[0078]

[0079] Where num is the total number of frames of the reconstructed flame, and is the correlation coefficient between the projection of the reconstructed i-th frame flame at the verification perspective and its actual projection.

[0080] According to statistics, the average correlation coefficient of this scheme is 0.987, which is higher than the SART algorithm (0.981).

Claims

1. A flame 4D chemiluminescence rapid reconstruction method, characterized in that: The following steps are involved: Step 1: Camera calibration; Step 2: Take chemiluminescent images of the flame at different angles; Step 3: Segment the voxels of the target area and calculate the weight matrix; Step 4: Train the custom fully connected neural network FCN; Step 5: Extracting network parameters is the flow field reconstruction value; Step 6: Reconstruction accuracy assessment.

2. A flame 4D chemiluminescence rapid reconstruction method according to claim 1, characterized in that: Step 1 specifically includes the following methods: Establish world coordinate system X w -Y w -Z w , camera coordinate system X c -Y c -Z c Established at the optical center of the lens, the pixel coordinates of the image plane are (u, v); the transformation relationship from the world coordinate system to the pixel coordinate system is: where f x 、f y is the focal length of the camera in the x and y directions; d x ,d y is the size of a single pixel; γ is the skew coefficient in the pixel coordinate system; K = [RT], is the rotation matrix, representing the direction of the camera, is the translation matrix, representing the position of the camera; X w , Y w , Z w are the coordinates in the world coordinate system; the matrices K and P are obtained using Zhang Zhengyou calibration method.

3. A flame 4D chemiluminescence rapid reconstruction method according to claim 1, characterized in that: Step 2 specifically includes the following methods: reconstructing the Bunsen burner flame at a 5kHz time resolution using an experimental device based on an image transmission bundle combined with a high-speed camera, including a high-speed camera and a nine-in-one optical fiber bundle; The camera model is Phtron SA5 high-speed camera, with a resolution of 1024×1024 and a COMS sensor size of 20.48mm×20.48mm; Each input end of the nine-in-one fiber optic imaging bundle contains 13,000 single-mode optical fibers, each with a core diameter of 13um. The size of each input end is 2mm×2mm, and the size of the output end is 6mm×6mm. The imaging bundle lens is an 8mm short-focus lens. The camera is connected to a macro lens with a 1:1 magnification ratio, and the resolution of each viewing angle is 100×100. Nine split optical fibers are used to capture views of the flame from nine different viewing angles and display them collectively on the other end face. A high-speed camera can be used to obtain high-time resolution views of the flame from different viewing angles.

4. A flame 4D chemiluminescence rapid reconstruction method according to claim 1, characterized in that: Step three specifically includes the following methods: The pixel division of the image plane depends on the camera resolution, and its size is consistent with the camera sensor pixel size; the target area is discretized into small cubes, and its side length is determined by the camera resolution and camera magnification; According to the camera internal and external parameters obtained by calibration in step 1, calculate the coordinates of the point in the world coordinate system on the image plane; project the 8 vertices of the i-th cube voxel to the image plane, and use the convex hull formed by these 8 points as the projection of the voxel on the image plane. The ratio of the area of ​​the intersection of the voxel projection on the image plane and the area of ​​the convex hull is used as the projection coefficient. The formula is as follows: Where S int is the projection area of ​​the j-th voxel on the i-th pixel, S con is the area of ​​the convex hull of the j-th voxel projection; The weight matrix W is processed using the coo_matrix class provided in scipy.sparse, compressing the 32GB weight matrix to 0.4GB; coo_matrix is ​​the simplest sparse matrix storage method, using triples to store information about non-zero elements in the matrix.

5. The flame 4D chemiluminescence rapid reconstruction method according to claim 1, characterized in that: Step 4 specifically includes the following methods: The flow field parameters are embedded into the neural network, and the reconstructed flow field is obtained through training, and multiple flow fields are reconstructed at the same time; the loss function is set to MSELoss, and the output reprojection Wf is obtained by inputting the weight matrix obtained in step 3 re ,The training process is to minimize the root mean square error between the re-projection and the actual projection and obtain the target flow field prediction value; The training process is divided into forward propagation and back propagation. Forward propagation is to reproject Wf according to the reconstructed flow field. re , back propagation is the update of the reconstructed flow field, and the update formula is: Where α is the learning rate, set to 0.0001, and p is the column vector composed of projections; By changing the number of output nodes and the number of network parameters, multiple flow fields can be reconstructed simultaneously, where s is the number of flow fields reconstructed simultaneously.

6. A flame 4D chemiluminescence rapid reconstruction method according to claim 1, characterized in that: Step 5 specifically includes the following methods: Read the parameters of the network trained in step 4 and rearrange them from 102400×1 to 32×32×100, which is the required three-dimensional structure of the flame.

7. A flame 4D chemiluminescence rapid reconstruction method according to claim 1, characterized in that: Step six specifically includes the following methods: Eight of the nine views are randomly selected for reconstruction, and the other view is used to evaluate the reconstruction accuracy. The evaluation criterion is the correlation coefficient R: in To predict the flow field f re In the projection of the verification perspective, p t is the actual projection of the viewing angle, m t is the number of pixels of the viewing angle.

Images