Ice crystal particle shape classification method based on interferometric defocus images

Abstract

This invention discloses a method for classifying the shape of ice crystal particles based on interferometric defocus images. The method involves preprocessing ice crystal particle images used to train a neural network, constructing a laser interferometric particle imaging system, and sequentially inputting the processed particle images into a DMD for interferometric imaging. A CCD located on the defocus plane receives the interferometric defocus images of the particles. The acquired interferometric defocus images are used to construct a convolutional network training set to train the classification network. New particle interferometric defocus images are then input into the classification network obtained in step 5 to achieve ice crystal particle shape classification. Compared with existing technologies, this invention achieves automatic particle shape classification; the method has the advantages of high speed and high reliability, providing a foundation for the application of laser interferometric imaging technology in the research and analysis of ice crystal particles.

Description

Technical Field

[0001] This invention relates to the field of ice crystal particle shape measurement technology, and in particular to an ice crystal particle shape classification method based on deep learning. Background Technology

[0002] Ice crystals, as a component of clouds, are diverse in type and shape. Different morphologies of ice crystals exhibit different scattering characteristics, influencing the radiation properties of clouds. Therefore, accurately understanding ice crystals in clouds is crucial for simulating radiation and temperature fields. Laser particle interferometry (LIFI) has been widely developed for measuring various spherical and irregular particles and can serve as a reliable method for detecting ice crystals. Defocused interferometric imaging extracts frequency information of the scattered light from particles to obtain relevant information such as particle size, offering better robustness compared to focused imaging.

[0003] The study of particle morphology using interferometric imaging methods is relatively mature. Patent CN108593528B, "Method for measuring the shape and size of non-spherical rough particles based on laser interferometry", discloses "establishing the relationship between the interferometric defocus image and the particle geometric size, and obtaining the particle size information through two-dimensional autocorrelation calculation of the interferometric defocus image".

[0004] To address the challenge of preserving ice crystals at room temperature, a digital micromirror (DMD) is used to simulate ice crystal particles during shape classification. A DMD is a spatial light modulation device composed of an array of square micromirrors a few micrometers wide. The number of micromirrors depends on the device resolution, with each micromirror representing a pixel. These micromirrors are supported diagonally by torsion hinges and integrated onto a single chip. Each micromirror has an individual address, is controlled by digital signals, and can deflect at a fixed angle around the diagonal. By varying the angle of each micromirror, the incident light is modulated. When a laser beam illuminates the DMD, micromirrors in the "on" state reflect the light into the imaging objective lens, while those in the "off" state reflect the light out of the imaging system and do not participate in the imaging process. By loading an image of an ice crystal particle onto the DMD and using the micromirrors to simulate scattering points on the particle surface, speckle interference images of the ice crystal particles can be acquired within the system.

[0005] The artificial ice crystal particle shape classification method is no longer applicable to the classification of ice crystal particle interference defocus images, and there is currently no automatic classification method for ice crystal interference defocus images. Summary of the Invention

[0006] This invention proposes a method for classifying the shape of ice crystal particles based on interferometric defocus images, using deep learning methods to classify the shape of ice crystal particles based on interferometric defocus images.

[0007] This invention is achieved using the following technical solution:

[0008] A method for classifying ice crystal particle shapes based on interferometric defocused images, the specific steps of which are as follows:

[0009] Step 1: Adjust the size of the ice crystal particle image used to train the neural network to m×n pixels;

[0010] Step 2: Extract the target contour region from the ice crystal particle image processed in Step 1, and randomly generate scattered points within the target contour region.

[0011] Step 3: Fill the boundary of the scatter plot with pixels 0 to a size of M×N pixels, so that the scatter plot of the particles is the same size as the DMD area.

[0012] Step 4: Build a laser interferometric particle imaging system, input the processed particle images into the DMD sequentially for interferometric imaging, and use the CCD located on the defocus image plane to receive the interferometric defocus image of the particles.

[0013] Step 5: Construct a convolutional network neural training set using the acquired interferometric defocus images to train the classification network;

[0014] Step 6: Input the new particle interference defocused image into the classification network obtained in step 5 to achieve ice crystal particle shape classification.

[0015] Compared with the prior art, the beneficial effects that the present invention can achieve are as follows:

[0016] 1) Automatic classification of particle shapes was achieved;

[0017] 2) It has the advantages of high speed and high reliability;

[0018] 3) It provides a foundation for the research and analysis of ice crystal particles using laser particle interferometry imaging technology. Attached Figure Description

[0019] Figure 1 This is a flowchart of the ice crystal particle shape classification method based on interferometric defocused images of the present invention;

[0020] Figure 2 A schematic diagram illustrating an example of ice crystal particle image preprocessing;

[0021] Figure 3 The diagram shows a laser interferometric particle imaging system. The labels are as follows: 1. Laser, 2. Beam expander lens, 3. Aperture, 4. Collimating lens, 5. Plane mirror, 6. DMD device, 7. Imaging objective lens, 8. Area array camera CCD.

[0022] Figure 4 Example image of the acquisition result of the defocused image of ice crystal particle interference;

[0023] Figure 5This is a schematic diagram of the classification network structure;

[0024] Figure 6 A schematic diagram of dense modules in the DenseNet-121 network;

[0025] Figure 7 This is a schematic diagram of the network training results. Detailed Implementation

[0026] The present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0027] like Figure 1 The figure shows the ice crystal particle shape classification method based on interferometric defocused images of the present invention. The specific steps are as follows:

[0028] Step 1: Stretch / compress the size of the ice crystal particle image used to train the neural network to m×n pixels, where m and n are the number of pixels horizontally and vertically, respectively.

[0029] Step 2: Extract the target contour region from the ice crystal particle image processed in Step 1, and then randomly generate scattered points within the target contour region.

[0030] Step 3: Since the original particle image pixel size m×n after processing in Step 1 and Step 2 is smaller than the size of the Digital Micromirror (DMD), this step fills the boundary of the particle scatter plot with pixels 0 to a size of M×N pixels, so that the particle scatter plot is the same size as the size of the Digital Micromirror (DMD), where M×N is the pixel size of the DMD, and M×N is generally much larger than m×n.

[0031] Step 4: Build a laser interferometric particle imaging system. Input the processed particle images into a digital micromirror (DMD) for interferometric imaging. Use a CCD array camera located on the defocused image plane to receive the interferometric defocused image of the particles.

[0032] Step 5: Construct a convolutional network neural training set using the acquired interferometric defocus images to train the classification network;

[0033] Step 6: Input the new particle interference defocused image into the classification network obtained in step 5 to achieve ice crystal particle shape classification.

[0034] like Figure 2 The image shown is a schematic diagram of an example of ice crystal particle image preprocessing. Similarly, the preprocessing result of this example is adapted to DMD.

[0035] like Figure 3The diagram shows a schematic of a laser interferometric particle imaging system. Laser 1 expands the beam through beam expander lens 2, then passes through aperture 3 to mask the interference ring formed by the beam expander lens, retaining the central bright spot. The beam is then collimated by collimating lens 4 and reflected by plane mirror 5 to DMD device 6. The scattered light from the DMD target area is sent to imaging objective lens 7 to achieve interferometric imaging. A CCD camera located on the defocused image plane receives the defocused interferometric image of the particles.

[0036] like Figure 4 The image shown is an example of the acquisition results of defocused interferometric images of ice crystal particles. A, b, and c are two-dimensional outlines of the ice crystal particles, and d, e, and f are the corresponding defocused interferometric images. The ice crystal particle images are from the ICDC ice crystal particle database, totaling 7282 images. The database classifies ice crystal particles into 10 categories: bud-like, rose-like, complex crystals with lateral surfaces (Cox), plate-like (Plt), fan-shaped plate-like (Ser), long columnar (Loc), short columnar (Shc), hollow columnar (Hoc), spherical (Sph), and small irregular bodies (Sir).

[0037] like Figure 5 The diagram shows a classification network structure. This invention preferentially uses the DenseNet-121 network. This network architecture includes convolutional layers (7×7 convolutions), pooling layers (3×3 pooling), 6 first dense modules (each consisting of 1×1 and 3×3 convolutions), a first transitive layer (including 1×1 and 2×2 pooling), 24 second dense modules (each consisting of 1×1 and 3×3 convolutions), a second transitive layer (including 1×1 and 2×2 pooling), a third transitive layer (including 1×1 and 2×2 pooling), 16 third dense modules (each consisting of 1×1 and 3×3 convolutions), and a classification layer (including 7×7 pooling, a fully connected layer, and an activation function).

[0038] like Figure 6 The diagram illustrates a dense module in the DenseNet-121 network, where H1-H4 are convolutional layers. In DenseNet, a dense module typically consists of multiple convolutional layers connected in this way. Within this module, the input to each layer is composed of the outputs of all previous layers. This structure provides each layer with a direct connection to the input data and the loss function, mitigating the gradient explosion and vanishing problems common in deep networks. Furthermore, due to the inter-layer connectivity, features can be reused, requiring only a smaller number of features per layer to achieve good training results, thus saving parameters and reducing computational cost.

[0039] Figure 7This diagram illustrates the network training results, where 'a' represents the accuracy and 'b' represents the loss function. After 30 iterations, the classification accuracy on the validation set reaches over 80%.

[0040] Specific embodiments of the present invention are described below:

[0041] The simulated particle measurement system uses a 532nm laser. The beam is expanded by a microscope objective (e.g., APSF13-1AT, 10× magnification) and a pinhole (10μm aperture). A convex lens (focal length f = 150mm) collimates the beam, which is then reflected by a mirror onto the surface of a digital micromirror (DMD) (e.g., DLP6500FYEHD, 1920×1080 resolution). The micromirror on the DMD, in its "open" state, reflects the light towards an imaging lens (e.g., PENTAXTV, f = 50mm, F = 1.4). A CCD area array camera (e.g., GRAS-14S5M-C, 1280×920 resolution) is located off-focus on the lens to receive the interference pattern. In the simulated particle interference imaging section, the object distance z1 = 65mm, the image distance z2 = 165mm, and the off-focus distance z1 = 51mm. The original images of ice crystal particles are uniformly stretched or compressed to 200×200 pixels and filled to 1920×1080 pixels to match the resolution of the DMD. The processed particle images are then sequentially input into the DMD for interferometric imaging. The CCD acquires an image of size 1280 pixels × 920 pixels, and a region with a center size of 500 pixels × 500 pixels is cropped to form the particle interferometric image dataset.

[0042] The neural network was built using PyTorch 1.8.1 based on Python 3.8. The initial learning rate was set to 0.001, and the learning rate was reduced to 1 / 10 of the original rate every 5 generations, for a total of 30 generations. The loss function was the cross-entropy loss function. The training set consisted of 5775 images, and the test set consisted of 1440 images. The ice crystal particles were divided into 10 categories according to their shapes.

[0043] It should be noted that although the present invention has been shown and described with reference to specific exemplary embodiments thereof, those skilled in the art should understand that the present invention is not limited to the above embodiments, and all modifications to the present invention fall within the scope of protection of the present invention.

Claims

1. An ice crystal particle shape classification method based on interference out-of-focus images, characterized by, The specific steps are as follows: Step 1, adjust the size of the ice crystal particle picture used for training the neural network to m x n pixels; Step 2, extract the target contour area from the ice crystal particle picture processed in step 1, and randomly generate scatter points in the target contour area; Step 3, fill the boundary of the scatter point diagram with pixel 0 to M x N pixels, so that the scatter point diagram of the particle is the same size as the DMD surface; Step 4, build a laser interference particle imaging system, input the processed particle image into the digital micro mirror DMD for interference imaging in turn, and use the CCD located at the defocus image plane to receive the interference defocus image of the particle; Step 5, use the collected interference defocus image to construct a convolution network neural training set and train the classification network; the classification network uses DenseNet-121 network, and the network structure includes: Convolution layer, specifically 7 x 7 convolution; Pooling layer, specifically 3 x 3 pooling; 6 first dense modules, specifically each dense module is composed of 1 x 1 convolution and 3 x 3 convolution; First transfer layer, specifically including 1 x 1 convolution and 2 x 2 pooling; 12 second dense modules, specifically each dense module is composed of 1 x 1 convolution and 3 x 3 convolution; Second transfer layer, specifically including 1 x 1 convolution and 2 x 2 pooling; 24 second dense modules, specifically each dense module is composed of 1 x 1 convolution and 3 x 3 convolution; Third transfer layer, specifically 1 x 1 convolution and 2 x 2 pooling; 16 third dense modules, specifically each dense module is composed of 1 x 1 convolution and 3 x 3 convolution; And, Classification layer, specifically including 7 x 7 pooling, fully connected layer and activation function; Step 6, input the new particle interference defocus image into the classification network obtained in step 5 to realize the shape classification of ice crystal particles.

Images