Particle position prediction method and system in particle field based on deep learning network

Abstract

The invention discloses a particle position prediction method and system in a particle field based on a deep learning network, and the method comprises the steps: obtaining a particle field hologram, and marking the particle position; constructing a data set according to the particle field hologram of the known particle position; dividing the data set into a training set, a verification set and a test set; inputting the training set into the deep learning network, training the network, verifying the trained network by using the verification set, and storing the trained network; inputting the test set into the trained network to obtain the coordinate position and confidence of each particle; wherein the deep learning network is realized through improved YOLOV5s; according to the improved YOLOV5s, a multi-spectrum channel attention module is added to a backbone network of an original YOLOV5s network, and the multi-spectrum channel attention module is used for improving the capturing capability of the network on particle stripes.

Description

Technical Field

[0001] The present invention relates to the technical field of particle position prediction, and in particular to a method and system for predicting particle positions in a particle field based on a deep learning network. Background Art

[0002] Three-dimensional (3D) particle field research has attracted widespread attention and has been applied in fields such as environment, energy, biological cell detection, and fluid mechanics. For example, particle field detection can be used to monitor the degree of fuel combustion by measuring the size and concentration of carbon particles raised in the air during combustion. Particle field research can also provide valuable data for biological research by examining the morphology, concentration, and size of cells. 3D particle field reconstruction technology also plays a key role in marine ecological monitoring, accurately measuring the spatial distribution of particles such as plankton, sediment, and microplastics, providing data support for marine environmental assessment and protection. Therefore, it is necessary to develop an in-situ particle detection system for particle field monitoring to provide an observational tool and data acquisition method for 3D particle field research.

[0003] Currently, the main methods for particle detection are inductive detection, acoustic detection, and optical detection.

[0004] Inductive detection is represented by the Coulter counter. When particles of different sizes pass through a small hole, the resistance will change. The volume of the particles is measured by measuring the change in resistance. However, the sample processing requirements are high, and it needs to be placed in an electrolyte, and the small hole is easily clogged.

[0005] Acoustic detection, represented by the acoustic Doppler flow profiler, uses the Doppler effect of sound waves to calculate the frequency shift of the return signal. Its advantage is that it can measure flow velocity and particle concentration at the same time. Its disadvantage is that the accuracy of measuring small particles is low.

[0006] Optical detection mainly includes turbidity meters, which determine turbidity by measuring the scattering and absorption of particles. They use LEDs or lasers to emit light beams, receive light passing through the water body, and then measure turbidity. The advantage is that the measurement process is simple and can be done in situ, but it can only measure overall turbidity and cannot measure particle size distribution.

[0007] Significant progress has been made in the application of digital holographic microscopy in particle detection. One study demonstrated its application in measuring water mist particles. By recording holographic diffraction patterns, this work enables precise measurement of the size, shape, and spatial position of water mist particles.

[0008] Another study focused on the application of digital holographic microscopy in the detection of non-spherical particles. This study used digital holographic microscopy to measure the morphology of both spherical polymer microparticles and non-spherical carbon fiber particles, and compared them, demonstrating the ability of digital holographic microscopy to measure irregularly shaped solid particles.

[0009] Another study explored the application of digital holographic microscopy in tumor cell detection. This study demonstrated a label-free, high-throughput tumor cell detection method based on inline digital holographic microscopy, using a machine learning classifier to accurately identify tumor cells in blood. These representative works fully demonstrate the diverse applications and enormous potential of digital holographic microscopy in the field of particle detection.

[0010] With the continuous advancement and innovation of technology, by combining digital holography and deep learning in particle field detection, it is possible to achieve automatic identification of particles and more accurate and rapid analysis of three-dimensional coordinates, providing more efficient and accurate technical means for scientific research and technological applications such as life health, environmental monitoring and industrial testing. Summary of the Invention

[0011] To address the shortcomings of the existing technology, the present invention provides a method and system for predicting particle positions in a particle field based on a deep learning network. This patent combines digital holography, computational optics, and a deep learning network to design a holographic deep learning network for particle detection. A simulated holographic particle field dataset is generated using a Fresnel diffraction algorithm, which is divided into a training set, a validation set, and a test set in an 8:1:1 ratio. A Matlab algorithm is used to automatically generate a txt file containing the particle's true value, eliminating the need for manual labeling and accurately and quickly providing the three-dimensional coordinates of the particles. The YOLOV5 network is used to train and predict the three-dimensional coordinate information of particles in the particle field. The network input is a two-dimensional hologram preprocessed with bilinear interpolation, and the output is the three-dimensional coordinates and confidence level of the particles in the hologram. This deep learning network is characterized by high accuracy and fast detection speed for particle field detection, and has broad application potential in particle detection research, life and health, environmental monitoring, and industrial testing.

[0012] On the one hand, a method for predicting particle positions in a particle field based on a deep learning network is provided, including:

[0013] Obtain a particle field hologram and mark the particle positions;

[0014] Construct a data set based on the particle field hologram with known particle positions; and divide the data set into a training set, a validation set, and a test set;

[0015] Input the training set into the deep learning network to train the network, use the validation set to verify the trained network, and save the trained network;

[0016] The test set is input into the trained network to obtain the coordinate position and confidence of each particle. The deep learning network is implemented by an improved YOLOV5s. The improved YOLOV5s adds a multi-spectral channel attention module to the backbone network of the original YOLOV5s network. The multi-spectral channel attention module is used to improve the network's ability to capture particle stripes.

[0017] On the other hand, a particle position prediction system in a particle field based on a deep learning network is provided;

[0018] The particle position prediction system in the particle field based on deep learning network includes:

[0019] An acquisition module is configured to: acquire a particle field hologram and mark particle positions;

[0020] A data set construction module is configured to: construct a data set based on a particle field hologram with known particle positions; and divide the data set into a training set, a validation set, and a test set;

[0021] A training module is configured to: input the training set into the deep learning network, train the network, verify the trained network using the validation set, and save the trained network;

[0022] The prediction module is configured to input the test set into the trained network to obtain the coordinate position and confidence of each particle. The deep learning network is implemented by an improved YOLOV5s. The improved YOLOV5s adds a multi-spectral channel attention module to the backbone network of the original YOLOV5s network. The multi-spectral channel attention module is used to improve the network's ability to capture particle streaks.

[0023] In another aspect, an electronic device is provided, comprising:

[0024] a memory for non-transitory storage of computer-readable instructions; and

[0025] a processor for executing said computer-readable instructions,

[0026] When the computer-readable instructions are executed by the processor, the method described in the first aspect is executed.

[0027] On the other hand, a storage medium is provided, which non-temporarily stores computer-readable instructions, wherein when the non-temporary computer-readable instructions are executed by a computer, the method described in the first aspect is executed.

[0028] On the other hand, a computer program product is provided, comprising a computer program, wherein the computer program is configured to implement the method described in the first aspect when running on one or more processors.

[0029] The above technical solution has the following advantages or beneficial effects:

[0030] Compared with existing technologies, digital holographic technology and computational optics are used to generate simulated particle field holograms, and deep learning networks are used to train and predict particle field information, overcoming the limitations of inductive detection sensors such as Coulter counters, which have high sample processing requirements, acoustic detection sensors such as acoustic Doppler flow profilers, which have low accuracy in measuring small particles, and optical detection sensors such as turbidity meters, which cannot measure particle size distribution.

[0031] Using the YOLOV5 object detection network, we predict 3D information from randomly generated particle fields in Matlab. This method requires significantly less time to generate particle field information than traditional digital coaxial holographic reconstruction algorithms. The input is a 2D hologram preprocessed with bilinear interpolation, and the output is the 3D coordinates of the particles in the hologram. We innovatively added the attention mechanism module FcaNet to the network, creating YOLOV5-FcaNet, which is used to predict 3D information of particle fields in digital holographic imaging.

[0032] YOLOV5-FcaNet achieved a map@0.5 accuracy of 99.1%, a precision of 98%, and a recall of 97.5%, respectively improvements of 1.4%, 0.2%, and 0.4% over the original YOLOV5 model. This holographic deep learning network combines digital holography technology with deep learning networks. By training an improved YOLOV5s model, it predicts particle field information. It is non-invasive, fast, highly accurate, and easy to use, and has broad application potential in particle detection research, as well as in life sciences, environmental monitoring, and industrial testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0034] Figure 1 This is a flow chart of the method of embodiment 1;

[0035] Figure 2 Schematic diagram of the internal structure of the deep learning network of Example 1;

[0036] FIG3( a ) is a simulated hologram randomly generated by Matlab in Example 1;

[0037] FIG3( b ) is a scatter diagram of the true value of the particle field of Example 1;

[0038] FIG3( c ) is the reconstructed particle field obtained from the network output in Example 1;

[0039] FIG3( d ) shows the reconstructed particle field and the true value particle field of the first embodiment. DETAILED DESCRIPTION

[0040] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0041] Example 1

[0042] This embodiment provides a method for predicting particle positions in a particle field based on a deep learning network;

[0043] The particle position prediction method in the particle field based on deep learning network includes:

[0044] S101: Obtain a particle field hologram and mark the particle positions;

[0045] S102: constructing a data set based on the particle field hologram of the known particle position; and dividing the data set into a training set, a validation set, and a test set;

[0046] S103: Input the training set into the deep learning network to train the network, use the validation set to validate the trained network, and save the trained network;

[0047] S104: Input the test set into the trained network to obtain the coordinate position and confidence of each particle; wherein, the deep learning network is implemented by an improved YOLOV5s; the improved YOLOV5s adds a multi-spectral channel attention module to the backbone network of the original YOLOV5s network, and the multi-spectral channel attention module is used to improve the network's ability to capture particle stripes.

[0048] Furthermore, the output dimension of the detection head (Head) of YOLOV5 is: (batch_size, num_anchors, grid_h, grid_w, 4+1+num_classes), where 4 represents the bounding box coordinates (cx, cy, width, height); 1 represents the objectness score target existence probability P obj , output by the Sigmoid function, the value range of the target existence probability is 0 to 1, and num_classes represents the probability P of each category classi, then the expression of confidence is:

[0049]

[0050] The present invention includes using a Fresnel diffraction algorithm to produce a simulated particle hologram, performing linear interpolation, and generating a true value TXT file using Matlab to form a simulated hologram particle field data set; dividing the data set into a training set, a validation set, and a test set; using YOLOV5s as a benchmark model to construct an improved YOLOV5s network model as a hologram particle field detection network model, and by adding a multi-spectral channel attention module to the backbone network, the network's ability to capture details of particle stripes in the hologram is improved; inputting the training set into the hologram particle field detection network model to perform model training; inputting the validation set into the trained hologram particle field detection network model; and inputting the test set into the trained hologram particle field detection network model to obtain a particle field detection result.

[0051] Furthermore, the step S101: obtaining a particle field hologram includes:

[0052] The simulated particle hologram is produced using the Fresnel diffraction algorithm. U(x0,y0) is the input value of the Fresnel diffraction algorithm, and U(x,y) is the output value of the Fresnel diffraction algorithm. The simulated hologram light field U(x,y) at a distance from particle z1 is expressed as:

[0053]

[0054] Where U(x0,y0) is the object light field of the particle at the plane (x0,y0), k is the wave number, λ is the wavelength, and z1 is the recording distance of the particle;

[0055] Convert formula (1) into convolution form:

[0056]

[0057] Taking Fourier transform on both sides of formula (2) we get:

[0058]

[0059] F is the symbol for Fourier transform, so:

[0060] U(x,y)=C·F -1 {F(U(x0,y0))·exp(iπz1λ(u 2 +v 2 ))}; (4)

[0061] Among them, F -1 is the inverse Fourier transform symbol, C is a constant, u, v are spatial frequencies,

[0062] The wavelength λ is 405nm, the width and length of the simulated particle field are 11.2mm, the number of particles is 16, and the particle size is 19.18μm; the particles in the particle field are randomly distributed between 60mm and 110mm from the image sensor, and the axial resolution is 1mm; the plane pixels perpendicular to the optical axis in the particle field space are set to 4096×4096, and the pixel size is 2.74um.

[0063] According to the simulated holographic light field U of a single particle i (x, y), propagate the object light field containing particle information in each plane with an axial spacing of 1 mm to the (x, y) plane to interfere with the reference light and perform light field superposition to obtain a simulated particle hologram U(x, y).

[0064] U(x,y)=U1(x,y)+U2(x,y)+...+U i (x,y)

[0065] Then, a linear interpolation algorithm was used to reduce the image size, which can shorten the training and verification time of the subsequent model. Matlab software was used to generate a true value TXT file to form a simulated hologram particle field dataset.

[0066] Furthermore, the step S101: marking the particle positions is to mark the three-dimensional spatial position of the particles in each image using a marking software.

[0067] Furthermore, the S102: constructing a data set according to the particle field hologram of the known particle position; and dividing the data set into a training set, a validation set and a test set, with the division ratio being 8:1:1.

[0068] Furthermore, the step S103 of inputting the training set into the deep learning network to train the network includes:

[0069] Classification loss L cls The calculation formula is:

[0070]

[0071] Bounding box regression loss L CIoU The calculation formula is:

[0072]

[0073] L CIoU =1-CIoU

[0074] Confidence loss L conf The calculation formula is:

[0075]

[0076] Total loss function L total , is to convert the classification loss L cls , bounding box regression loss L CIoU and confidence loss L conf Perform weighted addition;

[0077] L total =λ CIoU L CIoU +λ conf L conf +λ cls L cls ;

[0078] Among them, λ CIoU is the weight of the bounding box regression loss, λ conf is the weight of the confidence loss, λ cls is the weight of the classification loss.

[0079] Classification loss measures the accuracy of the model's prediction of the target class. In YOLOV5, binary cross-entropy loss is typically used to calculate classification loss. For each predicted bounding box, the model outputs a probability distribution indicating the confidence that the bounding box belongs to each class. Classification loss compares the predicted class probability distribution with the distribution of the true labels, calculating the difference between the two and guiding the model to optimize classification performance.

[0080] The bounding box regression loss measures the difference in position and size between the predicted bounding box and the ground-truth bounding box. YOLOV5 uses the complete intersection-over-union (CIoU) loss, which combines three factors: intersection-over-union (IoU), center point distance, and aspect ratio. This allows for simultaneous optimization of the bounding box's shape, size, and position.

[0081] Confidence loss is used to measure the accuracy of the model's confidence in the presence of an object in a bounding box. Confidence loss is similar to classification loss and is also calculated using binary cross entropy loss.

[0082] Furthermore, the S103: and use the validation set to verify the trained network, and save the trained network; verify the evaluation indicators, including Precision (%), Recall (%), mAP value calculated when the map@0.5 (%) IoU threshold is 0.5, map@[0.5:0.95] (%) mAP average F1 calculated when the IoU threshold is from 0.5 to 0.95 (step size is 0.05), and ScoreF1 score.

[0083] Furthermore, the S104: the improved YOLOV5s includes:

[0084] The backbone network, neck network and head network are connected in sequence;

[0085] The backbone network includes: a first convolutional layer, a second convolutional layer, a first C3 layer, a third convolutional layer, a second C3 layer, a fourth convolutional layer, a third C3 layer, a fifth convolutional layer, a multi-spectral channel attention module, an SPPF module and a sixth convolutional layer connected in sequence;

[0086] The neck network includes: a first upsampling layer, a first splicing unit, a fourth C4 layer, a seventh convolutional layer, a second upsampling layer, a second splicing unit, a fifth C3 layer, an eighth convolutional layer, a third splicing unit, a sixth C3 layer, a ninth convolutional layer, a fourth splicing unit, and a seventh C3 layer connected in sequence;

[0087] The head network includes: a first detection head, a second detection head and a third detection head;

[0088] The input end of the first upsampling layer is connected to the output end of the sixth convolutional layer; the input end of the first splicing unit is also connected to the output end of the third C3 layer; the input end of the second splicing unit is also connected to the output end of the second C3 layer; the input end of the third splicing unit is also connected to the input end of the seventh convolutional layer; the input end of the fourth splicing unit is also connected to the input end of the sixth convolutional layer;

[0089] The input end of the first detection head is also connected to the input end of the fifth C3 layer; the input end of the second detection head is also connected to the input end of the sixth C3 layer; the input end of the third detection head is also connected to the input end of the seventh C3 layer.

[0090] Furthermore, the SPPF module includes:

[0091] The tenth convolutional layer, the first maximum pooling layer, the second maximum pooling layer, the third maximum pooling layer, the fifth splicing unit, and the eleventh convolutional layer are connected in sequence;

[0092] The output end of the tenth convolutional layer, the output end of the first maximum pooling layer, and the output end of the second maximum pooling layer are all connected to the input end of the fifth splicing unit.

[0093] Furthermore, the internal structures of the first C3 layer, the second C3 layer, the third C3 layer, the fourth C3 layer, the fifth C3 layer, the sixth C3 layer and the seventh C3 layer are consistent.

[0094] The first C3 layer includes: a twelfth convolutional layer, a bottleneck module, a sixth splicing unit and a thirteenth convolutional layer connected in sequence, and the output end of the twelfth convolutional layer is also connected to the input end of the sixth splicing unit.

[0095] Furthermore, the first detection head, the second detection head and the third detection head are all implemented through a fully connected layer.

[0096] Furthermore, the functions of the first, second, third, fourth, fifth and sixth splicing units are the same, and are all used to implement serial splicing of input data.

[0097] Furthermore, the bottleneck module is located at different positions, and its internal structure is different;

[0098] A bottleneck module in the backbone network includes: a first CBS module, a second CBS module, and an adder connected in sequence, wherein an input end of the adder is residually connected to an input end of the first CBS module;

[0099] The bottleneck module in the neck network includes a third CBS module and a fourth CBS module connected in sequence.

[0100] The two structural designs of the bottleneck module enable a more effective combination of deep features and shallow features, which can express strong semantic information from top to bottom and positioning information from bottom to top, thereby improving the performance of the model.

[0101] Furthermore, the internal structures of the first CBS module, the second CBS module, the third CBS module and the fourth CBS module are consistent. The first CBS module includes: a convolutional layer (Conv), a batch normalization layer (BatchNorm2d) and an activation function (SiLU) connected in sequence.

[0102] Furthermore, the multi-spectral channel attention module includes:

[0103] Assume that the feature map X has dimensions C×H×W. If it is divided into n parts in the channel dimension, then each feature map X i The size of the channel dimension is

[0104] Perform two-dimensional discrete cosine transform (2D-DCT) of different frequency components on each feature map to obtain n compressed C′-dimensional vectors Freq i :

[0105]

[0106] Among them, 2DDCT represents two-dimensional discrete cosine transform, u i Indicates the frequency index in the horizontal direction, v i Indicates the frequency index in the vertical direction.

[0107] The n compressed C′-dimensional vectors Freq i Splicing, get C-dimensional multi-spectral vector Freq = concat ([Freq 0 , Freq 0 ,…,Freq n-1 ]);

[0108] Freq is processed through a fully connected layer operation and Sigmoid activation operation to generate a one-dimensional multi-spectral attention mechanism feature map M F (X);

[0109] M F (X) performs element-wise multiplication with the input feature map X:

[0110]

[0111] Among them, X′ represents the output feature of the input feature map X after spectral attention adjustment.

[0112] It should be understood that the multi-spectral channel attention module is used to achieve enhanced capture of the spatial frequency of particle fringes in the hologram.

[0113] Using YOLOV5s as a baseline model, an improved YOLOV5s network model was constructed as a holographic particle field detection network model. By adding a multi-spectral channel attention module to the backbone network, the network's ability to capture detailed particle streaks in holograms was enhanced. The backbone network of the existing YOLOV5s model primarily consists of a Conv module, a C3 module, and a Spatial Pyramid Pooling (SPPF) module. This module extracts input image features through convolutional layers and propagates them to the neck network. The neck network, consisting of a Feature Pyramid Network (FPN) and a Path Aggregation Network (PAN), extracts and fuses low-level spatial features with high-level semantic features for multi-scale object detection. The head network processes the results of the neck network's feature extraction, generates detection boxes, and provides information about the location, category, and confidence level of the detected object. The improved YOLOV5s network model optimizes prediction performance on the dataset.

[0114] Specifically, if Figure 2 , a multi-spectral channel attention module (FcaNet) is added to the backbone network to improve the ability to capture the details of particle stripes in the hologram.

[0115] The development environment required to run the improved model is shown in Table 1. These toolkits and Python libraries can be imported into the created Anaconda environment using the PyCharm terminal. Entering "pip install" automatically installs the toolkit and library by their names. The study employed transfer learning to accelerate model training. Transfer learning involves migrating a model trained on a large dataset to a smaller dataset. This allows the trained model to achieve good detection performance even with fewer samples, thereby reducing training time and accelerating model training. The study used YOLOV5s as the baseline model and used the YOLOV5s.pt weights generated from training on the COCO dataset as pre-trained weights for the improved model.

[0116] The hyperparameters used in training are shown in Table 2. The validation set was input into the trained holographic particle field detection network model; the model was evaluated using precision P, recall R, and mean average precision (mAP).

[0117] According to the experimental results in Table 3, the improved model with the addition of the FcaNet module improves the detection accuracy of holographic particle fields. However, the addition of the CBAM, SE, and ASFF modules reduces the detection accuracy to some extent. Because the interference fringes of holographic particles have frequency characteristics, the FCA module can be used to introduce more frequency components to learn information.

[0118] Compared to the original YOLOV5 model, the FcaNet module with the attention mechanism alone improves precision, recall, map@0.5, and map@[0.5:0.95] by 1.4%, 0.2%, 0.4%, and 0.4%, respectively. CBAM incorporates both spatial and channel-wise attention mechanisms.

[0119] Since the spatial and channel features of the hologram are not obvious except for the frequency domain, the precision and recall rates of adding the CBAM module are 74.9% and 89.9%, respectively, and the map@0.5 and map@[0.5:0.95] are 84% and 84%, respectively, and the F1 Score is 81.7%.

[0120] Compared to the original YOLOV5 model, precision and recall decreased by 21.7% and 7.4%, respectively. Map@0.5 and map@[0.5:0.95] decreased by 14.7% and 14.7%, respectively, and the F1 score decreased by 15.2%. The SE module achieves adaptive channel weight allocation through transformation, squeeze, excitation, and scale operations.

[0121] The precision and recall rates are 83.4% and 89.0%, respectively, while map@0.5 and map@[0.5:0.95] rates are 90.5% and 84%, respectively, and the F1 score is 90.5%. Compared to the original YOLOV5 model, the precision and recall rates are reduced by 13.2% and 8.3%, respectively, while map@0.5 and map@[0.5:0.95] rates are reduced by 8.2% and 8.2%, respectively, and the F1 score is reduced by 10.8%. The ASFF module dynamically selects and fuses multiple layers of feature maps to better capture multi-scale information of the object.

[0122] After adding the ASFF module, the precision and recall rates are 95.9% and 97.4%, map@0.5 and map@[0.5:0.95] are 98.6% and 98.6% respectively, and the F1 Score is 96.6%.

[0123] Compared with the original YOLOV5 model, the precision rate decreased by 0.7%, the recall rate increased by 0.1%, map@0.5 and map@[0.5:0.95] decreased by 0.1% and 0.1% respectively, and the F1 Score decreased by 0.3%.

[0124] Therefore, the FcaNet module was selected and added to the baseline network YOLOV5s. The performance of the resulting holographic particle field detection network model was improved compared to the original network. The test set was input into the trained holographic particle field detection network model to obtain the particle field detection results.

[0125] The experiment used a hologram test set containing a randomly generated particle field using Matlab. After linear interpolation, the resulting particle field hologram (Figure 3(a)) contains 16 particles with a diameter of 19.18 μm. Figure 3(b) shows the ground truth scatter plot of the particle field generated based on Figure 3(a), with the ground truth particles marked by hollow circles. Figure 3(a) was fed into the trained YOLOV5-FcaNet network for prediction, yielding the output image. The network detected all target particles and identified the depth category of each particle. Based on the network's output, a txt file containing the three-dimensional particle information, the three-dimensional particle field (Figure 3(c)) was reconstructed, with the predicted particles marked by plus signs. Figure 3(d) compares the reconstructed particle field with the ground truth particle field, with hollow circles representing the true particles and plus signs representing the predicted particles. As can be seen from the figure, the reconstructed particle field and the original ground truth particle field have a high degree of overlap, indicating that YOLOV5-FcaNet performs well in prediction.

[0126] In the task of particle field detection, in order to overcome the limitations of inductive detection sensors such as Coulter counters, which have high sample processing requirements, acoustic detection sensors such as acoustic Doppler flow profilers, which have low accuracy in measuring small particles, and optical detection sensors such as turbidity meters, which cannot measure particle size distribution, the present invention provides a holographic deep learning network for particle field detection. Based on digital holographic technology, the information of particles in space is recorded and then stored in a computer, which has the characteristics of easy operation, non-intrusiveness and pollution-free. Holograms stored by traditional digital holographic technology need to go through a reproduction process to reconstruct the amplitude and phase information of the object, which requires a certain amount of space and time costs. Applying deep learning networks to holographic particle field detection tasks can automatically complete the identification and three-dimensional position annotation of particles, and the calculation process requires less storage space and short prediction time. The study adds the FcaNet attention mechanism module to the YOLOV5s benchmark network, so that the improved network has a stronger ability to capture particle stripe information and higher prediction accuracy. The hologram deep learning network combines digital holographic technology and deep learning network. It predicts particle field information by training the improved YOLOV5s model. It has the characteristics of non-intrusive, fast, high precision and easy operation.

[0127] Table 1 Improved model development environment

[0128]

[0129]

[0130] Table 2 Hyperparameters for improving model training

[0131] parameter Numerical Learning rate 0.1 momentum 0.937 Weight decay 0.0005 IOU threshold 0.2 Batch size 16 Iteration rounds 300 Pre-trained weights yolov5s.pt

[0132] Table 3 Comparison of model performance after adding different modules

[0133]

[0134]

[0135] Example 2

[0136] This embodiment provides a particle position prediction system in a particle field based on a deep learning network;

[0137] The particle position prediction system in the particle field based on deep learning network includes:

[0138] An acquisition module is configured to: acquire a particle field hologram and mark particle positions;

[0139] A data set construction module is configured to: construct a data set based on a particle field hologram with known particle positions; and divide the data set into a training set, a validation set, and a test set;

[0140] A training module is configured to: input the training set into the deep learning network, train the network, verify the trained network using the validation set, and save the trained network;

[0141] The prediction module is configured to input the test set into the trained network to obtain the coordinate position and confidence of each particle. The deep learning network is implemented by an improved YOLOV5s. The improved YOLOV5s adds a multi-spectral channel attention module to the backbone network of the original YOLOV5s network. The multi-spectral channel attention module is used to improve the network's ability to capture particle streaks.

[0142] It should be noted that the acquisition module, dataset construction module, training module, and prediction module described above correspond to steps S101 to S104 in Example 1. The examples and application scenarios implemented by the modules and corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the modules described above, as part of a system, can be executed in a computer system, such as a set of computer-executable instructions.

[0143] The descriptions of the various embodiments in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0144] The proposed system can be implemented in other ways. For example, the system embodiment described above is merely illustrative. For example, the above module division is only a logical function division. In actual implementation, other division methods may be used. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not implemented.

[0145] Example 3

[0146] This embodiment also provides an electronic device, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes the method described in the above embodiment one.

[0147] It should be understood that in this embodiment, the processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0148] The memory may include a read-only memory and a random access memory, and provides instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0149] During implementation, each step of the above method may be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software.

[0150] The method in Example 1 can be directly implemented as being executed by a hardware processor, or by a combination of hardware and software modules within the processor. The software module can be located in a storage medium well-established in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not given here.

[0151] Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in conjunction with this embodiment can be implemented using electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0152] Embodiment 4: This embodiment further provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the method described in embodiment 1 is completed.

[0153] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A particle position prediction method in a particle field based on a deep learning network, characterized by: include: Obtain a particle field hologram and mark the particle positions; Construct a data set based on the particle field hologram with known particle positions; And divide the data set into training set, validation set and test set; Input the training set into the deep learning network to train the network, use the validation set to verify the trained network, and save the trained network; The test set is input into the trained network to obtain the coordinate position and confidence of each particle. The deep learning network is implemented by an improved YOLOV5s. The improved YOLOV5s adds a multi-spectral channel attention module to the backbone network of the original YOLOV5s network. The multi-spectral channel attention module is used to improve the network's ability to capture particle stripes.

2. The method for predicting particle positions in a particle field based on a deep learning network as claimed in claim 1, wherein: Get particle field holograms, including: The simulated particle hologram is produced using the Fresnel diffraction algorithm. U(x0,y0) is the input value of the Fresnel diffraction algorithm, and U(x,y) is the output value of the Fresnel diffraction algorithm. The simulated hologram light field U(x,y) at a distance from particle z1 is expressed as: Where U(x0,y0) is the object light field of the particle at the plane (x0,y0), k is the wave number, λ is the wavelength, and z1 is the recording distance of the particle; Convert formula (1) into convolution form: Taking Fourier transform on both sides of formula (2) we get: F is the symbol for Fourier transform, so: U(x,y)=C·F -1 {F(U(x0,y0))·exp(iπz1λ(u 2 +v 2 ))};(4) Among them, F -1 is the inverse Fourier transform symbol, C is a constant, u, v are spatial frequencies, 3. The method for predicting particle positions in a particle field based on a deep learning network as claimed in claim 1, wherein: Input the training set into the deep learning network to train the network, including: Classification loss L cls The calculation formula is: Bounding box regression loss L CIoU The calculation formula is: L CIoU =1-CIoU Confidence loss L conf The calculation formula is: Total loss function L total , is to convert the classification loss L cls , bounding box regression loss L CIoU and confidence loss L conf Perform weighted addition; L total =λ CIoU L CIoU +λ conf L conf +λ cls L cls ; Among them, λ CIoU is the weight of the bounding box regression loss, λ conf is the weight of the confidence loss, λ cls is the weight of the classification loss.

4. The method for predicting particle positions in a particle field based on a deep learning network as claimed in claim 1, wherein: The improved YOLOV5s includes: The backbone network, neck network and head network are connected in sequence; The backbone network includes: a first convolutional layer, a second convolutional layer, a first C3 layer, a third convolutional layer, a second C3 layer, a fourth convolutional layer, a third C3 layer, a fifth convolutional layer, a multi-spectral channel attention module, an SPPF module and a sixth convolutional layer connected in sequence; The neck network includes: a first upsampling layer, a first splicing unit, a fourth C4 layer, a seventh convolutional layer, a second upsampling layer, a second splicing unit, a fifth C3 layer, an eighth convolutional layer, a third splicing unit, a sixth C3 layer, a ninth convolutional layer, a fourth splicing unit, and a seventh C3 layer connected in sequence; The head network includes: a first detection head, a second detection head and a third detection head; The input end of the first upsampling layer is connected to the output end of the sixth convolutional layer; the input end of the first splicing unit is also connected to the output end of the third C3 layer; the input end of the second splicing unit is also connected to the output end of the second C3 layer; the input end of the third splicing unit is also connected to the input end of the seventh convolutional layer; the input end of the fourth splicing unit is also connected to the input end of the sixth convolutional layer; The input end of the first detection head is also connected to the input end of the fifth C3 layer; the input end of the second detection head is also connected to the input end of the sixth C3 layer; the input end of the third detection head is also connected to the input end of the seventh C3 layer.

5. The method for predicting particle positions in a particle field based on a deep learning network as claimed in claim 4, wherein: The internal structures of the first C3 layer, the second C3 layer, the third C3 layer, the fourth C3 layer, the fifth C3 layer, the sixth C3 layer and the seventh C3 layer are consistent; The first C3 layer includes: a twelfth convolutional layer, a bottleneck module, a sixth splicing unit, and a thirteenth convolutional layer connected in sequence, wherein the output end of the twelfth convolutional layer is also connected to the input end of the sixth splicing unit; The bottleneck module is located in different positions, and its internal structure is different; A bottleneck module in the backbone network includes: a first CBS module, a second CBS module, and an adder connected in sequence, wherein an input end of the adder is residually connected to an input end of the first CBS module; The bottleneck module in the neck network includes a third CBS module and a fourth CBS module connected in sequence.

6. The method for predicting particle positions in a particle field based on a deep learning network as claimed in claim 4, wherein: The multi-spectral channel attention module includes: Assume that the feature map X has dimensions C×H×W. If it is divided into n parts in the channel dimension, then each feature map X i The size of the channel dimension is Perform two-dimensional discrete cosine transform on each feature map at different frequency components to obtain n compressed C′-dimensional vectors Freq i : Among them, 2DDCT represents two-dimensional discrete cosine transform, u i Indicates the frequency index in the horizontal direction, v i Indicates the frequency index in the vertical direction; The n compressed C′-dimensional vectors Freq i Splicing, get C-dimensional multi-spectral vector Freq = concat ([Freq 0 , Freq 0 ,…,Freq n-1 ]); Freq is processed through a fully connected layer operation and Sigmoid activation operation to generate a one-dimensional multi-spectral attention mechanism feature map M F (X); M F (X) performs element-wise multiplication with the input feature map X: Among them, X′ represents the output feature of the input feature map X after spectral attention adjustment.

7. The method for predicting particle positions in a particle field based on a deep learning network as claimed in claim 1, wherein: The expression of confidence is:

8. A particle position prediction system in a particle field based on a deep learning network, characterized by including: An acquisition module is configured to: acquire a particle field hologram and mark particle positions; A data set construction module is configured to: construct a data set based on a particle field hologram with known particle positions; and divide the data set into a training set, a validation set, and a test set; A training module is configured to: input the training set into the deep learning network, train the network, verify the trained network using the validation set, and save the trained network; The prediction module is configured to input the test set into the trained network to obtain the coordinate position and confidence of each particle. The deep learning network is implemented by an improved YOLOV5s. The improved YOLOV5s adds a multi-spectral channel attention module to the backbone network of the original YOLOV5s network. The multi-spectral channel attention module is used to improve the network's ability to capture particle streaks.

9. An electronic device, comprising: a memory for non-transitory storage of computer-readable instructions; as well as a processor for executing said computer-readable instructions, When the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is executed.

10. A storage medium, characterized in that: Non-transitory storage of computer-readable instructions, wherein when the non-transitory computer-readable instructions are executed by a computer, the method according to any one of claims 1 to 7 is performed.

Images