A method and device for detecting red tide algae in three dimensions based on holographic imaging

Abstract

The application discloses a kind of red tide algae three-dimensional detection methods based on holographic imaging, comprising: build the coaxial light path based on Mach-Zehnder interferometer structure;Determine the reconstruction distance under different object distance of this system;Establish the original holographic image database of red tide algae;Select original hologram, and two different reconstruction distance reappearing hologram are used as the three-dimensional space sampling of this original hologram;Algae in holographic three-dimensional space sampling is marked, and the kind and boundary box of algae are recorded;The target detection network is trained using the three-dimensional space sampling of hologram and the file of marked;The kind and quantity information of plankton in imaging water body are obtained by using trained target detection neural network to detect the holographic three-dimensional space sampling of algae.The application further provides a kind of red tide algae three-dimensional detection device.The method provided by the application can improve the detection speed and accuracy of red tide algae species.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of red tide algae monitoring, and relates to a three-dimensional detection method and device for red tide algae based on holographic imaging. BACKGROUND

[0002] The frequent occurrence of harmful red tide algae has caused serious ecological damage and brought huge economic losses to the marine aquaculture industry. The current main methods for monitoring red tide algae include optical microscopy, flow cytometry, fluorescence spectroscopy, and holographic microscopy.

[0003] The depth of field of a conventional optical microscope is very small, and it can only clearly image red tide algae on the focal plane, resulting in a small sample volume for each detection and a small amount of information that can be obtained.

[0004] Flow cytometry is an instrument for detecting cells in a fluid. Bubbles generated by a peristaltic pump can be identified as algae by the instrument, causing errors. In addition, red tide algae may have damaged cell walls after passing through the peristaltic pump, resulting in errors in the identification results.

[0005] The fluorescence detection method has certain requirements for the abundance of algae in seawater, and cannot detect algae in low-density conditions.

[0006] The existing holographic microscopy method for detecting red tide algae needs to extract the target area of the algae first, then focus automatically, and then classify, but the algorithm is complex and time-consuming, and the accuracy of automatic focusing is high.

[0007] Patent document CN114418995A discloses a cascading algae cell statistical method based on a microscope image, which includes: collecting and labeling algae image sample data, and constructing a deep learning model; training the deep learning model to obtain a deep learning detection model; identifying the labeled image sample data based on the deep learning detection model to obtain an identification result. This method has an identification accuracy problem. Since there may be bubbles or impurities in the sample liquid, they may be identified as algae cells.

[0008] Patent document CN215727660U A kind of ocean red tide rapid detection system based on spectral imaging, including main controller and the spectral imaging device of sample culture dish imaging;The spectral imaging device includes industrial camera, also includes background plate and light source in dark box;The sample culture dish is placed at background plate and is illuminated by light source;The light source includes halogen light source and the filter at light input end is provided with the light line of the characteristic wavelength of red tide algae related and can be transmitted;The filter is installed on the runner, and the runner is driven by stepper motor;The main controller acquires sample image in culture dish by controlling industrial camera and stepper motor to carry out red tide detection for red tide detection.The method is identified to the spectral image feature of red tide algae, but it needs to spend a lot of time for the acquisition and processing of spectrum, and the accuracy is not high. SUMMARY

[0009] To solve the above problems, the present application provides a simple operation red tide algae three-dimensional detection method, which can improve the detection speed and accuracy of red tide algae species.

[0010] A kind of red tide algae three-dimensional detection method based on holographic imaging, comprising:

[0011] Step 1, based on the light path form of Mach-Zehnder interferometer, construct digital holographic imaging system;

[0012] Step 2, using the digital holographic imaging system, take hologram of different red tide algae, construct corresponding red tide algae original hologram data set;

[0013] Step 3, according to the preset reconstruction distance, the hologram of red tide algae is reconstructed to obtain corresponding two holographic reconstruction images with different reconstruction distances, and the hologram of red tide algae and the corresponding two holographic reconstruction images form three-dimensional space sampling data;

[0014] Step 4, according to the reconstruction process of step 3, process the red tide algae original hologram image data set obtained in step 2 to obtain the corresponding three-dimensional space sampling data set;

[0015] Step 5, label the red tide algae in three-dimensional space sampling data set to obtain the corresponding label set;

[0016] Step 6, using the three-dimensional space sampling data set in step 4 and the label set obtained in step 5, train the pre-constructed target detection model to obtain the red tide algae detection model for detecting red tide algae;

[0017] Step 7, input the three-dimensional space sampling data of the red tide algae to be detected into the red tide algae detection model, output the detection result of the red tide algae to be detected, and the detection result includes the species, quantity and position information of the red tide algae.

[0018] The application is based on the three-dimensional space sampling data of the hologram of red tide algae and two corresponding holographic reconstruction images, a detection model is trained to obtain a corresponding red tide algae detection model, and the detection model is used for analysis and detection of the algae to be detected to output the detection result of the red tide algae.

[0019] Specifically, the red tide algae original holographic image data set is obtained by shooting a cuvette with a back wall fixed on the focal plane of the object optical axis of the digital holographic imaging system and an optical path of 200 mu m.

[0020] Specifically, in step 3, the reconstruction distances are 0.08 m and 0.16 m, respectively.

[0021] Specifically, the holographic reconstruction image adopts an angular spectrum reconstruction algorithm to obtain the hologram of red tide algae.

[0022] Specifically, the angular spectrum reconstruction algorithm is specifically expressed as follows:

[0023]

[0024] In the formula, denotes Fourier transform, denotes inverse Fourier transform, Filter{·} denotes a frequency filter for obtaining the +1 order spectrum and the-1 order spectrum of the sample, λ denotes the wave field of the light source, z denotes the reconstruction distance, G(f x ,f y ,z) denotes the optical transfer function in the frequency domain corresponding to the propagation distance.

[0025] Specifically, in step 3, the three-dimensional space sampling data is a three-dimensional matrix data composed of the hologram of red tide algae and two holographic reconstruction images in the order of reconstruction distance from small to large.

[0026] Specifically, in step 5, the label set includes the species of red tide algae, the boundary box position and the size, and the boundary box position is the XY two-dimensional plane position.

[0027] The application also provides a red tide algae three-dimensional detection device, which comprises a computer memory, a computer processor and a computer program stored in the computer memory and executable on the computer processor, and the computer memory adopts the red tide algae detection model described above.

[0028] When the computer processor executes the computer program, the following steps are realized: inputting the three-dimensional space sampling data of the red tide algae to be detected, analyzing and detecting the red tide algae detection model, and outputting the detection result of the red tide algae to be detected.

[0029] Compared with the prior art, the application has the following beneficial effects:

[0030] This invention eliminates the need for extracting and cropping the image region of red tide algae. It only requires synthesizing the original hologram with two corresponding holographic reconstruction images to form three-dimensional spatial sampling data, which is then used to detect red tide algae through a trained red tide algae detection model, thereby improving the accuracy and speed of red tide algae identification. Attached Figure Description

[0031] Figure 1 A schematic flowchart of the three-dimensional detection method for red tide algae based on holographic imaging provided by the present invention;

[0032] Figure 2 This is a schematic diagram of the coaxial optical path based on the Mach-Zehnder interferometer structure provided in this embodiment;

[0033] Figure 3 for Figure 2 Enlarged side view of structure A in the middle;

[0034] Figure 4 This is a schematic diagram of the reconstructed planar position of the holographic reconstruction image group provided in this embodiment;

[0035] Figure 5 Holographic images of some of the red tide algae provided in this embodiment. Detailed Implementation

[0036] To facilitate understanding by those skilled in the art, the structure of the present invention will now be described in further detail with reference to the accompanying drawings:

[0037] like Figure 1 As shown, a three-dimensional detection method for red tide algae based on holographic imaging includes the following steps:

[0038] Step 1, according to Figure 2 The Mach-Zehnder interferometer structure shown is used to build the optical path. In order to detect micro- and small planktonic organisms, a microscope objective is introduced behind the sample for magnification, thus forming a digital holographic imaging system.

[0039] Step 2: Use a resolution plate and a linear slide to determine the hologram reconstruction distance at different object distances. First, fix the resolution plate as follows: Figure 3 On the manual linear slide shown, observe the hologram captured by the photodetector on the computer, determine the focal plane on the object axis, adjust the manual linear slide to gradually adjust the position of the resolution plate from the focal plane to 200μm in front of the focal plane, and record the holographic images when the resolution plate is in different positions.

[0040] Manually determine whether the resolution plate in the image is in focus, and determine the holographic reconstruction distance when in focus. Replace the resolution plate with a cuvette (sample cuvette: containing the sample) and fix it on a manual linear slide. Figure 3As shown, the inner wall of the back side of the cuvette is fixed on the focal plane. The same cuvette (reference cuvette: pure water inside) is placed at the same position of the reference arm on the other side;

[0041] In this embodiment, the optical path of the sample cuvette is 200 μm, and several kinds of red tide algae are selected for experiment. The hologram of the red tide algae is photographed, and the original hologram image dataset of the red tide algae is established.

[0042] Step 3, according to the reconstruction distance and the optical path of the cuvette obtained by the experiment, the reconstruction distance is calculated as Figure 4 As shown, the reconstruction distance of 0.08 m and 0.16 m is selected to reconstruct the original hologram, and two holographic reconstruction images are obtained. According to the angular spectrum reconstruction algorithm, the hologram is reconstructed:

[0043]

[0044] In the formula, denotes the Fourier transform, denotes the inverse Fourier transform, Filter{·} denotes the frequency filter for obtaining the sample +1 order spectrum and -1 order spectrum, λ denotes the wave field of the light source, z denotes the reconstruction distance, G(f x ,f y ,z) denotes the optical transfer function in the frequency domain corresponding to the propagation distance;

[0045] The original hologram and the two holographic reconstruction images form a three-dimensional matrix in order of increasing reconstruction distance. This three-dimensional matrix is the three-dimensional spatial sampling data of the original hologram. The matrix is exported and saved in RGB picture format.

[0046] Step 4, according to the reconstruction process of step 3, the original hologram image dataset of the red tide algae obtained in step 2 is processed to obtain the corresponding three-dimensional spatial sampling dataset.

[0047] Step 5, label the three-dimensional spatial sampling dataset with the species of red tide algae, the position and size of the bounding box, and obtain the corresponding label set.

[0048] Step 6, group the three-dimensional spatial sampling dataset and the corresponding label set into a sample set, and divide the sample set into a training set, a validation set and a test set according to a certain proportion.

[0049] Use the training set to train different target detection models, and evaluate the models through the test set. The evaluation parameters are average precision (AP) and recall, and the formulas are as follows:

[0050]

[0051]

[0052] In the formula, TP (true positive) represents the number of samples that are judged as positive samples by the model and are actually positive samples, FN (false negative) represents the number of samples that are judged as negative samples by the model but are actually positive samples, and FP (false positive) represents the number of samples that are judged as positive samples by the model but are actually negative samples;

[0053] IOU refers to the ratio of the overlapping area of two bounding boxes to the combined area. AP is an evaluation index that comprehensively considers recall rate and accuracy. They are expressed as:

[0054]

[0055]

[0056] In the formula, TB represents a real bounding box, PB represents a predicted bounding box, t represents a preset system number, when IOU is greater than t, it is assumed that the model has correctly detected the target, and p(t) represents the precision when the recall rate is r(t);

[0057]

[0058] In the formula, F1 represents the final evaluation index.

[0059] According to the above evaluation index, the model is evaluated, so as to select a red tide algae detection model that meets the requirements.

[0060] Step 7, input the three-dimensional space sampling data of the red tide algae to be detected into the red tide algae detection model, and output the species, quantity and position information of the red tide algae to be detected.

[0061] The embodiment also provides a red tide algae three-dimensional detection device, which comprises a computer memory, a computer processor and a computer program stored in the computer memory and executable on the computer processor, and the computer memory adopts the red tide algae detection model described above;

[0062] When the computer processor executes the computer program, the following steps are implemented: input the three-dimensional space sampling data of the red tide algae to be detected, analyze and detect the red tide algae to be detected by the red tide algae detection model, and output the detection result of the red tide algae to be detected.

[0063] In the embodiment, common toxic red tide algae: Alexandrium tamarense (AT), Karenia mikimotoi (KM), Prorocentrum donghaiense (PD), Prorocentrum lima (PL), Prorocentrum micans (PM), Karenia brevis (KV), Heterosigma akashiwo (HA) and Karenia mikimotoi (KM) are taken as examples, and two detection models are selected for testing.

[0064] AsFigure 5 Figure 1 shows a hologram of part of a red tide algae.

[0065] YOLOv4-tiny and YOLOv7 models were built and trained using the dataset obtained in the above steps. The results are shown in Tables 1 and 2.

[0066] Table 1 Detection results of YOLOv4-tiny

[0067]

[0068] Table 2 Detection results of YOLOv7

[0069]

[0070] For the same red tide algae image data, the time required by the YOLOv4-tiny and YOLOv7 models is as follows:

[0071]

[0072] In addition, the mean average precision of the above two models reached 98.5% and 99.55%, respectively.

Claims

1. A method for detecting red tide algae in three dimensions based on holographic imaging, characterized in that, The method comprises the following steps: Step 1, constructing a digital holographic imaging system based on the optical path form of a Mach-Zehnder interferometer; Step 2, using the digital holographic imaging system to shoot holograms of different red tides, and constructing a corresponding red tide original holographic image dataset; Using a resolution plate and a linear slide to determine the hologram reconstruction distance at different object distances, first, fix the resolution plate on the manual linear slide, observe the hologram shot by the photoelectric detector on the computer, determine the focal plane on the object optical axis, adjust the manual linear slide, and gradually adjust the position of the resolution plate from the focal plane to 200 μm in front of the focal plane, and record the hologram image when the resolution plate is at different positions; Artificially determine whether the resolution plate in the image is focused to determine the hologram reconstruction distance when focused; replace the resolution plate with a sample cuvette fixed on the manual linear slide, and fix the inner wall of the back side of the sample cuvette on the focal plane; the same reference cuvette is placed at the same position of the reference arm on the other side; Using the optical path of 200 μm where the sample cuvette is placed, select several kinds of red tides for experiment, shoot holograms of the red tides, and establish a red tide original holographic image dataset; Step 3, according to the preset reconstruction distance, reconstructing the hologram of the red tide to obtain two corresponding holographic reconstruction images with different reconstruction distances, and combining the hologram of the red tide with the two corresponding holographic reconstruction images to form three-dimensional space sampling data; Selecting the reconstruction distances of 0.08 m and 0.16 m to reconstruct the original hologram to obtain two holographic reconstruction images, and according to the angular spectrum reconstruction algorithm, the hologram is reconstructed: wherein denotes the Fourier transform, denotes the inverse Fourier transform, Filter{·} denotes a frequency filter for obtaining the +1 and -1 order spectra of the sample, λ denotes the wave field of the light source, z denotes the reconstruction distance, G(f x ,f y ,z) denotes the optical transfer function in the frequency domain corresponding to the propagation distance; Step 4, according to the reconstruction process of step 3, processing the red tide original holographic image dataset obtained in step 2 to obtain a corresponding three-dimensional space sampling dataset; Step 5, labeling the red tides in the three-dimensional space sampling dataset to obtain a corresponding label set; Step 6, using the three-dimensional space sampling dataset in step 4 and the label set obtained in step 5, training a pre-constructed target detection model to obtain a red tide detection model for detecting red tides; Step 7, inputting the three-dimensional space sampling data of the red tide to be detected into the red tide detection model, and outputting the detection result of the red tide to be detected, wherein the detection result includes the type, quantity and position information of the red tide.

2. The method according to claim 1, wherein the method is a holographic imaging based method for detecting red tide algae in three dimensions. In step 2, the red tide original holographic image dataset is obtained by shooting a cuvette with a rear wall fixed on the focal plane of the object optical axis of the digital holographic imaging system and an optical path of 200 μm.

3. The method according to claim 1, wherein the method is characterized by, In step 3, the three-dimensional space sampling data is a three-dimensional matrix data composed of the hologram of the red tide and two holographic reconstruction images in order of reconstruction distance from small to large.

4. The method according to claim 1, wherein the method is characterized by, In step 5, the label set includes the type, bounding box position and size of the red tide, and the bounding box position is the XY two-dimensional plane position.

5. A device for detecting red tide algae in three dimensions, comprising a computer memory, a computer processor, and a computer program stored in the computer memory and executable on the computer processor, wherein, When the computer processor executes the computer program, the method for detecting red tides in three dimensions based on holographic imaging according to any one of claims 1-4 is realized.

Images