Method for realizing artificial intelligence classification and identification of blood cells based on Fourier laminated microscopic imaging
Through binocular stereoscopic vision and neural network coding illumination methods, combined with deep learning and microfluidic chip design, the problem of inaccurate position of the illumination array in Fourier stacked microscopy is solved, and high-resolution and quantitative phase imaging are achieved. Improve the accuracy and efficiency of blood cell classification recognition.
Patent Information
- Application Number
- CN202411600497.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-11
- Publication Date
- 2025-01-03
AI Technical Summary
The inaccurate position of the illumination array in Fourier stacked microscopy results in inaccurate position during aperture synthesis and reduced reconstruction quality. The time resolution of traditional FPM algorithms is limited, resulting in poor blood cell image quality.
Using LED pose calibration method based on binocular stereo vision and encoding illumination method based on neural network, the intensity images at different illumination angles are collected for spatial domain constraints, and aperture synthesis is performed in the spectrum, combining deep learning object detection algorithm and Microfluidic chip design enables high resolution and quantitative phase imaging.
It improves the accuracy and efficiency of blood cell classification recognition, avoids the reduction in reconstruction quality caused by inaccurate position of the illumination array, and achieves large field of view, high resolution and quantitative phase imaging.
Smart Images
Figure BDA0005128175790000061 
Figure BDA0005128175790000071
Abstract
Description
Technical Field
[0001] The present invention relates to the field of blood cell identification and classification, and in particular to a method for realizing blood cell artificial intelligence classification and identification based on Fourier stack microscopy imaging. Background Art
[0002] Blood cells, also known as "blood corpuscles", are cells that exist in the blood and can spread throughout the body with the flow of blood. For mammals, blood cells mainly include the following three types: red blood cells: the main function is to transport oxygen, white blood cells: mainly play an immune role. When pathogens invade the human body, white blood cells can pass through the capillary walls, concentrate at the site of invasion, surround the pathogens and then engulf them, platelets: play an important role in the hemostasis process. Blood cells account for about 45% of the blood volume, including red blood cells, white blood cells and platelets. Under normal physiological conditions, blood cells and platelets have certain morphological structures and relatively stable numbers.
[0003] When performing microscopic imaging of blood cells through Fourier stacking, the inaccurate position of the illumination array will lead to inaccurate positions of each sub-region during aperture synthesis, which in turn leads to a decrease in reconstruction quality. In addition, the traditional FPM algorithm needs to sequentially collect intensity images under the illumination of a single LED light at different positions, which limits the temporal resolution of its imaging, resulting in poor image quality.
[0004] Therefore, the present invention proposes a method for realizing artificial intelligence classification and identification of blood cells based on Fourier stack microscopy imaging. Summary of the invention
[0005] The main purpose of the present invention is to provide a method for realizing artificial intelligence classification and identification of blood cells based on Fourier stack microscopy imaging, which can effectively solve the problems in the background technology.
[0006] To achieve the above object, the technical solution adopted by the present invention is:
[0007] The method for realizing artificial intelligence classification and recognition of blood cells based on Fourier stack microscopy imaging includes the following steps:
[0008] S1: Cell culture and acquisition: including red blood cells and white blood cells. The culture and acquisition of red blood cells include the following steps: dilute 500 μL of whole blood with an equal volume of blood cell analysis diluent, spread 1 ml of blood cell analysis diluent on 3 mL of separation solution, centrifuge at 800g acceleration for 30 minutes, and then there will be obvious stratification, from top to bottom, there are plasma layer, lymphocyte layer, separation solution layer and red blood cell layer, use a pipette to absorb red blood cells from the red blood cell layer, wash with blood cell diluent 2-3 times, and then add diluent to resuspend;
[0009] The culturing and obtaining of white blood cells includes the following steps: lysing 2 mL of blood with 6 mL of red blood cell lysis buffer and incubating on ice for 15 minutes, centrifuging the incubated blood at 4°C at an acceleration of 450g for 10 minutes, removing the supernatant, and recollecting the precipitate with 4 mL of lysis buffer, centrifuging under the same conditions for 10 minutes, removing the supernatant, obtaining the white blood cell precipitate, washing it 2-3 times with a blood cell analysis diluent, and then resuspending it;
[0010] S2: Microscopic imaging of cells based on Fourier stacking: construct an illumination array of any shape through lighting equipment; use binocular stereo vision method to obtain the relative position of the lamp post of the lighting equipment; calculate the position using the light plate relationship of the edge image of the bright and dark fields; reconstruct the image according to the Fourier stacking microscopic imaging algorithm, and image the cells based on this;
[0011] S3: Design and preparation of microfluidic chip: The microfluidic structure is designed by L-EDIT and printed on a transparent chrome mask. The master mold of the microfluidic channel is made using a standard photolithography process: SU8-2025 photoresist is used on a 4-inch silicon wafer. Polydimethylsiloxane, curing agent and activating agent are mixed at a mass ratio of 1000:100:1 and poured on the master mold of the microfluidic channel. After the bubbles are discharged under a vacuum environment, the PDMS cover is cured at 90°C to form a PDMS cover. Finally, the PDMS cover and the glass slide surface are oxidized and bonded using the principle of oxygen plasma bonding to complete the preparation of the microfluidic chip.
[0012] S4: Dataset creation: The images in the dataset come from the images of cells in step S2. The dataset is divided into training set, validation set and test set. During the division process, cell images from the same sample will not appear in the training set, validation set and test set at the same time. The open source image annotation tool Colabeler is used to annotate the cells in the image. The number of cells in each image in the dataset ranges from 1 to 15. Each cell is manually labeled as RBC or WBC according to its true value. The training set contains 8,000 images, including 4,000 RBC images and 4,000 WBC images; the validation set contains 1,000 images, including 500 RBC images and 500 WBC images; the test set contains 1,000 images, including 500 RBC images and 500 WBC images;
[0013] S5: Cell target detection model: The target detection algorithm of the model is derived from YOLO. Through a deep convolutional neural network, the position and category information of the target object can be obtained from the original image data, and the target detection problem is converted into a regression problem. The training and verification of the cell detection model are both run on a workstation with an Ubuntu 20.04 operating system. The model is trained using a pre-trained model in the form of transfer learning. During the training process, the network uses asynchronous stochastic gradient descent with a momentum term of 0.94. Each batch contains 64 images, the initial learning rate of the model is 0.001, the classification confidence threshold is 0.25, the intersection-over-union threshold is 0.5, and the maximum number of iterations is 40,000. During the training process, the average accuracy of the model is calculated using the verification set after every two generations of data learning, thereby completing the intelligent classification and recognition of blood cells.
[0014] Preferably, step S2 includes two parts: one is to generate a specific illumination pattern according to a physical neural network based on an ideal imaging process; the other is to reconstruct the amplitude and phase of the target using a multi-parameter physical neural network based on the collected image.
[0015] Preferably, in step S3, the microchannel is sealed for 30 minutes using a 1% volume fraction bovine serum albumin solution before use to reduce adhesion between cells and the channel wall, wherein the inlet of the microfluidic chip adopts a distributed structure design, and pillars and serpentine structures are designed at the inlet and outlet of the microfluidic chip to improve the overall flow resistance of the chip. The middle part of the microfluidic chip also involves a channel, and the imaging area of the channel is designed as a long straight channel with a cross-section of 400 μm×20 μm and a length of 600 μm.
[0016] Preferably, in step S4, the image resolution in the data set is unified as 416×416.
[0017] Compared with the prior art, the present invention has the following beneficial effects:
[0018] In the present invention, on the basis of Fourier stack microscopy imaging, a method of LED posture calibration based on binocular stereo vision is proposed to calibrate irregular LED arrays. The study also introduces a coded illumination method based on a neural network to improve the temporal resolution of FPM. By collecting intensity images under different illumination angles as spatial domain constraints and performing aperture synthesis in the spectrum, large field of view, high resolution and quantitative phase imaging are simultaneously achieved. The phenomenon that the inaccurate position of each sub-region during aperture synthesis due to the inaccurate posture of the illumination array will be avoided, thereby causing the reconstruction quality to decrease. In addition, the traditional FPM algorithm needs to collect intensity images under the illumination of a single LED lamp at different positions in sequence, which limits the temporal resolution of its imaging. The present invention can avoid the occurrence of such problems. The classification and recognition of cells based on the images obtained by this method can effectively improve its accuracy and efficiency.
[0019] The present invention combines the target detection algorithm in deep learning with the manufacturing process of microfluidic chips, and proposes a method for classifying and counting whole blood cells. The microfluidic chip adopts a decentralized inlet and wide channel design to facilitate obtaining a higher test throughput. The cell images in the data set basically cover all angles of flipping of red and white blood cells in the microchannel, so as to obtain a higher detection accuracy. DETAILED DESCRIPTION
[0020] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0021] The present invention relates to a method for realizing artificial intelligence classification and identification of blood cells based on Fourier stack microscopy imaging, comprising the following steps:
[0022] S1: Cell culture and acquisition: including red blood cells and white blood cells. The culture and acquisition of red blood cells include the following steps: dilute 500 μL of whole blood with an equal volume of blood cell analysis diluent, spread 1 ml of blood cell analysis diluent on 3 mL of separation solution, centrifuge at 800g acceleration for 30 minutes, and then there will be obvious stratification, from top to bottom, there are plasma layer, lymphocyte layer, separation solution layer and red blood cell layer, use a pipette to absorb red blood cells from the red blood cell layer, wash with blood cell diluent 2-3 times, and then add diluent to resuspend;
[0023] The culturing and obtaining of white blood cells includes the following steps: lysing 2 mL of blood with 6 mL of red blood cell lysis buffer and incubating on ice for 15 minutes, centrifuging the incubated blood at 4°C at an acceleration of 450g for 10 minutes, removing the supernatant, and recollecting the precipitate with 4 mL of lysis buffer, centrifuging under the same conditions for 10 minutes, removing the supernatant, obtaining the white blood cell precipitate, washing it 2-3 times with a blood cell analysis diluent, and then resuspending it;
[0024] S2: Microscopic imaging of cells based on Fourier stacking: It includes two parts. One is to generate a specific lighting pattern based on a physical neural network based on an ideal imaging process; the other is to reconstruct the amplitude and phase of the target using a multi-parameter physical neural network based on the collected image. It includes two parts. One is to generate a specific lighting pattern based on a physical neural network based on an ideal imaging process; the other is to reconstruct the amplitude and phase of the target using a multi-parameter physical neural network based on the collected image. The lighting array of any shape is constructed through lighting equipment; the relative position of the lamp post of the lighting equipment is obtained using the binocular stereo vision method; the position is calculated using the light plate relationship of the bright and dark field edge image; the image is reconstructed according to the Fourier stacking microscopic imaging algorithm, and the image of the cell is imaged based on this;
[0025] S3: Design and preparation of microfluidic chip: The microfluidic structure was designed by L-EDIT and printed on a transparent chrome mask. The master mold of the microfluidic chip was made using a standard photolithography process: SU8-2025 photoresist was used on a 4-inch silicon wafer. Polydimethylsiloxane, curing agent and activating agent were mixed at a mass ratio of 1000:100:1 and poured on the master mold of the microfluidic chip. After the bubbles were discharged under a vacuum environment, the PDMS cover was cured at 90°C to form a cover. Finally, the PDMS cover was bonded to the glass surface using the principle of oxygen plasma bonding. The microfluidic chip can be made by performing oxidation treatment and completing bonding. Before using the microfluidic channel, a bovine serum albumin solution with a volume fraction of 1% is used to seal the microfluidic channel for 30 minutes to reduce the adhesion between cells and the channel wall. The inlet of the microfluidic chip adopts a decentralized structure design. Pillars and serpentine structures are designed at the inlet and outlet of the microfluidic chip to improve the overall flow resistance of the chip. The middle part of the microfluidic chip also involves a channel, and the imaging area of the channel is designed as a long straight channel with a cross-section of 400μm×20μm and a length of 600μm.
[0026] S4: Dataset creation: The images in the dataset come from the images of cells in step S2. The dataset is divided into training set, validation set and test set. During the division process, cell images from the same sample will not appear in the training set, validation set and test set at the same time. The open source image annotation tool Colabeler is used to annotate the cells in the image. The number of cells in each image in the dataset ranges from 1 to 15, and each cell is manually labeled as RBC or WBC according to its true value. The training set contains 8,000 images, including 4,000 RBC images and 4,000 WBC images; the validation set contains 1,000 images, including 500 RBC images and 500 WBC images; the test set contains 1,000 images, including 500 RBC images and 500 WBC images. The image resolution in the dataset is unified to 416×416.
[0027] S5: Cell target detection model: The target detection algorithm of the model is derived from YOLO. Through a deep convolutional neural network, the position and category information of the target object can be obtained from the original image data, and the target detection problem is converted into a regression problem. The training and verification of the cell detection model are both run on a workstation with an Ubuntu 20.04 operating system. The model is trained using a pre-trained model in the form of transfer learning. During the training process, the network uses asynchronous stochastic gradient descent with a momentum term of 0.94. Each batch contains 64 images, the initial learning rate of the model is 0.001, the classification confidence threshold is 0.25, the intersection-over-union threshold is 0.5, and the maximum number of iterations is 40,000. During the training process, the average accuracy of the model is calculated using the verification set after every two generations of data learning, thereby completing the intelligent classification and recognition of blood cells. Specific embodiment:
[0029] Based on the operation steps of the present invention, a corresponding experiment was set up to verify the effectiveness of the method in blood cell counting. In the experiment, whole blood was diluted with a volume ratio of 1:50, and 10 groups of blood samples were tested under the same experimental conditions. The blood cell counting results finally obtained are shown in the following table:
[0030]
[0031]
[0032] Based on the comparison between the above table and the adult red and white blood cell count ratios in the clinical blood routine analysis report, the distribution range of the measured blood cell count ratios is basically consistent with the distribution range in the blood routine analysis report. Therefore, it can be seen that the test results of the present invention have clinical reference value.
[0033] In the present invention, on the basis of Fourier stack microscopy imaging, a method of LED posture calibration based on binocular stereo vision is proposed to calibrate irregular LED arrays. The study also introduces a coded illumination method based on a neural network to improve the temporal resolution of FPM. By collecting intensity images under different illumination angles as spatial domain constraints and performing aperture synthesis in the spectrum, large field of view, high resolution and quantitative phase imaging are simultaneously achieved. The phenomenon that the inaccurate position of each sub-region during aperture synthesis due to the inaccurate posture of the illumination array will be avoided, thereby causing the reconstruction quality to decrease. In addition, the traditional FPM algorithm needs to collect intensity images under the illumination of a single LED lamp at different positions in sequence, which limits the temporal resolution of its imaging. The present invention can avoid the occurrence of such problems. The classification and recognition of cells based on the images obtained by this method can effectively improve its accuracy and efficiency.
[0034] The present invention combines the target detection algorithm in deep learning with the manufacturing process of microfluidic chips, and proposes a method for classifying and counting whole blood cells. The microfluidic chip adopts a decentralized inlet and wide channel design to facilitate obtaining a higher test throughput. The cell images in the data set basically cover all angles of flipping of red and white blood cells in the microchannel, so as to obtain a higher detection accuracy.
[0035] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A method for realizing artificial intelligence classification and identification of blood cells based on Fourier stack microscopy imaging, characterized in that: The steps include: S1: Cell culture and acquisition: These include red blood cells and white blood cells. The culture and acquisition of red blood cells includes the following steps: Dilute 500 μL of whole blood with an equal volume of blood cell analysis diluent, spread 1 ml of blood cell analysis diluent on 3 mL of separation solution, centrifuge at 800 g for 30 min, and obvious stratification will appear, including plasma layer, lymphocyte layer, separation solution layer and red blood cell layer from top to bottom. Use a pipette to draw red blood cells from the red blood cell layer, wash with blood cell diluent 2-3 times, and then add diluent to resuspend; The culturing and obtaining of white blood cells includes the following steps: lysing 2 mL of blood with 6 mL of red blood cell lysis buffer and incubating on ice for 15 minutes, centrifuging the incubated blood at 4°C at an acceleration of 450g for 10 minutes, removing the supernatant, and recollecting the precipitate with 4 mL of lysis buffer, centrifuging under the same conditions for 10 minutes, removing the supernatant, obtaining the white blood cell precipitate, washing it 2-3 times with a blood cell analysis diluent, and then resuspending it; S2: Microscopic imaging of cells based on Fourier stacking: construct an illumination array of any shape through lighting equipment; use binocular stereo vision method to obtain the relative position of the lamp post of the lighting equipment; calculate the position using the light plate relationship of the edge image of the bright and dark fields; reconstruct the image according to the Fourier stacking microscopic imaging algorithm, and image the cells based on this; S3: Design and preparation of microfluidic chip: The microfluidic structure is designed by L-EDIT and printed on a transparent chrome mask. The master mold of the microfluidic channel is made using a standard photolithography process: SU8-2025 photoresist is used on a 4-inch silicon wafer. Polydimethylsiloxane, curing agent and activating agent are mixed at a mass ratio of 1000:100:1 and poured on the master mold of the microfluidic channel. After the bubbles are discharged under a vacuum environment, the PDMS cover is cured at 90°C to form a PDMS cover. Finally, the PDMS cover and the glass slide surface are oxidized and bonded using the principle of oxygen plasma bonding to complete the preparation of the microfluidic chip. S4: Dataset creation: The images in the dataset come from the images of cells in step S2. The dataset is divided into training set, validation set and test set. During the division process, cell images from the same sample will not appear in the training set, validation set and test set at the same time. The open source image annotation tool Colabeler is used to annotate the cells in the image. The number of cells in each image in the dataset ranges from 1 to 15. Each cell is manually labeled as RBC or WBC according to its true value. The training set contains 8,000 images, including 4,000 RBC images and 4,000 WBC images; the validation set contains 1,000 images, including 500 RBC images and 500 WBC images; the test set contains 1,000 images, including 500 RBC images and 500 WBC images; S5: Cell target detection model: The target detection algorithm of the model is derived from YOLO. Through a deep convolutional neural network, the position and category information of the target object can be obtained from the original image data, and the target detection problem is converted into a regression problem. The training and verification of the cell detection model are both run on a workstation with an Ubuntu 20.04 operating system. The model is trained using a pre-trained model in the form of transfer learning. During the training process, the network uses asynchronous stochastic gradient descent with a momentum term of 0.
94. Each batch contains 64 images, the initial learning rate of the model is 0.001, the classification confidence threshold is 0.25, the intersection-over-union threshold is 0.5, and the maximum number of iterations is 40,000. During the training process, the average accuracy of the model is calculated using the verification set after every two generations of data learning, thereby completing the intelligent classification and recognition of blood cells.
2. The method for realizing artificial intelligence classification and identification of blood cells based on Fourier stack microscopy imaging according to claim 1 is characterized in that: The step S2 includes two parts: one is to generate a specific illumination pattern according to a physical neural network based on an ideal imaging process; the other is to reconstruct the amplitude and phase of the target using a multi-parameter physical neural network according to the collected image.
3. The method for realizing artificial intelligence classification and identification of blood cells based on Fourier stack microscopy imaging according to claim 1, characterized in that: In the step S3, the microchannel is sealed for 30 minutes with a bovine serum albumin solution having a volume fraction of 1% before use to reduce adhesion between cells and the channel wall, wherein the inlet of the microfluidic chip adopts a distributed structure design, and pillars and serpentine structures are designed at the inlet and outlet of the microfluidic chip to improve the overall flow resistance of the chip. The middle part of the microfluidic chip also involves a channel, and the imaging area of the channel is designed as a long straight channel with a cross-section of 400 μm×20 μm and a length of 600 μm.
4. The method for realizing artificial intelligence classification and identification of blood cells based on Fourier stack microscopy imaging according to claim 1, characterized in that: In step S4, the image resolution in the data set is unified as 416×416.