Cell sorting method and system based on image acoustophoretic cell sorting model

Abstract

The application discloses a cell sorting method and system based on an image acoustic flow control cell sorting model and a computer readable storage medium, the image acoustic flow control cell sorting model comprises a cell image recognition module and a cell ejection module, and the cell sorting method comprises the following steps: acquiring an original cell image set and a predetermined sample cell image set; inputting the sample cell image set into the cell image recognition module for feature extraction to obtain image category information of the cell image; inputting the image category information and the sample image set into the image acoustic flow control cell sorting model for training; inputting the original cell image into the trained image acoustic flow control cell sorting model for image prediction to determine a target image corresponding to the image category; and ejecting a cell corresponding to the target image to a preset collection area based on the cell ejection module. In the embodiment of the application, the automatic classification of cells can be performed by using a label-free method, so that the purification and collection of target cells are realized.

Description

Technical Field

[0001] This invention relates to the field of cell observation technology, and in particular to a cell sorting method, system, and computer-readable storage medium based on an image-based acoustic fluid control cell sorting model. Background Technology

[0002] With the development of cell therapy and gene therapy technologies, cell sequencing and cell culture require increasingly higher purity of obtained cells. Cell sorting, in particular, involves separating a specific cell population from a mixture of multiple cell populations based on their characteristics. Traditional separation methods include fluorescence-activated cell sorting (FACS) and immunomagnetic cell sorting. FACS works by staining the target cells with a specific fluorescent dye and placing them in a sample tube. The cells are then introduced into a flow chamber filled with sheath fluid under gas pressure. This requires complex labeling treatment beforehand, which can affect cell viability. Immunomagnetic cell sorting uses antibodies bound to magnetic beads to label target cells. These cells then move under the influence of a magnetic field, separating them from cells without the beads. While this method is fast, it can only sort specific cell types. Therefore, both methods typically require complex pretreatment processes and additional signal tags to identify target cells, which can reduce cell viability. Summary of the Invention

[0003] The present invention aims to at least solve one of the technical problems existing in the prior art, and provides a cell sorting method, system and computer-readable storage medium based on an image-based acoustic fluid control cell sorting model, which can automatically classify cells using a label-free method, thereby achieving the purification and collection of target cells.

[0004] In a first aspect, the present invention provides a cell sorting method based on an Image Acoustofluidics Cell Sorter (IACS) model, wherein the image acoustic fluid control cell sorting model includes a cell image recognition module and a cell ejection module, and the cell sorting method includes:

[0005] Acquire a raw cell image set and a predetermined sample cell image set, wherein the sample cell image set includes multiple cell images carrying detection markers, and the raw cell image set includes multiple raw cell images;

[0006] The sample cell image set is input into the cell image recognition module, which then performs feature extraction on multiple cell images in the sample cell image set based on a preset convolutional neural network and the detection identifier to obtain image type information of the cell images.

[0007] The image type information and the sample cell image set are input into the image acoustic fluid control cell sorting model for training;

[0008] The original cell images are input into the trained image acoustic fluid control cell sorting model for image prediction to determine the image category of the original cell images. The original cell image set is then filtered according to the image category to determine the target image corresponding to the image category.

[0009] The cell ejection module ejects cells corresponding to the target image to a preset collection area.

[0010] The cell sorting method based on an image-acoustic flow-controlled cell sorting model provided by the present invention has at least the following beneficial effects: First, it acquires an original cell image set and a pre-determined sample cell image set carrying detection markers, facilitating the training of the image-acoustic flow-controlled cell sorting model. The sample cell image set is then input into the cell image recognition module of the image-acoustic flow-controlled cell sorting model. This allows the cell image recognition module to extract features from multiple cell images in the sample cell image set based on a pre-defined convolutional neural network and detection markers, thereby obtaining image type information for the cell images. This accurately determines the image type information of the cell images, achieving precise classification of the cell images. Subsequently, the image type information and sample cell images are combined... The cell image set is input into the image acoustic fluid control cell sorting model for training, thereby improving the accuracy of image classification. Then, the original cell images are input into the trained image acoustic fluid control cell sorting model for image prediction to determine the image category of the original cell images. Based on the image category, the images in the original cell image set are filtered to obtain the target images corresponding to the image categories, thereby achieving precise screening of image categories and improving the accuracy of cell classification. Finally, based on the cell ejection module, the cells corresponding to the target images are ejected to the preset collection area to complete the cell sorting. This enables automatic cell classification using a label-free method, thereby achieving the purification and collection of target cells.

[0011] According to some embodiments of the present invention, the sample cell image set is obtained by the following steps:

[0012] Obtain a mixed sample of cells;

[0013] Cells in the mixed cell sample are separated based on acoustic radiation force to obtain a sample cell set;

[0014] The sample cell set was image acquired to obtain cell images;

[0015] The cell image is resized;

[0016] Image labeling is performed on the cell images after size adjustment to obtain a sample cell image set carrying detection markers, which facilitates subsequent training of the image acoustic fluid control cell sorting model. The model achieves precise cell separation through acoustic radiation force, realizing non-contact cell separation and improving cell separation efficiency.

[0017] According to some embodiments of the present invention, the cell image recognition module includes a feature extractor; the cell image recognition module performs feature extraction on cell images in the sample cell image set based on a preset convolutional neural network and the detection identifier to obtain image type information of the cell images, including:

[0018] The cell images are input into the convolutional layer of the preset convolutional neural network for encoding to obtain image features of multiple cell images;

[0019] The image features of all the cell images are input into the feature extractor, which performs dimensionality reduction on the cell images based on the detection identifier and the convolution kernel, and performs feature prediction on the dimensionality-reduced image features to obtain the prediction result;

[0020] Based on the prediction results and the image features, the image type information of the cell image is obtained, which facilitates the subsequent prediction of the category of the original image and improves the accuracy of the prediction of the original image.

[0021] According to some embodiments of the present invention, the preset convolutional neural network includes a channel dimension and a feature layer; the step of inputting the cell image into the convolutional layer of the preset convolutional neural network for encoding to obtain image features of multiple cell images includes:

[0022] The cell image is input into the preset convolutional neural network, which upsamples the cell image to obtain multiple predicted feature maps.

[0023] Channel splicing is performed on the channel dimensions to obtain the predicted branches;

[0024] Based on the predicted branch and the feature layer, tensor concatenation is performed on multiple predicted feature maps to obtain image features of multiple cell images, thereby improving the efficiency of image feature extraction.

[0025] According to some embodiments of the present invention, the step of tensor concatenating multiple predicted feature maps based on the predicted branch and the feature layer to obtain image features of multiple cell images includes:

[0026] The predicted feature map is input into the feature layer for calculation to obtain the predicted feature value;

[0027] The predicted feature values ​​are integrated based on the channel dimension of the predicted branch to obtain the image features of the cell image, thereby improving the accuracy of image category judgment.

[0028] According to some embodiments of the present invention, the detection identifier includes the position coordinate information of the cell in the cell image and preset category information; the step of inputting the image features of all the cell images into the feature extractor, such that the feature extractor performs dimensionality reduction processing on the cell images according to the detection identifier and the convolutional layer, and performs feature prediction on the dimensionality-reduced image features to obtain a prediction result, includes:

[0029] The image features of the cell image are input into the feature extractor, so that the feature extractor generates a target anchor box carrying a detection label on the cell image according to the position coordinates and the preset category information;

[0030] The cell image is dimensionality reduced using a convolutional layer, and features are predicted based on the target anchor box to obtain the prediction result, thereby improving the accuracy of feature prediction for the cell image.

[0031] According to some embodiments of the present invention, the step of inputting the image category information and the sample cell image set into the image acoustic fluid control cell sorting model for training includes:

[0032] The image category information and the sample cell image set are input into the image acoustic fluid control cell sorting model, so that the image acoustic fluid control cell sorting model calculates the position coordinate information and the preset category information to obtain a confidence value;

[0033] The confidence score value is normalized based on the target anchor frame to obtain the confidence index value;

[0034] The confidence index value is compared with a preset threshold to obtain the comparison result;

[0035] The image acoustic fluid control cell sorting model is trained based on the comparison results to improve its ability to predict cell categories and achieve accurate prediction of cell images.

[0036] According to some embodiments of the present invention, the step of inputting the original cell image into the trained image acoustic fluid control cell sorting model for image prediction and determining the image category of the original cell image includes:

[0037] The original cell image is input into the trained image acoustic fluid control cell sorting model, so that the cell image recognition module predicts the original cell image and obtains the predicted location information and predicted type information of the original cell image.

[0038] The prediction anchor box of the original cell image is determined based on the prediction location information and the prediction type information, and the prediction index value is obtained based on the prediction anchor box.

[0039] The image category of the original cell image is determined based on the predicted index value, thereby achieving accurate judgment of the image category and avoiding misjudgment.

[0040] Secondly, the present invention provides a cell sorting system based on an image-based acoustic fluid control cell sorting model, comprising:

[0041] The sample acquisition module is used to acquire a raw cell image set and a predetermined sample cell image set, wherein the sample cell image set includes multiple cell images carrying detection markers, and the raw cell image set includes multiple raw cell images;

[0042] A cell image recognition module is used to receive the sample cell image set and extract features from multiple cell images in the sample cell image set based on a preset convolutional neural network and the detection identifier to obtain image type information of the cell images;

[0043] The model training module is used to input the image type information and the sample cell image set into the image acoustic fluid control cell sorting model for training;

[0044] The image determination module is used to input the original cell image into the trained image acoustic fluid control cell sorting model for image prediction, determine the image category of the original cell image, filter the original cell image set according to the image category, and determine the target image corresponding to the image category;

[0045] The cell ejection module is used to eject cells corresponding to the target image to a preset collection area.

[0046] Thirdly, the present invention provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the cell sorting method based on the image-acoustic flow control cell sorting model as described in the first aspect.

[0047] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the description and the drawings. Attached Figure Description

[0048] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

[0049] Figure 1 This is a schematic diagram of the architecture of a sorting system based on an image-based acoustic fluid control cell sorting model provided in one embodiment of the present invention;

[0050] Figure 2 This is a flowchart of a cell sorting method based on an image-based acoustic fluid control cell sorting model provided in one embodiment of the present invention;

[0051] Figure 3 This is a flowchart of obtaining a sample cell image set according to an embodiment of the present invention;

[0052] Figure 4 yes Figure 2 The flowchart of the specific method for step S200 in the process;

[0053] Figure 5 yes Figure 4 The flowchart of the specific method for step S210 in the process;

[0054] Figure 6 yes Figure 5 The flowchart of the specific method for step S213 in the process;

[0055] Figure 7 yes Figure 4 The flowchart of the specific method for step S220 in the process;

[0056] Figure 8 yes Figure 2 The flowchart of the specific method for step S300 in the process;

[0057] Figure 9 yes Figure 2 The flowchart of the specific method for step S400 in the process;

[0058] Figure 10 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0060] It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0061] This invention provides a cell sorting method, system, and computer-readable storage medium based on an image-based acoustic fluid control cell sorting model. First, an original set of cell images and a pre-determined set of sample cell images carrying detection markers are acquired to facilitate training the image-based acoustic fluid control cell sorting model. The sample cell image set is then input into the cell image recognition module of the model. The cell image recognition module extracts features from multiple cell images in the sample cell image set based on a pre-defined convolutional neural network and detection markers to obtain cell image type information, thereby accurately determining the cell image type and achieving precise cell image classification. Finally, the image type information and sample cell images are processed together. The cell image set is input into the image acoustic fluid control cell sorting model for training, thereby improving the accuracy of image classification. Then, the original cell images are input into the trained image acoustic fluid control cell sorting model for image prediction to determine the image category of the original cell images. Based on the image category, the images in the original cell image set are filtered to obtain the target images corresponding to the image categories, thereby achieving precise screening of image categories and improving the accuracy of cell classification. Finally, based on the cell ejection module, the cells corresponding to the target images are ejected to the preset collection area to complete the cell sorting. This enables automatic cell classification using a label-free method, thereby achieving the purification and collection of target cells.

[0062] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0063] like Figure 1 As shown, Figure 1 This is a schematic diagram of the architecture of a sorting system based on an image-based acoustic fluid control cell sorting model provided in one embodiment of the present invention.

[0064] exist Figure 1 In the example, the sorting system includes, but is not limited to, a sample acquisition module 100, a cell image recognition module 200, a model training module 300, an image determination module 400, and a cell ejection module 500.

[0065] In some embodiments, the sample acquisition module 100 is used to acquire a raw cell image set and a predetermined sample cell image set, wherein the sample cell image set includes multiple cell images carrying detection markers, and the raw image set includes multiple raw cell images. The images in both the raw cell image set and the sample cell image set are obtained by taking 20x magnification images of cells in a flowing cell mixture solution in a polydimethylsiloxane chip using a CCD (Charge Coupled Device) camera under a microscope.

[0066] It should be noted that the cell mixture solution can be a mixture of multiple cell types, such as breast cancer cells, cancer cells, prostate cells, platelet cells, etc. This embodiment does not impose specific limitations.

[0067] In some embodiments, the cell image recognition module 200 is used to receive a sample cell image set and extract features from multiple cell images in the sample cell image set based on a preset convolutional neural network and detection labels to obtain image type information of the cell images. This accurate acquisition of image type information of the cell images facilitates subsequent training of the image acoustic fluid control cell sorting model and improves the classification ability of the image acoustic fluid control cell sorting model.

[0068] It should be noted that the cell image recognition module is capable of recognizing and classifying cell images based on deep learning.

[0069] In some embodiments, the model training module 300 is used to input image category information and sample cell image set into the image acoustic fluid control cell sorting model for training, thereby improving the image classification performance of the image acoustic fluid control cell sorting model and enhancing the accuracy of image classification.

[0070] In some embodiments, the image determination module 400 is used to input the original cell image into the trained image acoustic fluid control cell sorting module for image prediction, thereby determining the image category of the original cell image, and then filtering the original cell image set according to the image category to determine the target image corresponding to the image category, thereby achieving accurate determination of the target image and realizing the identification of unlabeled cell types.

[0071] It should be noted that the image identification module 400 identifies images by using a deep network with an improved residual network structure. The deep network integrates different network layers to extract features such as cell size, shape, and contour in the image, thereby improving the accuracy of identifying the image type.

[0072] In some embodiments, the cell ejection module 500 is used to eject cells corresponding to the target image to a preset collection area, thereby completing the cell sorting process, realizing the sorting of specific cells, and completing the separation and purification of specific cells.

[0073] It should be noted that the cell ejection module is a focused interdigitated transducer (FIDT) used to generate focused traveling surface acoustic waves (FTSAW). The FIDT generates acoustic radiation force (ARF) under the drive of a high-voltage pulse signal emitted by a signal generator and amplified by a power amplifier, and drives the cells to move along the direction of the acoustic radiation force, thereby achieving cell sorting.

[0074] Understandably, FIDT can generate surface acoustic waves with higher intensity and narrower beamwidth compared to SIDT. Higher energy intensity can generate higher cell sorting driving force, thereby improving cell sorting efficiency.

[0075] The sorting system and application scenarios described in the embodiments of the present invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided in the embodiments of the present invention.

[0076] Based on the above sorting system, the following are various embodiments of the cell sorting method based on the image acoustic fluid control cell sorting model of the present invention.

[0077] like Figure 2 As shown, Figure 2 This is a flowchart of a cell sorting method based on an image acoustic fluid control cell sorting model provided in an embodiment of the present invention. The cell sorting method based on the image acoustic fluid control cell sorting model includes, but is not limited to, steps S100 to S500.

[0078] It should be noted that the image-acoustic fluid control cell sorting model includes a cell image recognition module and a cell ejection module.

[0079] Step S100: Obtain the original cell image set and the predetermined sample cell image set;

[0080] It should be noted that the sample cell image set includes multiple cell images carrying detection markers, while the raw cell image set includes multiple raw cell images.

[0081] In some embodiments, the images in the original cell image set and the sample cell image set are obtained by taking images of cells in a flowing cell mixture solution in a PDMS chip under a microscope at 20x magnification using a CCD camera.

[0082] Step S200: Input the sample cell image set into the cell image recognition module, so that the cell image recognition module extracts features from multiple cell images in the sample cell image set based on a preset convolutional neural network and detection labels to obtain the image type information of the cell images;

[0083] In some embodiments, a sample cell image set is input into a cell image recognition module, which then extracts features from the cell images based on a preset convolutional neural network to obtain the image types of the cell images, facilitating subsequent training of the image-based acoustic fluid control cell sorting model.

[0084] It should be noted that the default convolutional neural network is the pre-trained YOLOv3 (You Only Look Once Version 3, YOLOv3) algorithm, and the cell image recognition module recognizes images through a deep network with an improved residual network structure. The deep network integrates different network layers to extract features such as cell size, shape, and contour in the image, thereby improving the accuracy of determining the image type.

[0085] Step S300: Input the image type information and sample cell image set into the image acoustic fluid control cell sorting model for training;

[0086] In some embodiments, image type information and sample cell image sets are input into the image acoustic fluid control cell sorting model for training, thereby improving the image acoustic fluid control cell sorting model's ability to predict cell images and achieving high-precision cell sorting.

[0087] Step S400: Input the original cell image into the trained image acoustic fluid control cell sorting model for image prediction, determine the image category of the original cell image, filter the original cell image set according to the image category, and determine the target image corresponding to the image category;

[0088] In some embodiments, the original cell image is input into a trained image acoustic fluid control cell sorting model for image prediction, thereby determining the image category of the original cell image. Finally, the images in the original cell image set are filtered according to the image category to determine all target images corresponding to the image category, thereby realizing the classification of different image categories in the original cell image set, which facilitates the subsequent sorting of cells.

[0089] Step S500: Based on the cell ejection module, eject the cells corresponding to the target image to the preset collection area.

[0090] In some embodiments, the cell ejection module based on the image-acoustic fluid control cell sorting model ejects cells corresponding to the target image to a preset collection area, thereby achieving label-free sorting of target cells.

[0091] It should be noted that after identifying the cells corresponding to the target image, the position of the cells needs to be determined. Once it is determined that the cells are located within the preset ejection area, the cells are ejected to the preset collection area to complete the sorting and collection of the target cells.

[0092] In some embodiments, the cell ejection module combines interdigital electrodes to generate acoustic radiation force for cell ejection and collection. The interdigital electrodes can be adjusted with different design structures and parameters to deflect different types of target cells, thereby achieving cell sorting.

[0093] like Figure 3 As shown, Figure 3 This is a flowchart of obtaining a sample cell image set according to an embodiment of the present invention. The method for obtaining the sample cell image set includes, but is not limited to, steps S110 to S150.

[0094] Step S110: Obtain a mixed cell sample;

[0095] Step S120: Separate cells from the mixed cell sample based on acoustic radiation force to obtain a sample cell set;

[0096] Step S130: Acquire images of the sample cell set to obtain cell images;

[0097] Step S140: Resize the cell image;

[0098] Step S150: Image labeling is performed on the size-adjusted cell images to obtain a sample cell image set carrying detection markers.

[0099] In some embodiments, obtaining a sample cell image set requires first acquiring a mixed cell sample, then separating the cells in the mixed cell sample based on acoustic radiation force to obtain a sample cell set, achieving precise cell separation. Subsequently, the sample cell set is imaged under a microscope using a CCD camera to obtain cell images. The cell images are then resized to adjust their size, thereby accelerating the subsequent image acquisition of cell images and improving the efficiency of feature extraction. Finally, the resized cell images are labeled to obtain a sample cell image set carrying detection markers.

[0100] It should be noted that the cell mixture sample can be a mixture of cells with different diameters, different cell types, or different cell functions. This embodiment does not impose any specific limitations.

[0101] like Figure 4 As shown, Figure 4 yes Figure 2 The flowchart of the specific method of step S200 is as follows, and step S200 includes but is not limited to steps S210-S230.

[0102] It should be noted that the cell image recognition module includes a feature extractor.

[0103] Step S210: Input the cell images into the convolutional layer of a preset convolutional neural network for encoding to obtain image features of multiple cell images;

[0104] In some embodiments, cell images are input into multiple convolutional layers of a preset convolutional neural network, whereby the first convolutional layer encodes the size and shape information of the cell images to obtain first image features. The first image features are then input into the second convolutional layer for encoding, and so on, layer by layer, until the last convolutional layer, thereby obtaining the image features of each cell image.

[0105] Step S220: Input the image features of all cell images into the feature extractor, so that the feature extractor performs dimensionality reduction on the cell images based on the detection labels and convolution kernels, and performs feature prediction on the dimensionality-reduced image features to obtain the prediction results;

[0106] In some embodiments, the image features of all cell images are input into a feature extractor, which performs dimensionality reduction on the cell images based on the convolution kernel of its internal convolution kernel structure, and performs feature prediction on the dimensionality-reduced image features according to the detection identifier to obtain the final prediction result. Dimensionality reduction can change the color of the cell image from color to black and white, and from three dimensions to one dimension, thereby accelerating the efficiency of feature extraction.

[0107] It should be noted that the convolutional kernel structure inside the feature extractor is a convolutional set. The feature extractor extracts information such as the contour, roundness, and transparency of the cell image, thereby increasing the accuracy of feature extraction.

[0108] Step S230: Obtain image type information of cell images based on prediction results and image features.

[0109] In some embodiments, the image type information of the cell image is finally determined based on the prediction results and image features, thereby accurately determining the type of cell image and facilitating subsequent training of the image acoustic fluid control cell sorting model.

[0110] like Figure 5 As shown, Figure 5 yes Figure 4 The flowchart of the specific method of step S210 is as follows: step S210 includes, but is not limited to, steps S211-S213.

[0111] It should be noted that the default convolutional neural network includes channel dimensions and feature layers.

[0112] Step S211: Input the cell image into a preset convolutional neural network, so that the preset convolutional neural network upsamples the cell image to obtain multiple predicted feature maps;

[0113] In some embodiments, a cell image is input into a preset convolutional neural network, such that the upsampling convolutional blocks in the preset convolutional neural network perform upsampling operations on the cell image to obtain multiple predicted feature maps, thereby reducing the dimensionality of the features and preserving the effective information of the cell image.

[0114] Step S212: Perform channel splicing on the channel dimension to obtain the predicted branches;

[0115] Step S213: Perform tensor concatenation on multiple predicted feature maps based on the predicted branches and feature layers to obtain image features of multiple cell images.

[0116] In some embodiments, the cell image is first input into a preset convolutional neural network, which then upsamples the cell image and outputs multiple predicted feature maps at different scales. Next, channel concatenation is performed to integrate the input of the current feature layer with the output of the previous feature layer, thereby expanding the channel dimension of the tensor and obtaining the predicted branch. Finally, the predicted feature maps are concatenated based on the predicted branch and the feature layer to obtain the image features of multiple cell images.

[0117] It should be noted that the backbone network of YOLOv3 in this embodiment adopts the Darknet-53 structure. The upsampled convolutional block (Darknetconv2d_BN_Leaky, DBL) is the smallest component of YOLOv3, consisting of a two-dimensional convolutional layer, a batch normalization (BN) layer, and activation function layers (Rectified Linear Units, LeakyReLu). The residual unit (also called Res-Unit) allows the network structure to be deeper, consisting of two DBLs with 1×1 and 3×3 convolutional kernels respectively.

[0118] like Figure 6 As shown, Figure 6 yes Figure 5 The flowchart of the specific method of step S213 is as follows: step S213 includes, but is not limited to, steps S2131-S2132.

[0119] Step S2131: Input the predicted feature map into the feature layer for calculation to obtain the predicted feature value;

[0120] Step S2132: Integrate the predicted feature values ​​according to the channel dimension of the predicted branch to obtain the image features of the cell image.

[0121] In some embodiments, the predicted feature map is input into the feature layer for calculation, so that the feature map calculates the entire predicted feature map and accurately distinguishes the target and background of the predicted feature map. The feature layer detects the predicted feature map based on a preset convolutional neural network, divides the entire image into regions, calculates the predicted feature map in each region to obtain the predicted feature value, and then integrates the predicted feature value according to the channel dimension of the predicted branch to obtain the image features of the cell image.

[0122] like Figure 7 As shown, Figure 7 yes Figure 4 The flowchart of the specific method of step S220 is as follows: step S220 includes, but is not limited to, steps S221-S222.

[0123] It should be noted that the detection identifier includes the cell's position coordinates in the cell image and preset category information.

[0124] Step S221: Input the image features of the cell image into the feature extractor, so that the feature extractor generates a target anchor box carrying a detection label on the cell image according to the position coordinates and preset category information;

[0125] Step S222: Perform dimensionality reduction on the cell image based on the convolutional layer, and perform feature prediction on the dimensionality-reduced cell image based on the target anchor box to obtain the prediction result.

[0126] In some embodiments, the image features of the cell image are input into a feature extractor, which generates an anchor box at the cell's location based on the cell's position coordinates. Then, a detection label is generated within the anchor box based on preset category information, thereby obtaining a target anchor box carrying the detection label. Subsequently, the cell image is dimensionality reduced based on the convolution kernel in the convolutional layer. Feature prediction is performed on the dimensionality-reduced cell image within a pre-divided region based on the target anchor box to obtain the prediction result. This improves the accuracy of cell image prediction and facilitates subsequent training of the image-based acoustic fluid control cell sorting model.

[0127] like Figure 8 As shown, Figure 8 yes Figure 2 The flowchart of the specific method of step S300 is as follows, and step S300 includes but is not limited to steps S310-S340.

[0128] Step S310: Input the image type information and sample cell image set into the image acoustic fluid control cell sorting model, so that the image acoustic fluid control cell sorting model can calculate the position coordinate information and preset category information to obtain the confidence value;

[0129] Step S320: Normalize the confidence scores based on the target anchor frame to obtain the confidence index value;

[0130] Step S330: Compare the confidence index value with the preset threshold to obtain the comparison result;

[0131] Step S340: Train the image acoustic fluid sorting model based on the comparison results.

[0132] In some embodiments, the training operation of the image acoustic fluid control cell sorting model first inputs the image category information and the sample cell image set into the image acoustic fluid control cell sorting model for training. This allows the image acoustic fluid control cell sorting model to calculate the cell position coordinate information and preset category information to obtain a confidence value. Then, the confidence value is normalized according to the target anchor box to obtain a confidence index value, thereby improving the training accuracy and the image category prediction ability of the image acoustic fluid control cell sorting model. Next, the confidence index value is compared with a preset threshold to obtain a comparison result. Finally, the image acoustic fluid control cell sorting model is trained according to the comparison result, thereby completing the training of the image acoustic fluid control cell sorting model.

[0133] It should be noted that after inputting image category information and a sample cell image set into the image-acoustic fluid control cell sorting model, the model will progressively extract image features from the cell images in the sample cell image set through convolutional layers and output multi-scale feature maps for prediction. The multi-scale feature maps are then calculated to obtain confidence scores. Based on the extracted features and loss calculation results, the coordinates of the target anchor boxes are progressively adjusted, and their categories are determined to obtain confidence index values. Finally, the prediction result for the target in the image is obtained. Training the image-acoustic fluid control cell sorting model mainly involves calculating the model loss using the model parameters in each iteration, i.e., the comparison between the confidence index value and a preset threshold. The loss value is then backpropagated to update the model parameters, allowing the model to gradually fit the input data.

[0134] like Figure 9 As shown, Figure 9 yes Figure 2 The flowchart of the specific method of step S400 is as follows, and step S400 includes but is not limited to steps S410-S430.

[0135] Step S410: Input the original cell image into the trained image acoustic fluid control cell sorting model, so that the cell image recognition module can predict the original cell image and obtain the predicted position information and predicted type information of the original cell image;

[0136] Step S420: Determine the prediction anchor box of the original cell image based on the prediction location information and prediction type information, and obtain the prediction index value based on the prediction anchor box;

[0137] Step S430: Determine the image category of the original cell image based on the predicted index value.

[0138] In some embodiments, the original cell image is input into a trained image acoustic fluid control cell sorting model, which then predicts the original cell image to obtain the predicted location information and predicted type information of the original cell image. Based on the predicted location information and predicted type information, the model determines the predicted anchor box of the original cell image, thereby achieving accurate cell identification. The model then obtains the predicted index value based on the predicted anchor box, and determines the image category of the original cell image based on the predicted index value, thereby achieving cell type identification.

[0139] To more clearly illustrate the process of the cell sorting method based on the image-based acoustic fluid control cell sorting model provided in this embodiment of the invention, specific examples are given below.

[0140] Example 1:

[0141] This example is a specific illustration of a cell sorting method based on an image-based acoustic fluid control cell sorting model.

[0142] Step 1: Obtain the original cell image set and the pre-determined sample cell image set;

[0143] It should be noted that the cell images were obtained by taking 20x magnification photographs of a mixture of leukocytes and cancer cells under a microscope using a CCD camera. Of these, 360 images were randomly selected as the sample cell image set, 120 as the validation set, and 120 as the original cell image set. The average number of cells per image was approximately 20.

[0144] In some embodiments, during the process of acquiring the original cell image set and the predetermined sample cell image set, it is necessary to uniformly convert the tagged image file format (TIFF) images captured by the CCD camera into a lossy image compression algorithm format (Joint Photographic Experts Group, JIP).

[0145] It should be noted that in order to make the data images suitable for the improved YOLOv3 model, the image size needs to be adjusted to 416×416. However, any stretching transformation may cause the target image to be deformed. In order to preserve image information as much as possible during convolution, this embodiment first scales the input images required for training and testing to 416×n (n<416), then centers them, and fills the top and bottom sides of the n-th side with black to generate an image of size 416×416. After scaling, the dataset is created. The transformed cell images are labeled using YOLO_mark to indicate the image category, and the x and y coordinates of the center point of the label box, as well as the width w and height h of the cropped image, are recorded to obtain a predetermined sample cell image set.

[0146] Step 2: Input the sample cell image set into the cell image recognition module, so that the cell image recognition module can extract features from multiple cell images in the sample cell image set based on the YOLOv3 network and detection labels to obtain the image type information of the cell images;

[0147] It should be noted that in this embodiment, the image features are continuously upsampled using DBL+ and then tensor-concatenated with the shallow network to obtain a larger feature map, thereby achieving better small target detection results. This embodiment sets up four feature output layers responsible for prediction: a 13×13 feature map for large targets, a 26×26 feature map for medium-sized targets, a 52×52 feature map for small targets, and a 104×104 feature map for very small targets. Although adding detection layers increases computation and slows down detection and classification, it significantly improves the recognition effect for small targets.

[0148] It is understandable that the core of a convolutional layer is to use a convolutional kernel to map the features of the previous layer to the next layer. The mathematical expression of the next layer is shown in the following formula (1):

[0149] H i =W i ·H i-1 +b i (1)

[0150] Among them, H i H represents the output after the convolution operation. i-1 W represents the input features. i Denotes the convolution kernel, b iTo indicate the deviation, in order to keep the output size of the convolution consistent with the input size, the same padding pattern is used in each convolutional layer to pad the outer edge of the input matrix with zero values, and the stride of the convolution kernel is set to 1 each time. During the convolution calculation, the mapping area of ​​the convolution kernel on the input image is called the receptive field, and its size is the same as that of the convolution kernel. By sliding the receptive field across the image, the inner product of the current receptive field and the convolution kernel matrix is ​​calculated each time as the feature value output at that position. Until the receptive field has traversed the entire image, the output feature map after the convolution operation on the image can be obtained. In order to eliminate redundant information and reduce overfitting, the ReLU activation function is used in the residual network, and its expression is shown in the following formula (2):

[0151] ReLU(x) = max(0,x) (2)

[0152] Step 3: Input image type information and sample image set into the image acoustic fluid control cell sorting model for training;

[0153] It should be noted that after inputting image category information and a sample cell image set into the image-based acoustic fluid control cell sorting model, the model progressively extracts image features from the cell images in the sample cell image set through convolutional layers and outputs multi-scale feature maps for prediction. The multi-scale feature maps are then used to calculate confidence scores. Based on the extracted features and loss calculation results, the coordinates of the target anchor boxes are progressively adjusted, and their categories are determined to obtain confidence index values. Finally, the prediction result for the target in the image is obtained. Training the image-based acoustic fluid control cell sorting model mainly involves calculating the model loss using the model parameters in each iteration—that is, the comparison between the confidence index value and a preset threshold. This loss value is then backpropagated to update the model parameters, allowing the model to gradually fit the input data. The final detection process involves inputting the image into the trained model and directly outputting the prediction result using the existing model parameters. Because there are a large number of pre-set anchor boxes, and a target may be identified as different categories with different confidence levels, a large number of predicted boxes are obtained directly. Moreover, a large portion of these outputs are bad or even wrong results. Therefore, it is necessary to design a filtering algorithm to extract the truly meaningful detection results.

[0154] In some embodiments, this embodiment uses the non-maximum suppression (NMS) method to filter out prediction boxes with a confidence level less than 0.5.

[0155] Step 4: Input the original cell images into the trained image acoustic fluid control cell sorting model for image prediction, determine the image category of the original cell images, filter the original cell image set according to the image category, and determine the target images corresponding to the image categories;

[0156] In some embodiments, since the model prediction process scales and pads the original image, the detection results are first biased, then inversely scaled, and rectangles and text are drawn. Cells are then tested using a pre-trained Re-YOLOv3 model.

[0157] It should be noted that recall (R) and precision (P) are important metrics for evaluating the performance of image acoustic fluid sorting models in target image detection. These two metrics are used to evaluate Re-YOLOv3, as shown in formulas (3) and (4) below:

[0158]

[0159]

[0160] Where TP represents true positive samples, that is, the number of samples with true values ​​of positive and classified as positive. FP represents false positive samples, that is, the number of samples with true values ​​of negative and classified as positive. FN represents false negative samples, that is, the number of samples with true values ​​of positive and classified as negative.

[0161] Step 5: Using the cell ejection module, eject the cells corresponding to the target image to the preset collection area.

[0162] In some embodiments, the cell ejection module uses a single-sided FIDT to eject cells. Unlike the bidirectional ARF of surface acoustic waves, the unidirectional force generated by surface acoustic waves continuously propels particles along the wave, improving the cell sorting performance.

[0163] It should be noted that when a sound wave encounters an obstacle during propagation, the scattering of the sound wave at the interface will generate a positive acoustic radiation pressure along the direction of sound propagation. In a fluid medium, the acoustic radiation pressure is proportional to the sound energy density and generates an acoustic radiation force on particles or cells exposed to the sound field, which can be expressed as shown in the following formula (5):

[0164]

[0165] Among them, Y T is the acoustic radiation factor, which depends on the droplet density, size, and sound velocity. d is the particle diameter. <e>denoted as time-averaged acoustic energy density.

[0166] In some embodiments, the geometry of the FIDT is determined by the radius R and radian of the innermost FIDT finger, and the energy intensity of the surface acoustic wave decreases as the width of the sorting signal decreases.

[0167] It should be noted that the FIDT radius in the embodiments can be 10°, 20°, or 30°, etc., and this embodiment does not impose specific limitations. Specifically, when the radius is 10°, tests were conducted with frequencies ranging from 10MHz to 70MHz and voltages from 1-5V. The particle ejection effect was extremely weak, indicating that the focal diameter was too short to support particle ejection at the designed position. When the radius is 20°, the frequency is 38.4MHz, and the voltage is 1V, it is sufficient to eject 12μm particles to the collection area. When the radius is 30°, the frequency is 39.6MHz, and the voltage is 2.2V, it is also sufficient to eject 12μm particles to the collection area. In summary, the FIDT with a radius of 20°, when applied to IACS, and with an FTSAW frequency of 38.4MHz, effectively ejects cells, thereby improving cell sorting efficiency.

[0168] In some embodiments, the cell image type information is first obtained by identifying and extracting features from the sample cell image set. Then, the image acoustic fluid control cell sorting model is trained based on the image type information and the sample cell image set. This allows deep network fusion to extract features such as cell size and morphology from different network layers, improving the detection accuracy of small targets. Finally, the original cell image is input into the trained image acoustic fluid control cell sorting model for image prediction and screening to determine the target image corresponding to the image type. Then, based on the cell ejection module, the cells corresponding to the target image are ejected to the preset collection area. This enables automatic cell classification using a label-free method, thereby achieving the purified collection of target cells.

[0169] Please see Figure 10 , Figure 10 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0170] Specifically, the electronic device includes: one or more processors and memory. Figure 10 Let's take a processor and memory as an example. The processor and memory can be connected via a bus or other means. Figure 10 Taking the example of a connection between China and Israel via a bus.

[0171] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs, such as the cell sorting method based on the image-acoustic flow control cell sorting model in the above embodiments of the present invention. The processor implements the cell sorting method based on the image-acoustic flow control cell sorting model in the above embodiments of the present invention by running the non-transitory software program and the program stored in the memory.

[0172] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function. The data storage area may store data required for executing the cell sorting method based on the image-acoustic flow control cell sorting model described in the above embodiments of the present invention. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0173] Furthermore, one embodiment of the present invention also provides a computer-readable storage medium storing computer-executable instructions that are executed by a processor or controller, for example, by the aforementioned... Figure 10 One of the processors can be executed, which can cause the processor to execute the cell sorting method based on the image acoustic fluid control cell sorting model in the above embodiments.

[0174] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0175] It will be understood by those skilled in the art that Figure 2-9 The technical solutions shown do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0176] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0177] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0178] The terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0179] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0181] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0183] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0184] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.< / e>

Claims

1. A cell sorting method based on an image-based acoustic fluid control cell sorting model, characterized in that, The image-based acoustic fluid control cell sorting model includes a cell image recognition module and a cell ejection module, and the cell sorting method includes: Acquire a raw cell image set and a predetermined sample cell image set, wherein the sample cell image set includes multiple cell images carrying detection markers, and the raw cell image set includes multiple raw cell images; The cell image recognition module includes a feature extractor; The cell images are input into the convolutional layer of a preset convolutional neural network for encoding to obtain image features of multiple cell images; The detection identifier includes the cell's position coordinates in the cell image and preset category information; The image features of the cell image are input into the feature extractor, so that the feature extractor generates a target anchor box carrying a detection label on the cell image according to the position coordinates and the preset category information; The cell image is dimensionality reduced based on the convolutional layer, and the feature prediction of the dimensionality-reduced cell image is performed based on the target anchor box to obtain the prediction result; Based on the prediction results and the image features, the image type information of the cell image is obtained; The image type information and the sample cell image set are input into the image acoustic fluid control cell sorting model for training; The original cell images are input into the trained image acoustic fluid control cell sorting model for image prediction to determine the image category of the original cell images. The original cell image set is then filtered according to the image category to determine the target image corresponding to the image category. The cell ejection module ejects cells corresponding to the target image to a preset collection area.

2. The cell sorting method based on the image-acoustic flow-controlled cell sorting model according to claim 1, characterized in that, The sample cell image set was obtained through the following steps: Obtain a mixed sample of cells; Cells in the mixed cell sample are separated based on acoustic radiation force to obtain a sample cell set; The sample cell set was image acquired to obtain cell images; The cell image is resized; The resized cell images are labeled to obtain a set of sample cell images carrying detection markers.

3. The method of claim 1, wherein the image-based acoustic cytometry model is a flow cytometry model. The preset convolutional neural network includes channel dimensions and feature layers; The step of inputting the cell images into the convolutional layer of the preset convolutional neural network for encoding to obtain image features of multiple cell images includes: The cell image is input into the preset convolutional neural network, which upsamples the cell image to obtain multiple predicted feature maps. Channel splicing is performed on the channel dimensions to obtain the predicted branches; Based on the predicted branch and the feature layer, tensor concatenation is performed on multiple predicted feature maps to obtain image features of multiple cell images.

4. The method of claim 3, wherein the image-based acoustic hydrodynamic cell sorting model is based on a model of the form: wherein the parameters are determined by the method of claim 1. The step of tensor concatenating multiple predicted feature maps based on the predicted branch and the feature layer to obtain image features of multiple cell images includes: The predicted feature map is input into the feature layer for calculation to obtain the predicted feature value; The predicted feature values ​​are integrated based on the channel dimension of the predicted branch to obtain the image features of the cell image.

5. The cell sorting method based on the image-acoustic flow-controlled cell sorting model according to claim 1, characterized in that, The step of inputting the image type information and the sample cell image set into the image acoustic fluid control cell sorting model for training includes: The image category information and the sample cell image set are input into the image acoustic fluid control cell sorting model, so that the image acoustic fluid control cell sorting model calculates the position coordinate information and the preset category information to obtain a confidence value; The confidence score value is normalized based on the target anchor frame to obtain the confidence index value; The confidence index value is compared with a preset threshold to obtain the comparison result; The image acoustic fluid sorting model is trained based on the comparison results.

6. The method of claim 5, wherein the image-based acoustic hydrodynamic cell sorting model is based on a model of the form: wherein the parameters are determined by the method of claim 1. The step of inputting the original cell image into the trained image acoustic fluid control cell sorting model for image prediction and determining the image category of the original cell image includes: The original cell image is input into the trained image acoustic fluid control cell sorting model, so that the cell image recognition module predicts the original cell image and obtains the predicted location information and predicted type information of the original cell image. The prediction anchor box of the original cell image is determined based on the prediction location information and the prediction type information, and the prediction index value is obtained based on the prediction anchor box. The image category of the original cell image is determined based on the predicted index value.

7. A cell sorting system based on an image acoustofluidic cell sorting model, characterized in that, include: The sample acquisition module is used to acquire a raw cell image set and a predetermined sample cell image set, wherein the sample cell image set includes multiple cell images carrying detection markers, and the raw cell image set includes multiple raw cell images; A cell image recognition module is used to receive the sample cell image set, and the cell image recognition module includes a feature extractor; The cell images are input into the convolutional layer of a preset convolutional neural network for encoding to obtain image features of multiple cell images; The detection identifier includes the cell's position coordinates in the cell image and preset category information; The image features of the cell image are input into the feature extractor, so that the feature extractor generates a target anchor box carrying a detection label on the cell image according to the position coordinates and the preset category information; The cell image is dimensionality reduced based on the convolutional layer, and the feature prediction of the dimensionality-reduced cell image is performed based on the target anchor box to obtain the prediction result; Based on the prediction results and the image features, the image type information of the cell image is obtained; The model training module is used to input the image type information and the sample cell image set into the image acoustic fluid control cell sorting model for training; The image determination module is used to input the original cell image into the trained image acoustic fluid control cell sorting model for image prediction, determine the image category of the original cell image, filter the original cell image set according to the image category, and determine the target image corresponding to the image category; The cell ejection module is used to eject cells corresponding to the target image to a preset collection area.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions for causing a computer to execute the cell sorting method based on the image acoustofluidic cell sorting model according to any one of claims 1 to 6.

Images