Encapsulated cell droplet sorting method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an encapsulated cell droplet sorting method and system, electronic equipment and a storage medium, and belongs to the technical field of intelligent biomedical engineering. The method comprises the following steps: acquiring a sorting area image of a micro-fluidic channel in a micro-fluidic chip; cutting the sorting area image according to a preset cutting frame to obtain a microfluidic droplet image; the microfluidic liquid drop image is input into a cell recognition model based on a convolutional neural network structure to obtain a cell recognition result, and the cell recognition result is used for judging whether cells exist in liquid drops or not; and determining a chip control signal of the micro-fluidic chip according to the cell recognition result so as to collect encapsulated cell droplets in the sorting area. According to the invention, the droplet cell encapsulation efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent biomedical engineering technology, and in particular to a method, system, electronic device and storage medium for sorting encapsulated cell droplets. Background Art

[0002] Cell encapsulation in microdroplets is primarily achieved through fluid dynamics and surface tension manipulation. Random cell encapsulation is an inherently complex problem faced when encapsulating cells using microfluidic droplet technology. The distribution of cells in the solution is typically uneven, and encapsulation is a random process. Generally speaking, most droplets will contain one or no cells, while only a few will contain multiple cells. To ensure that droplets encapsulate cells, some researchers have adopted a random encapsulation followed by sorting, which involves sorting out cell-encapsulated droplets by identifying the presence of cells in the droplets.

[0003] Traditional cell identification methods rely primarily on technologies such as flow cytometry and fluorescence-activated cell sorting. While these methods can achieve cell identification to a certain extent, they have many limitations. For example, flow cytometry requires high cell labeling requirements and can only identify cells based on a few physical and chemical characteristics, resulting in low cell identification accuracy. Fluorescence-activated cell sorting requires fluorescent labeling of cells, which may affect the physiological functions of cells. In addition, the fluorescent labeling process is cumbersome and costly.

[0004] With the development of computer and image processing technologies, image-based cell recognition and sorting methods, applied to the isolation of cell-encapsulated droplets, have become a research hotspot. However, existing image-based cell recognition and sorting schemes mostly use traditional image processing algorithms, such as threshold segmentation and edge detection. These algorithms often fail to accurately detect cells in complex droplet images, resulting in low droplet encapsulation rates. Summary of the Invention

[0005] The main purpose of the embodiments of the present application is to provide a method, system, electronic device and storage medium for sorting encapsulated cell droplets, aiming to improve the droplet cell encapsulation rate.

[0006] To achieve the above objectives, one aspect of the embodiments of the present application provides a method for sorting encapsulated cell droplets, comprising the following steps:

[0007] Acquire the sorting area image of the microfluidic channel in the microfluidic chip;

[0008] Cropping the image of the selected area according to a preset cropping frame to obtain a microfluidic droplet image;

[0009] Inputting the microfluidic droplet image into a cell recognition model based on a convolutional neural network structure to obtain a cell recognition result, wherein the cell recognition result is used to determine whether a cell exists in the droplet;

[0010] A chip control signal of the microfluidic chip is determined according to the cell identification result to collect the encapsulated cell droplets in the sorting area.

[0011] In some embodiments, the step of acquiring a sorting area image of a microfluidic channel in a microfluidic chip comprises the following steps:

[0012] The images of the sorting areas are collected by a microscope camera at a preset frame rate, wherein the microscope camera is directed toward the microfluidic channel of the microfluidic chip to shoot.

[0013] In some embodiments, the microfluidic chip includes a microfluidic channel, an electrode array, and at least two collection channels, wherein the electrode arrays are distributed on both sides of the microfluidic channel. The chip control signal of the microfluidic chip is determined according to the cell recognition result to collect the encapsulated cell droplets in the sorting area, comprising the following steps:

[0014] When it is determined that a cell exists according to the cell identification result, determining that the chip control signal is a first control signal, and adjusting the electric field of the electrode array in the microfluidic chip according to the chip control signal so that the droplets in the sorting area in the microfluidic channel move toward the collection channel in the first direction;

[0015] When it is determined that no cells exist based on the cell identification result, the chip control signal is determined to be a second control signal, and the electric field of the electrode array in the microfluidic chip is adjusted according to the chip control signal to make the droplets in the sorting area in the microfluidic channel move toward the collection channel in the second direction.

[0016] In some embodiments, before the step of inputting the microfluidic droplet image into a cell recognition model based on a convolutional neural network structure to obtain a cell recognition result, the encapsulated cell droplet sorting method further includes the following steps:

[0017] performing denoising processing on the microfluidic droplet image to obtain a first preprocessed image;

[0018] performing grayscale normalization processing on the first preprocessed image to obtain a second preprocessed image;

[0019] Perform image enhancement processing on the second preprocessed image to obtain a preprocessed microfluidic droplet image.

[0020] In some embodiments, the cell recognition model is trained by the following steps:

[0021] Acquire a training data set, wherein the training data set includes a plurality of image samples, and the image samples include a sample label indicating whether the image includes cells;

[0022] Initialize the cell recognition model based on the pre-trained image classification model based on the convolutional neural network structure;

[0023] The initialized cell recognition model is trained according to the training data set to obtain a trained cell recognition model.

[0024] In some embodiments, obtaining a training data set comprises the following steps:

[0025] Get multiple channel shooting images;

[0026] Cropping each channel captured image according to a preset cropping frame, and performing a preprocessing operation on the cropped region image to obtain a preprocessed region image;

[0027] Performing transformation processing on the pre-processed region image according to a preset image transformation rule to obtain a transformed image;

[0028] A training data set is formed according to the plurality of pre-processed region images and the transformed images.

[0029] In some embodiments, the cell recognition model includes a backbone network, a neck network, and a detection module connected in sequence;

[0030] The backbone network is used to extract features from the input image to obtain multi-scale features; the backbone network adopts a spatial channel separation downsampling structure, and the convolution operation of the backbone network adopts distributed shift convolution;

[0031] The neck network is used to fuse the multi-scale features using a pyramid spatial attention mechanism to obtain fused features;

[0032] The detection module is used to classify the fusion features to obtain cell recognition results.

[0033] To achieve the above objectives, another aspect of the present invention provides an encapsulated cell droplet sorting system, comprising:

[0034] The first module is used to obtain a sorting area image of a microfluidic channel in a microfluidic chip;

[0035] The second module is used to crop the selection area image according to a preset cropping frame to obtain a microfluidic droplet image;

[0036] The third module is used to input the microfluidic droplet image into a cell recognition model based on a convolutional neural network structure to obtain a cell recognition result, and the cell recognition result is used to determine whether there is a cell in the droplet;

[0037] The fourth module is used to determine the chip control signal of the microfluidic chip according to the cell identification result to collect the encapsulated cell droplets in the sorting area.

[0038] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes an electronic device, which includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the method described in the above embodiment is implemented.

[0039] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method described in the above embodiment.

[0040] The encapsulated cell droplet sorting method, system, electronic device and storage medium proposed in the present application obtain the sorting area image of the microfluidic channel in the microfluidic chip, crop the sorting area image according to the preset cropping frame to obtain the microfluidic droplet image, and then input the microfluidic droplet image into the cell recognition model based on the convolutional neural network structure to obtain the cell recognition result. The cell recognition result is used to determine whether there are cells in the droplet, and then the chip control signal of the microfluidic chip is determined according to the cell recognition result to collect the encapsulated cell droplets in the sorting area. The embodiment of the present application adopts a deep learning model based on a convolutional neural network to identify cells, which can improve the accuracy of detecting whether the droplets contain cells, and then sort out the droplets encapsulating cells in combination with the cell recognition results, which is convenient for subsequent cell research. In addition, before model recognition, the microfluidic droplet image is cropped out first, so that the model can focus on the droplet area image, reduce the influence of pixels in other areas on model recognition, and further improve the accuracy and efficiency of model recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flow chart of the encapsulated cell droplet sorting method provided in an embodiment of the present application;

[0042] Figure 2 Schematic diagram of a hollow droplet in a microfluidic channel provided in an embodiment of the present application;

[0043] Figure 3 Schematic diagram of encapsulated cell droplets in a microfluidic channel provided in an embodiment of the present application;

[0044] Figure 4 This is a schematic diagram of the backbone network structure provided by an embodiment of the present application;

[0045] Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0047] It should be noted that although the system is divided into functional modules and the flowcharts illustrate a logical sequence, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowcharts. The terms "first," "second," and so on in the specification, claims, and drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0049] The embodiments of the present application provide a method, system, electronic device, and storage medium for sorting encapsulated cell droplets, aiming to improve the droplet cell encapsulation rate.

[0050] The encapsulated cell droplet sorting method, system, electronic device, and storage medium provided in the embodiments of the present application are specifically illustrated by the following examples. First, the encapsulated cell droplet sorting method in the embodiments of the present application is described.

[0051] The encapsulated cell droplet sorting method provided in the embodiment of the present application relates to the field of intelligent biomedical engineering technology. The encapsulated cell droplet sorting method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or distributed system composed of multiple physical servers, or can be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the encapsulated cell droplet sorting method, etc., but is not limited to the above forms.

[0052] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0053] According to some embodiments of the present application, the encapsulated cell droplet sorting method of the present application embodiment can be applied to the following hardware systems:

[0054] The hardware system includes a microscope camera, a processor, and a microfluidic chip. The microfluidic chip includes a microfluidic channel, an electrode array, at least two collection channels, and a control circuit for the electrode array. The electrode arrays are distributed on both sides of the microfluidic channel. The microfluidic channel and the collection channel are interconnected. The area where the microfluidic channel connects to one end of the collection channel is defined as the sorting zone. The width and height of the microfluidic channel can be optimized according to the size of the cells so that the cells can flow stably in the channel. The microscope camera and the microfluidic chip are fixed relative to each other, and the microscope camera's shooting direction is toward the sorting zone. The microscope camera is used to capture images of the sorting zone and send the sorting zone images to the processor for analysis and processing. The processor can be spatially integrated with the microscope camera or the control circuit, or it can be spatially separated, and the embodiments of the present application are not specifically limited. The processor is used to perform preprocessing operations on the sorting zone images captured by the microscope camera, and then use a cell recognition model based on a convolutional neural network structure to detect the preprocessed images to obtain a cell recognition result of whether cells are present. Based on the cell recognition result, a corresponding chip control signal is generated. According to the chip control signal, the state of the control circuit is changed so that the control circuit generates corresponding voltage parameters. The control circuit is electrically connected to the electrode array and is used for injecting electric charges into the electrode array so that the electrode array generates an electric field.

[0055] In actual application, after the cells enter the microfluidic channel, microfluidic droplet technology is used to drip the liquid to encapsulate the cells in the droplets. Figure 2 and Figure 3 , Figure 2 Schematic diagram of hollow droplets in microfluidic channels. Figure 3 This is a schematic diagram of cell droplets encapsulating in a microfluidic channel. Droplets pass through the microfluidic channel one by one and reach the sorting area of ​​the microfluidic channel. Droplet images are captured at the sorting area, and the processor identifies whether cells are detected based on the captured image. If cells are detected, it means that the sorting area is as follows: Figure 3 If the encapsulated cell droplets are detected, the corresponding control signal is generated, and the electric field force is used to precisely control the droplets, so that they move in the desired direction and enter the corresponding encapsulated cell droplet collection channel to achieve cell sorting; on the contrary, if no cells are detected, it means that the sorting area is as follows Figure 2 The empty packet droplets shown generate corresponding control signals to make them move in another desired direction and enter the corresponding empty packet droplet collection channel.

[0056] Figure 1 This is an optional flow chart of the encapsulated cell droplet sorting method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S104.

[0057] Step S101, acquiring a sorting area image of a microfluidic channel in a microfluidic chip;

[0058] Step S102, cropping the selected area image according to a preset cropping frame to obtain a microfluidic droplet image;

[0059] Step S103: Inputting the microfluidic droplet image into a cell recognition model based on a convolutional neural network structure to obtain a cell recognition result, which is used to determine whether there is a cell in the droplet;

[0060] Step S104 , determining a chip control signal of the microfluidic chip according to the cell identification result, so as to collect the encapsulated cell droplets in the sorting area.

[0061] In the steps S101 to S104 shown in the embodiment of the present application, by acquiring the sorting area image of the microfluidic channel in the microfluidic chip, the sorting area image is cropped according to the preset cropping frame to obtain a microfluidic droplet image, and then the microfluidic droplet image is input into the cell recognition model based on the convolutional neural network structure to obtain a cell recognition result. The cell recognition result is used to determine whether there are cells in the droplet, and then the chip control signal of the microfluidic chip is determined according to the cell recognition result to collect the encapsulated cell droplets in the sorting area. The embodiment of the present application uses a deep learning model based on a convolutional neural network to identify cells, which can improve the accuracy of detecting whether the droplets contain cells, and then sort out the droplets encapsulating cells in combination with the cell recognition results, thereby improving the droplet cell encapsulation rate. In addition, before model recognition, the microfluidic droplet image is first cropped, so that the model can focus on the droplet area image, reduce the influence of pixels in other areas on model recognition, and further improve the accuracy and efficiency of model recognition.

[0062] In step S101 of some embodiments, the sorting zone is part of the microfluidic channel and is defined as the end of the microfluidic channel connected to the collection channel. A camera is directed toward the sorting zone to capture an image of the sorting zone. After the cell solution is dripped, droplets in the solution are introduced into the microfluidic channel one by one at a predetermined rate and then into the sorting zone. A microscope camera can be used to capture the sorting zone at a predetermined rate to obtain an image of the sorting zone.

[0063] According to some embodiments of the present application, step S101 may include but is not limited to the following steps:

[0064] Step S201 : collecting images of the sorting areas by using a microscope camera at a preset frame rate, wherein the microscope camera is directed toward the microfluidic channel of the microfluidic chip to shoot.

[0065] In this embodiment, a high-resolution and high-speed microscope camera can be used to capture images of the sorting area. The sorting area includes droplets and cells. By using a microscope camera, the cell morphology and details can be clearly captured, thereby improving the accuracy of subsequent recognition. Specifically, the microscope camera captures images of the sorting area at a certain frame rate. For example, a high-resolution microscope camera with a resolution of 1440*1080 pixels and a frame rate of 318 frames per second is selected, and a 4x objective lens is used to capture images of the droplets in the sorting area. During the acquisition process, the cells need to be placed in a suitable culture environment, such as temperature, humidity, and culture fluid composition, to ensure the normal morphology and physiological state of the cells. The preset frame rate of the embodiment of the present application can be determined according to the droplet flow rate.

[0066] In step S102 of some embodiments, Figure 2 and Figure 3 As shown in the figure, droplets are generally confined to a certain area, and the collected sorting area image includes many areas outside the channel, that is, the part of the sorting area image that is useful for cell recognition analysis only occupies a part of the whole sorting area image. In order to reduce the interference of other areas in other images on the model analysis and improve the efficiency of model operation, the cropping frame (including the preset cropping frame size and position) can be preset in combination with the droplet confinement area, and then the image is cut according to the preset cropping frame (such as Figure 2 and Figure 3 The image of the selected area is cropped (within the dotted box in FIG) to obtain a microfluidic droplet image. For example, a 320*320 window is used to crop the image of the selected area to obtain a local droplet image.

[0067] According to some embodiments of the present application, before step S103, the encapsulated cell droplet sorting method of the embodiment of the present application may further include but is not limited to the following steps:

[0068] Step S301, performing denoising processing on the microfluidic droplet image to obtain a first pre-processed image;

[0069] Step S302, performing grayscale normalization processing on the first preprocessed image to obtain a second preprocessed image;

[0070] Step S303 : performing image enhancement processing on the second pre-processed image to obtain a pre-processed microfluidic droplet image.

[0071] In this embodiment, after obtaining the microfluidic droplet image, the microfluidic droplet image can be further preprocessed to improve image quality, thereby improving the accuracy of the model in identifying encapsulated cell droplets. The preprocessing process includes denoising, grayscale normalization, and image enhancement, as follows:

[0072] Denoising is performed on the microfluidic droplet image to obtain a first preprocessed image. Denoising is performed using an adaptive median filter algorithm. This algorithm dynamically adjusts the filter window size and median calculation method based on the distribution of pixel values ​​in the neighborhood of each pixel in the image, effectively removing salt and pepper noise and Gaussian noise from the image while preserving cell edges and detail information.

[0073] The grayscale value normalization processing is performed on the first pre-processed image to obtain the second pre-processed image. The grayscale value normalization processing is to normalize the grayscale value of the first pre-processed image to the interval [0,1]. The calculation formula is:

[0074]

[0075] Among them, I is the original image pixel value, I min and I max are the minimum and maximum values ​​of the image pixel values, I norm is the normalized pixel value.

[0076] Image enhancement processing is performed on the second preprocessed image to obtain a preprocessed microfluidic droplet image. The image enhancement process utilizes a histogram equalization method to enhance the contrast of the second preprocessed image. First, the grayscale histogram of the second preprocessed image is calculated. Then, the histogram is equalized to obtain a balanced histogram. Finally, the pixel values ​​of the second preprocessed image are remapped based on the balanced histogram to obtain a preprocessed microfluidic droplet image. This significantly improves the contrast of the microfluidic droplet image and makes cell features more clearly visible.

[0077] The pre-processed microfluidic droplet image is obtained through the above process, and then the microfluidic droplet image is input into the cell recognition model for cell recognition analysis.

[0078] The embodiment of the present application processes the collected sorting area images, including mosaic data enhancement, optimization of hyperparameter adjustments such as contrast enhancement and brightness, reducing the lighting differences between different images, making the cell features more obvious, and facilitating subsequent processing and analysis.

[0079] In step S103 of some embodiments, the cell recognition model based on the convolutional neural network structure can be a target detection model of the YOLO series. The target detection model of the YOLO series is a deep learning algorithm that can be used for image recognition in computer vision technology. The YOLO model converts the target detection problem into a regression problem, that is, given an input image, the bounding box of the target and its classification category are directly regressed at multiple positions in the image. YOLO models include but are not limited to Yolov3, Yolov4, Yolov5, Yolov10 (all different versions of YOLO), etc. Different models have different weights, network structure diagrams, and algorithms, and the area sampling methods used are also different. The cell recognition model of the embodiment of the present application outputs cell recognition results based on microfluidic droplet images. The cell recognition results are used to determine whether there are cells in the droplets. Specifically, please refer to Figure 2 and Figure 3 The cell recognition results include a droplet detection frame and a detection frame classification category. The detection frame classification category can include cell presence and cell absence. The droplet detection frame of the cell presence category is marked "0", and the droplet detection frame of the cell absence category is marked "1". In other embodiments, based on different training tasks, the detection frame classification categories can include cell absence, single cell, and multi-cell. The microfluidic chip is equipped with three collection channels, respectively for collecting empty droplets, single-cell droplets, and multi-cell droplets. Both single-cell droplets and multi-cell droplets are droplets that encapsulate cells.

[0080] In step S104 of some embodiments, the processor determines the chip control signal of the microfluidic chip based on the cell recognition result, and sends the chip control signal to the sorting execution device to collect the encapsulated cell droplets, thereby improving the droplet encapsulation rate. The collection work can be achieved by the sorting execution device, which is designed based on microfluidic technology and electrodynamics principles. The sorting execution device can be a microfluidic chip. When the cell passes through the microfluidic channel, according to the chip control signal sent by the processor, an electric field force is generated by applying different voltages, and the electric field force is used to precisely manipulate the encapsulated cell droplets, causing them to deviate from their original motion trajectory and enter the corresponding collection channel to achieve droplet sorting. The sorting execution device has the advantages of fast sorting speed, high precision, and little damage to cells, and can meet the needs of large-scale cell sorting.

[0081] The principle of electrodynamic sorting is that as cells pass through a microfluidic channel, a corresponding voltage is applied to the electrode array based on chip control signals from the processor. Because cells have different electrical properties, the electric field forces droplets containing cells in a specific direction and magnitude, causing them to deviate from their original trajectory and enter the corresponding collection channel, enabling droplet sorting. By precisely controlling the magnitude and direction of the electric field, high-precision droplet sorting can be achieved.

[0082] According to some embodiments of the present application, step S104 may include but is not limited to the following steps:

[0083] Step S401, when it is determined that a cell exists according to the cell identification result, determining that the chip control signal is a first control signal, and adjusting the electric field of the electrode array in the microfluidic chip according to the chip control signal to cause droplets in the sorting area of ​​the microfluidic channel to move toward a collection channel in a first direction;

[0084] Step S402, when it is determined that no cells exist according to the cell identification result, the chip control signal is determined to be a second control signal, and the electric field of the electrode array in the microfluidic chip is adjusted according to the chip control signal to make the droplets in the sorting area in the microfluidic channel move toward the collection channel in the second direction.

[0085] In this embodiment, depending on the different cell recognition results, a first control signal or a second control signal can be selected as the chip control signal. The first control signal and the second control signal are used to cause the electrode array to generate electric fields in different directions. For example, the first control signal can be a high-level signal, and the second control signal can be a low-level signal. When the cell recognition result is that a cell is present, the processor sends a high-level chip control signal to the control circuit of the electrode array, thereby generating an electric field in one direction in the electrode array; when the cell recognition result is that a cell is not present, the processor sends a low-level chip control signal to the control circuit of the electrode array, thereby generating an electric field in another direction in the electrode array. The collection channel in the first direction is used to collect encapsulated cell droplets, and the collection channel in the second direction is used to collect empty packet droplets. If the cell identification results indicate that cells are present, it indicates that the current droplets may be encapsulated cell droplets. In this case, the first control signal is used as the chip control signal, and the chip control signal is input into the control circuit of the electrode array to generate an electric field force of a preset direction and magnitude in the electrode array, and under the action of the electric field force, the encapsulated cell droplets move toward the collection channel in the first direction. If the cell identification results indicate that the cell is not present, it indicates that the current droplets may be empty droplets. In this case, the second control signal is used as the chip control signal, and the chip control signal is input into the control circuit of the electrode array to generate an electric field force of another preset direction and magnitude in the electrode array, and under the action of the electric field force, the empty droplets move toward the collection channel in the second direction. In another example, if the cell identification results indicate that the cell is not present, combined with the structural design of the microfluidic chip, the chip control signal may not be generated, that is, no electric field force is generated in the power array, and instead gravity is used to separate the empty droplets, which can reduce the energy consumption of the entire system.

[0086] In another example, based on different training tasks, the detection frame classification categories can include absence of cells, single cells, and multi-cells. The microfluidic chip is equipped with three collection channels for collecting empty droplets, single-cell droplets, and multi-cell droplets, respectively. The processor generates corresponding chip control signals based on the three different detection frame classification categories. Different chip control signals represent voltage values ​​and circuit states. The control circuit can generate electric field forces of different magnitudes and directions in the electrode array based on different chip control signals, thereby precisely controlling the movement path of droplets in the channel and achieving separation of different encapsulated droplets.

[0087] According to some embodiments of the present application, the cell recognition model in step S103 is trained by the following steps:

[0088] Step S501: obtaining a training data set, the training data set including a plurality of image samples, the image samples including a sample mark indicating whether the image includes cells;

[0089] Step S502, initializing a cell recognition model according to a pre-trained image classification model based on a convolutional neural network structure;

[0090] Step S503 , training the initialized cell recognition model according to the training data set to obtain a trained cell recognition model.

[0091] In this embodiment, the training dataset includes multiple image samples, each of which includes a sample label indicating whether the image contains cells. For example, for an image containing droplets, the droplets are framed and labeled "1" or "0" based on the presence of cells in the droplets, with "1" indicating the presence of cells in the droplets and "0" indicating the absence of cells. It is understood that the images in the image samples can also be subjected to the aforementioned cropping, denoising, grayscale normalization, and contrast enhancement processes to improve model training efficiency.

[0092] In this example, transfer learning technology is introduced. The parameters of a convolutional neural network-based image classification model, pre-trained on a natural image dataset, are used to initialize the cell recognition model. Fine-tuning this initial value accelerates model convergence and improves training efficiency. Specifically, for two identical network components, the parameters of the image classification model are used to initialize the network parameters of that portion of the cell recognition model. The remaining network components in the cell recognition model can be initialized based on experience.

[0093] In this embodiment, the initialized cell recognition model is trained according to the training data set to obtain a trained cell recognition model. The model can use the TensorRT algorithm for accelerated reasoning. TensorRT is a high-performance deep learning inference engine developed by NVIDIA, designed for NVIDIA GPUs, and used to accelerate the reasoning process of deep learning models. During the training process, the cross entropy loss function is used as the loss function of the model, and the model is trained using an auto optimizer. The learning rate is dynamically adjusted according to the training effect of the model, and a learning rate decay strategy is adopted to gradually reduce the learning rate as the number of training rounds increases, thereby avoiding overfitting of the model in the later stages of training. At the same time, the early stopping method is used to prevent the model from overtraining. When the loss function on the validation set no longer decreases, the training is stopped and the optimal model is saved.

[0094] According to some embodiments of the present application, step S501 may include but is not limited to the following steps:

[0095] Step S601, acquiring multiple channel shooting images;

[0096] Step S602, cropping each channel captured image according to a preset cropping frame, and performing a preprocessing operation on the cropped regional image to obtain a preprocessed regional image;

[0097] Step S603, transforming the pre-processed region image according to a preset image transformation rule to obtain a transformed image;

[0098] Step S604: forming a training data set based on the plurality of pre-processed region images and the transformed images.

[0099] In this embodiment, the process of acquiring the training image set is as follows: a microscope camera is used to continuously capture multiple channel images. The multiple channel images captured by the microscope camera may or may not contain droplets. The channel images are then preprocessed using the above-mentioned cropping, denoising, grayscale normalization, and contrast enhancement processes to preprocess the region images. For images with droplets, the droplets are selected and marked with "1" or "0" based on the cell status of the droplets. "1" indicates that there are cells in the droplets, and "0" indicates that there are no cells in the droplets. Data augmentation technology is then used to expand the data set, and the preprocessed region images that have been obtained are processed using image transformation rules such as rotation, scaling, flipping, and brightness to generate diverse large-scale training samples and enhance the generalization ability of the model.

[0100] According to some embodiments of the present application, the cell recognition model of the embodiments of the present application includes a backbone network, a neck network and a detection module connected in sequence.

[0101] Please refer to Figure 4 The schematic diagram of the backbone network structure shown in the figure shows that the backbone network is used to extract features from the input image and obtain multi-scale features. The backbone network adopts the CSPDarknet infrastructure of the YOLO series. The model eliminates the image scaling (resize) operation of the pre-processing part in the reasoning at the edge end, and directly analyzes based on the 320*320 microfluidic droplet image, that is, the input image of the backbone network is the microfluidic droplet image of the above embodiment. The embodiment of the present application also introduces the C2fUIB (cross-stage partial feature fusion) unit on the basis of the CSPDarknet infrastructure to improve the feature extraction efficiency by reducing redundant calculations; at the same time, spatial channel separation downsampling (SCDown) is used to replace the traditional convolution downsampling to reduce the amount of calculation while retaining more detailed information. The backbone network uses three channels with different feature extraction depths to extract features from the input image, and finally outputs three features of different scales in the three Detect modules respectively. For the convolution operation in the backbone network, the traditional CNN is replaced by distributed shift convolution (DSConv) to improve the memory efficiency and speed of the convolution layer, so that the model can learn richer cell features.

[0102] The neck network is used to fuse multi-scale features using a pyramid spatial attention mechanism to generate fused features. The pyramid spatial attention mechanism enhances the feature representation of droplet cells in the image through adaptive weight distribution, improving the accuracy and pertinence of feature extraction.

[0103] The detection module is used to classify the fusion features and obtain cell recognition results. The output layer of the detection module includes classification, positioning and confidence branches.

[0104] According to some embodiments of the present application, the overall system integration and optimization process of the embodiments of the present application is as follows:

[0105] Hardware Integration: Image acquisition equipment, processors, microfluidic chips, and sorting execution devices are integrated to build a complete cell recognition and sorting system. A high-performance processor serves as the system's core control unit, responsible for image data processing, model calculation, and sorting decision-making. High-speed data transmission interfaces enable rapid data transfer between the image acquisition equipment and processor, as well as control signal transmission between the processor and the sorting execution device.

[0106] Software Integration and Optimization: Develop a Python-based software system that integrates algorithms for various modules, including image acquisition and preprocessing, feature extraction and recognition, sorting decision-making, and droplet sorting execution. Multi-threaded programming techniques enable parallel processing of image acquisition, data processing, and sorting execution, improving system efficiency. Simultaneously, software system optimization reduces memory usage and computing resource consumption, ensuring system stability over extended periods of operation.

[0107] System Testing and Optimization: Comprehensively test the integrated cell recognition and sorting system, including image acquisition quality, droplet cell recognition accuracy, sorting speed, and precision. Based on the test results, optimize the system's hardware and software. For example, adjust the image acquisition device parameters to improve image quality; optimize the deep learning model structure and training parameters to increase cell recognition accuracy; and adjust the microfluidic chip design and sorting actuator control parameters to improve sorting speed and precision, ensuring the system meets the needs of practical applications.

[0108] System Control and Monitoring: A specialized control system was developed to provide real-time control and monitoring of the sorting execution device. Based on signals from the sorting decision module, the control system precisely controls the voltage of the electrode array and the flow rate of the microfluidic chip, ensuring accurate and stable cell sorting. Furthermore, by real-time monitoring of the droplet flow state and sorting results within the microfluidic channel, control parameters can be adjusted promptly to ensure proper system operation.

[0109] According to some embodiments of the present application, the embodiments of the present application have at least one of the following beneficial effects:

[0110] High accuracy: By adopting advanced deep learning models and effective data enhancement and transfer learning technologies, it can accurately extract droplet and cell features and achieve high-precision recognition of droplets in different situations.

[0111] High efficiency: The system integrates high-speed image acquisition equipment and efficient algorithms, which can realize the rapid processing and sorting of large numbers of droplets, improving experimental efficiency.

[0112] Low damage: The cell sorting execution device based on microfluidics technology and electrodynamic principles causes minimal damage to cells and can maintain cell activity and physiological functions to the greatest extent. It is suitable for experimental research with high requirements on cell activity.

[0113] Wide adaptability: Through training on large-scale data sets and transfer learning technology, the model has good generalization ability and can subsequently adapt to different types of cells and diverse experimental conditions, and has broad application prospects.

[0114] The present application also provides an encapsulated cell droplet sorting system, comprising:

[0115] The first module is used to obtain a sorting area image of a microfluidic channel in a microfluidic chip;

[0116] The second module is used to crop the selection area image according to the preset cropping frame to obtain the microfluidic droplet image;

[0117] The third module is used to input the microfluidic droplet image into the cell recognition model based on the convolutional neural network structure to obtain the cell recognition result, which is used to determine whether there is a cell in the droplet;

[0118] The fourth module is used to determine the chip control signal of the microfluidic chip according to the cell recognition result to collect the encapsulated cell droplets in the sorting area.

[0119] It can be understood that the contents of the above-mentioned encapsulated cell droplet sorting method embodiment are all applicable to the present system embodiment. The functions specifically implemented by the present system embodiment are the same as those of the above-mentioned encapsulated cell droplet sorting method embodiment, and the beneficial effects achieved are also the same as those achieved by the above-mentioned encapsulated cell droplet sorting method embodiment.

[0120] The present application also provides an electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for communication between the processor and the memory. When the program is executed by the processor, the aforementioned encapsulated cell droplet sorting method is implemented. The electronic device can be any smart terminal, including a tablet computer and an in-vehicle computer.

[0121] See also Figure 5 , Figure 5 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0122] The processor 901 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0123] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called by the processor 901 to execute the encapsulated cell droplet sorting method of the embodiments of this application.

[0124] Input / output interface 903, used to implement information input and output;

[0125] Communication interface 904, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0126] Bus 905 , which transmits information between various components of the device (e.g., processor 901 , memory 902 , input / output interface 903 , and communication interface 904 );

[0127] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0128] An embodiment of the present application further provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned encapsulated cell droplet sorting method.

[0129] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0130] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0131] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0132] The system embodiment described above is merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0133] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0134] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0135] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the above units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0137] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0138] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0139] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0140] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A method for sorting encapsulated cell droplets, characterized in that: The following steps are involved: Acquire the sorting area image of the microfluidic channel in the microfluidic chip; Cropping the image of the selected area according to a preset cropping frame to obtain a microfluidic droplet image; Inputting the microfluidic droplet image into a cell recognition model based on a convolutional neural network structure to obtain a cell recognition result, wherein the cell recognition result is used to determine whether a cell exists in the droplet; A chip control signal of the microfluidic chip is determined according to the cell identification result to collect the encapsulated cell droplets in the sorting area.

2. The method for sorting encapsulated cell droplets according to claim 1, wherein: The method of obtaining the sorting area image of the microfluidic channel in the microfluidic chip comprises the following steps: The images of the sorting areas are collected by a microscope camera at a preset frame rate, wherein the microscope camera is directed toward the microfluidic channel of the microfluidic chip to shoot.

3. The method for sorting encapsulated cell droplets according to claim 1, wherein: The microfluidic chip includes a microfluidic channel, an electrode array, and at least two collection channels. The electrode arrays are distributed on both sides of the microfluidic channel. The chip control signal of the microfluidic chip is determined according to the cell recognition result to collect the encapsulated cell droplets in the sorting area. Preferably, the method includes the following steps: When it is determined that a cell exists according to the cell identification result, determining that the chip control signal is a first control signal, and adjusting the electric field of the electrode array in the microfluidic chip according to the chip control signal so that the droplets in the sorting area in the microfluidic channel move toward the collection channel in the first direction; When it is determined that no cells exist based on the cell identification result, the chip control signal is determined to be a second control signal, and the electric field of the electrode array in the microfluidic chip is adjusted according to the chip control signal to make the droplets in the sorting area in the microfluidic channel move toward the collection channel in the second direction.

4. The method for sorting encapsulated cell droplets according to claim 1, wherein: Before the step of inputting the microfluidic droplet image into a cell recognition model based on a convolutional neural network structure to obtain a cell recognition result, the encapsulated cell droplet sorting method further includes the following steps: performing denoising processing on the microfluidic droplet image to obtain a first preprocessed image; performing grayscale normalization processing on the first preprocessed image to obtain a second preprocessed image; Perform image enhancement processing on the second preprocessed image to obtain a preprocessed microfluidic droplet image.

5. The method for sorting encapsulated cell droplets according to claim 1, wherein: The cell recognition model is trained by the following steps: Acquire a training data set, wherein the training data set includes a plurality of image samples, and the image samples include a sample label indicating whether the image includes cells; Initialize the cell recognition model based on the pre-trained image classification model based on the convolutional neural network structure; The initialized cell recognition model is trained according to the training data set to obtain a trained cell recognition model.

6. The method for sorting encapsulated cell droplets according to claim 5, characterized in that: The step of obtaining a training data set comprises the following steps: Get multiple channel shooting images; Cropping each channel captured image according to a preset cropping frame, and performing a preprocessing operation on the cropped region image to obtain a preprocessed region image; Performing transformation processing on the pre-processed region image according to a preset image transformation rule to obtain a transformed image; A training data set is formed according to the plurality of pre-processed region images and the transformed images.

7. The method for sorting encapsulated cell droplets according to claim 1, wherein: The cell recognition model includes a backbone network, a neck network and a detection module connected in sequence; The backbone network is used to extract features from the input image to obtain multi-scale features; the backbone network adopts a spatial channel separation downsampling structure, and the convolution operation of the backbone network adopts distributed shift convolution; The neck network is used to fuse the multi-scale features using a pyramid spatial attention mechanism to obtain fused features; The detection module is used to classify the fusion features to obtain cell recognition results.

8. An encapsulated cell droplet sorting system, characterized in that: include: The first module is used to obtain a sorting area image of a microfluidic channel in a microfluidic chip; The second module is used to crop the selection area image according to a preset cropping frame to obtain a microfluidic droplet image; The third module is used to input the microfluidic droplet image into a cell recognition model based on a convolutional neural network structure to obtain a cell recognition result, and the cell recognition result is used to determine whether there is a cell in the droplet; The fourth module is used to determine the chip control signal of the microfluidic chip according to the cell identification result to collect the encapsulated cell droplets in the sorting area.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for implementing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. A storage medium, which is a computer-readable storage medium and is used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any one of claims 1 to 7.

Images