Efficient, robust and fast neural networks for cell image classification
A shallow neural network architecture with a precision-recall trade-off module addresses the speed and accuracy limitations of existing methods, enabling high-speed cell image classification with 99% accuracy and 2500 images per second processing.
Patent Information
- Application Number
- JP2023530264
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2021-07-09
- Filing Date
- 2021-11-19
- Publication Date
- 2024-12-09
- Estimated Expiration
- 2041-11-19
AI Technical Summary
Existing cell image analysis methods are not suitable for real-time high-speed applications such as cell sorting due to their complexity, limiting the use of deep neural networks in ultra-high speed requirements.
A shallow two-convolutional layer neural network architecture that uses a precision-recall trade-off module based on softmax scores to classify cells into pseudo-classes or specific classes, achieving high precision and recall by comparing the difference between the highest and second-highest softmax scores, with a threshold determined by machine learning or user input.
The method enables high-speed cell image classification with approximately 99% accuracy and 2500 images per second processing, achieving high precision and recall by classifying ambiguous cells into pseudo-classes and clear cells into specific classes.
Smart Images

Figure 0007596534000002 
Figure 0007596534000003 
Figure 0007596534000004
Abstract
Description
[Technical field]
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority under 35 U.S.C. §119(e) to U.S. Provisional Patent Application No. 63 / 116,088, filed November 19, 2020, entitled "EFFICIENT AND ROBUST HIGH-SPEED NEURAL NETWORKS FOR CELL IMAGE CLASSIFICATION," the entire disclosure of which is incorporated herein by reference for all purposes.
[0002] The present invention relates to cell sorting, more particularly to image-based cell sorting. [Background technology]
[0003] Cell image analysis plays an increasingly important role in biological and medical research, but existing methods cannot be utilized for real-time, high-speed applications such as cell sorting. Deep neural networks are typically built to achieve high accuracy classification, but their complexity limits their use in real-time applications that require ultra-high speed. Summary of the Invention [Problem to be solved by the invention]
[0004] Described herein is an efficient and robust fast neural network for cell image classification. The neural network for cell image classification utilizes the difference between softmax scores to determine whether a cell is ambiguous or not. If the cell is ambiguous, it classifies the cell into a pseudo-class, and if the cell is not ambiguous, it classifies the cell into the class corresponding to the highest class score. The neural network for cell image classification allows for implementation with high speed, high precision and high recall.
[0005] State-of-the-art methods are based on boosting and manually designed features. Deep learning-based methods (e.g., DeepFlow, ResNet) have also been proposed. Not all of these existing methods meet the speed requirements of typical applications, including image-activated cell sorting. [Means for solving the problem]
[0006] In one aspect, the method includes processing a cell image using a neural network that includes generating a plurality of softmax scores, each corresponding to a class; comparing the highest softmax score of the plurality of softmax scores with a second highest softmax score to determine a difference; classifying the cell image into a pseudo class when the difference is below a threshold; and classifying the cell image into a class corresponding to the highest softmax score when the difference is equal to or greater than the threshold. The neural network includes two convolutional layers and one fully connected layer. The softmax score is greater than or equal to 0 and less than or equal to 1. The threshold is set based on historical information to generate high precision and high recall. The threshold is determined by machine learning and artificial intelligence. The threshold is manually determined by a user. The method further includes obtaining the cell image.
[0007] In another aspect, the apparatus includes a non-transitory memory for storing an application for processing a cell image using a neural network including generating a plurality of softmax scores each corresponding to a class, comparing a highest softmax score of the plurality of softmax scores with a second highest softmax score to determine a difference, classifying the cell image into a pseudo class when the difference is below a threshold, and classifying the cell image into a class corresponding to the highest softmax score when the difference is equal to or above the threshold, and a processor coupled to the memory and configured to process the application. The neural network includes two convolutional layers and one fully connected layer. The softmax score is greater than or equal to 0 and less than or equal to 1. The threshold is set based on historical information to generate high precision and high recall. The threshold is determined by machine learning and artificial intelligence. The threshold is manually determined by a user. The apparatus is further configured to acquire the cell image.
[0008] In another aspect, the system includes a first device configured to acquire a cell image, and a second device configured to process the cell image using a neural network including generating a plurality of softmax scores each corresponding to a class, compare a highest softmax score of the plurality of softmax scores with a second highest softmax score to obtain a difference, classify the cell image into a pseudo class when the difference is below a threshold, and classify the cell image into a class corresponding to the highest softmax score when the difference is equal to or exceeds the threshold. The neural network includes two convolutional layers and one fully connected layer. The softmax score is between 0 and 1. The threshold is set based on historical information to generate high precision and high recall. The threshold is determined by machine learning and artificial intelligence. The threshold is manually determined by a user. [Brief description of the drawings]
[0009] [Figure 1] FIG. 1 illustrates a flowchart of a method for implementing a neural network for cell image classification according to some embodiments. [Diagram 2] FIG. 2 is a diagram of a neural network architecture according to some embodiments. [Diagram 3] FIG. 1 illustrates a flowchart for trading recall for precision based on a softmax score in accordance with some embodiments. [Figure 4] FIG. 1 illustrates a chart of exemplary classifications according to some embodiments. [Diagram 5] FIG. 1 is a block diagram of an exemplary computing device configured to implement a neural network for cell image classification according to some embodiments. [Figure 6] FIG. 1 is a schematic diagram illustrating an overall configuration of a biological sample analyzer according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] A shallow two-convolutional layer neural network architecture suitable for ultrafast real-time cell image classification is designed and established, and the algorithm can achieve high precision and high recall while minimizing the common trade-off pitfall between precision and recall.
[0011] Neural networks, including cell image classification, address the unmet needs of many cell image classification applications - mainly two important requirements: high accuracy and high speed. Existing classical machine learning-based methods (e.g., boosting, SVM with manually designed features) and deep learning-based methods cannot meet the speed requirements. The method described herein not only operates at a very high speed, but also achieves high accuracy due to the shallow neural network architecture designed by architecture search techniques and a novel precision-recall tradeoff module.
[0012] The neural network with cell image classification can be used in image-based flow cytometry systems, including high throughput and high accuracy. The neural network with cell image classification can also be used in other cell image analysis systems, such as cell image organization, searching, etc.
[0013] 1 shows a flowchart of a method for implementing a neural network for cell image classification according to some embodiments. In step 100, a cell image is acquired. The cell image can be acquired in any manner, such as using a system including a microscope and a camera.
[0014] In step 102, the neural network is used to process the cell images. The neural network can be any type of neural network, for example, a neural network including two convolutional layers and one fully connected layer. The neural network generates a softmax score, which is a classification score for the cell. For example, the softmax score is a number between 0 and 1, where the closer the number is to 1 for a class, the more likely the cell is in that class, and the closer the number is to 0 for a class, the less likely the cell is in that class. Multiple softmax scores (e.g., one score per class) can be generated for a single cell. For example, the neural network analyzes the cells to generate softmax scores for all five classes. For example, when classifying white blood cells, there are five classes / classifications (e.g., neutrophils, eosinophils, basophils, monocytes, lymphocytes). The scores from one example image for the different classes sum to 1. For example, [0.5, 0.2, 0.1, 0.1, 0.1] is a valid score for the five classes. The softmax score is a standard calculation (the softmax function) for determining the predicted probability of each class among multiple classes. An exemplary softmax function includes the following formula: TIFF0007596534000001.tif2062In the above formula, l i,c is the logit for example i, per class c.
[0015] In step 104, a precision-recall tradeoff is applied to achieve high speed and high accuracy to classify cells into classes or pseudo-classes. As described herein, instead of simply classifying cells based on the highest softmax score, the two highest softmax scores are compared (e.g., highest score - second highest score), and if the difference between the two highest softmax scores is equal to or exceeds a threshold, the cell (image) is classified into the class containing the highest softmax score. If the difference is less than the threshold, the cell (image) is classified into the pseudo-class. In some embodiments, the order of the steps is changed. In some embodiments, fewer or additional steps are implemented.
[0016] FIG. 2 shows a diagram of a neural network architecture according to some embodiments. A convolutional neural network receives a cell image input. The convolutional neural network uses a rectified linear activation function (ReLU) to directly output the input if positive, otherwise the output is zero. Then, max pooling is used to downsample the 2x2 data to one value. This value is the maximum value, which is the representative image of 2x2. There is another layer of convolutional neural network and ReLU. Then, global average pooling is implemented. Finally, a soft maximum (softmax) score is achieved. A softmax is a normalized score similar to a probability. The softmax score can be used as a confidence level threshold. The softmax score is a number between 0 and 1. The softmax score is the standard output of any multi-class classifier.
[0017] Figure 3 shows a flowchart for trading recall for precision based on softmax scores according to some embodiments. At step 300, generate and / or receive softmax scores. At step 302, determine whether the classification based on the determination of the softmax scores is ambiguous. If S1 - S2 < t, the determination is ambiguous. In this formula, S1 is the largest softmax score, S2 is the second largest softmax score, and t is a threshold determined by the test set and the target precision. For example, based on previous tests and the desired precision, the threshold can be determined to achieve the desired precision (e.g., previous tests / historical data determine that a threshold of 0.5 results in a precision of 90%, while a threshold of 0.6 results in a precision of 99%, which is the goal in that example). In another example, the threshold can be determined and set by the user. In another example, learning and artificial intelligence can be used to adjust the threshold. At step 304, if it is ambiguous which class the cell belongs to, classify the cell into a pseudo-class. At step 306, if it is not ambiguous which class the cell belongs to, classify the cell into the class corresponding to the largest score. In some embodiments, the order of the steps is changed. In some embodiments, fewer or additional steps are implemented.
[0018] Figure 4 shows an exemplary classification chart according to some embodiments. In the first row under the heading, the exemplary data has a difference of S1 - S2 of 0.9, which is very high (and above the threshold), so it is clear and the system is confident that it can classify the cell (e.g., into class 3). In the second row, the exemplary data has a difference of S1 - S2 of 0.1, which is very low, so it is ambiguous and difficult to classify, and the cell is classified into a pseudo-class. By rejecting cells with a low difference between softmax scores, a precision of approximately 99% and a recall of 98.5% can be achieved.
[0019] FIG. 5 illustrates a block diagram of an exemplary computing device configured to implement a neural network for cell image classification according to some embodiments. The computing device 500 can be used to acquire, store, compute, process, communicate, and / or display information, such as images and videos. The computing device 500 can implement any of the aspects of the neural network for cell image classification. In general, a hardware structure suitable for implementing the computing device 500 includes a network interface 502, a memory 504, a processor 506, an I / O device 508, a bus 510, and a storage device 512. The selection of the processor is not critical as long as a suitable processor with sufficient speed is selected. The memory 504 can be any conventional computer memory known in the art. The storage device 512 can include a hard drive, CDROM, CDRW, DVD, DVDRW, high definition disk / drive, ultra high definition drive, flash memory card, or any other storage device. The computing device 500 can include one or more network interfaces 502. Examples of network interfaces include a network card connected to an Ethernet or other type of LAN. The I / O devices 508 may include one or more of the following: a keyboard, a mouse, a monitor, a screen, a printer, a modem, a touch screen, a button interface, and other devices. A neural network application 530 used to implement the neural network for cell image classification is likely to be stored in the storage device 512 and memory 504 and processed as an application is typically processed. The computing device 500 may include more or less components than those shown in FIG. 5. In some embodiments, neural network hardware 520 is included. Although the computing device 500 of FIG. 5 includes an application 530 and hardware 520 for the neural network for cell image classification, the neural network for cell image classification may be implemented in the computing device as hardware, firmware, software, or any combination thereof.For example, in some embodiments, the neural network application 530 is programmed into memory and executed using a processor. In another example, in some embodiments, the neural network hardware 520 is programmed hardware logic that includes gates that are specifically designed to implement a neural network for cell image classification.
[0020] In some embodiments, neural network application 530 includes several applications and / or modules. In some embodiments, a module also includes one or more sub-modules. In some embodiments, fewer or additional modules may be included.
[0021] Examples of suitable computing devices include a microscope, a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a handheld computer, a personal digital assistant, a cellular / mobile phone, a smart appliance, a game console, a digital camera, a digital camcorder, a camera phone, a smartphone, a portable music player, a tablet computer, a mobile device, a video player, a video disc writer / player (e.g., DVD writer / player, high definition disc writer / player, ultra high definition disc writer / player), a television, a home entertainment system, an augmented reality device, a virtual reality device, smart jewelry (e.g., a smart watch), a vehicle (e.g., a self-driving vehicle), or any other suitable computing device.
[0022] FIG. 6 shows a schematic diagram of the overall configuration of a biological sample analyzer according to some embodiments.
[0023] FIG. 6 shows an exemplary configuration of the biological sample analyzer of the present disclosure. The biological sample analyzer 6100 shown in FIG. 6 includes a light irradiation unit 6101 that irradiates light onto the biological sample S flowing in a flow path C, a detection unit 6102 that detects light generated by irradiating the biological sample S, and an information processing unit 6103 that processes information about the light detected by the detection unit. The biological sample analyzer 6100 is, for example, a flow cytometer or an imaging cytometer. The biological sample analyzer 6100 may include a sorting unit 6104 that sorts out a specific biological particle P in the biological sample. The biological sample analyzer 6100 including the sorting unit is, for example, a cell sorter.
[0024] (Biological samples) The biological sample S may be a liquid sample containing biological particles. The biological particles may be, for example, cells or non-cellular biological particles. The cells may be living cells, more specific examples of which include blood cells such as red blood cells and white blood cells, and reproductive cells such as sperm and fertilized eggs. The cells may also be collected directly from a sample such as whole blood, or may be cultured cells obtained after culturing. The non-cellular biological particles may be extracellular vesicles, particularly exosomes and microvesicles. The biological particles may be labeled with one or more labeling substances (e.g., dyes (particularly fluorescent dyes) and antibodies labeled with fluorescent dyes). It should be noted that the biological sample analyzer of the present disclosure may analyze particles other than biological particles, such as beads for calibration, etc.
[0025] (Flow path) The flow channel C is designed to form a flow of the biological sample S. In particular, the flow channel C can be designed to form a flow in which biological particles contained in the biological sample are substantially arranged in a line. The flow channel structure including the flow channel C can be designed to form a laminar flow. In particular, the flow channel structure is designed to form a laminar flow in which the flow of the biological sample (sample flow) is surrounded by the flow of the sheath liquid. The design of the flow channel structure can be appropriately selected by those skilled in the art, or a known structure can be adopted. The flow channel C can be formed in a flow channel structure such as a microchip (a chip having a flow channel on the order of micrometers) or a flow cell. The width of the flow channel C is 1 mm or less, and in particular, can be 10 μm or more and 1 mm or less. The flow channel C and the flow channel structure including the flow channel C can be formed of a material such as plastic or glass.
[0026] The biological sample analyzer of the present disclosure is designed so that the biological sample flowing in the flow path C, particularly the biological particles in the biological sample, is irradiated with light from the light irradiation unit 6101. The biological sample analyzer of the present disclosure can be designed so that the irradiation point of light on the biological sample is located within the flow path structure in which the flow path C is formed, or can be designed so that the irradiation point is located outside the flow path structure. An example of the former case can be a configuration in which light is emitted onto the flow path C in a microchip or flow cell. In the latter case, light can be irradiated onto the biological particles after they have left the flow path structure (particularly, the nozzle portion thereof), and for example, a jet-in-air type flow cytometer can be adopted.
[0027] (Light irradiation unit) The light illumination unit 6101 includes a light source unit that emits light and a light guide optical system that guides the light to an illumination point. The light source unit includes one or more light sources. The type of light source is, for example, a laser light source or an LED. The wavelength of the light to be emitted from each light source can be any wavelength of ultraviolet light, visible light, and infrared light. The light guide optical system includes optical components such as, for example, a beam splitter, a mirror, or an optical fiber. The light guide optical system can also include a lens group for focusing the light, including, for example, an objective lens. There can be one or more illumination points where the biological sample and the light intersect. The light illumination unit 6101 can be designed to collect light emitted from one light source or different light sources onto one illumination point.
[0028] (Detection unit) The detection unit 6102 includes at least one photodetector that detects light generated by emitting light on the bioparticle. The light to be detected can be, for example, fluorescence or scattered light (one or more of forward scattered light, back scattered light, and side scattered light, etc.). Each photodetector includes one or more light receiving elements, and has, for example, a light receiving element array. Each photodetector can include, as the light receiving element, one or more photomultiplier tubes (PMTs) and / or photodiodes such as APDs and MPPCs. The photodetector includes, for example, a PMT array in which a plurality of PMTs are arranged in a one-dimensional direction. The detection unit 6102 can also include an image sensor such as a CCD or CMOS. Using the image sensor, the detection unit 6102 can acquire an image of the bioparticle (for example, a bright-field image, a dark-field image, or a fluorescent image, etc.).
[0029] The detection unit 6102 includes a detection optical system that allows light of a predetermined detection wavelength to reach a corresponding photodetector. The detection optical system includes a spectroscopic unit such as a prism or a diffraction grating, or a wavelength separation unit such as a dichroic mirror or an optical filter. The detection optical system is designed, for example, to disperse light generated by light irradiation of the bioparticles and detect the dispersed light using a number of photodetectors that is greater than the number of fluorescent dyes used to label the bioparticles. A flow cytometer including such a detection optical system is called a spectral flow cytometer. Furthermore, the detection optical system is designed, for example, to separate light corresponding to a fluorescence wavelength band of a specific fluorescent dye from the light generated by light irradiation of the bioparticles and cause the corresponding photodetector to detect the separated light.
[0030] The detection unit 6102 may also include a signal processing unit that converts the electrical signal acquired by the photodetector into a digital signal. The signal processing unit may include an A / D converter as a device that performs the conversion. The digital signal acquired by the conversion performed by the signal processing unit may be transmitted to the information processing unit 6103. The digital signal may be treated by the information processing unit 6103 as data related to light (hereinafter also referred to as "light data"). The light data may be, for example, light data including fluorescence data. More specifically, the light data may be data of light intensity, and the light intensity may be light intensity data of light including fluorescence (the light intensity data may include features such as area, height, and width).
[0031] (Information Processing Unit) The information processing unit 6103 includes a processing unit that performs processing of various kinds of data (e.g., light data) and a storage unit that stores, for example, various kinds of data. When the processing unit acquires light data corresponding to fluorescent dyes from the detection unit 6102, the processing unit can perform a fluorescence spillover correction (compensation process) on the light intensity data. In the case of a spectral flow cytometer, the processing unit also performs a fluorescence separation process on the light data to acquire light intensity data corresponding to the fluorescent dyes. The fluorescence separation process can be performed, for example, by an unmixing method disclosed in JP 2011-232259 A. When the detection unit 6102 includes an image sensor, the processing unit can acquire morphological information about the bioparticles based on the image acquired by the image sensor. The storage unit can be designed to store the acquired light data. The storage unit can be designed to further store spectral reference data to be used in the unmixing process.
[0032] When the biological sample analyzer 6100 includes a sorting unit 6104 described below, the information processing unit 6103 can determine whether to sort the biological particles based on the optical data and / or the morphological information. The information processing unit 6103 can then control the sorting unit 6104 based on the result of the determination, so that the biological particles can be sorted by the sorting unit 6104.
[0033] The information processing unit 6103 can be designed to output various types of data (e.g., light data and images, etc.). For example, the information processing unit 6103 can output various types of data generated based on the light data (e.g., two-dimensional plots or spectral plots, etc.). The information processing unit 6103 can also be designed to accept input of various types of data, for example, accepting a gating process on the plot by a user. The information processing unit 6103 can include an output unit (e.g., a display, etc.) or an input unit (e.g., a keyboard, etc.) and can perform output or input.
[0034] The information processing unit 6103 can be designed as a general-purpose computer, for example, as an information processing device including a CPU, a RAM, and a ROM. The information processing unit 6103 can be included in a housing including the light irradiation unit 6101 and the detection unit 6102, or can be located outside the housing. Furthermore, various processes or functions to be performed by the information processing unit 6103 can be realized by a server computer or a cloud connected via a network.
[0035] (Sorting unit) The sorting unit 6104 performs sorting of the bioparticles according to the result of the judgment performed by the information processing unit 6103. The sorting method can be a method in which droplets containing bioparticles are generated by vibration, a charge is added to the droplets to be collected, and the moving direction of the droplets is controlled by an electrode. The sorting method can be a method in which the bioparticles are separated by controlling the moving direction of the bioparticles in the flow channel structure. The flow channel structure has, for example, a control mechanism based on pressure (injection or suction) or charge. An example of the flow channel structure can be a chip (for example, the chip disclosed in JP 2020-076736 A) having a flow channel structure in which the flow channel C branches into a recovery flow channel and a waste liquid flow channel on the downstream side, and specific bioparticles are collected in the recovery flow channel.
[0036] To utilize the neural networks for cell image classification described herein, a device such as a microscope including a camera can be used to capture content and the device can process the captured content. The neural networks for cell image classification can be implemented with user assistance or automatically without user involvement.
[0037] In operation, the neural network for cell image classification allows for high accuracy, high speed, and high recall. High accuracy can be approximately 99%, high speed includes classifying 2500 images per second, and high recall is approximately 99%. Other implementations do not match the accuracy, speed, and / or recall.
[0038] In terms of accuracy for cytometry, the accuracy threshold is typically around 99%, meaning that the system must be correct 99% of the time to be considered accurate. In an exemplary implementation, there was a precision rate of 99% and a recall rate of 98.5% on a publicly available dataset. In terms of speed, the system processes, for example, 2500 images per second on a Titan X GPU (latency of less than 0.4 ms).
[0039] Some embodiments of efficient and robust fast neural networks for cell image classification 1. A method comprising: processing the cell images using a neural network, the neural network comprising generating a plurality of softmax scores each corresponding to a class; comparing the highest softmax score and the second highest softmax score among the plurality of softmax scores to obtain a difference; classifying the cell image into a pseudo-class when the difference is below a threshold; classifying the cell image into the class corresponding to the highest softmax score when the difference is equal to or exceeds the threshold; The method includes:
[0040] 2. The method of claim 1, wherein the neural network includes two convolutional layers and one fully connected layer.
[0041] 3. The method of claim 1, wherein the softmax score is greater than or equal to 0 and less than or equal to 1.
[0042] 4. The method of claim 1, wherein the threshold is set based on historical information to produce high precision and high recall.
[0043] 5. The method of claim 1, wherein the threshold is determined by machine learning and artificial intelligence.
[0044] 6. The method of claim 1, wherein the threshold is manually determined by a user.
[0045] 7. The method of claim 1, further comprising the step of acquiring an image of the cell.
[0046] 8. An apparatus comprising: processing the cell images using a neural network, the neural network comprising generating a plurality of softmax scores each corresponding to a class; comparing the highest softmax score and the second highest softmax score among the plurality of softmax scores to determine a difference; classifying the cell image into a pseudo-class when the difference is below a threshold; classifying the cell image into the class corresponding to the highest softmax score when the difference is equal to or exceeds the threshold. a non-transitory memory for storing an application for a processor coupled to the memory and configured to process the application; An apparatus comprising:
[0047] 9. The apparatus of claim 8, wherein the neural network includes two convolutional layers and one fully connected layer.
[0048] 10. The apparatus of claim 8, wherein the softmax score is greater than or equal to 0 and less than or equal to 1.
[0049] 11. The apparatus of claim 8, wherein the thresholds are set based on historical information to produce high precision and high recall.
[0050] 12. The apparatus of claim 8, wherein the threshold is determined by machine learning and artificial intelligence.
[0051] 13. The apparatus of claim 8, wherein the threshold is manually determined by a user.
[0052] 14. The apparatus of claim 8, wherein the apparatus is further configured to acquire an image of the cells.
[0053] 15. A system comprising: a first device configured to acquire a cell image; processing the cell images using a neural network, the neural network comprising generating a plurality of softmax scores each corresponding to a class; comparing the highest softmax score and the second highest softmax score among the plurality of softmax scores to determine a difference; classifying the cell image into a pseudo-class when the difference is below a threshold; classifying the cell image into the class corresponding to the highest softmax score when the difference is equal to or exceeds the threshold. A second device configured to: A system including:
[0054] 16. The system of claim 15, wherein the neural network includes two convolutional layers and one fully connected layer.
[0055] 17. The system of claim 15, wherein the softmax score is greater than or equal to 0 and less than or equal to 1.
[0056] 18. The system of claim 15, wherein the thresholds are set based on historical information to produce high precision and high recall.
[0057] 19. The system of claim 15, wherein the threshold is determined by machine learning and artificial intelligence.
[0058] 20. The system of claim 15, wherein the threshold is manually determined by a user.
[0059] The present invention has been described with respect to specific embodiments incorporating details to facilitate an understanding of the principles of the construction and operation of the invention. Such references herein to specific embodiments and their details are not intended to limit the scope of the claims appended hereto. Those skilled in the art will readily appreciate that various other modifications can be made to the embodiments selected for illustration without departing from the spirit and scope of the invention as defined by the claims. [Explanation of symbols]
[0060] Acquire 100 cell images 102 Processing cell images using neural networks 104 Apply precision-recall tradeoff to classify cells into classes or pseudo-classes 300 Generate and / or receive softmax scores 302 Determine whether the classification is ambiguous 304 Classify cells into pseudo-classes 306 Classify cells into the class corresponding to the highest score 500 Computer equipment 502 Network Interface 504 Memory 506 Processor 508 I / O Devices 510 Bus 512 Storage device 520 Neural Network Hardware 530 Neural Network Applications 6100 Biological Sample Analyzer 6101 Light irradiation unit 6102 Detection Unit 6103 Information Processing Unit 6104 Sorting Unit C Flow Path P bioparticles S Biological sample
Claims
1. 1. A method comprising: Irradiating the biological sample flowing through the flow path with light; detecting light generated by the illuminated biological sample; processing the cellular images of the biological sample using a neural network, the neural network comprising generating a plurality of softmax scores each corresponding to a class; sorting the plurality of softmax scores from highest to lowest; comparing the highest softmax score and the second highest softmax score among the plurality of softmax scores to obtain a difference; classifying the cell image into a pseudo-class when the difference is below a threshold; classifying the cell image into the class corresponding to the highest softmax score when the difference is equal to or exceeds the threshold; The method according to claim 1, further comprising:
2. The method of claim 1 , wherein the neural network includes two convolutional layers and one fully connected layer.
3. The method of claim 1 , wherein the softmax score is greater than or equal to 0 and less than or equal to 1.
4. 10. The method of claim 1, wherein the threshold is set based on historical information to produce high precision and high recall.
5. The method of claim 1 , wherein the threshold is determined by machine learning and artificial intelligence.
6. The method of claim 1 , wherein the threshold is manually determined by a user.
7. The method of claim 1 , further comprising the step of acquiring the cell image.
8. An apparatus comprising: a light irradiation unit configured to irradiate light onto the biological sample flowing through the flow path; a detection unit configured to detect light generated by the illuminated biological sample; processing the cellular images of the biological sample using a neural network, the neural network comprising generating a plurality of softmax scores each corresponding to a class; sorting the plurality of softmax scores from highest to lowest; comparing the highest softmax score and the second highest softmax score among the plurality of softmax scores to determine a difference; classifying the cell image into a pseudo-class when the difference is below a threshold; classifying the cell image into the class corresponding to the highest softmax score when the difference is equal to or exceeds the threshold. a non-transitory memory for storing an application for a processor coupled to the memory and configured to process the application; An apparatus comprising:
9. The apparatus of claim 8 , wherein the neural network includes two convolutional layers and one fully connected layer.
10. The apparatus of claim 8 , wherein the softmax score is greater than or equal to 0 and less than or equal to 1.
11. The apparatus of claim 8 , wherein the thresholds are set based on historical information to produce high precision and high recall.
12. The apparatus of claim 8 , wherein the threshold is determined by machine learning and artificial intelligence.
13. The apparatus of claim 8 , wherein the threshold is manually determined by a user.
14. The apparatus of claim 8 , further configured to acquire an image of the cells.
15. 1. A system comprising: A first device configured to acquire a cell image, the first device further including a light irradiation unit configured to irradiate light onto a biological sample flowing through a flow path, and a detection unit configured to detect light generated by the irradiated biological sample; processing the cellular images of the biological sample using a neural network, the neural network comprising generating a plurality of softmax scores each corresponding to a class; sorting the plurality of softmax scores from highest to lowest; comparing the highest softmax score and the second highest softmax score among the plurality of softmax scores to determine a difference; classifying the cell image into a pseudo-class when the difference is below a threshold; classifying the cell image into the class corresponding to the highest softmax score when the difference is equal to or exceeds the threshold. and a second device configured to A system comprising:
16. 16. The system of claim 15, wherein the neural network includes two convolutional layers and one fully connected layer.
17. 16. The system of claim 15, wherein the softmax score is greater than or equal to 0 and less than or equal to 1.
18. 16. The system of claim 15, wherein the thresholds are set based on historical information to produce high precision and high recall.
19. 16. The system of claim 15, wherein the threshold is determined by machine learning and artificial intelligence.
20. 16. The system of claim 15, wherein the threshold is manually determined by a user.
Citation Information
Patent Citations
Methods and Systems for Disease Classification
US20170091528A1
Constrained Classification and Ranking via Quantiles
US20190266513A1
Deep Learning-Based Framework for Identifying Sequence Patterns that Cause Sequence-Specific Errors (SSEs)
US20200251183A1
Platform, device and process for annotation and classification of tissue specimens using convolutional neural network
US20200272864A1
Systems and methods for automatically interpreting images of microbiological samples
US20200357516A1
