Methods and systems for processing images of biological samples

Abstract

Systems and method are provided for processing images representing biological samples captured using off-the-shelf (OTS) hardware. A computing device may receive a set of images from a mobile device over a network. The set of images may include images of biological samples captured by a camera of the mobile device. In some instances, the mobile device may be secured to an eyepiece of a microscope using an adapter to enable consistent imaging of the biological samples. The computing device may execute a neural network using the set of images. The neural network may be configured to generate one or more characteristics of the biological samples represented in the set of images such as, but not limited to, cell confluency. The computing device may then transmit the one or more characteristics of the biological samples to a computing device.

Description

METHODS AND SYSTEMS FOR PROCESSING IMAGES OF BIOLOGICALSAMPLESCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The present patent application claims the benefit of priority to U. S. Provisional Patent Application 63 / 518,234, filed August 8, 2023, and Canadian Patent Application No. 3,208,655 filed on August 8, 2023, which are incorporated herein by reference in their entirety for all purposes.TECHNICAL FIELD

[0002] This disclosure relates to processing images of biological samples, and more particularly to continuous processing of biological images captured using mobile devices.BACKGROUND

[0003] Accurate measurement of cell fitness is essential for understanding cell growth patterns in experimental biology and medical research. Current processes for measuring characteristics of cell growth patterns use reagents or specialized equipment for live cell imaging (e.g., such as specially designed microscopes, imaging devices, etc.) that may be expensive to acquire, take time to acquire, and / or, difficult to operate (e.g., prone to introducing error, etc.).SUMMARY

[0004] Methods are described herein for processing images of biological samples to derive characteristics of the biological samples. The methods may include: receiving a request to initiate a monitoring session associated with a user profile of a user device, wherein the request includes at least one monitoring process; receiving one or more images captured over a time interval defined by the at least one monitoring process, wherein the one or more images are generated by an imaging device configured to capture images through a lens of a microscope, and wherein the one or more images include a representation of a biological sample; executing a neural network using the one or more images, wherein the neural network is configured to derive one or more features from at least one image of the one or more images; predicting, using the neural network, a presence of one or more characteristics of the biological sample inthe at least one image, wherein the one or more characteristics are defined by the at least one monitoring process; and transmitting the one or more characteristics of the biological sample and a mask generated by the neural network, wherein when received by the user device, the user device generates a statistical analysis based on the one or more characteristics of the biological sample and the at least one monitoring process

[0005] The systems described herein for processing images of biological samples. The systems may include one or more processors and a non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the one or more processors to perform any of the methods as previously described.

[0006] The non-transitory computer-readable media described herein may store instructions which, when executed by one or more processors, cause the one or more processors to perform any of the methods as previously described.

[0007] These illustrative examples are mentioned not to limit or define the disclosure, but to aid understanding thereof. Additional embodiments are discussed in the Detailed Description, and further description is provided there.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Features, embodiments, and advantages of the present disclosure are better understood when the following Detailed Description is read with reference to the accompanying drawings.

[0009] FIG. 1 illustrates a block diagram of example image processing network configured to process sets of biological images from varying sources according to aspects of the present disclosure.

[0010] FIG. 2 illustrates a diagram of an example adapter configured to position and stabilize an imaging device relative to an eyepiece of a microscope according to aspects of the present disclosure.

[0011] FIG. 3 illustrates a flowchart of an example process for processing images of biological samples according to aspects of the present disclosure.

[0012] FIG. 4 illustrates a block diagram of example machine-learning model with dynamically configurable skip connections configured to process biological images from varying sources according to aspects of the present disclosure.

[0013] FIG. 5 illustrates a block diagram of example machine-learning system including one or more machine-learning stages configured to process biological images from varying sources according to aspects of the present disclosure.

[0014] FIG. 6 A and 6B illustrate graphs of accuracy metrics of the machine-learning models described herein according to aspects of the present disclosure.

[0015] FIG. 7 illustrates a flowchart of another example process for processing images of biological samples according to aspects of the present disclosure.

[0016] FIG. 8 illustrates an example computing device architecture of an example computing device that can implement the various techniques described herein according to aspects of the present disclosure.DETAILED DESCRIPTION

[0017] Current methods for characterizing live cells (e.g., for research, experimentation, diagnostics, etc.) include the use of reagents and / or manual analysis of samples of the live cells. For instance, a researcher may analyze the samples through a microscope and record subjective observations from which characteristics of the live samples may be derived. In some instances, specially designed microscopes and / or imaging devices may be utilized to capture images of the samples from which the researcher may record subjective observations from which characteristics of the live samples may be derived. The use of reagents and specially designed microscopes and / or imaging devices may be difficult to obtain, costly, and difficult to operate (thereby potentially introducing user error, which may invalidate results, etc.).

[0018] The systems and methods described herein include processes for capturing images of biological samples using off-the-shelf (OTS) hardware. The images can be processed using the pattern recognition techniques and processes described herein to derive accurate characteristics of the biological samples without affecting the samples. The OTS hardware may be configured for use with most microscopes thereby providing an efficient system to capture images of biological samples. The images may be processed using an image processing network that includes a set of trained neural networks configured to utilize pattern recognition to generate features (e.g., characteristic, etc.) of the biological samples represented by the images and provide metadata useable for diagnostic and verification of the images and for further training of the set of neural networks. The image processing network may process images in real time and / or in batches and transmit the generated features to the OTS hardware and / or to anotherdevice. The combination of OTS hardware and the image processing network provides a resource efficient, user friendly, and accurate assessment of live cells.

[0019] The image processing network may operate in conjunction with an application executing on a computing device. The computing device may be a mobile device (e.g., such as a smartphone, etc.) that includes processing hardware (e.g., central processing unit, memory, network interfaces, etc.) and a camera. When the application executes for a first time, the application may register the mobile device with the image processing network by transmitting a registration request. The registration request may include characteristics of the mobile device (e.g., a device identifier, an identification of hardware and / or software installed on the mobile device, an Internet Protocol (IP) address of the mobile device, a Media Access Control (MAC) address of the mobile device, etc.), an identification of a user that owns or operates the mobile device, an organization that is associated with the mobile device, combinations thereof, or the like. The image processing network may determine, based on the information included in the request, whether a profile exists for the mobile device, the user, and / or the organization. If the image processing network determines that a profile exists for the mobile device, the user, and / or the organization, then the image processing network may transmit an identification of the existing profile to the application. If the image processing network determines that no profile exists for the mobile device, the user, and / or the organization, then the image processing network generates anew profile and transmits an identifier of the new profile to the application. A profile may be associated with one or more devices, one or more users, and / or one or more organizations.

[0020] Once registered, the application may define a monitoring process associated with a particular experiment, group (e.g., group of experiments, group of samples being monitored), replicate (e.g., an experiment, assay, etc. being reproduced), a particular sample, or the like. A monitoring process may be a configuration that indicates how data associated with one or more samples will be collected and processed. The monitoring process may be defined based on one or more predefine monitoring types (e.g., based on particular experiments, biological samples, etc.), previous monitoring processes, one or more machine-learning models, user input, etc. In some instances, defining a monitoring process may include an identifier of an experiment or assay, an identification of a biological sample or material being monitored, an identification of a time interval over which the biological sample or biological material is being monitored, a frequency in which images are to be captured over the time interval, a quantity of images to be captured each time images are captured according to the frequency, features to derive fromcaptured images (e.g., such as, but not limited to, cell confluency, cell count, cell migration and motility, cell size, changes in cell morphology, cell differentiation, sperm cell analysis, clonogenicity, bacterial clone count, yeast count, fermentation, combinations thereof, or any feature or information that can be identified or derived from the images), combinations thereof, or the like. The application may then associate images captured and any information derived therefrom with the monitoring process.

[0021] The mobile device may be affixed to a microscope (or other optical magnifying device) using an adapter that may ensure the mobile device can capture consistent images over the time interval of the monitoring process. The application may cause the mobile device to capture images according to the frequency and quantity defined by the monitoring process. For example, the mobile device may capture X images of a biological sample every Y hours over the time interval. The images may be transmitted to the image processing network as the images are captured (e.g., in real time). Alternatively, or additionally, the images may be transmitted to the image processing network in batches (e.g., such as every day at predetermined time, etc.).

[0022] The image processing network may include one or more pattern recognition processes, algorithms, machine-learning models, combinations thereof, or the like to process the images and derive the identified characteristics of the monitoring process. The one or more machine-learning models may be neural networks (e.g., such as deep learning networks, convolutional neural networks, faster convolutional neural networks, region-based convolutional neural networks, fast or faster convolutional neural networks, You Only Look Once (Y OLO), Naive Bayes, densely connected networks, and / or any machine-learning model configured to detect features represented in an image, etc.). Each machine-learning model may be configured to train pattern recognition algorithms that derive one or more features from an input image or set of input images. For example, a first machine-learning model may train a pattern recognition algorithm to derive a cell count of cells represented in an image and a second machine-learning model may be train a pattern recognition algorithm to derive a cell migration (e.g., cell velocity, directionality, travel distance, etc.) from cells represented in sets of images. The image processing network may train pattern recognition algorithms according to one or more predefined features for which monitoring processes are frequently established. If the image processing network does not have a pattern recognition algorithm trained to derive a particular feature from an input image or set of images, the image processing network may train a new pattern recognition algorithm to derive the particular feature.

[0023] The image processing network may train pattern recognition algorithms by identifying a set of training images associated with a target feature and corresponding features. The corresponding features may include metadata, features of the training images (e.g., image dimensions, image resolution, fde sizes, quantity of training images, types of training images, an identification of a file type, etc.), labels (e.g., for training involving supervised learning, etc.), etc. The image processing network may store images in a training database that correspond to historical monitoring processes, data received from one or more remote sources or databases, procedurally generated data, images generated by machine-learning models, manually generated or labeled images, and / or the like. The training images may be assigned one or more keywords corresponding to a feature that the image may be used to train a machinelearning model to derive. The keywords may be used to identify sets of training images usable to train a machine-learning model to derive a target feature.

[0024] The image processing network may train the pattern recognition algorithms for a predetermined time interval, a predetermined quantity of iterations, or until one or more accuracy metrics reach threshold values. Examples of accuracy metrics include, but are not limited to, accuracy, precision, recall, Fl score, confusion matric, logarithmic loss, mean absolute error, mean squared error, area under the curve (ROC), and / or the like). In some instances, the one or more accuracy metrics and corresponding thresholds may be selected based on user input or predetermined. The pattern recognition algorithms may also be trained using reinforcement learning with each time a machine-learning model is executed using an input image. Alternatively, a reinforcement learning iteration may be executed when an error is detected in the output of the pattern recognition algorithm. Reinforcement learning may enable the pattern recognition algorithms to learn each time the machine-learning model processes an image.

[0025] The image processing network may select one or more pattern recognition algorithms to derive the features identified by the monitoring process. The image processing network may define an execution schedule that includes order in which to execute the one or more pattern recognition algorithms. The one or more pattern recognition algorithms may be scheduled to execute in series, in parallel, or combination of in series and in parallel. Each pattern recognition algorithm may receive the same input images. Some pattern recognition algorithms may receive more images as input than other machine learning models. Based on the particular pattern recognition algorithm being executed and an output generated by the pattern recognition algorithm, the images may be preprocessed differently before being input (e.g.,based on the particular pattern recognition algorithm and / or output to be generated, etc.). Examples of preprocessing operations that may be executed on the images include, but are not limited to, thresholding (e.g., modifying pixel luminance and / or chrominance based on one or more thresholds), erosion (e.g., decreasing a size of regions of pixels with a luminance greater than a threshold and increasing a size of regions of pixels with a luminance less than a threshold), dilation (e.g., increasing a size of regions of pixels with a luminance greater than a threshold and decreasing a size of regions of pixels with a luminance less than a threshold), closing (e.g., executing dilation after erosion to reduce small low luminance pixel regions), normalization (e.g., rescaling pixel values into a predefine range of pixel values), augmentation (e.g., linear transformation, affine transformation, etc. configured to regularize the feature of an image, generate alternate versions of training images to improve variety of training images and training robustness, etc.). The image processing network (or a user) may determine what preprocessing to perform for each pattern recognition algorithm. In some instances, the preprocessing operations may be performed by the mobile device before the images are transmitted to the image processing network.

[0026] The image processing network may execute the pattern recognition algorithms according to the schedule to derive the features of the monitoring process. The features derived by the one or more pattern recognition algorithms may be transmitted to the application executing on the mobile device. In some instances, the image processing network may also transmit metadata associated with the one or more pattern recognition algorithms to the application. The metadata may characterize the operation of the pattern recognition algorithms, or the features output from the pattern recognition algorithms usable to assess accuracy, reproducibility, etc. of the pattern recognition algorithms. The metadata may include, but is not limited to, accuracy metrics (as previously described), confidence values associated with features derived by the pattern recognition algorithms, masks (e.g., pixel-segmentation of the pixels representing objects of interest), an identification of boundary boxes, an identification of the monitoring process (e.g., in instances where the image processing network is processing images associated with multiple monitoring processes for a same profile), combinations thereof, or the like.

[0027] The application may store the features and metadata received from the image processing network in association with the monitoring process. The application may use the features to generate tables, graphs, derive additional features or features, generate reports, etc. to provide a framework for a statistical analysis of the monitoring process over the timeinterval. The application may, for example, track the cell confluence over time to derive cell growth patterns of a cell line, generate time lapses from the images over the first time interval, predict the cell confluence at different time points over the time interval of the monitoring process and / or over future time intervals, define quality control metrics (e.g., by comparing cell confluence values over time, cell confluence values between images of a same set of images, etc.), define experimental designs for cell confluence assays (e.g., the application may define experimental conditions, experimental guidelines, etc. for subsequent cell assays), combinations thereof, or the like. In some instances, the operations performed by the application may be performed by the image processing network (in addition to or in place of the application). For example, real-time reports can be generated that include a growth curve of cells represented in received images, statistical analyses such as mean values, standard errors, standard deviations, confidence intervals, and / or the like.

[0028] The application may store representative images (or all images) associated with a particular monitoring process along with the features derived from the representative images received from the image processing network. The application may be used to track monitoring current and previous monitoring processes as well as review previous and / or historical experiments, groups, replicates, events, etc. In some instances, historical experiments, groups, replicates, events, etc. may include experiments, groups, replicates, events, etc. associated with other profiles (e.g., different mobile devices, users, organizations, etc. Upon termination of the time interval of the monitoring process, the application may prevent subsequent images and features derived therefrom from begin associated with the monitoring process to prevent data contamination or corruption from affect the monitoring process. The application may be configured to export the monitoring process (e.g., raw data, images, representative images, derived features, tables, charts, graphs, and / or any information received or derived by the application that is associated with the monitoring process) in any suitable format (e.g., a set of data structures, images, spreadsheets, word processing documents, data files, etc.). For example, the mobile device may export the monitoring process by transmitting the monitoring process to one or more computing devices and / or other devices. Alternatively, or additionally, the monitoring process may be exported by the image processing network. In those instances, if the image processing network stores the monitoring process, then the image processing network may transmit the monitoring process to the one or more computing devices and / or other devices. If the image processing network does not store the monitoring process (or not all of the data of the monitoring process), the image processing network may transmit any dataof the monitoring process stored by the image processing network. The image processing network may then cause the application executing on the mobile device to transmit any data not stored by the image processing data to the image process network, which will retransmit the data to the one or more computing devices and / or other devices or directly to the one or more computing devices and / or other devices.

[0029] In an illustrative example, an image processing framework may be utilized to process images of samples to derive characteristics of the samples. The image processing framework may include a computing device that may be configured to process images of a sample received from a mobile device (e.g., such as a smartphone, etc.) that captured the images. The mobile device may be affixed to a microscope enabling the mobile device to capture images of the sample magnified by the microscope. For example, the sample may be a biological sample (e.g., cells, etc.) associated with an experiment. The mobile device may be configured to capture one or more images of the sample at regular intervals over a time interval. In some examples, the quantity of images captured may be based on a quantity of wells prepared for the microscope. For instance, if a 96-well plate is prepared for the microscope, then the mobile device may be configured to capture at least one image per well (e.g., for n*96 images, etc.).

[0030] The received images may be in an unaltered state (e.g., in the same form and format as captured by the camera) or in a preprocessed state. For instance, an application executing on the mobile device may perform some preprocessing of the images before the images are transmitted to the computing device. For example, the mobile device may convert images to a different form or format, upscale the images (e.g., using interpolation, a machine-learning model, etc. to increase a resolution), downscale the images (e.g., reducing a resolution), change the dimensions, perform thresholding, perform erosion, perform, dilation, perform closing, perform normalization, augment the images, combinations thereof, or the like. Alternatively, one or more of the preprocessing operations may be performed by the computing device (addition to or in place of the preprocessing operations performed by the mobile device).

[0031] The computing device execute a neural network using the set of images. The neural network may be a convolutional neural network configured to process images and output one or more features of the sample represented in the images and metadata characterizing the images and / or the one or more feature (e.g., such as confidences, accuracy metrics, masks, etc.). The one or more features may include, but are not limited to, cell confluency, cell count, cell size, cell migration or motility (e.g., cell position, cell velocity, etc.), an identification ofmorphological changes, cell differentiation (e.g., classification of cells, etc.), cell analysis (e.g., such as for sperm cells, yeast cells, etc.), clonogenicity, fermentation analysis, etc.

[0032] The computing device may then transmit the one or more features and / or the metadata to another computing device (e.g., such as a computer, server, database, the mobile device, or any other processing device). The computing device may then transmit the one or more features of the biological samples to a computing device.

[0033] The mobile device or other computing device may execute an application configured to track the monitoring of the sample over the time interval of the experiment. The application may aggregate the images captured over the regular intervals during the time interval and the output received from the computing device in response to processing those images. The mobile device may be configured to generate one or more of time lapses (e.g., using automatically and / or manually selected sampling of the captured images), charts, tables, graphs, additional characteristics or features, predictions, interpolations, combinations thereof, or the like associated with the sample over the time interval. Alternatively, or additionally, the computing device (or a device associated with the computing device) may be configured to generate one or more of time lapses, charts, tables, graphs, additional characteristics or features, predictions, interpolations, combinations thereof, or the like associated with the sample over the time interval.

[0034] FIG. 1 illustrates a block diagram of example image processing network configured to process sets of biological images from varying sources according to aspects of the present disclosure. Imaging processing framework 100 may include operations executed by mobile devices and / or image processing networks to process images captured by mobile devices (as described in connection to this FIG. 1 as well as throughout the present disclosure). For example, Mobile devices 104A-104C may be utilized to capture images of a sample (e.g., such as, but not limited to, a biological sample such as cells cultures, etc.) in regular intervals over a time interval. The images may be processed by the mobile device that captured the images or the images may be transmitted over network 112 (e.g., local area network, wide area network, the Internet, etc.) to image processing network 116. Image processing network 116 may include one or more computing devices (e.g., computers, servers, databases, networks, etc.) configured to process images received from mobile devices and transmit the results of processing the images back to the transmitting mobile device or to other devices (e.g., such as computing device 108, etc.). In some instances, each computing device of image processing network 116may be a discrete processing node configured to process images received from mobile devices 104A-104C. In other instances, image processing network 116 may be a distributed network (e.g., such as cloud network, or the like), in which image processing network 116 may distribute image processing tasks across the computing devices of image processing network 116 based on load of the computing devices of image processing network 116.

[0035] Image processing network 116 may be configured to process images received from one or more mobile devices (e.g., mobile devices 104A-104C). Mobile devices 104A-104C may be associated with a same user and / or organization (e.g., such that images captured by each of mobile device 104A-104C may be associated with a same experiment, assay, image processing operation, etc.) or associated with different users and / or organizations. Image processing network 116 may execute image processing operations for any number of users and / or organizations each may correspond it images received from one or more mobile devices.

[0036] Image processing network 116 may include session manager 120 that defines and manages sessions. A session may correspond to an image processing operation requested by a mobile device and facilitated by image processing network 116. A session may be defined based on a monitoring process specification (e.g., defining an image processing operation). A monitoring process specification may be a configuration of a monitoring process that indicates how data associated with one or more samples will be collected and processed. The monitoring process specification may be defined based on one or more predefined types (e.g., based on particular experiments, samples, etc.), previous monitoring processes specifications, one or more machine-learning models, user input, etc. Examples of parameters included in a monitoring process specification may include, but is not limited to, an identification of an experiment or assay, an identification of a sample or material represented in images to be processed by image processing network 116, an identification of a time interval over which the sample or material is being monitored, a frequency in which images are to be captured over the time interval, a quantity of images to be captured each time images are captured according to the frequency, features to derive from captured images (e.g., such as, but not limited to, features including or associated with cell confluency, cell count, cell migration and motility, cell size, changes in cell morphology, cell differentiation, sperm cell analysis, clonogenicity, bacterial clone count, yeast count, fermentation, combinations thereof, or any feature or information that can be identified or derived from the images), combinations thereof, or the like.

[0037] Session manager 120 may manage communications between external devices (e.g., mobile devices 104A-104C. computing device 108, etc.) and internal components of image processing network 116. For example, session manager 120 may be configured to translate incoming communications into a form or format native to image processing network 116, generate outgoing communications into a form or format native to a recipient device, route communications within image processing network 116 to an appropriate computing device, component, or process, and / or the like. Session manager 120 may include one or more network interfaces, application programming interfaces (e.g., to enable communications using different protocols, etc.), input / output interfaces (e.g., physical and / or software interfaces configured to process connections with peripherals and devices), combinations thereof, or the like.

[0038] A mobile device may transmit a communication to image processing network 116 associated with a monitoring process specification. Image processing network 116 may use the communication to determine if mobile device 104 (and / or a user thereof, an organization associated therewith, etc.) is registered with image processing network 116. For example, image processing network 116 may identify access credentials, a token, an IP or MAC address of mobile device 104, etc. in the communication and use the access credentials, token, IP or MAC address of mobile device 104, etc. to identify a profile in profiles 122 associated with mobile device 104. If a profile is identified in profiles 122 (e.g., indicating that mobile device 104, the user thereof, the organization associated therewith, etc. has previously registered with image processing network 116), then image processing network may associate a new image processing operation with an identifier of the profile. If a profile is not identified in profiles 122 (e.g., indicating that mobile device 104, the user thereof, the organization associated therewith, etc. has not previously registered with image processing network 116), then image processing network 116 may generate a new profile and associate a new image processing operation with an identifier of the new profile.

[0039] The profile may include, but is not limited to, an identification of mobile device 104 (e.g., an identifier associated with mobile device 104 such as a universally unique identifier (UUID) assigned by mobile device 104 or image processing network 116), characteristics of mobile device 104 (e.g., such as processing capabilities, hardware and / or software installed therein, an IP address, a MAC address, combinations thereof, or the like), an identification of a user and / or organization associated with mobile device 104 (e.g., such as an owner or operator of mobile device 104, a laboratory, etc.), an identification of image processing operations associated with the profile (e.g., current and / or historical image processing operations, etc.),pointers to a current image processing operation and / or historical image processing operations (e.g., memory address or link to a region of memory storing information associated with current and / or historical image processing operation), combinations thereof, or the like.

[0040] Mobile device 104A may request a new session by transmitting a monitoring process specification to image processing network 116. Session manager 120 may receive the monitoring process specification and establish a new session using the monitoring process specification and the profile associated with mobile device 104 A. If the monitoring process specification is incomplete (e.g., missing one or more parameters, etc.), then image processing network may use the profile associated with mobile device 104A to define the missing portions of the monitoring process specification.

[0041] Session manager 120 may pass a communication to ML process 124 enable processing images of the image processing operation. ML process 124 may be configured to manage pattern recognition algorithms and machine-learning models configured to perform various image processing tasks. ML process 124 may manage one or more pattern recognition algorithms configured to process received images, derive features from received images, etc. ML process 124 may manage one or more machine-learning models configured to train pattern recognition algorithms and / or perform pattern recognition on input images to derive features from received images, etc. ML process 124 may train new pattern recognition algorithms using machine-learning models, train new machine-learning models, retrain (or reinforce) existing pattern recognition algorithms and / or machine-learning models, terminate (e.g., delete) pattern recognition algorithms and / or machine-learning models, and / or the like. In some instances, ML process 124 may manage a variety of pattern recognition algorithms and machine-learning models (e.g., such as, but not limited to, convolutional neural networks, deep neural networks, classifiers, etc.) each being configured to generate a same, similar, or different output (e.g., one or more features, a classification, etc.). Each request to process an image from session manager 120 may include an identification of a particular pattern recognition algorithm and / or machinelearning model or include a requested output (e.g., one or more features, a classification, etc.) to enable ML process 124 to route the request to an appropriate pattern recognition algorithm and / or machine-learning model or instantiate and train a new pattern recognition algorithms and / or machine-learning model configured to satisfy the request.

[0042] If ML process 124 cannot identify a trained pattern recognition algorithm configured to process the request, then ML process 124 may instantiate and train one or more patternrecognition algorithms configured to process the request. For instance, ML process 124 may instantiate a first pattern recognition algorithm configured to output a cell confluency value and a second pattern recognition algorithm configured to characterize cell mobility. The first pattern recognition algorithm and the second pattern recognition algorithm may both be or be trained by convolutional neural networks. Although the first pattern recognition algorithm and the second pattern recognition algorithm may be a same type of algorithm, the training of each algorithm and / or model may be targeted based on a particular expected output. The first pattern recognition algorithm may not be configured to generate a same output as the second pattern recognition algorithm and vice versa. ML process 124 may instantiate and train other types of algorithms and / or models to perform other operations. Examples of types of algorithms and / or models include, but are not limited to, classifiers (e.g., regression models, support vector machines, statistical models etc.), clustering models, etc.

[0043] ML process 124 may select one or more pattern recognition algorithms and / or machine-learning models based on the request and / or an expected output. ML process 124 may then use feature extractor 128 to generate training datasets for the new pattern recognition algorithms and / or machine-learning models. Feature extractor 128 may define training dataset using historical session data 140. Training image data 132 may store images and / or features usable to train and retrain pattern recognition algorithms and / or machine-learning models. The images and / or features may include images and / or features from previous image processing operations, procedurally generated images and / or features (e.g., , augmented versions of other images including affine transformed images, interpolated features, etc.), manually generated images and / or features, labels (e.g., for supervised learning), images and / or features received from one or more remote devices, combinations thereof, or the like.

[0044] Feature extractor 128 may define a training dataset based on the algorithm or model type of the pattern recognition algorithm and / or machine-learning model to be trained and the expected output to be generated by the pattern recognition algorithm and / or machine-learning model to be trained. Feature extractor 128 may pass the training dataset to ML process 124. Alternatively, feature extractor 128 may pass pointers to one or more locations within memory of image processing network 116 where the training data is stored to ML process 124. ML process 124 may then use the pointers to access the training data and train the machine-learning modes.

[0045] ML process 124 may initiate a training phase for each of the one or more pattern recognition algorithms and / or machine-learning models. The one or more pattern recognition algorithms and / or machine-learning models may be trained using supervised learning, unsupervised learning, self-supervised learning, or the like. The one or more pattern recognition algorithms and / or machine-learning models may be trained based on a predetermined time interval, a predetermined quantity of iterations, until one or more target accuracy metrics have exceeded a corresponding threshold function (e.g., accuracy, precision, area under the curve, logarithmic loss, Fl score, weighted human disagreement rate, cross entropy, mean absolute error, mean square error, etc.), user input, combinations thereof, or the like. Once trained, ML process 124 may validate and / or test the trained pattern recognition algorithm and / or machine-learning models using additional training datasets. At runtime (e.g., during execution of an image processing operation), ML process 124 may continue training pattern recognition algorithms and / or machine-learning models using reinforcement learning. A reinforcement learning iteration can be executed each time a pattern recognition algorithm and / or machine-learning model executed, each time feedback associated with an output from a pattern recognition algorithm and / or machine-learning model is received, based on one or more accuracy metrics, upon receiving particular user input, upon detection of an event, in regular time intervals, etc.

[0046] ML process 124 may store the trained pattern recognition algorithms and / or machinelearning models in machine-learning models 138. Once the pattern recognition algorithms and / or machine-learning models are trained, ML process 124 may then manage the operation of the one or more pattern recognition algorithms and / or machine-learning models. If the one or more pattern recognition algorithms and / or machine-learning models are trained in response to receiving a monitoring process specification from mobile device 104A, then session manager 120 may transmit a notification indicating that image processing network 116 can begin processing images corresponding to the monitoring process specification. Mobile device 104A may begin transmitting images to image processing network 116 for image processing. Session manager 120 may receive the images and route the images to ML process 124 and / or feature extractor 128. Feature extractor 128 may be configured to extract features and / or feature vectors from the images that can be unput into machine-learning models (e.g., for those machine-learning models that use feature vectors rather than images as input).

[0047] Feature extractor 128 may also preprocess images received from mobile device 104A. Examples of preprocessing operations that can be executed on the received images include, butare not limited to, thresholding (e.g., modifying pixel luminance and / or chrominance based on one or more thresholds), erosion (e.g., decreasing a size of regions of pixels with a luminance greater than a threshold and increasing a size of regions of pixels with a luminance less than a threshold), dilation (e.g., increasing a size of regions of pixels with a luminance greater than a threshold and decreasing a size of regions of pixels with a luminance less than a threshold), closing (e.g., executing dilation after erosion to reduce small low luminance pixel regions), normalization (e.g., rescaling pixel values into a predefine range of pixel values), augmentation (e.g., linear transformation, affine transformation, etc. configured to regularize the feature of an image, generate alternate versions of training images to improve variety of training images and training robustness, etc.). Feature extractor 128 (or a user) may determine what preprocessing operations to perform for each pattern recognition algorithm and / or machinelearning model (e.g., based on the type of pattern recognition algorithm and / or machinelearning model, the expected output from the pattern recognition algorithm and / or machinelearning model, etc.). In some instances, one or more preprocessing operations may be performed by mobile device 104A before mobile device 104A transmits the images to image processing network 116.

[0048] Image processing network 116 may store trained pattern recognition algorithms and / or machine-learning models in machine-learning models 138. When images are received from mobile device 104A, ML process 124 may use ML model selector 136 to identify a set of pattern recognition algorithms and / or machine-learning models from machine-learning models 138 configured to process the images and generate the output features requested (e.g., the output features identified by the monitoring process specification, etc.).

[0049] ML process 124 may then execute the set of pattern recognition algorithms and / or machine-learning models to generate the output features. In some instances, ML process 124 may define an execution schedule for the set of pattern recognition algorithms and / or machinelearning models. The execution schedule may indicate an order in which the set of pattern recognition algorithms and / or machine-learning models are to be executed. Some pattern recognition algorithms and / or machine-learning models of the set of pattern recognition algorithms and / or machine-learning models may be executed in parallel with other pattern recognition algorithms and / or machine-learning models. Some pattern recognition algorithms and / or machine-learning models of the set of pattern recognition algorithms and / or machinelearning models may be executed in series with other pattern recognition algorithms and / or machine-learning models (e.g., such as a pattern recognition algorithm and / or machine-learning model that may use an output from another pattern recognition algorithm and / or machine-learning model as input, etc.). ML process 124 may pass the output features to session manager 120, which may transmit the output features to mobile device 104A and / or to devices associated with mobile devices 104A (e.g., as specified in the monitoring process specification).

[0050] ML process 124 may facilitate the processing of images in real time and / or in batches. For example, ML process 124 may facilitate real time processing of images to generate output features with second or minutes. In one example, cell fitness can be determined from an input image 20 times faster that reagent-based assays. ML process 124 may process received images as the images are received by image processing network 116 while ML process 124 has the processing resources to do so. If a processing load (e.g., CPU, memory, GPU, network, etc.) is greater than a load threshold. ML process 124 may begin processing some or all received images in batches (e.g., where the batch size and batch execution schedule may be determined by a load balancer, machine-learning model, user input, combinations thereof, or the like.

[0051] In some instances, image processing network 116 may aggregate the output features generated over the time interval of the monitoring process specification for analysis, to derive additional features, etc.). For example, image processing network 116 may generate one or more of time lapses (e.g., using an automatically and / or manually selected sampling of the received images), charts, tables, graphs, additional characteristics or features, predictions, interpolations, combinations thereof, or the like associated with the sample represented in the received images over the time interval. In other instances, an application of mobile device 104A may receive the output features and generate one or more of the time lapses, charts, tables, graphs, additional characteristics or features, predictions, interpolations, combinations thereof, or the like associated with the sample over the time interval.

[0052] Mobile device 104A may continue transmitting images to image processing network 116 over the time interval identified by the monitoring process specification. In some instances, mobile device 104A may indicate that the time interval is to be extended or terminated earlier. Once the time interval expires, the image processing network may not accept new images from mobile device 104A to prevent contaminating or corrupting the data associated with the session.

[0053] FIG. 2 illustrates a diagram of an example adapter configured to position and stabilize an imaging device relative to an eyepiece of a microscope according to aspects of the presentdisclosure. An adapter may be provided to enable an imaging device to capture images of samples through a microscope. The adapter may be fully adjustable to allow any off the shelf imaging device (e.g., smartphone, camera, etc.) to be used with any microscope. Using the disclosed adapter, imaging of samples can be performed without using costly specialized imaging devices or microscopes.

[0054] Imaging device 204 (e.g., a mobile device such as a smartphone, camera, etc.) may be removably affixed to an adapter to allow the imaging device to capture images of samples being magnified by an optical magnifying device (e.g., such as a microscope, or the like). The adapter may be affixed to eyepiece 208 of an optical magnifying device. Holding element 212 may be configured to secure imaging device 204 to the adapter with a camera of imaging device 204 facing eyepiece 208. Holding element 212 may secure imaging device 204 using an adjustable bracket 216. Adjustable bracket 216 may be secured to prevent movement of imaging device 204 relative to holding element 212.

[0055] Holding element 212 may be secured to lens adjusting element 220 with one or more fasteners to enable holding element to be positioned relative to lens adjusting element 220 (e.g., enabling different sized imaging devices to be aligned with eyepiece 208). Eyepiece 208 may be secured to lens adjusting element 220 via fastener 224 (e.g., a thumb screw or other adjustable fastener, etc.).

[0056] Lens 228 may be positioned proximate to an aperture within lens adjusting element 220. The position of lens 228 may be adjusted (e.g., by exerting a rotational force, a longitudinal force, etc.) relative to eyepiece 208 to accommodate different sized eyepieces. The position of lens 228 may be adjusted to align lens 228 with eyepiece 208. The position of holding element 212 relative to lens adjusting element 220 may be adjusting using the one or more fasteners to align the camera of an imaging device (such as mobile device 104, an analog or digital camera, etc.) with lens 228 and eyepiece 208. The fastener may be tightened to prevent holding element 212 from moving relative to lens adjusting element 220 (e.g., preventing the camera of imaging device 204 to become out of alignment with lens 228 and eyepiece 208). The adapter (once the fasteners are tightened) may also stabilize the imaging device to prevent forces from inducing vibrations or affecting the quality of images captured by imaging device 204.

[0057] Once aligned, imaging device 204 may be operated to capture images of samples through eyepiece 208. The images may be transmitted to an image processing network (e.g.,such as image processing network 116 of FIG. 1 , or the like). The adapter may be manufactured using plastics (e.g., such as, but not limited to, polylactic acid, or the like), metals, composites, and / or any other materials. In some instances, the adapter may be manufactured in separate pieces. In other instances, the adapter may be a manufactured a monolithic (e.g., single) component.

[0058] FIG. 3 illustrates a flowchart of an example process of processing images of biological samples according to aspects of the present disclosure. At block 304, a computing device may receive a set of images from a mobile device over a network. The computing device may be a component of an image processing network (e.g., such as image processing network of FIG. 1 as previously described) or a discrete device configured to process images for one or more mobile devices. In some instances, the computing device may receive metadata associated with the set of images. The metadata may provide additional information and / or context of one or more images of the set of images. For example, the metadata associated with an image may identify an area of interest (e.g., such as an area including a representation of cells) to improve the efficiency and / or accuracy of the image processing. The metadata may include, but is not limited to, an identifier of the image, a timestamp of when the image was taken, an identification of one or more expected output features, an identification of the biological sample, an identification of one or more cells and / or materials represented in the image, one or more areas of interest, annotations, combinations thereof, or the like.

[0059] The mobile device may be affixed to a microscope enabling the mobile device to capture images of a sample magnified by the microscope. For instance, the mobile device may be affixed to the microscope using an adapter such as adapter 200 of FIG. 2. The sample may be a biological sample (e.g., cells, etc.) associated with an experiment, assay, monitoring process (e.g., in which the biological sample is monitored over time), etc. The mobile device may be configured to capture one or more images of the sample at regular intervals over a time interval. In some examples, the quantity of images captured may be based on a quantity of wells prepared for the microscope. For instance, if a 96-well plate is prepared for the microscope, then the mobile device may be configured to capture at least one image per well (e.g., for n*96 images, etc.) during each image capture interval. If images are to be captured once a day for 30 days, then the mobile device may capture at the selected quantity of images every day for the 30 days.

[0060] In some instances, the images may be preprocessed (e.g., by the mobile device and / or by the computing device). For example, preprocessing may include, but is not limited to, converting images to a different form or format, upscaling the images (e.g., using interpolation, a machine-learning model, etc. to increase a resolution), downscaling the images (e.g., reducing a resolution), change the dimensions, performing thresholding, performing erosion, performing, dilation, performing closing, performing normalization, augmenting the images, identifying features (e.g., such as regions of interest, objects to be classified, characteristics of the images, extract features, etc.), combinations thereof, or the like.

[0061] In some examples, the computing device may assess received images to determine if the received images can be processed by a pattern recognition algorithm, neural network, or other processes of the computing device using a quality score (e.g., based on resolution, sharpness, contrast, luminosity, clarity etc.). If an image cannot be used (e.g., a quality score is below a threshold), then the computing device may transmit a communication to the mobile device with an identifier of the image to allow the mobile device to capture another image of the sample. The new image may be assigned a same identifier as the image that cannot be used to replace the image that cannot be used.

[0062] At block 308, the computing device may execute a pattern recognition algorithm and / or neural network using the set of images. In some instances, the pattern recognition algorithm may be trained by the neural network to detect particular patterns and / or pattern types indicative of various characteristics of the biological sample. The pattern recognition algorithms and / or neural network may be configured to generate one or more characteristics of the biological samples represented in the set of images. In other instances, the neural network may be a convolutional neural network configured to process images and output one or more features (e.g., one or more characteristics, etc.) and metadata associated with the images, the neural network, and / or the features output from the neural network. For example, the metadata may include confidence values associated with output features, accuracy metrics, masks, etc.

[0063] The one or more features may include, but are not limited to, values corresponding to cell confluency, cell count, cell size, cell migration or motility (e.g., cell position, cell velocity, etc.), an identification of morphological changes, cell differentiation (e.g., classification of cells, etc.), cell analysis (e.g., such as for sperm cells, yeast cells, etc.), cell viability (e.g., a determination of whether the cells represented in the image are alive or dead), clonogenicity,fermentation analysis, combinations thereof, or any characteristic or information that can be derived from the images.

[0064] In some instances, the computing device may derive additional characteristics based on any of the aforementioned characteristics. For instance, the computing device may derive a first one or more characteristics from an image. The computing device may then execute the pattern recognition algorithm and / or the neural network using the image and / or the first one or more characteristics to derive a second one or more characteristics. Alternatively, the computing device may derive the second one or more characteristics through other processes, algorithms, models, combinations thereof, and / or the like. For instance, the computing device may derive a cell confluency, a cell count, and / or the like. The computing device may then use one or more of the cell confluency, a cell count, and / or the like to derive a cell viability metric. The cell viability metric may be an indication of a quantity of live cells represented in an image, a quantity of dead cells represented in the image, a percentage of live cells represented in the image, a ratio of live cells to dead cells represented in the image, combinations thereof, and / or the like.

[0065] At block 312, the computing device may transmit the one or more characteristics of the biological sample (and / or the metadata) to another computing device. The other computing device may be a computer, server, database, service (e.g., operating with a device or network), cloud network, etc. In some instances, the mobile device and the computing device may be the same device (e.g., such that the computing device transmits the one or more characteristics of the biological sample to the mobile device). The computing device may also transmit metadata associated with the one or more characteristics such as, but not limited to, annotations, areas of interest, boundary boxes, masks, labels, combinations thereof, or the like.

[0066] In some instances, the computing device may aggregate the one or more characteristics generated over the time interval for further analysis, to derive additional features, etc. For example, the computing device may generate one or more time lapses (e.g., using an automatically and / or manually selected sampling of the received images), charts, tables, graphs, additional characteristics or features, predictions, interpolations, combinations thereof, or the like associated with the biological sample represented in the received images over the time interval. In other instances, an application of the mobile device may receive the output features and generate the time lapses, charts, tables, graphs, additional characteristics or features, predictions, interpolations, combinations thereof, or the like associated with thesample over the time interval. In still yet other instances, the computing device may be configured to generate one or more of the time lapses, charts, tables, graphs, additional characteristics or features, predictions, interpolations, combinations thereof, or the like and the mobile device may be configured to generate one or more of the time lapses, charts, tables, graphs, additional characteristics or features, predictions, interpolations, combinations thereof.

[0067] The computing device may transmit notifications and / or alerts based on the one or more characteristics and / or the one or more time lapses (e.g., using an automatically and / or manually selected sampling of the received images), charts, tables, graphs, additional characteristics or features, predictions, interpolations, combinations thereof, or the like. The one or more notifications and / or alerts may include a status of the biological sample (and / or the experiment, assay, etc. associated with the biological sample). In some instances, the status may indicate that intervention or adjustment may be needed to biological sample (and / or the experiment, assay, etc. associated with the biological sample).

[0068] In some instances, the computing device may receive an indication from the other computing device that one or more of the one or more characteristics derived from an image of the set of images are incorrect or include errors. For example, the computing device, the other computing device, and / or any users thereof (of either device) may compare the images, the one or more characteristics output from the images, and / or the metadata to determine an accuracy of the one or more characteristics. If one or more characteristics derived from an image are determined to be incorrect and / or a processing error associated with an image is detected, the computing device and / or other computing device may request that the image is reprocessed to generate a new one or more characteristics for the image.

[0069] The following example applications of the image processing framework (e.g., including the functionality of an image processing network and / or a mobile device) are intended for illustrative purposes and are not intended to be limiting:

[0070] CELL CONFLUENCE

[0071] An imaging processing framework can be utilized in various aspects of cell confluence assays, which can be used to measure the degree to which cells have covered a substrate surface in a culture dish or well.

[0072] The imaging processing framework can be used to determine the confluence of cells in an assay over time. By analyzing the growth patterns of cells in a time-lapse series of images,cell growth, doubling time, cell survival, and the like can be determined. In addition, the confluence of cells can be predicted at different time points. This can help researchers optimize experimental protocols, plan experiments more efficiently, make predictions about cell behavior, and / or the like.

[0073] The imaging processing framework can be used for quality control in cell confluence assays. By comparing the confluence of cells in experimental samples with predefined thresholds, the image processing network and / or the mobile device can be configured to automatically flag samples that fall outside of the expected confluence range. This can help researchers identify potential issues with their experiments and improve the reliability of their results.

[0074] The imaging processing framework can be used to optimize experimental design in cell confluence assays. By analyzing large datasets and identifying relevant features, such as cell density, growth rate, and confluence dynamics, the imaging processing framework can provide recommendations for experimental conditions, such as seeding density, time points for measurement, and appropriate controls. This can help researchers design more efficient and effective experiments.

[0075] In some examples, cell confluence may be used to determine a cell viability metric of a biological sample represented in an image. The cell viability metric may be an indication of a quantity of live cells represented in an image, a quantity of dead cells represented in the image, a percentage of live cells represented in the image, a ratio of live cells to dead cells represented in the image, combinations thereof, and / or the like. The cell viability metric may be predicted using a machine-learning model configured to process the image of the biological sample and / or a cell confluence value derived from the image. In some instances, the machinelearning model may use one or more additional features such as, but not limited to cell migration, cell motility, cell count, and / or any other feature described herein.

[0076] CELL MIGRATION AND MOTILITY

[0077] An imaging processing framework can be used in various aspects of cell migration assays, which may be usable to analyze the movement of cells in biological research.

[0078] The imaging processing framework can be used to automatically extract features from cell images such as, but not limited to, cell velocity, directionality, distance traveled,combinations thereof, or the like. The extracted features can be used to quantify cell migration behavior and generate large amounts of quantitative data for statistical analysis.

[0079] Cell migration assays typically involve capturing time-lapse microscopy images or videos of cells moving, and manually tracking individual cells to analyze their migration behavior. The imaging processing framework can track cells in real-time or from recorded images / videos, reducing or eliminating manual tracking and increasing the accuracy and speed of data analysis.

[0080] The imaging processing framework can be used to recognize and classify different types of cell migration behavior, such as random migration, directed migration, and collective migration. This can help researchers identify specific migration patterns and understand how different factors, such as drugs or genetic modifications, affect cell migration.

[0081] CELL COUNT

[0082] An imaging processing framework can determine cell count from images without staining by utilizing the inherent characteristics of the cells themselves. The imaging processing framework can use bright-field microscopy images to identify and segment individual cells using the pattern recognition algorithms and / or machine-learning models described herein. A count of the number of cells in the image can be determined based on the number of identified cell segments. The imaging processing framework can also use phasecontrast microscopy images, which provide contrast between the cytoplasm and nucleus count cells based on the refractive index differences.

[0083] The imaging processing framework may process fluorescence microscopy images without staining to count live cells by exploiting natural cellular autofluorescence to detect and count live cells in culture.

[0084] The imaging processing framework can also exploit the fluorescence signals from certain cellular components to recognize the distinctive fluorescence patterns of different cell types, such as stem cells, neurons, and cancer cells.

[0085] The imaging processing framework can analyze microscope images of blood cells to detect and count different types of cells such as red blood cells, white blood cells, and platelets. The system can also analyze cell morphology, which can be used to detect abnormalities associated with certain diseases.

[0086] In some examples, cell count may be used to determine a cell viability metric of a biological sample represented in an image. The cell viability metric may be an indication of a quantity of live cells represented in an image, a quantity of dead cells represented in the image, a percentage of live cells represented in the image, a ratio of live cells to dead cells represented in the image, combinations thereof, and / or the like. The cell viability metric may be predicted using a machine-learning model configured to process the image of the biological sample and / or a cell count value derived from the image. In some instances, the machine-learning model may use one or more additional features such as, but not limited to cell migration, cell motility, cell confluence, and / or any other feature described herein.

[0087] CELL SIZE

[0088] An imaging processing framework can be used to determine cell size from microscopy series of images by recognizing cell boundaries and measuring the size of the cells in each image. Changes in cell size can be tracked over time or compared between different experimental conditions.

[0089] CELL MORPHOLOGY CHANGES

[0090] An imaging processing framework can detect and identify cell morphology changes, which may be usable for drug discovery, disease diagnosis, biotechnology, and / or the like.

[0091] The imaging processing framework can analyze images of blood cells, tissues, organs, or the like to identify abnormal cell morphology, which is a common characteristic of many diseases. For instance, in sickle cell anemia, the red blood cells have a distinct sickle shape due to a genetic mutation. The imaging processing framework can detect such morphological changes and help diagnose the disease.

[0092] The imaging processing framework can be used to analyze the morphological changes of cells treated with potential drugs. The technology can identify changes in cell shape or size, indicating the drug's effectiveness. It can also help identify potential side effects by detecting abnormal cell morphology changes. By analyzing changes in cell size over time, researchers can gain insights into the mechanisms that control cell growth, cell division, and attachment. For instance, cells undergoing mitosis or blocked in mitosis are round and semidetached. Cells undergoing apoptosis or mitotic catastrophe will have a distinct shape that can be detected by the pattern recognition algorithms and / or machine-learning models of the imaging processing framework.

[0093] The imaging processing framework can analyze cell morphology to detect toxicity levels in drugs or other chemical compounds. For example, images of cells exposed to drugs, toxins, harmful substances, etc. can be analyzed by the imaging processing framework to determine a toxicity level of the drugs.

[0094] The imaging processing framework can monitor and analyze cell morphology during bioprocessing (e.g., the production of biologies such as vaccines, antibodies, recombinant proteins, etc.). The imaging processing framework can detect any abnormal cell morphology changes during the production process, ensuring product quality and consistency.

[0095] CELL DIFFERENTIATION

[0096] The imaging processing framework differentiate cells represented in images of cells at various stages of development, allowing for the identification and classification of different cell types. For instance, imaging processing framework can process images of tumor cells and distinguish various types of cancer cells. It can be predicted how cancer cells may respond to different treatments based on their differentiation state, thereby allowing doctors to personalize treatment plans for individual patients and enhance treatment outcomes. In addition, the imaging processing framework can be used to analyze images of stem cells and track their differentiation into different cell types, such as neurons or muscle cells. This can assist stem cell scientists in determining the differentiation stage and differentiation route, optimizing experiments and product development.

[0097] SPERM CELL ANALYSIS

[0098] An imaging processing framework can differentiate and count sperm cells from images of sperm cells. The imaging processing framework can process images of semen samples of a post-vasectomy patient to accurately and efficiently detect the presence of sperm cells to assess the effectiveness of the vasectomy. The imaging processing framework may also provide a more reliable and objective assessment of semen quality and detect instances of male infertility. Sperm count and differentiation may be indicators of male fertility. The imaging processing framework can identify patterns and correlations that may not be apparent to human observers, leading to more accurate diagnoses and personalized treatment plans.

[0099] The imaging processing framework can be used to select the best sperm cells for artificial insemination for animal husbandry. By analyzing images of sperm cells from different animals, the imaging processing framework can identify sperm cells with the most desirablecharacteristics, such as high motility or morphology, and use this information to optimize breeding programs. This can improve the genetic diversity and health of breeding populations, leading to healthier and more productive animals.

[0100] CLONOGENICITY / BACTERIAL CLONE COUNT

[0101] Cell Clonogenic Assay: cells may be plated and allowed to grow into colonies, which are then counted manually to assess the cell's ability to form new colonies. However, this process can be time-consuming and prone to errors. The imaging processing framework can automate the process of counting colonies by analyzing the images of the plates.

[0102] In microbiology, bacterial clones are often used to study gene expression, protein function, and other biological processes. To count the number of bacterial clones, scientists typically use a process called colony-forming unit (CFU) counting. The imaging processing framework can count the number of bacterial clones by analyzing images of bacterial colonies.

[0103] YEAST COUNT AND FERMENTATION

[0104] An imaging processing framework can count the number of cells present in a sample at various stages of a brewing process, such as during the initial inoculation, a growth phase, and a fermentation phase. The imaging processing framework can derive the growth rate of the yeast and monitor the fermentation process in real-time. The imaging processing framework may also identify changes in yeast cell morphology that may indicate contamination or other issues.

[0105] The imaging processing framework can track the movement of yeast and other microorganisms during the fermentation process using images of fermentation tanks. This can help identify areas where the fermentation is not progressing as expected, allowing for early intervention to prevent spoilage or contamination.

[0106] The imaging processing framework can detect defects or inconsistencies in the final product using images of the final products. For example, the pattern recognition algorithms and / or machine-learning models described herein can detect changes in color, texture, or the like that may indicate contamination or improper brewing techniques.

[0107] FIG. 4 illustrates a block diagram of example machine-learning model with dynamically configurable skip connections configured to process biological images from varying sources according to aspects of the present disclosure. Machine-learning model 400may be an autoencoder, convolutional autoencoder, or convolutional neural network configured to process images. In examples where machine-learning model 400 is an autoencoder, machine-learning model 400 may include one or more encoding layers (e.g., blocks 404-408) and one or more decoding layers (e.g., blocks 420-428). In examples where machine-learning model 400 is a convolutional autoencoder or convolutional neural network, blocks 404-416 of machine-learning model 400 may be encoding layers or convolutional layers and blocks 416-428 may be decoding layers or deconvolutional layers.

[0108] Each encoding layer extracts features from the input data (e.g., an image) while compressing the input data. At block 416 the input data is in a compressed representation (e.g., as compressed as the machine-learning model is configured to compress the input data). The decoding layers reconstructs the input data from the compressed representation. Since some data may be lost during the compression and / or decompression of the input data, one or more skip connections may be implemented. A skip is a connection from a layer to a subsequent layer that may skip one or more layers of machine-learning model 400. For example, skip connection 432 connects layer 404 to layer 428 skipping layers 408-424. Skip connections 432- 440 may pass feature maps directly to a decoding layer to improve the performance of the decoding layers. The feature maps include more information (e.g., due to the features maps being in a less compressed state before block 416) then the compressed representation of the input data being processed by the decoding layer.

[0109] Machine-learning model 400 may include one or more hyperparameters determine when to establish skip connections and the layers to be connected by a particular skip connection. The hyperparameters may be static (e.g., predetermined, defined based on training conditions, defined from user input, defined by post-training conditions, combinations thereof, and / or the like), or dynamic (e.g., generated and / or removed at runtime, etc.). Dynamic hyperparameters may be based on feedback (e.g. user input, accuracy metrics, a loss function, combinations thereof, and / or the like), metadata (e.g., data associated with the input image including, but not limited to, a resolution of the image, a camera that captured the image, a device connected to the camera, a user profile, an identification of cells represented by the image, combinations thereof, and / or the like), an execution status of machine-learning model 400 (e.g., based a feature map output from an encoding layer, a feature map of block 416, feature maps generated by layers during processing of a previous image or images, processing resources of the computing device executing machine-learning model 400 as part of a load balancing process, combinations thereof, and / or the like), combinations thereof, and / or the like.For example, skip connection 432 may be established when processing particular types of images and disconnected for other types of images.

[0110] In some examples, machine-learning model 400 may include one or more fully connected layers configured to classify the decoded image data. Machine-learning model 400 may be configured to classify the presence or absence of features in the image, identify regions of the image including particular features, and / or the like. Machine-learning model 400 may be trained to output any of the aforementioned features including, but not limited to, cell confluency, cell migration, cell motility, cell count, cell size, cell morphological changes, sperm cell characteristics, bacterial clone count, yeast count, cell differentiation, cell identification, combinations thereof, and / or the like. In some instances, machine-learning model 400 may be trained to detect a first one or more features of an input image. A second machine-learning model (e.g., of a same type as machine-learning model 400 or another type of machine-learning model) may be trained to detect second one or more features of the input image. Any number of machine-learning models may be trained to operate in parallel (e.g., in which inputs and outputs from a machine-learning model are distinct from the inputs and outputs of other machine-learning model), in series (e.g., in which the inputs of a machinelearning model may be, or be derived from, the outputs of another machine-learning model or the inputs of a machine-learning model may be derived from the outputs of another machinelearning model), or partially in series and partially in parallel (e.g., in which the inputs of a machine-learning model may be the outputs of another machine-learning model, derived from the outputs of another machine-learning model, or distinct from the outputs of a machinelearning model while inputs of other machine-learning models may be the outputs from another machine-learning model, etc.).

[0111] Machine-learning model 400 may be trained using supervised learning, unsupervised learning, metaleaming, reinforcement learning, self-supervised learning, adversarial learning, and / or the like. For example, during training machine-learning model 400 may process input images and output a predicted feature. Another machine-learning model or a user may annotate the output. The annotated output along with the input image may be used to retrain machinelearning model 400. The machine-learning model 400 may be trained for a predetermined quantity of iterations, until one or more target accuracy metrics have exceeded a corresponding threshold function (e.g., accuracy, precision, area under the curve, logarithmic loss, Fl score, weighted human disagreement rate, cross entropy, mean absolute error, mean square error, etc.), combinations thereof, and / or the like.

[0112] FIG. 5 illustrates a block diagram of example machine-learning system including one or more machine-learning stages configured to process biological images from varying sources according to aspects of the present disclosure. Machine-learning system 500 may include one or more machine-learning models that trained to detect one or more features within an input image. Input image 504 may be an image of a biological sample including one or more cells. Input image 504 may be preprocessed (e.g., converted to greyscale, up-sampled or down- sampled to a particular resolution, etc.). Input image 504 may include metadata or other input identify a region of interest within input image 504 that may include cells or other features for detection by machine-learning system 500.

[0113] Input image 504 may be received by machine-learning model 400 (or alternatively in any of the aforementioned machine-learning models previously described). Machine-learning model 400 may be trained out output one or more matrices characterizing features of input image 504. For instance, machine-learning system 500 may process input image 504 to output cell count (e.g., an integer corresponding to a quantity of cell detected in an input image). Machine-learning model 400 predict a cell probability matrix 508 (e.g., a binary indication of whether each pixel within the region of interest (if defined) corresponds to any cell or a preselected cell, etc.). Machine-learning model 400 may also predict vertical gradient matrix 512 and / or horizontal gradient matrix 516 of a topological map of input image 504. Cell probability matrix 508, gradient matrix 512, and / or horizontal gradient matrix 516 may be postprocessed to define a set of pixel regions of the region of interest (if defined) of input image 504 that correspond to a cell. Post-processed cell mask 520 includes an identification of the set of pixel regions. The set of pixel regions may be counted to derive the cell count (or other features) and output as output features 528.

[0114] Alternatively, or additionally, cell probability matrix 508, vertical gradient matrix 512, and / or horizontal gradient matrix 516 may be processed by machine-learning model 524. Machine-learning model 524 may be machine-learning model 400, a same type of machinelearning model as machine-learning model 400, or a different type of model from machinelearning model 524. Machine-learning model 524 may receive post-processed cell mask 520 and predict the cell count or other features. In some examples, depending on the features to be identified by machine-learning model 524, machine-learning model 524 may receive additional input. For instance, when generating a prediction of cell viability, machine-learning model 524 may receive post-processed cell mask 520 and input image 504. Alternative, or additionally, in instances where machine-learning model 524 is a separate model from machine-learning model400, machine-learning model 524 may receive input from machine-learning model 400 (e.g., such as confidence values, feature maps, hyperparameters, etc.). Machine-learning model 524 may output predicted features (e.g., such as cell count and / or cell viability described in the examples above and / or any other features previously described) as output features 528.

[0115] In some instances, the one or more machine-learning models may include a single machine-learning model (e.g., such as machine-learning model 400 of FIG. 4, any of the aforementioned machine-learning models, and / or the like). In those instances, post-processed cell mask 520 may be passed back to machine-learning model 400 and processed by machinelearning model 400 to generate output features 528 (e.g., machine-learning model 524 may be omitted).

[0116] Machine-learning system (e.g., machine-learning model 400 and machine-learning model 524) may be trained using supervised learning, unsupervised learning, metaleaming, reinforcement learning, self-supervised learning, adversarial learning, and / or the like. For example, during training machine-learning system 500 may process input images and output a predicted feature. Another machine-learning model or a user may annotate the output. The annotated output along with the input image may be used to retrain machine-learning system 500. The machine-learning system 500 may be trained for a predetermined quantity of iterations, until one or more target accuracy metrics have exceeded a corresponding threshold function (e.g., accuracy, precision, area under the curve, logarithmic loss, Fl score, weighted human disagreement rate, cross entropy, mean absolute error, mean square error, etc.), combinations thereof, and / or the like.

[0117] In some examples, machine-learning system 500 may be trained using adversarial training. In those is examples, machine-learning model 400 may process input images and generate an output of predicted features. The output may be passed to machine-learning model 524, which may process the output (and optionally the input image) to generate annotations (e.g., corrections, etc.). The annotations may be passed back to machine-learning model 400, which may be retrained completing a cycle. In some instances, a predetermined quantity of cycles may be executed (e.g., using the same input image or another image) or cycles may be executed until one or more target accuracy metrics have exceeded a corresponding threshold function. Alternatively, or additionally, Output features 528 may be passed to machine-learning model 400, which may process the output features 528 (and optionally the input image) to generate annotations (e.g., corrections, etc.). The annotations may be passed back to machine-learning model 524, which may be retrained completing a cycle. In some instances, a predetermined quantity of cycles may be executed (e.g., using the same input image or another image) or cycles may be executed until one or more target accuracy metrics have exceeded a corresponding threshold function.

[0118] FIG. 6A and 6B illustrate graphs of accuracy metrics of the machine-learning models described herein according to aspects of the present disclosure. The input images were captured using off-the-shelf hardware including an imaging device (e.g., a mobile device and / or digital camera) affixed to a microscope using an adapter (e.g., such as adapter 200 of FIG. 2). The use of off the shelf hardware may substantially reduce the costs associated with executing monitor processes of biological samples at the expense of introducing variability into the captured images. Generally, such variability is unacceptable and can introduce significant error. However, the image processing techniques described herein can process input image subject to such variability while maintaining a high degree of accuracy. For example, FIG. 6 A illustrates four accuracy metrics associated with processing images of live and dead cells. Machinelearning system 500 processed the images to predict cell viability (e.g., identify the live and / or dead cells) from the input images. Machine-learning system 500 reached an accuracy score of accuracy of 94%, a precision of 94%, a recall of 90%, an Fl score of 92% indicating machinelearning system 500 can accurately identify live and dead cells within input images. FIG. 6B illustrates four accuracy metrics associated with processing images for cell confluency using machine-learning model 400. Machine-learning model 400 reached an accuracy score of accuracy of 92%, a precision of 93%, a recall of 91%, an Fl score of 92% indicating machinelearning model 400 can accurately identify confluency within input images.

[0119] FIG. 7 illustrates a flowchart of another example process for processing images of biological samples according to aspects of the present disclosure. At block 704, a computing device may receive a request to initiate a monitoring session associated with a user profile of a user device. The request may include at least one monitoring process. A monitoring session may define an analysis of one or more biological samples and include one or more monitoring session that implement the monitoring of the one or more monitoring sessions. A user may define a monitoring session by identifying the one or more biological samples, defining parameters of one or more monitoring processes, define the parameters of the monitoring session, combinations thereof, and / or the like. The parameters of a monitoring process may include, but are not limited to, a time interval of each monitoring process, a data type to be collected (e.g., image data, test data, chemical data, biopsy data, etc.), a frequency of the datacollection, a quantity of data to be collected during each data collection (e.g., a quantity of images, tests executed, etc.), a time interval between monitoring processes, an identification of features to be extracted from the collected data, an identification of neural networks or machine-learning models (or algorithms, etc.) to be executed on the collected data to generated the features, combinations thereof, and / or the like. The parameters of a monitoring session may include, but are not limited to, an identification of one or more experiments or analyses being performed, a time interval of the monitoring session, an identification of the user profile and / or one or more usernames of users authorized to access the monitoring session, a quantity of monitoring processes to be executed, an identification of laboratory conditions and / or equipment, an identification of processing devices, an identification of features to be extracted from the collected data, an identification of neural networks or machine-learning models (or algorithms, etc.) to be executed on the collected data to generated the features, combinations thereof, and / or the like. The parameters of the monitoring session and the parameters of the one or more monitoring processes may be predefined, defined based on a previous monitoring session or monitoring process, received from user input, received from a remote device, automatically defined (e.g., algorithmically, via a machine-learning model configured to predict parameter values based on the experiments or analyses being performed and / or based on the one or more biological samples, and / or the like), combinations thereof (e.g., partially user selected and partially automatically defined, etc.), and / or the like.

[0120] For example, a computing device (e.g., mobile device such as a smartphone or tablet, processing device such as a computer or server, etc.) may execute a monitoring application. A user may interact with the monitoring application to define a new monitoring session. The monitoring session may include one or more monitoring processes with each monitoring process configured to identify particular features from data collected by the monitoring process. For instance, the user may define a first monitoring process to identify cell count within images of a biological sample over time and a second monitoring process to identify cell viability within the images of the biological sample overtime, etc. Each monitoring process may execute over a same or different time interval.

[0121] At block 708, the computing device may receive one or more images captured over a time interval defined by the at least one monitoring process. In some instances, the one or more images may be captured by a camera of the computing device. In some instances, the computing device may be affixed to a lens of a microscope using, for example, an adapter such as adapter 200 of FIG. 2. The monitoring application may cause the camera to capture imagesaccording to the parameters defined in the monitoring session. For example, the monitoring application may cause the camera to capture images in regular intervals and / or capture a predetermined quantity of images, etc. In other instances, the computing device may be in communication with a user device (e.g., a mobile device, analog or digital camera, tablet, etc.). The user device may be affixed to the lens of the microscope using, for example, an adapter such as adapter 200 of FIG. 2. The user device may operate an instance of the monitoring application which may cause the user device to capture images according to the monitoring process and transmit the images to the computing device and / or to a storage location (e.g., network area storage, cloud network, a server, other remote device, etc.).

[0122] At block 712, the computing device may execute a neural network using the one or more images. Examples of neural networks include, but are not limited to, deep learning networks, convolutional neural networks, faster convolutional neural networks, region-based convolutional neural networks, fast or faster convolutional neural networks, You Only Look Once (Y OLO), Naive Bayes, densely connected networks, and / or any machine-learning model configured to detect features represented in an image. The neural network may be trained to predict a feature represented in the one or more images. In some instances, the neural network may be trained to predict multiple features represented in the one or more images such as, but not limited to, cell confluency, cell migration, cell motility, cell viability, cell count, cell size, cell morphological changes, sperm cell characteristics, bacterial clone count, yeast count, cell differentiation, combinations thereof, and / or the like. In other instances, the computing device may include multiple neural networks with each neural network being trained to detect one or more of biological features described herein. For example, a first neural network may be trained to derive a cell count of cells represented in an image and a second neural network may derive a cell migration (e.g., cell velocity, directionality, travel distance, etc.) from cells represented in sets of images.

[0123] Alternatively, or additionally, the neural network may be configured to train one or more pattern recognition algorithms that derive the one or more features from at least one image of the one or more images.

[0124] In some instances, the computing device may derive additional features based on any of the aforementioned features. For instance, the computing device may derive a first one or more features from an image. The computing device may then execute the neural network using the image and / or the first one or more features to derive a second one or more features.Alternatively, the computing device may derive the second one or more features through other processes, algorithms, pattern recognition algorithms, models, combinations thereof, and / or the like. For instance, the computing device may derive a cell confluency, a cell count, and / or the like. The computing device may then use one or more of the cell confluency, a cell count, and / or the like to derive a cell viability metric. The cell viability metric may be an indication of a quantity of live cells represented in an image, a quantity of dead cells represented in the image, a percentage of live cells represented in the image, a ratio of live cells to dead cells represented in the image, combinations thereof, and / or the like.

[0125] At block 716, the computing device may predict, using the neural network, a presence of one or more features (e.g., characteristics) of a biological sample in the at least one image. The one or more features may be defined by the at least one monitoring process.

[0126] At block 720, the computing device may transmit the one or more features of the biological sample and a mask generated by the neural network. In some instances, the computing device may transmit the one or more features to the device that generated the one or more images (e.g., when the computing device is not the device that captured the images) such as to the user device, etc. In some instances, the computing device may transmit the one or more features to a remote device such as, but not limited to, a remote database, a server, a cloud network, etc. In other instances, the computing device may be the user device (e.g., the computing device was the device that captured the images, etc.). In those instances, the computing device may transmit the one or more features to local or remote storage.

[0127] Upon being received, an analysis may be executed based on the one or more features of the biological sample and the at least one monitoring process. For example, the analysis may be a statistical analysis usable to identify features of the biological sample.

[0128] The blocks of FIG. 7 may be configured to process images in real time (e.g., approximately immediately after the images are received) to provide the output one or more features to the user device (or any other device as previously described). The blocks 708-720 may be executed any number of times, in series and / or in parallel to execute the purpose of the monitoring session and / or any monitoring process. For instance, a monitoring process may cause images to be received in sets ever the time interval of the monitoring process (e.g., 100 images every hour for 14 days). The computing device may begin processing the images as soon as the images are received. The computing device may segment the received images intobatches and process each batch using blocks 708-720. The batches may be processed in series, in parallel, or partially in series or in parallel.

[0129] FIG. 8 illustrates an example computing device according to aspects of the present disclosure. For example, computing device 800 can implement any of the systems or methods described herein. In some instances, computing device 800 may be a component of or included within a media device. The components of computing device 800 are shown in electrical communication with each other using connection 806, such as a bus. The example computing device 800 includes processor 804 (e.g., CPU, processor, or the like) and connection 806 (e.g., such as a bus, or the like) that is configured to couple components of computing device 800 such as, but not limited to, memory 820, read only memory (ROM 818), random access memory (RAM 816), and / or storage device 808, to processor 804.

[0130] Computing device 800 can include a cache 802 of high-speed memory connected directly with, in close proximity to, or integrated within processor 804. Computing device 800 can copy data from memory 820 and / or storage device 808 to cache 802 for quicker access by processor 804. In this way, cache 802 may provide a performance boost that avoids delays while processor 804 waits for data. Alternatively, processor 804 may access data directly from memory 820, ROM 817, RAM 816, and / or storage device 808. Memory 820 can include multiple types of homogenous or heterogeneous memory (e.g., such as, but not limited to, magnetic, optical, solid-state, etc.).

[0131] Storage device 808 may include one or more non-transitory computer-readable media such as volatile and / or non-volatile memories. A non-transitory computer-readable medium can store instructions and / or data accessible by computing device 800. Non-transitory computer-readable media can include, but is not limited to magnetic cassettes, hard-disk drives (HDD), flash memory, solid state memory devices, digital versatile disks, cartridges, compact discs, random access memories, read only memory (ROM), combinations thereof, or the like.

[0132] Storage device 808, may store one or more services, such as service 1 810, service 2 812, and service 3 814, that are executable by processor 804 and / or other electronic hardware. The one or more services include instructions executable by processor 804 to: perform operations such as any of the techniques, steps, processes, blocks, and / or operations described herein; control the operations of a device in communication with computing device 800; control the operations of processor 804 and / or any special-purpose processors; combinations therefor; or the like. Processor 804 may be a system on a chip (SOC) that includes one or more cores orprocessors, a bus, memories, clock, memory controller, cache, other processor components, and / or the like. A multi-core processor may be symmetric or asymmetric.

[0133] Computing device 800 may include one or more input devices 822 that may represent any number of input mechanisms, such as a microphone, a touch-sensitive screen for graphical input, keyboard, mouse, motion input, speech, media devices, sensors, combinations thereof, or the like. Computing device 800 may include one or more output devices 824 that output data to a user. One or more output devices 824 may include, but are not limited to, a media device, projector, television, speakers, combinations thereof, or the like. In some instances, multimodal computing devices can enable a user to provide multiple types of input to communicate with computing device 800. Communications interface 826 may be configured to manage user input and computing device output. Communications interface 826 may also be configured to managing communications with remote devices (e.g., establishing connection, receiving / transmitting communications, etc.) over one or more communication protocols and / or over one or more communication media (e.g., wired, wireless, etc.).

[0134] Computing device 800 is not limited to the components as shown if FIG. 8. Computing device 800 may include other components not shown and / or components shown may be omitted.

[0135] The following examples illustrate various aspects of the present disclosure. As used below, any reference to a series of examples is to be understood as a reference to each of those examples disjunctively (e.g., "Examples 1-4" is to be understood as "Examples 1, 2, 3, or 4").

[0136] Example 1 is a method comprising: receiving a request to initiate a monitoring session from a user profile of a user device, wherein the request includes at least one monitoring process; receiving, from a user device over a network, the one or more images captured over a time interval defined by the at least one monitoring process, wherein the user device is affixed with hardware configured to capture images for the monitoring session; storing the one or more images in association with the monitoring process; executing a neural network using the one or more images, wherein the neural network is configured to train pattern recognition algorithms that derive one or more features from an input image; deriving, using the neural network, one or more characteristics of a biological sample identified by the neural network in the one or more images, wherein the one or more characteristics are defined by the at least one monitoring process; and transmitting the one or more characteristics of the biological sample and a mask generated by the neural network, wherein when received by the user device, theuser device generates a statistical analysis based on the one or more characteristics of the biological sample and the at least one monitoring process, wherein the statistical analysis is stored in association with the monitoring process, and wherein the monitoring process and associated data is transmitted to a computing device.

[0137] Example 2 is the method of any of example(s) 1 and 3-19, wherein the neural network is a convolutional neural network or autoencoder.

[0138] Example 3 is the method of any of example(s) 1-2 and 4-19, wherein the one or more characteristics include cell confluency.

[0139] Example 4 is the method of any of example(s) 1-3 and 5-19, wherein the one or more characteristics include cell migration and cell motility.

[0140] Example 5 is the method of any of example(s) 1-4 and 6-19, wherein the one or more characteristics include cell count.

[0141] Example 6 is the method of any of example(s) 1-5 and 7-19, wherein the one or more characteristics include cell size.

[0142] Example 7 is the method of any of example(s) 1-6 and 8-19, wherein the one or more characteristics include cell morphological changes.

[0143] Example 8 is the method of any of example(s) 1-7 and 9-19, wherein the one or more characteristics include cell differentiation.

[0144] Example 9 is the method of any of example(s) 1-8 and 10-19, wherein the one or more characteristics include sperm cell characteristics.

[0145] Example 10 is the method of any of example(s) 1-9 and 11-19, wherein the one or more characteristics include a bacterial clone count.

[0146] Example 11 is the method of any of example(s) 1-10 and 12-19, wherein the one or more characteristics include a yeast count.

[0147] Example 12 is the method of any of example(s) 1-11 and 13-19, wherein the set of images are annotated with one or more regions of interest.

[0148] Example 13 is the method of example(s) of any of example(s) 1-12 and 14-19, further comprising: transmitting with the one or more characteristics of the biological sample, a mask generated by the neural network.

[0149] Example 14 is the method of any of example(s) 1-13 and 15-19, further comprising: receiving, from the computing device, metadata associated with a mask output by the neural network, the metadata indicating an accuracy of the one or more characteristics; and executing a training iteration of the neural network based on the metadata.

[0150] Example 15 is the method of any of example(s) 1-14 and 16-19, wherein the computing device is the mobile device.

[0151] Example 16 is the method of any of example(s) 1-15 and 17-19, wherein the mobile device is affixed with the hardware to a microscope.

[0152] Example 17 is the method of any of example(s) 1-16 and 18-19, wherein the hardware is compatible with off the shelf imaging devices, including at least one of a smartphone and digital camera.

[0153] Example 18 is the method of any of example(s) 1-17 and 19, wherein the one or more characteristics include a cell viability metric.

[0154] Example 19 is the method of any of example(s) 1-18, further comprising: predicting a cell viability metric using the one or more characteristics, wherein the one or more characteristics include a cell count or a cell confluence.

[0155] Example 20 is a system comprising: one or more processors; a non-transitory computer-readable medium storing instructions that when executed by the one or more processors, cause the one or more processors to perform the methods of any of example(s)s 1- 19.

[0156] Example 21 is a non-transitory computer-readable medium storing instructions that when executed by one or more processors, cause the one or more processors to perform the methods of any of example(s)s 1-19.

[0157] The term “computer-readable medium” includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other mediums capable of storing, containing, or carrying instruct! on(s) and / or data. A computer-readable medium may include a non-transitory medium in which data can be stored in a form that excludes carrier waves and / or electronic signals. Examples of a non-transitory medium may include, but are not limited to, a magnetic disk or tape, optical storage media such as compact disk (CD) or digital versatile disk (DVD), flash memory, memory or memory devices. A computer-readablemedium may have stored thereon code and / or machine-executable instructions that may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, or the like.

[0158] Some portions of this description describe examples in terms of algorithms and symbolic representations of operations on information. These operations, while described functionally, computationally, or logically, may be implemented by computer programs or equivalent electrical circuits, microcode, or the like. Furthermore, arrangements of operations may be referred to as modules, without loss of generality. The described operations and their associated modules may be embodied in software, firmware, hardware, or any combinations thereof.

[0159] Any of the steps, operations, or processes described herein may be performed or implemented with one or more hardware or software modules, alone or in combination with other devices. In some examples, a software module can be implemented with a computer- readable medium storing computer program code, which can be executed by a processor for performing any or all of the steps, operations, or processes described.

[0160] Some examples may relate to an apparatus or system for performing any or all of the steps, operations, or processes described. The apparatus or system may be specially constructed for the required purposes, and / or it may comprise a general-purpose computing device selectively activated or reconfigured by a computer program stored in memory of computing device. The memory may be or include a non-transitory, tangible computer readable storage medium, or any type of media suitable for storing electronic instructions, which may be coupled to a bus. Furthermore, any computing systems referred to in the specification may include a single processor or multiple processors.

[0161] While the present subject matter has been described in detail with respect to specific examples, it will be appreciated that those skilled in the art, upon attaining an understanding of the foregoing, may readily produce alterations to, variations of, and equivalents to such embodiments. Numerous specific details are set forth herein to provide a thoroughunderstanding of the claimed subject matter. However, those skilled in the art will understand that the claimed subject matter may be practiced without these specific details. In other instances, methods, apparatuses, or systems that would be known by one of ordinary skill have not been described in detail so as not to obscure claimed subject matter. Accordingly, the present disclosure has been presented for purposes of example rather than limitation, and does not preclude the inclusion of such modifications, variations, and / or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art.

[0162] For clarity of explanation, in some instances the present disclosure may be presented as including individual functional blocks including functional blocks comprising devices, device components, steps or routines in a method embodied in software, or combinations of hardware and software. Additional functional blocks may be used other than those shown in the figures and / or described herein. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form in order not to obscure the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.

[0163] Individual examples may be described herein as a process or method which may be depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. A process is terminated when its operations are completed but may have additional steps not shown. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination can correspond to a return of the function to the calling function or the main function.

[0164] Processes and methods according to the above-described examples can be implemented using computer-executable instructions that are stored or otherwise available from computer-readable media. Such instructions can include, for example, instructions and data which cause or otherwise configure a general-purpose computer, special purpose computer, or a processing device to perform a certain function or group of functions. Portions of computer resources used can be accessible over a network. The computer executableinstructions may be, for example, binaries, intermediate format instructions such as assembly language, firmware, source code, etc.

[0165] Devices implementing the methods and systems described herein can include hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof, and can take any of a variety of form factors. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks (e.g., a computer-program product) may be stored in a computer-readable or machine-readable medium. The program code may be executed by a processor, which may include one or more processors, such as, but not limited to, one or more digital signal processors (DSPs), general purpose microprocessors, an application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Such a processor may be configured to perform any of the techniques described in this disclosure. A processor may be a microprocessor; conventional processor, controller, microcontroller, state machine, or the like. A processor may also be implemented as a combination of computing components (e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration). Accordingly, the term “processor,” as used herein may refer to any of the foregoing structure, any combination of the foregoing structure, or any other structure or apparatus suitable for implementation of the techniques described herein. Functionality described herein also can be embodied in peripherals or add-in cards. Such functionality can also be implemented on a circuit board among different chips or different processes executing in a single device, by way of further example.

[0166] In the foregoing description, aspects of the disclosure are described with reference to specific examples thereof, but those skilled in the art will recognize that the disclosure is not limited thereto. Thus, while illustrative examples of the disclosure have been described in detail herein, it is to be understood that the inventive concepts may be otherwise variously embodied and employed, and that the appended claims are intended to be construed to include such variations. Various features and aspects of the above-described disclosure may be used individually or in any combination. Further, examples can be utilized in any number of environments and applications beyond those described herein without departing from the broader spirit and scope of the disclosure. The disclosure and figures are, accordingly, to be regarded as illustrative rather than restrictive.

[0167] The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, firmware, or combinations thereof. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0168] Unless specifically stated otherwise, it is appreciated that throughout this specification discussions utilizing terms such as “processing,” “computing,” “calculating,” “determining,” and “identifying” or the like refer to actions or processes of a computing device, such as one or more computers or a similar electronic computing device or devices, that manipulate or transform data represented as physical electronic or magnetic quantities within memories, registers, or other information storage devices, transmission devices, or media devices of the computing platform. The use of “adapted to” or “configured to” herein is meant as open and inclusive language that does not foreclose devices adapted to or configured to perform additional tasks or steps. Additionally, the use of “based on” is meant to be open and inclusive, in that a process, step, calculation, or other action “based on” one or more recited conditions or values may, in practice, be based on additional conditions or values beyond those recited. Headings, lists, and numbering included herein are for ease of explanation only and are not meant to be limiting.

[0169] The foregoing detailed description of the technology has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the technology to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. The described embodiments were chosen in order to best explain the principles of the technology, its practical application, and to enable others skilled in the art to utilize the technology in various embodiments and with various modifications as are suited to the particular use contemplated. It is intended that the scope of the technology be defined by the claim.

Claims

CLAIMS1. A method comprising: receiving a request to initiate a monitoring session associated with a user profile of a user device, wherein the request includes at least one monitoring process; receiving one or more images captured over a time interval defined by the at least one monitoring process, wherein the one or more images are generated by an imaging device configured to capture images through a lens of a microscope, and wherein the one or more images include a representation of a biological sample; executing a neural network using the one or more images, wherein the neural network is configured to derive one or more features from at least one image of the one or more images; predicting, using the neural network, a presence of one or more characteristics of the biological sample in the at least one image, wherein the one or more characteristics are defined by the at least one monitoring process; and transmitting the one or more characteristics of the biological sample and a mask generated by the neural network, wherein when received by the user device, the user device generates a statistical analysis based on the one or more characteristics of the biological sample and the at least one monitoring process.

2. The method of claim 1, wherein the imaging device is configured to capture images through the lens of the microscope by affixing the imaging device to the microscope using an adapter.

3. The method of claim 2, wherein the adapter is compatible with off-the-shelf imaging devices, including at least one of a smartphone and digital camera.

4. The method of claim 1, wherein the neural network is a convolutional neural network or autoencoder.

5. The method of claim 1, wherein the one or more characteristics include at least one of cell confluency, cell migration, cell motility, cell viability, cell count, cell size, cell morphological changes, sperm cell characteristics, bacterial clone count, yeast count, and cell differentiation.

6. The method of claim 1, wherein the one or more images are annotated with one or more regions of interest.

7. The method of claim 1, further comprising: receiving metadata associated with a mask output by the neural network, the metadata indicating an accuracy of the one or more characteristics; and executing a training iteration of the neural network based on the metadata.

8. The method of claim 1, wherein the imaging device is the user device.

9. A system comprising: one or more processors; and a non-transitory computer-readable medium storing instructions that when executed by the one or more processors, cause the one or more processors to perform operations including: receiving a request to initiate a monitoring session associated with a user profde of a user device, wherein the request includes at least one monitoring process; receiving one or more images captured over a time interval defined by the at least one monitoring process, wherein the one or more images are generated by an imaging device configured to capture images through a lens of a microscope, and wherein the one or more images include a representation of a biological sample; executing a neural network using the one or more images, wherein the neural network is configured to derive one or more features from at least one image of the one or more images; predicting, using the neural network, a presence of one or more characteristics of the biological sample in the at least one image, wherein the one or more characteristics are defined by the at least one monitoring process; and transmitting the one or more characteristics of the biological sample and a mask generated by the neural network, wherein when received by the user device, the user device generates a statistical analysis based on the one or more characteristics of the biological sample and the at least one monitoring process.

10. The system of claim 9, wherein the imaging device is configured to capture images through the lens of the microscope by affixing the imaging device to the microscope using an adapter.

11. The system of claim 10, wherein the adapter is compatible with off-the-shelf imaging devices, including at least one of a smartphone and digital camera.

12. The system of claim 9, wherein the one or more characteristics include at least one of cell confluency, cell migration, cell motility, cell viability, cell count, cell size, cell morphological changes, sperm cell characteristics, bacterial clone count, yeast count, and cell differentiation.

13. The system of claim 9, wherein the one or more images are annotated with one or more regions of interest.

14. The system of claim 9, wherein the instructions further cause the one or more processors to: receive metadata associated with a mask output by the neural network, the metadata indicating an accuracy of the one or more characteristics; and executing a training iteration of the neural network based on the metadata.

15. The system of claim 9, wherein the imaging device is the user device.

16. A non-transitory computer-readable medium storing instructions that when executed by one or more processors, cause the one or more processors to perform operations including: receiving a request to initiate a monitoring session associated with a user profde of a user device, wherein the request includes at least one monitoring process; receiving one or more images captured over a time interval defined by the at least one monitoring process, wherein the one or more images are generated by an imaging device configured to capture images through a lens of a microscope, and wherein the one or more images include a representation of a biological sample; executing a neural network using the one or more images, wherein the neural network is configured to derive one or more features from at least one image of the one or more images;predicting, using the neural network, a presence of one or more characteristics of the biological sample in the at least one image, wherein the one or more characteristics are defined by the at least one monitoring process; and transmitting the one or more characteristics of the biological sample and a mask generated by the neural network, wherein when received by the user device, the user device generates a statistical analysis based on the one or more characteristics of the biological sample and the at least one monitoring process.

17. The non-transitory computer-readable medium of claim 16, wherein the imaging device is configured to capture images through the lens of the microscope by affixing the imaging device to the microscope using an adapter.

18. The non-transitory computer-readable medium of claim 17, wherein the adapter is compatible with off-the-shelf imaging devices, including at least one of a smartphone and digital camera.

19. The non-transitory computer-readable medium of claim 16, wherein the one or more characteristics include at least one of cell confluency, cell migration, cell motility, cell viability, cell count, cell size, cell morphological changes, sperm cell characteristics, bacterial clone count, yeast count, and cell differentiation.

20. The non-transitory computer-readable medium of claim 16, wherein the instructions further cause the one or more processors to: receive metadata associated with a mask output by the neural network, the metadata indicating an accuracy of the one or more characteristics; and executing a training iteration of the neural network based on the metadata.

Images