Methods and systems for developing an imaging configuration to optimize performance of a microscope system

Abstract

A method and system for operating a microscope system are disclosed. A first image of a sample acquired using a first imaging configuration is received. A second image is generated from the first image, wherein the second image is associated with a second imaging configuration. A series of image processing steps are applied to the first image to develop a first classification of a first object represented in the first image; and the series of image processing steps are applied to the second image to develop a second classification of a second object represented in the second image. A score associated with the second imaging configuration is obtained, the score representing the difference between the first classification and the second classification. The image acquisition time or component requirements for operating the microscope system using the second imaging configuration are less than those for operating the microscope system using the first imaging configuration.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. Provisional Patent Application 63 / 081,019, filed September 21, 2020, entitled "Method and System of Developing an Imaging Configuration to Optimize Performance of a Microscopy System" by Cohen et al., the entire contents of which are incorporated herein by reference. Technical Field

[0003] This subject matter relates to microscope systems, and more specifically, to methods and systems for developing imaging configurations to optimize the operation of microscope systems. Background Technology

[0004] Microscopy systems, such as high-content imaging systems, can be used in experiments to acquire and analyze microscopic images of biological samples. Image analysis systems can be used to process such images to identify pixels in the image associated with objects of interest (cells, organelles, etc.) represented in the image, to classify or characterize different types of objects of interest represented in the image, and to obtain measures about such objects of interest or parts thereof. For each biological sample, such measures may include, for example, the count of objects of interest in the sample, the count of each type of object of interest represented in the biological sample, the size of the objects of interest (area, perimeter, volume), statistics on the size of the objects of interest (mean, mode, median, standard deviation, etc.), and characteristics of the objects of interest (e.g., whether the object is alive or dead, whether the object contains a specific protein, etc.).

[0005] To conduct experiments, researchers can develop experimental protocols that include experimental imaging configurations and image analysis steps. The experimental imaging configurations specify values ​​associated with imaging parameters (e.g., objective diameter, magnification, exposure time, focusing accuracy, etc.) for operating the microscope system to acquire one or more images of biological samples. The image analysis steps are used to analyze these images. Such imaging parameter values ​​can specify magnification levels (e.g., high magnification levels), the number of images of the biological sample to be acquired, the number of focal planes for acquiring images of the biological sample (i.e., at different z-planes), exposure times (e.g., high exposure times using the full dynamic range of the microscope system), the use of confocal imaging, the use of specialized optics (e.g., immersion or oil immersion optics), etc. Furthermore, if experimental groups of biological samples are arranged in the wells of a microplate, the imaging parameters can specify acquiring one or more images of different portions of each well at high resolution and / or different focal positions.

[0006] Following the development of experimental assay protocols, production assay protocols were developed, suitable for acquiring and analyzing production images of biological samples from production groups to validate the findings of experiments conducted using experimental assays. As will be understood by those skilled in the art, a production group of biological samples comprises a significantly larger number of samples than a test group of biological samples. In some cases, experimental imaging configurations used for acquiring high-quality images of test groups of biological samples may be impractical for acquiring images of larger production groups. For example, acquiring multiple images of each biological sample in a production group at high resolution, using long exposure times, and multiple images per location could be too time-consuming. Furthermore, the acquisition of images of larger groups may be distributed across multiple microscope systems, and equipping all such microscope systems with expensive, high-quality components, such as confocal imagers specified by the experimental imaging configuration, specialized optics, etc., may be infeasible. Summary of the Invention

[0007] According to one aspect, a method of operating a microscope system includes the steps of: receiving a first image of a sample acquired using a first imaging configuration; generating a second image from the first image, wherein the second image is associated with a second imaging configuration. The method further includes the steps of: applying a series of image processing steps to the first image to develop a first classification of a first object represented in the first image; applying the series of image processing steps to the second image to develop a second classification of a second object represented in the second image; and generating a score associated with the second imaging configuration, the score representing the difference between the first classification and the second classification. The image acquisition time or component requirements for operating the microscope system using the second imaging configuration are less than those for operating the microscope system using the first imaging configuration.

[0008] According to another aspect, a system for operating a microscope system includes an imaging configuration optimizer, a degradation image generator, an image analyzer, and an imaging configuration scorer, all operating on one or more processors. The imaging configuration optimizer receives a first image of a sample acquired using a first imaging configuration. The degradation image generator produces a second image from the first image, wherein the second image is associated with a second imaging configuration. The image analyzer applies a series of image processing steps to the first and second images to develop a first classification, wherein the first classification is associated with a first object represented in the first image. The imaging configuration scorer applies the series of image processing steps to the second image to develop a second classification associated with a second object represented in the second object, and derives a score associated with the second imaging configuration, the score representing the difference between the first and second classifications. The image acquisition time or component requirements for operating the microscope system using the second imaging configuration are less than those for operating the microscope system using the first imaging configuration.

[0009] Other aspects and advantages will become apparent after considering the following detailed description and accompanying drawings, in which the same reference numerals denote the same structures throughout the specification. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of a microscope system;

[0011] Figure 2 This is a block diagram of an image analysis system that can be used to develop operations... Figure 1 Imaging configuration of the microscope system;

[0012] Figure 3 It is by Figure 2 A flowchart of the steps taken by the image analysis system to develop the imaging configuration;

[0013] Figure 4 yes Figure 2 A block diagram of the imaging configuration optimizer for an image analysis system;

[0014] Figure 5 It is by Figure 4 A flowchart of the steps taken by the imaging configuration optimizer to develop an imaging configuration; and

[0015] Figure 6 It is by Figure 2 The graphical user interface generated by the imaging analysis system. Detailed Implementation

[0016] According to the embodiments disclosed herein, the image analysis system facilitates the development of experimental measurement schemes. The experimental measurement scheme includes an experimental imaging configuration and a series of image analysis steps for analyzing images from a microscope system operating according to the experimental imaging configuration.

[0017] In addition, the image analysis system receives high-quality images of biological samples acquired using the experimental imaging configuration. The imaging configuration optimizer of the image analysis system develops multiple additional imaging configurations (i.e., candidate production imaging configurations) from the experimental imaging configuration and the high-quality images. Each candidate production imaging configuration allows the microscope system to be operated using the candidate production imaging configuration with less time and one or two fewer pieces of specialized equipment to acquire images compared to operating it using the experimental imaging configuration. Furthermore, for each candidate production imaging configuration, the imaging configuration optimizer generates a degraded image from the high-quality image, simulating the image that would be acquired if the microscope system were operated using the candidate production imaging configuration. Moreover, the imaging configuration optimizer evaluates each candidate production imaging configuration to generate a score that indicates how well the object or object characteristics can be identified in the image acquired using the candidate production imaging configuration compared to the image acquired using the experimental imaging configuration. Specifically, the imaging configuration optimizer analyzes the degraded image associated with the candidate production imaging configuration to identify a first characteristic of the object therein and compares the first characteristic of the object identified in the degraded image with a second characteristic of the object identified in the high-quality image. The score associated with the candidate production imaging configuration indicates how close the first characteristic is to the second characteristic.

[0018] For example, a score can indicate the percentage of features of interest identified in a high-quality image that are also correctly identified in a degraded image associated with a candidate production imaging configuration. These features of interest can include cells, cells with specific properties, organelles, proteins, etc. In some embodiments, the score can indicate the accuracy of classifying cells (e.g., live cells vs. dead cells, treated cells vs. control cells, type I cells vs. type II cells, etc.) in the degraded image compared to those classified in the high-quality image. In some embodiments, an imaging configuration optimizer analyzes the scores associated with candidate production imaging configurations and uses the candidate production imaging configurations to select a recommended production imaging configuration to estimate the saved acquisition time using a production assay protocol.

[0019] To develop candidate production imaging configurations, the image analyzer applies a series of image analysis steps to high-quality images to develop classifications of the objects represented in the high-quality images. Such classifications may include the identification of specific types of objects (e.g., cells, organelles, proteins, etc.) and / or characteristics of these objects (e.g., size, quantity, type, mortality, etc.).

[0020] Subsequently, the imaging configuration optimizer generates a degraded image from the high-quality image based on each candidate production imaging configuration. The degraded image simulates the following image, which uses the candidate production imaging configuration associated with this degraded image to configure the microscope system to be acquired.

[0021] An imaging configuration optimizer automatically analyzes each degraded image to identify and classify objects associated with the experimental biological sample represented in such an image. For each degraded image, the classification of objects identified in the degraded image is compared to the classification of objects identified in a high-quality image to derive a score representing the accuracy with which objects can be identified and classified in the degraded image. In some embodiments, the imaging configuration optimizer selects a candidate production imaging configuration associated with the best score as a production imaging configuration for use in a production assay. In other embodiments, the imaging configuration optimizer presents to the researcher a list of one or more candidate production imaging configurations and a score associated with each such candidate production imaging configuration, and in response, receives a selection of one of the presented candidate production imaging configurations for use in a production assay.

[0022] refer to Figure 1 As those skilled in the art will understand, a microscope system 100, such as a high-content microscope system, may include an XY stage 102, one or more objectives 104, one or more illumination sources 106, an image capturing device 110, and a controller 112. The microscope system 100 may also include one or more mirrors (not shown) that direct light from the illumination source 106 to a sample tray or microplate 116 disposed on the XY stage 102. The light then passes through such a microplate 116, through the objectives 104, and reaches the image capturing device 110. In some embodiments, the microplate 116 includes a plurality of wells 118, and a biological sample (e.g., biological cells) to be imaged by the microscope system 100 may be disposed in each of such wells 118.

[0023] During operation, the microplate 116 can be placed manually or automatically on the XY stage 102. Additionally, the controller 112 can configure the microscope system 100 according to the imaging configuration (e.g., the experimental or production imaging configuration described above) to use a combination of specific objectives 104, illumination generated by the illumination source 106, etc. For example, the controller 112 can operate a positioning device (not shown) to place the selected objective 104 in the optical path between the microplate 116 and the image capture device 110. The controller 112 can also guide the illumination source 106 to illuminate the microplate 116 with light of a specific wavelength. In some cases, the sample in the microplate 116 may contain fluorescent molecules, either naturally occurring or generated or present in the sample due to treatment. The wavelength of the light generated by the illumination device can be the excitation wavelength associated with such fluorescent molecules, and the image capture device will capture only the emission spectrum of such fluorescent materials. One or more wavelengths can be used to continuously or simultaneously illuminate the same sample and produce an image.

[0024] In some embodiments, the controller 112 may operate the focusing mechanism 120 so that the image capturing device 110 may acquire focused images of the biological samples arranged in the microplate 116 at different focal planes.

[0025] Subsequently, the controller 112 can operate the XY stage 102 such that the aperture 118 or a portion thereof is within the field of view of the image acquisition device 110, and actuate the image acquisition device 110 to acquire an image of the aperture 118 or a portion thereof. The controller 112 can repeat the operation of the XY stage 102 and the image acquisition device 110 in this manner until images of all apertures 118 of interest on the microplate 116 have been captured. Furthermore, the controller 112 can capture several images of the same aperture 118 or the same portion thereof, wherein each such image is captured using different objectives 104, illumination wavelengths, and / or focal positions of the image biological sample.

[0026] Figure 1 The microscope system shown is exemplary, and other types of imaging or microscope systems that will be apparent to those skilled in the art can be used to capture high-quality images of experimental or production biological samples.

[0027] refer to Figure 1 and Figure 2 Researchers can use an image analysis system 150 that communicates with the microscope system 100 to develop experimental assays to analyze biological samples placed on a tray of the microscope system 100. As described above, the experimental assay includes: an experimental imaging configuration for acquiring one or more high-quality images of the biological sample, and a series of image processing steps for analyzing the acquired images.

[0028] Specifically, the image analysis system includes a user interface 152 that communicates with a user computer 154. As will be apparent to those skilled in the art, the user interface 152 instructs the user computer 154 to display a graphical user interface (GUI) to allow researchers to interact with the image analysis system 150. The image analysis system 150 also includes: a microscope system interface 156 that communicates with a microscope system 100; an image data storage 158; an image analyzer 160; a machine learning system 162; a training parameter data storage 164 including a set of training parameters that can be used to configure the machine learning system 162 from an untrained state to a trained state; and an imaging configuration optimizer 166. It will be apparent to those skilled in the art that by reinitializing such a machine learning system, the trained machine learning system 162 can return to an untrained state.

[0029] Figure 3 A flowchart 200 illustrates the steps taken by the image analysis system 150 in developing experimental and production measurement protocols. (Reference) Figures 1 to 3 In step 202, the user interface 152 receives from the user computer 154 instructions to specify an experimental imaging configuration and to capture one or more high-quality images of the biological sample set in the microscope system. In some embodiments, an experimental imaging configuration may be selected from a plurality of predetermined imaging configurations based on, for example, the type of cells in the biological sample, the characteristics of the object to be analyzed or the object in the image of the biological sample, the target of the ongoing measurement, etc.

[0030] In step 204, user interface 152 provides the experimental imaging configuration to microscope system interface 156, which then instructs controller 112 of microscope system 100 ( Figure 1 The various components are configured according to the experimental imaging configuration. Then, in step 206, the microscope system interface 156 instructs the controller 112 to acquire one or more high-quality images of the biological sample according to the experimental imaging configuration, and transmits the acquired high-quality images to the microscope system interface 156. The microscope system interface 156 also stores the acquired high-quality images in the image data storage 158 in step 206. In some embodiments, in step 208, the user interface 152 instructs the user computer 154 to display the acquired high-quality images.

[0031] After capturing a high-quality image, in step 210, the user interface 152 receives instructions on a series of image processing steps from the user computer 154.

[0032] In step 212, the image analyzer 160 performs a series of image processing steps, including those received in step 210, to analyze the acquired high-quality image, thereby identifying objects represented in the image and / or characteristics of those objects. U.S. Patent No. 8,577,079, entitled “IMAGE PROCESSING SYSTEM PROVIDING SELECTIVE ARRANGEMENT AND CONFIGURATION FOR AN IMAGE ANALYSIS SEQUENCE”, and U.S. Patent No. 10,706,259, entitled “SYSTEM AND METHOD FOR IMAGE ANALYSIS OF MULTI-DIMENSIONAL DATA”, by Cohen et al., disclose systems and methods for specifying and performing such a series of image processing steps to analyze an image. The entire contents of these patents are incorporated herein by reference. A series of image processing steps is provided to the image analyzer 160, which performs the specified image processing steps on one or more high-quality images. Such image processing steps may include: selecting an image captured using a specific illumination source; thresholding the image; applying one or more filters (e.g., unsharpening filter, smoothing filter, median filter, etc.); and masking or combining one or more images that were captured using different imaging configurations and / or obtained by applying imaging processing steps to the images.

[0033] In some embodiments, a series of image processing steps may specify image processing steps using the machine learning system 162 to analyze high-quality images. In such embodiments, the training parameter data storage 164 may have one or more predetermined sets of training parameters developed to train the machine learning system 162 to predict the presence of a particular object or the characteristics of an object (i.e., the classification of such an object) in images of a particular type of biological sample. Examples of developing such sets of training parameters are disclosed in the following applications: U.S. Patent Application Serial No. 16 / 128,798, entitled “SYSTEM AND METHOD FOR LABEL-FREE IDENTIFICATION AND CLASSIFICATION OF BIOLOGICAL SAMPLES”, Cohen et al.; and U.S. Patent No. 10,706,261, entitled “SYSTEM AND METHOD FOR AUTOMATICALLY ANALYZING PHENOTYPICAL RESPONSES OF CELLS”, Cohen et al. The entire contents of these applications and patents are incorporated herein by reference.

[0034] Refer again Figures 1 to 3 In order to perform the image analysis step using the specified machine learning system 162, the image analyzer 160 also configures the untrained machine learning system 162 according to specified training parameters in step 212 to develop a trained machine learning system 162, and operates the trained machine learning system 162 with pixels of one or more images (i.e., one or more high-quality images and / or images generated by previous image processing steps) as input, and produces an output that predicts the probability that a corresponding pixel of an image is associated with a specific object type or an object type with specific characteristics (i.e., pixels associated with object classification). The output generated by the trained machine learning system 162 can then be used as input in another image analysis step designated as part of a trial-and-error determination scheme.

[0035] It should be understood that a series of image processing steps, including the experimental measurement scheme, may include multiple image processing steps using a machine learning system configured with a corresponding set of training parameters available in the training parameter database 164.

[0036] In step 214, the image analyzer 160 instructs the user interface 152 to direct the user computer 154 to display the output generated by applying a series of image processing steps to the high-quality image acquired in step 206. Therefore, researchers can evaluate the results of capturing high-quality images using an experimental imaging configuration and analyze the high-quality images using a specified series of image processing steps.

[0037] In step 216, user interface 152 determines whether the researcher has used user computer 154 to adjust the experimental imaging configuration and / or a series of image processing steps, for example, to improve how effectively objects are classified (i.e., identified or determined to determine the characteristics of such objects) as represented in biological images. If yes, image analysis system 150 returns to step 204; otherwise, image analysis system 150 proceeds to step 218. The researcher may instruct image analysis system 150 to repeat steps 204 to 216 in this manner until the experimental imaging configuration and series of image processing steps (i.e., experimental assay protocol) for capturing and analyzing images of experimental biological samples are developed as desired by the researcher.

[0038] After developing the experimental measurement program, the researchers used the user computer 154 to guide the image analysis system 150 in developing a production imaging configuration that includes the production measurement program.

[0039] As described in more detail below, in response, in step 218, the imaging configuration optimizer 166 of the image analysis system 150 automatically evaluates the trial measurement scheme to derive multiple candidate production imaging configurations and a score for each candidate production imaging configuration. This score represents how well an object and / or object characteristics can be identified in a high-quality image by applying a series of image processing steps to an image captured using a candidate production imaging configuration. Thereafter, one of the candidate production imaging configurations (e.g., the candidate production imaging configuration with the highest score) can be selected as the recommended production imaging configuration.

[0040] Compared to the experimental imaging configurations developed through steps 204 to 216, each of the multiple candidate production imaging configurations developed in step 218 requires less time and / or fewer specialized components to capture images of biological samples using the microscope system 100.

[0041] refer to Figure 4 The imaging configuration optimizer 166 includes: a recommendation generator 248 for generating recommended production imaging configurations, a candidate configuration generator 250 for developing multiple candidate production imaging configurations, a degraded image generator 252, and an imaging configuration scorer 254.

[0042] Figure 5 A flowchart 300 illustrates the steps taken by the imaging configuration optimizer 166 to develop recommended production imaging configurations. In step 302, the recommendation generator 248 guides the candidate configuration generator 250 to develop multiple candidate production imaging configurations from the experimental imaging configurations. Specifically, the candidate configuration generator 250 identifies imaging parameters of the experimental imaging configurations that are associated with long imaging times (e.g., magnification, exposure time, etc.), the use of specialized imaging equipment (e.g., immersion or other specific optics, specialized filters, specific illumination sources, precise focusing, etc.) and / or the use of specialized microscopy techniques or sample processing (e.g., confocal microscopy, fluorescence imaging, sample labeling, etc.), and develops candidate production imaging configurations where the values ​​of the imaging parameters are replaced by another imaging parameter value that requires less imaging time, uses conventional imaging equipment and microscopy techniques and / or does not require specialized sample processing.

[0043] For example, if the test imaging configuration specifies a magnification of 40X, the candidate configuration generator 250 develops candidate production imaging configurations specifying magnifications of 20X, 10X, and 4X. Similarly, if the test imaging configuration specifies an exposure time of 100 milliseconds, the candidate configuration generator 250 develops candidate production imaging configurations specifying exposure times of 80 milliseconds, 50 milliseconds, 20 milliseconds, and 10 milliseconds.

[0044] Other examples of imaging parameter substitution include, for instance, using an air objective with higher excitation power and exposure time instead of a water immersion objective to avoid longer laser autofocus times and water requirements; using transmitted light imaging instead of fluorescence imaging; and using wide-field imaging, lower magnification, shorter exposure times, and additional image processing steps instead of confocal imaging with high magnification. It will be apparent to those skilled in the art that the candidate configuration generator 250 can be configured to substitute specific combinations of imaging parameters in the experimental imaging configuration with other combinations of parameters to develop candidate production imaging configurations.

[0045] In some embodiments, multiple imaging parameters of the test imaging configuration can be replaced to develop a candidate production imaging configuration. For example, the candidate configuration generator 250 can generate a candidate production imaging configuration based on the test imaging configuration by replacing the high magnification and long exposure time specified in the test imaging configuration with a lower magnification and a shorter exposure time. Those skilled in the art will understand that the candidate configuration generator 250 can be configured to replace specific combinations of imaging parameters specified in the test imaging configuration to develop candidate production imaging configurations.

[0046] In some embodiments, the candidate configuration generator 250 may develop candidate production imaging configurations by selecting values ​​of imaging parameters of the experimental imaging configuration to replace those of the biological sample being analyzed, based on the type of cells or the size of the object detected using the experimental assay protocol. For example, such information may be used to select objectives that may be suitable for replacement. Similarly, the intensity values ​​of pixels in a high-quality image associated with the identified object may be used to select the exposure time.

[0047] Furthermore, the candidate configuration generator 250 can be configured with information relating to the effect of substituting specific values ​​of imaging parameters of the experimental imaging configuration into the values ​​of other imaging parameters, and candidate production imaging configurations can be developed based on such information. For example, changing the objective lens may have known effects on exposure time, focusing, Z-step (i.e., focal plane increment) requirements, and acquisition mode.

[0048] After developing multiple candidate production imaging configurations, in step 304, the candidate configuration generator 250 selects one of the candidate production imaging configurations, and in step 306, instructs the degradation image generator 252 to form a degradation image associated with the candidate production imaging configuration from the high-quality image. Specifically, the degradation image generator 252 performs image processing steps associated with substitutions to one or more imaging parameters of the test image configuration to develop candidate production imaging configurations, thereby generating a degradation image associated with such candidate production imaging configurations. The degradation image predicts the image that the microscope system 100 will produce if operated using the candidate production imaging configuration.

[0049] In some embodiments, the image processing step taken by the degraded image generator 252 in step 306 includes applying one or more filters to a high-resolution image to generate a degraded image. For example, a degraded image associated with a magnification variation between a first objective and a second objective can be created by scaling a high-quality image according to a ratio of magnification associated with a first objective specified by a test imaging configuration and a second objective specified by a candidate production imaging configuration. Similarly, a Gaussian blur filter can be applied to a high-quality image to form a degraded image representing a variation in the numerical aperture used to capture the high-quality image and specified in the candidate production imaging configuration. If multiple imaging parameters vary between the test imaging configuration and the candidate production imaging configuration, image processing functions associated with each of these imaging parameter variations can be sequentially applied to the high-quality image to produce a degraded image.

[0050] In other embodiments, the machine learning training parameter data storage 164 includes a set of training parameters associated with substitutions of specific imaging parameters in a trial imaging configuration to develop candidate production imaging configurations. In such a case, the degradation image generator 252 selects a set of training parameters associated with this substitution, uses the selected set of training parameters to configure an untrained machine learning system 162 to develop a trained machine learning system 162, and operates the trained machine learning system 162 using a high-resolution image as input. In response, the trained machine learning system 162 generates a degradation image associated with the candidate production imaging configuration. For example, data storage 165 may include a set of training parameters for training the machine learning system 162 to generate degradation images from high-quality images, representing images captured by replacing a high-magnification water immersion objective with a lower-magnification air objective, or a fluorescence microscope with a transmission light microscope, etc.

[0051] In some embodiments, also in step 306, the degraded image generator 252 stores the degraded image associated with the selected candidate production imaging configuration in the image data storage 158.

[0052] After generating (and storing) the degraded image in step 306, the imaging configuration scorer 254 analyzes the degraded image to derive a score for the candidate production imaging configuration selected in step 304 for generating the degraded image. Specifically, in step 308, the imaging configuration scorer 254 uses the image analyzer 160 to apply a series of image processing steps specified by the test assay protocol to the degraded image to identify the objects and / or characteristics of the objects represented in the degraded image.

[0053] Subsequently, in step 310, the imaging configuration scorer 254 compares the characteristics of objects and / or objects identified in the degraded image with the characteristics of objects and / or objects identified in the high-quality image to derive a score. In some embodiments, the imaging configuration scorer 254 calculates a first Z-prime statistic associated with the experimental imaging configuration. As will be understood by those skilled in the art, the Z-prime statistic assesses the degree to which the high-quality image predicts differences in the cells of the experimental group represented therein. These differences may include, for example, cells in the experimental cell group that have been treated versus cells in the treated control group, including dead cells in the experimental group versus live cells in the control group, etc. Furthermore, in step 310, the imaging configuration scorer 254 also calculates a second Z-prime statistic associated with the candidate production imaging configuration, which indicates the degree to which the degraded image generated according to the candidate production imaging configuration predicts differences in the cells of the experimental group. The imaging configuration scorer 254 derives a score associated with the candidate production imaging configuration based on the first and second Z-prime scores. This score may be the difference between the two Z-prime scores, the ratio of the two Z-prime scores, etc.

[0054] In other embodiments, a series of image processing steps specified by the experimental determination scheme generate the output image. In such embodiments, those skilled in the art will understand that in step 212 ( Figure 3 The image analyzer generates a first output image by applying a series of image processing steps to a high-quality image. Furthermore, a second output image is generated in step 308 when the imaging configuration scorer 254 applies a series of image processing steps to a degraded image. In such an embodiment, in step 310, the imaging configuration scorer 252 calculates the error (e.g., root mean square error, etc.) between each pixel of the first output image and each pixel of the second output image to derive a score.

[0055] In some embodiments, at step 310, the imaging configuration scorer compares a first metric (e.g., area, volume, perimeter, pixel intensity, etc.) of an object identified in a high-quality image with a second metric of a corresponding object identified in a degraded image associated with a candidate production imaging configuration, and derives a score associated with the candidate production imaging configuration based on the error between the first and second metrics (e.g., root mean square error).

[0056] In some cases, in step 212 ( Figure 3 A series of image processing steps, specified by the experimental design, are applied to the microplate 116 disposed in the microscope system 100. Figure 1One or more high-quality images of the wells of the microplate are generated to produce information about each well (e.g., whether the biological sample placed in the well has been processed, the ratio of live to dead cells in each microplate, the presence of a specific protein in each well, etc.). In this case, in step 306, images of the microplate 116 ( Figure 1 The image configuration scorer 254 applies a series of image processing steps to the degraded images corresponding to such high-quality images in each well of the microwell plate 116. Then, in step 310, the image configuration scorer 254 calculates a score representing the result of applying the series of image processing steps to predict the degraded images relative to the wells set in the microwell plate 116. Figure 1 The accuracy of the information of the biological sample in each well.

[0057] In step 311, the imaging configuration scorer 254 estimates how much time will be required to form one or more images of the biological sample based on the candidate production imaging configuration. This time estimate may reflect the acquisition time of one or more images of the well 118, a portion of the well 118, or the entire microplate 116 in which the biological sample is situated. It will be apparent to those skilled in the art that the time estimate can be calculated based on imaging parameters specified by the candidate production imaging configuration, including: the exposure time for acquiring images at each focal plane at each site of the biological sample, the number of focal planes at which images of that site are to be acquired, the amount of time required to focus each focal plane at each site according to the objective lens used, the time to activate any dedicated hardware for image acquisition, the time to move the microplate 116 to bring the site into the field of view of the objective lens, the number of sites of the biological sample to be imaged, etc. The estimated time is compared to the amount of time required to acquire high-quality images according to the experimental imaging configuration, and an estimated time saving is calculated for the candidate production imaging configuration.

[0058] In step 312, candidate configuration generator 250 determines whether scores have been generated for all candidate production imaging configurations developed in step 302. If yes, candidate configuration generator 250 proceeds to step 314; otherwise, candidate configuration generator 250 proceeds to step 304 to select another candidate production imaging configuration.

[0059] In step 313, the recommendation generator 248 analyzes the scores and estimated time savings associated with the candidate production imaging configurations developed by the candidate configuration generator 250, and selects the candidate production imaging configuration with the best score and the maximum time savings as the recommended production imaging configuration. In some cases, the recommendation generator 248 selects the candidate production imaging configuration with the maximum time savings from those candidate production imaging configurations with scores exceeding a predetermined amount. In other cases, the recommendation generator 248 selects the candidate production imaging configuration with the maximum time savings from a predetermined portion of all candidate production imaging configurations as the recommended production imaging configuration. The predetermined portion may be a predetermined number (e.g., three, five, ten, etc.) or percentage (five percent, ten percent, etc.) of all candidate production imaging configurations with the best score. It will be apparent to those skilled in the art that the criteria used by the recommendation generator 248 to select the best score depends on the metric used to derive the score associated with the candidate production imaging configuration. For example, the best score may be the score with the maximum value. Alternatively, the best score may be the score with the lowest value, the maximum absolute value, the lowest absolute value, the value closest to a predetermined value (e.g., 0 or 1), etc.

[0060] In step 314, the recommendation generator 248 instructs the user interface 152 to direct the user computer 154 to display candidate production imaging configurations, scores associated with each candidate production imaging configuration, and estimates of how much time could be saved by optionally using a candidate production imaging configuration instead of a trial imaging configuration. In some embodiments, the user interface 152 may also instruct the user computer 154, also in step 314, for each candidate production imaging configuration, to display a degraded image associated with the production imaging configuration, objects identified in the degraded image, and / or a classification of the objects identified in the degraded image.

[0061] Also in step 313, the recommendation generator 248 instructs the user interface 152 to instruct the user computer 154 to display an indicator that recognizes the candidate production imaging configuration selected in step 313 as the recommended production imaging configuration.

[0062] refer to Figure 6In one embodiment, user interface 152 instructs user computer 154 to generate GUI 400, which may display information about recommended production imaging configurations and other candidate production imaging configurations. Specifically, a high-quality image is displayed in region 402 of GUI 400. An estimate of the amount of time required to capture the high-quality image according to the experimental imaging configuration is overlaid on the high-quality image in region 404 of GUI 400, and information icons are overlaid in region 406 of GUI 400. If an investigator (or other operator) clicks (or hovers over) region 406, user interface 152 instructs user computer 154 to display imaging parameters specified by the experimental imaging configuration. These imaging parameters may be displayed, for example, in a pop-up window (not shown), overlaid on region 402 of the GUI, or displayed in another region of GUI 400 (not shown).

[0063] Additionally, one or more degraded images are displayed in areas 408a to 408f of GUI 400. Although in Figure 6 The image shows eight degraded images surrounding a high-quality image, but it will be apparent to those skilled in the art that more or fewer degraded images could be shown, and that these degraded images are positioned in different locations relative to the high-quality image.

[0064] For each degraded image displayed in region 408 of GUI 400, step 311 is displayed in region 410 of GUI 400. Figure 3 The estimated time savings are calculated in the image and displayed as an information icon in area 412 of the GUI 400. Selecting or hovering over the information icon displays information about the imaging parameters of the candidate production imaging configuration associated with the degraded image displayed in area 408. In some embodiments, when the information icon is selected or hovered over it, the difference between the imaging parameters of the candidate production imaging configuration associated with the degraded image and the imaging parameters of the test imaging configuration is displayed.

[0065] An indicator is displayed in region 414 to indicate that the recommendation generator 248 has selected a candidate production imaging configuration associated with the degraded image displayed in region 408f as the recommended production imaging configuration.

[0066] Refer again Figure 1 and 3In some embodiments, in step 316, the user interface 152 receives instructions from the user computer 154, for example, from a researcher (or another operator), to develop a production measurement scheme using a recommended production imaging configuration or by selecting another candidate production imaging configuration to be used in the production measurement scheme. In response, the imaging configuration optimizer 166 develops a production measurement scheme in step 318 that includes the selected candidate production imaging configuration and a series of imaging steps from the experimental imaging scheme. The image analysis system 150 can then use this production measurement scheme to automatically analyze additional microplates 116 disposed in the microscope system 100. Furthermore, this production measurement scheme can be used with an additional image analysis system 150 that communicates with the corresponding microscope system 100 to automatically analyze microplates 116 disposed in such microscope systems.

[0067] In some cases, by conducting Figure 3 The experimental assay protocol developed in steps 202 to 216 can specify the use of multiple corresponding experimental imaging configurations to acquire multiple high-quality images of biological samples, and a series of image processing steps can analyze the multiple high-quality images at 212. It will be apparent to those skilled in the art that the imaging configuration optimizer 166 can create one or more candidate production imaging configurations for each of the multiple imaging configurations, develop one or more degraded images based on the one or more candidate production imaging configurations, and analyze each degraded image to develop a score for the candidate production imaging configuration associated with that degraded image.

[0068] Those skilled in the art will understand that the microscope system 100 can be located remotely from the image analysis system 150, and communication between the microscope system 100 and the microscope system interface 156 can be conducted via private or public networks (e.g., the Internet, Virtual Private Network, Local Area Network, Cellular Network, etc.). Similarly, the user computer 154 can also be located remotely from the image analysis system 150, and communication between the user computer 154 and the user interface 152 can be conducted via private or public networks. Furthermore, components of the image analysis system 150 can operate on a single computing device or on multiple distributed computing devices communicating with each other via private or public networks. For example, the machine learning system 162 can be provided by a cloud service and accessed by the image analyzer 160 via the Internet. Similarly, the image data storage 158 and / or the training parameter data storage 164 can be implemented using cloud-based storage services.

[0069] It will be apparent to those skilled in the art that the image analysis system 150 described herein can be implemented using any combination of hardware and / or software. It will be understood and appreciated that, in conjunction with… Figures 1 to 5The described one or more processes, subprocesses, and process steps can be executed by hardware, software, or a combination of hardware and software on one or more electronic control devices or digital control devices. The software may reside in a software memory (not shown) in a suitable electronic processing unit or system, such as... Figures 1 to 5 The schematic diagram illustrates one or more of the functional systems, controllers, devices, components, modules, or submodules. The software memory may include an ordered list of executable instructions for implementing logical functions (i.e., "logic" may be implemented in digital form, such as digital circuits or source code, or in analog form, such as analog sources, such as analog electrical, sound, or video signals). The instructions may be in a processing module or controller (e.g., Figure 2 User interface 152, microscope system interface 156, image analyzer 160, machine learning system 162, and imaging configuration optimizer 166, and Figure 4 The system executes within a recommender generator 248, a candidate configuration generator 250, a degraded image generator 252, and an imaging configuration scorer 254, and includes, for example, one or more microprocessors, general-purpose processors, combinations of processors, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), or application-specific integrated circuits (ASICs). Furthermore, the schematic diagrams depict the logical partitioning of functions with physical (hardware and / or software) implementations that are not limited by functional architecture or physical layout. The example systems described in this application can be implemented in various configurations and operate as a single hardware / software unit or as hardware / software components within co-located or distributed separate hardware / software units.

[0070] Executable instructions can be implemented as a computer program product in which instructions are stored, which, when executed by a processing module of an electronic system, direct the electronic system to perform the instructions. The computer program product can be selectively embodied in any non-transitory computer-readable storage medium for use by or in conjunction with an instruction execution system, apparatus, or device, such as a computer-based system, a processor-containing system, or other system that selectively acquires and executes instructions from and from an instruction execution system, apparatus, or device. In the context of this document, a computer-readable storage medium is any non-transitory device that can store a program used by or in conjunction with an instruction execution system, apparatus, or device. Non-transitory computer-readable storage media can optionally be, for example, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices. A non-exhaustive list of more specific examples of non-transitory computer-readable media includes: electrical connections having one or more wires (electronic); portable computer disks (magnetic); random access memory, i.e., volatile memory (electronic); read-only memory (electronic); erasable programmable read-only memory, such as flash memory (electronic); optical disc storage, such as CD-ROM, CD-R, CD-RW (optical); and digital universal disc storage, i.e., DVD (optical).

[0071] It will also be understood that the reception and transmission of signals or data as used herein means that two or more systems, devices, components, modules, or submodules are able to communicate with each other via signals traveling along some type of signal path. The signal can be a communication, electrical, data, or energy signal that can transmit information, power, or energy from one system, device, component, module, or submodule to another along a signal path between the first system, device, component, module, or submodule and the second system, device, component, module, or submodule. The signal path can include physical connections, electrical connections, magnetic connections, electromagnetic connections, electrochemical connections, optical connections, wired connections, or wireless connections. The signal path can also include additional systems, devices, components, modules, or submodules between the first system, device, component, module, or submodule and the second system, device, component, module, or submodule.

[0072] All references cited in this article, including publications, patent applications and patents, are incorporated herein by reference to the extent that each reference is individually and specifically indicated to be incorporated herein by reference and discussed in its entirety.

[0073] The terms “a (and an)” and “the”, and similar references used in the context of describing this invention (especially in the context of the claims), are to be interpreted as encompassing both the singular and plural, unless otherwise indicated herein or clearly contradicted by the context. Unless otherwise stated herein, descriptions of numerical ranges are intended only as a shorthand method for individually referring to each individual value falling within that range, and each individual value is incorporated into the specification as if it were individually described herein. All methods described herein may be performed in any suitable order unless otherwise stated herein or clearly contradicted by the context. Unless otherwise required, the use of any and all instances or exemplary language (e.g., “such as”) provided herein is intended only to better illustrate this disclosure and not to limit the scope of this disclosure. The language in the specification should not be construed as indicating that any unclaimed element is necessary for the practice of this disclosure.

[0074] In view of the foregoing description, many modifications to this disclosure will be apparent to those skilled in the art. It should be understood that the illustrated embodiments are merely exemplary and should not be considered as limiting the scope of this disclosure.

Claims

1. A method for operating a microscope system, comprising the following steps: Receive a first image of a sample acquired using a first imaging configuration; A second image is generated from the first image, wherein the second image is associated with a second imaging configuration; A series of image processing steps are applied to the first image to develop a first classification of the first object represented in the first image; Wherein, the first classification of the first object is associated with the identification of one or more objects represented in the first image, the identification of one or more objects having specific characteristics, or a metric associated with one or more objects represented in the first image; The series of image processing steps are applied to the second image to develop a second classification of the second object represented in the second image; A score is derived associated with the second imaging configuration, the score representing the difference between the first classification and the second classification, wherein the score includes the first score; and The image acquisition time or the number of specialized components required to operate the microscope system using the second imaging configuration is less than the image acquisition time or the number of specialized components required to operate the microscope system using the first imaging configuration. The method further includes the following steps: A third image is generated from the first image, wherein the third image is associated with a third imaging configuration; The aforementioned series of image processing steps are applied to the third image to develop a third classification. A second score representing the difference between the first category and the third category is derived; and The system automatically selects a recommended production imaging configuration, wherein if the first score is better than the second score, the recommended production imaging configuration is the second imaging configuration, and if the second score is better than the first score, the recommended production imaging configuration is the third imaging configuration. The selected recommended production imaging configuration is used in the production measurement scheme, and The production measurement scheme is used to automatically analyze the microplates set in the microscope system.

2. The method according to claim 1, further comprising the following step: Select the training parameter set according to the image processing steps in the series of image processing steps; Configure an untrained machine learning system with the selected set of training parameters to develop a trained machine learning system; as well as The trained machine learning system is operated to develop the first classification.

3. The method of claim 1, further comprising the step of generating the second imaging configuration from the first imaging configuration.

4. The method according to claim 3, wherein, The first imaging configuration specifies a first value associated with the imaging parameter, and the second imaging configuration specifies a second value associated with the imaging parameter, wherein the first value and the second value are different.

5. The method according to claim 4, further comprising the following step: Based on the difference between the first imaging configuration and the second imaging configuration, the training parameters of the untrained machine learning system are selected; The untrained machine learning system is trained using the selected training parameters to develop a trained machine learning system. as well as The trained machine learning system is operated using the first image as input to generate the second image.

6. The method of claim 1, wherein the step of receiving the first image comprises: The steps of acquiring the first image using the first microscope system; The step of receiving the first image further includes the step of acquiring a fourth image using a second microscope system according to the second imaging configuration.

7. The method of claim 1, wherein the second image simulates an image of a sample acquired if the microscope system operates in the second imaging configuration.

8. The method of claim 1, wherein the second imaging configuration is one of a plurality of candidate production imaging configurations, wherein a score is derived for each of the plurality of candidate production imaging configurations, the method further comprising: The step of selecting the recommended production imaging configuration from candidate production imaging configurations with scores exceeding a predetermined number.

9. The method of claim 1, wherein the second imaging configuration is one of a plurality of candidate production imaging configurations, and for each of the plurality of candidate production imaging configurations, a development classification and a score are derived, the method further comprising the steps of: Select the recommended production imaging configuration from the plurality of candidate production imaging configurations; as well as The computer is instructed to display information about each of the plurality of candidate production imaging configurations, and to display an indicator for identifying the candidate production imaging configuration selected as the recommended production imaging configuration; The information displayed for each candidate production imaging configuration includes one or more of the following associated with the candidate production imaging configuration: score, estimate of time savings, image, and object classification.

10. The method of claim 9, further comprising the step of: The computer receives a selection of one of the candidate production imaging configurations, and the microscope system is configured using the candidate production imaging configuration.

11. A system for operating a microscope system, comprising: An imaging configuration optimizer operating on one or more processors, the imaging configuration optimizer receiving a first image of a sample acquired using a first imaging configuration; A degradation image generator operating on one or more processors, the degradation image generator generating a second image from the first image, wherein the second image is associated with a second imaging configuration; An image analyzer operating on one or more processors applies a series of image processing steps to a first image and a second image to develop a first classification, wherein the first classification is associated with a first object represented in the first image, and wherein the first classification of the first object is associated with the identification of one or more objects represented in the first image, the identification of one or more objects having specific characteristics, or a metric associated with one or more objects represented in the first image; and An imaging configuration scorer operating on one or more processors applies the series of image processing steps to the second image to develop a second classification associated with a second object represented in the second image and to derive a score associated with the second imaging configuration, the score representing the difference between the first classification and the second classification, wherein the score includes a first score; The image acquisition time or the number of specialized components required to operate the microscope system using the second imaging configuration is less than the image acquisition time or the number of specialized components required to operate the microscope system using the first imaging configuration. The degradation image generator generates a third image from the first image, wherein the third image is associated with a third imaging configuration, and The imaging configuration scorer applies the series of image processing steps to the third image to develop a third classification and derives a second score representing the difference between the first and third classifications; and The system also includes a recommendation generator operating on the one or more processors, the recommendation generator automatically selecting a recommended production imaging configuration. Wherein, if the first score is better than the second score, the recommended production imaging configuration is the second imaging configuration; and if the second score is better than the first score, the recommended production imaging configuration is the third imaging configuration. The selected recommended production imaging configuration is used in the production measurement scheme, and The production measurement scheme is used to automatically analyze the microplates set in the microscope system.

12. The system of claim 11, wherein the first classification of the first object is associated with the identification of one or more objects represented in the first image, the identification of characteristics of one or more objects, or a metric associated with one or more objects represented in the first image.

13. The system of claim 11, further comprising an untrained machine learning system, wherein the image analyzer selects training parameters associated with a step in the series of image processing steps, uses the training parameters to configure the untrained machine learning system to develop a trained machine learning system, and operates the trained machine learning system to develop the first classification.

14. The system of claim 11, further comprising a candidate configuration generator operating on the one or more processors, the candidate configuration generator automatically generating the second imaging configuration based on the first imaging configuration.

15. The system of claim 14, wherein the first imaging configuration specifies a first value associated with the imaging parameters, and the second imaging configuration generated by the candidate configuration generator automatically specifies a second value associated with the imaging parameters, and wherein the first value and the second value are different.

16. The system of claim 15, further comprising an untrained machine learning system, wherein the degraded image generator selects a set of training parameters based on a difference between the first imaging configuration and the second imaging configuration; trains the untrained machine learning system with the selected training parameters to develop a trained machine learning system; and operates the trained machine learning system using the first image as input to generate the second image.

17. The system of claim 11, wherein the microscope system comprises a first microscope system and a second microscope system, wherein the first microscope system operates according to the first imaging configuration to acquire the first image, and the second microscope system operates according to the second imaging configuration to acquire a fourth image.

18. The system of claim 11, wherein the second image simulates an image of the sample acquired if the microscope system operates in the second imaging configuration.

19. The system of claim 11, wherein the second imaging configuration is one of a plurality of candidate production imaging configurations, the system further comprising a candidate configuration generator and a recommendation generator, wherein, The candidate configuration generator develops the plurality of candidate production imaging configurations, the imaging configuration scorer assigns a score to each of the candidate production imaging configurations, and the recommendation generator selects a recommended production imaging configuration from the candidate production imaging configurations whose scores exceed a predetermined number.

20. The system of claim 11, wherein the second imaging configuration is one of a plurality of candidate production imaging configurations, and for each of the plurality of candidate production imaging configurations, a development classification and a score are derived, the system further comprising: A recommendation generator selects a recommended production imaging configuration from the plurality of candidate production imaging configurations; as well as A user interface generator instructs a computer to display information about each of the plurality of candidate production imaging configurations, and to display an indicator for identifying the candidate production imaging configuration selected as the recommended production imaging configuration. The information displayed for each candidate production imaging configuration includes one or more of the following associated with the candidate production imaging configuration: score, estimate of time savings, image, and object classification.

21. The system of claim 20, wherein the user interface generator receives from the computer a selection of one of the candidate production imaging configurations, and uses the candidate production imaging configuration to configure the microscope system.

Images