System and method for measurement of and compensation for optical distortion

Abstract

A system implements techniques for measurement and compensation of optical distortion by accessing a first image of a target object captured of via an imaging instrument, and repeating the process of moving the target object a known distance via an actuator and capturing other images of the target object after the target object has been moved a number of times until a plurality of images have been captured. A correspondence is then established between a feature of the first image and a corresponding feature of each of the other images to identify corresponding features in consecutive images and the coordinates of each pair of identified corresponding features is measured. The measured displacement vector between each pair of features is used to determine parameters for optical distortion of the imaging instrument which is then used to modify the first image to compensate for optical distortion, and output the modified image.

Description

PATENT NovoTecliIP Docket No. 240613A-002PCTSYSTEM AND METHOD FOR MEASUREMENT OF AND COMPENSATION FOR OPTICAL DISTORTIONBACKGROUND

[0001] Optical distortion is a common occurrence in many imaging systems, particularly in high numerical aperture (NA) systems such as high NA microscopes and widefield imaging systems. Optical distortion may change over time or due to environmental conditions such as temperature and homogeneity of imaging media. While the presence of optical distortion in any imaging context is undesirable, it is highly deleterious in the context of spatial biology where transcripts may appear with low frequency and need to be detected with high sensitivity and ascribed to the correct cell. Furthermore, biological systems are often highly heterogeneous and information rich. As a result, optical distortion can lead to duplication or misattribution of rare biological transcripts which can in turn greatly affect downstream cell classification and tissue organization. This can result in errors in interpretation of disease etiology and mechanism. Hence, there is a need for improved systems and methods for measurement of. calibration and / or compensation for optical distortion in imaging systems.BRIEF DESCRIPTION OF THE DRAWINGS

[0002] The drawing figures depict one or more implementations in accord with the present teachings, by way of example only, not by way of limitation. In the figures, like reference numerals refer to the same or similar elements. Furthermore, it should be understood that the drawings are not necessarily to scale.

[0003] FIG. 1 is a diagram of an example system in which the techniques for measurement, calibration and compensation of optical distortion herein are implemented.

[0004] FIG. 2 depicts multiple example images of the same target object being captured by an imaging instrument when the target object is displaced by an actuator.

[0005] FIG. 3 depicts an example feature matching between features of example images of the same target object when the target object is displaced by an actuator.

[0006] FIG. 4 depicts diagrams of example distorted and undistorted images.

[0007] FIG. 5 is a flow chart of an example process for measurement of and compensation for optical distortion according to the techniques disclosed herein.1-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT

[0008] FIG. 6 is a block diagram showing an example software architecture, various portions of which may be used in conjunction with various hardware architectures herein described, which may implement any of the described features.

[0009] FIG. 7 is a block diagram showing components of an example machine configured to read instructions from a machine-readable medium and perform any of the features described herein.

[0010] FIGS. 8-10 show successful results from the distortion measurement procedures disclosed herein.DETAILED DESCRIPTION

[0011] Systems and methods for measurement of. calibration and / or compensation for optical distortion in imaging systems are disclosed. These techniques provide a technical solution to the technical problems associated with the presence of optical distortion in images generated by imaging systems. Optical distortion is prevalent in many imaging systems, particularly those systems used in the field of spatial biology such as high NA microscopes and widefield systems. In widefield image-forming optical systems, optical distortion refers to a change in magnification with field of view. In some fields, distortion is a primary Seidel aberration described by the Zemike mode (1,±1 ) or “tilt,” and can be classified as “pincushion,” “barrel,” or “moustache” depending on the sign of the change in magnification. This aberration results in a nonlinear transformation of the image from world-coordinates to image-coordinates, which causes undesirable deformation of image geometry, curvature of edges, inaccurate spot localization, and / or erroneous length and area measurement. Distortion also hinders image registration and stitching in large area scanning microscopes.Additionally, in single molecule imaging systems, distortion can result in the appearance of duplicate cells and transcripts which require resolution in downstream analysis. Thus, the presence of optical distortion is significantly detrimental in the field of spatial biology where transcripts need to be detected with a high level of accuracy and sensitivity.

[0012] Furthermore, as discussed above, biological systems are highly heterogeneous and information-rich, and duplication or misattribution of biological transcripts can greatly affect downstream cell classification and tissue organization. This means that the presence of even minor optical distortion in such systems can lead to errors in interpretation of disease etiology and mechanism. Taken together, these effects degrade the performance of imaging systems and diminish user experience. Yet, detection of and compensation for optical distortion in imaging systems used in spatial biology is a challenging undertaking, as such systems require2-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT a high degree of accuracy and sensitivity. Furthermore, traditional distortion measurement methods require expensive, precise calibration test targets which are typically fabricated as chrome-on-glass by photolithography. These tools limit the measurement of distortion to specific scenarios such as production, field service, and refurbishment, and prohibit regular and automated calibration. Moreover, the geometry chosen for such systems is often carefully tuned to the imaging mode, resolution, and field of view (FOV) size of the imaging system, and therefore needs to be designed anew given a different magnification or sensor size.

[0013] Some recent systems have been able to measure distortion using sharp edges that are naturally acquired by certain imaging systems in some extended scenes. However, many imaging systems are not used in applications where straight edges are available for imaging. Similarly, some prior systems use two-camera or two-view techniques for the measurement of distortion and the correction of images into rectilinear spaces. However, these methods rely on detecting known image geometry for the establishment of scale, which is error prone and reliant on the presence of standard geometries. Additionally, these methods rarely use a known, precise offset between images, causing the accumulation of error when applied to image metrology in world coordinates. Thus, there exists a technical problem of lack of mechanisms for accurate, efficient and consistent detection, measurement, calibration and compensation for optical distortion in systems used for optical metrology' in spatial biology'.

[0014] To address these technical problems and more, in an example, this description provides technical solutions for improved measurement, calibration and compensation of optical distortion in imaging systems. This involves capturing an image of a high-contrast target object having any geometry' using an imaging system, utilizing a high-precision linear actuator to accurately and precisely move the target according to a known displacement, recapturing the image of the target after it has been displaced, and providing a known displacement in world-coordinate between corresponding measured image coordinates. These steps may be done iteratively to generate sufficient data for calibration of the system. An algorithm is then used to estimate the amount of transformation between the images and map the transformations to undistorted world coordinates. This information is then used to calibrate for the distortion and to provide for nonlinear transformation of image data from distorted image coordinates to the undistorted world coordinates. These technical solutions improve an imaging system by accurately measuring and standardizing magnification and facilitating correction of optical distortion. This provides a precise, error-tolerant, low-cost, and highly adaptable to nearly any imaging system and imaging target solution and enables regular and automated measurement and calibration of distortion. These and other technical3-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT benefits of the techniques disclosed herein will be evident from the discussion of the example implementations that follow.

[0015] It should be noted that while the solutions disclosed herein are discussed in relation to managing optical distortion in the field of spatial biology, the solutions can also be applied to scenes depicting any high-contrast geometry' .

[0016] FIG. 1 is a diagram of an example system 100 in which the techniques for measurement, calibration and compensation of optical distortion disclosed herein are implemented. The example system 100 includes an imaging instrument 102, a target object 104, an actuator 106 and a distortion compensation system 108. The example system 100 shows one possible configuration of an environment that may be used to implement the techniques disclosed herein. Other implementations may include additional components instead of or in addition to one or more of the components shown in the example implementation of FIG. 1.

[0017] The imaging instrument 102 is a device configured to capture and / or magnify images of objects. In an example, the imaging instrument 102 is an imaging device, such as a microscope, used to examine and analyze biomolecular samples. For example, the imaging instrument 102 may be a high NA microscope or fluorescence microscope. In another example, the imaging instrument 102 is a widefield microscope. In yet another example, the imaging instrument 102 is a telescope. In some implementations, the imaging instrument 102 is an epifluorescence widefield microscope which, in some cases, provides four excitation / emission pairs corresponding to standard high-performance fluorescent dyes, in addition to a near-UV excitation source used for UV-excitable dyes and species. In an example, the imaging instrument 102 leverages a water immersion objective with a numerical aperture (NA) of 1. 1, nominal magnification of 23x, and a field diameter of 720 pm. This objective imparts a distortion at the edge of captured images that is up to 5% or 18 microns of error, and as such requires correction. Other microscopes may provide different NA, magnification, and field diameters and as such may generate different optical distortions. While the level of optical distortion may vary, most imaging instruments generate some level of distortion which when left uncorrected can result in inaccurate depiction of the target object.

[0018] As discussed above, in some implementations, the imaging instrument 102 captures and collects magnified images of biological samples. These images can be collected through a microscope objective lens with an imaging sensor of the imaging instrument 102.4-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCTAs such, the imaging instrument 102 may be used to perform assays on various ty pes of biomedical samples, including identifying cells in some samples.

[0019] The imaging instrument 102 can be used to capture a magnified image of a target object 104. As discussed above, the target object 104 may be a biological or biomedical sample. For example, a target object can include a tissue sample, including a tissue sample taken from a subject. A target object can include fresh-frozen (FF) tissue, such as a fresh- frozen biopsy sample or a surgical sample taken from a subject. A target object can include formalin-fixed paraffin-embedded (FFPE) tissue, such as a biopsy sample or a surgical sample. In some situations, a tissue can be stored for some time (months or years) prior to use. A target object can include a section taken from a sample and positioned on a microscope slide using standard techniques in histochemistry and pathology laboratories. A target object can include cultured cells, such as a cell culture or a cell culture pellet. Cultured cells can be FF or FFPE preserved, depending on the embodiment. In some embodiments, a target object can include a fresh cell culture.

[0020] In an example, the target object 104 is a high-contrast target. A high contrast target may be used to ensure optical distortions can be easily detected and measured, which enables calibration of the imaging instrument. In some examples, the high contrast imaging target includes fluorescent diffraction-limited polysty rene beads. In some examples, these beads are approximately 200 pm in diameter and are impregnated with a variety of fluorescent dyes which allows them to fluoresce strongly in the excitation / emission pairs used in the microscope. In some implementations, the fluorescent diffraction-limited polystyrene beads are deposited randomly on a microscope slide at a high densify (e.g., a densify of about 8000-12000 beads per square millimeter). Such a densify provides sufficient contrast and coverage of the field while reducing the error rate in the algorithms used for feature matching, as discussed in more detail below. Further, the target object could be any fluorescently7labelled object with intensify7patterns, such as a tissue sample, a cell pellet array, or a tissue microarray.

[0021] Once the target object 104 is positioned at a desired location (e.g., positioned on a microscope slide and the microscope slide is placed on a microscope stage), a first image of the target object 104 is captured and collected by the imaging instrument 102. After the first image is captured, the actuator 106 is used to displace the target object 104 a known distance with respect to the imaging instrument 102. In some implementations, the actuator 106 is a highly accurate linear actuator that offers high precision. The actuator 106 may include a high-resolution encoder to ensure precise control of speed and / or position of the target object5-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT104. Furthermore, the actuator 106 may include a stage for positioning and moving the target object 104. In an example, the actuator 106 is a two-axis linear positioning stage actuator device. This linear positioning device may provide a three-phase pulse-width modulation (PWM) drive and, in some implementations, is capable of micro accuracy (e.g., 10-16 pm accuracy) and an encoder with nano precision accuracy (e.g., nm). The actuator stage is governed by an encoder feedback loop and controlled by an on-board high-performance controller, which results in the actuator’s high accuracy and in-position stability. A technical benefit of using such an actuator is that it provides the level of performance and accessibility needed for standardization of magnification.

[0022] Once the actuator 106 moves the target object 104 a desired distance in a predetermined direction, the imaging instrument 102 is used to capture another image of the target object 104. This process is repeated iteratively a predetermined number of times until a desired number of images have been captured. In some implementations, the actuator stage is moved the same specific distance each time the actuator moves the target object 104. In an example, the stage is moved a distance that is approximately one-third of the objective FOV diameter of the imaging instrument (e.g., 250 pm for a microscope with a field diameter of 720 pm) during the iterative imaging and translation process. These large steps reduce errors in the final distortion measurement by minimizing the localization and encoder precision relative to the accuracy of the stage during the large step size. The accuracy of the stage is improved by lost motion compensation, which eliminates small errors due to slop and deformation of the mechanical components. Each time the stage is moved, the imaging instrument 102 is used to capture an image of the target object 104. The required number of images may vary based on the type of imaging instrument used, the type of target object used, the level of contrast the target object provides, the level of accuracy desired and the like. In an example, three images are acquired, and the stage is moved twice, once in the x-direction and then in the y-direction.

[0023] The captured images, along with information about the coordinates of the target object 104, and the distance the target object 104 was moved each time are transmitted from the imaging instrument 102 to the distortion compensation system 108. In some implementations, the images are transmitted from the imaging instrument 102 and the coordinate / movement data is transmitted via the actuator 106.

[0024] In some implementations, the distortion compensation system 108 includes a distortion measurement engine 110, a calibration datastore 112. a calibration engine 114. and a processing engine 116. The distortion measurement engine 110 uses various mechanisms6-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT for measuring the amount of distortion in the captured images. In an implementation, the distortion measurement engine 110 utilizes feature matching, spot localization and / or nonlinear least square optimization techniques to measure the amount of distortion. This may be achieved by, first, template matching large portions (e.g. large image patches) of subsequently captured images to one another based on the known displacement distance of the target object and / or based on the nominal magnification of the imaging instrument 102. Next, a correspondence between the matched large portions of the subsequent images is refined by template matching smaller portions of the images to provide full-field information about the distorted image coordinates. Additionally, a direct correspondence between identical spots in the images is made by using a spot detection algorithm. The spot detection algorithm may first identify various spots in each image, and then determine which spot in each image corresponds to a similar spot in the next image.

[0025] The template matching information as well as the identified matching spots are then used to measure the amount of distortion in the images. That is because the actual distance between identical spots in two subsequent images is known based on the distance the actuator moved the stage. The distance between two identical spots in the subsequently captured images can also be measured based on the template matching and spot detection information. The X and Y-positions in the image (and therefore the imaging instrument sensor) are then transformed according to an initialized form of an analytical distortion model, thus providing estimated world-coordinates for undistorted spots. The distance between the distorted spots is then compared to the known displacement distance in worldcoordinates to help determine the amount of optical distortion. This information may be provided by the distortion measurement engine 110 to the calibration engine 114. In some implementations, the distortion measurement engine 110 calculates distortion based on the following equations.In equation (1), Kodenotes the measured magnification at the center of the image, while Krdenotes the rate of magnification change with field. Additionally. rdrepresents the radius of one feature of the distorted image, measured from the optical axis, and rurepresents the radius of the same feature in the undistorted version of the same image. X and Y denote the X and Y coordinate positions in the image, and (xc, yc) represents the optical center coordinates of the image. By using the above equations, the distortion term can be calculated for use in7-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT calibration of the images. In some implementations, the initial estimates generated by this analytical distortion model is iteratively refined until it provides a transformation of the distorted image into a rectilinear space with magnification informed by the known move size of the target when the error is minimized.

[0026] Previously used distortion models do not account for the Koparameter. As a result, these models are unable to properly standardize magnification used for metrology or perform image stitching. The distortion measurement engine 1 10 addresses these technical problems as it allows for an analytical inversion of the transformation, generating simplified and efficient mechanisms for determining both the distortion transformation and the un-distortion transformation. This provides the technical benefit of being empirically more appropriate for measurement of distortion in the field of spatial biology and provides improved feature recombination at the edges of the field.

[0027] In some implementations, the measured distortion parameters are transmitted from the distortion measurement engine 110 to the calibration datastore 112 for storage and future use. The calibration datastore 112 is a persistent datastore in the memory of the distortion compensation system 108 that stores calibration data as it relates to the imaging instrument 102. The processing engine 116 then utilizes the measured distortion parameters retrieved from the calibration datastore 112 to determine how the X and Y positions in an image captured by the imaging instrument 102 should be transformed to minimize distortion. In some implementations, the processing engine 116 utilizes a standard image transformation model such as a nonlinear image transformation algorithm (e.g., image warping algorithm) to compensate for distortion and generate undistorted images. The undistorted images are then stitched together to generate a modified image. Moreover, the processing engine 116 utilizes the undistorted images to duplicate features at the edges of the field to ensure accuracy at the edges.

[0028] Once the image is processed to compensate for distortion and perform any other desired processing to generate an improved image, the distortion compensation system 108 provides the improved image 118 as an output. The image 118 may be provided to imaging instrument 102 for analysis and / or display to a user. In some implementations, each time a target object is being examined vis the imaging instrument 102, the actuator 106 is employed to take multiple images which are then analyzed via the distortion compensation system 108 to determine how- to properly compensate for any distortion in the images. In other implementations, the actuator 106 is used to facilitate imaging of one or more sample target objects and calibration parameters for the imaging instrument 102 are calculated based on the8-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT sample target objects, such that future imaging does not require the use of the actuator and / or multiple captured images to compensate for distortion.

[0029] FIG. 2 depicts multiple example images of the same target object being captured by an imaging instrument when the target object is displaced by an actuator. Image 202 in FIG. 2 represents an image which displays multiple spots when an object is located at an initial position with X and Y coordinates of zero. Image 204 depicts an image of the same target object, when the target object is moved in the vertical direction (Y direction) a known distance. In this case, the X and Y coordinates change to X = Axand Y = Ar, where Axand Arrepresent the measured amount of movement in the horizontal and vertical directions. The magnitude of Axis typically non-zero due to distortion, even though the physical movement is only along the Y axis. Image 206 represents an image of the same target object, when the target object is moved in the horizontal direction (X direction) a known distance. For image 206, the X and Y coordinates change to X = Ax', Y = A / , where Ax' and Ay' represent the amount of movement in the horizontal and vertical directions. Similar to the case with vertical movement, horizontal movement will typically result in a non-zero j, due to distortion. As can be seen, when the target image moves in a direction, the image features (e.g., the spots in each image) move a corresponding distance. In an undistorted image, the changes in the X and Y coordinates of each spot in a subsequent image correspond to the amount of physical movement of the target object (e.g., Axor Ar). However, when the image is distorted, this change will not be as expected. Furthermore, the change may not be uniform for all of the image portions. For example, the areas closer to the edges of the images may experience more distortion and as such larger changes in coordinates.

[0030] FIG. 3 depicts an example feature matching between features of example images of the same target object when the target object is moved by an actuator. FIG. 3 depicts an example where the images 202, 204 and 206 are overlayed on each other so that features in each image can be matched with features in the other images. The features may be matched based on template matching and spot detection techniques, as discussed above. Once the features are detected and matched, displacement vectors between matching features are measured. For example, distance 302 represents the distance betw een the same feature in images 202 and 206 (where the feature moves in the horizontal direction), while distance 304 represents the distance between the same feature in images 202 and 204, where the feature moves in the vertical direction. By measuring the displacement vector betw een matching9-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT features in the multiple images, an amount of distortion for each feature in the distorted images can be determined which can then be used to determine the distortion in each image.

[0031] FIG. 4 depicts diagrams of example distorted and undistorted images. The element 402 in FIG. 4 depicts a physical grid having an image area 410. The element 402 represents an actual depiction of a biomedical target within a physical grid. The biomedical target includes various reporter spots 412, 414, 416 and 418, as well as biological cells 420 and 422. As can be seen, the reporter spot 418 and cell 420 are positioned outside the grid area 410 of the target object. Element 404 represents an image of the target object captured via an imaging instrument, such as the imaging instrument 102 of FIG. 1. The image in element 404 is a distorted image, which may be caused by the magnification of the imaging instrument. As a result of the distortion, the cell 420 and reporter 418 are depicted within the grid area 424 of the image. Thus, the distorted image displays a duplicate cell and a duplicate reporter spot, resulting in an inaccurate image and as such potentially inaccurate analysis of the target object. This ty pe of optical distortion can lead to duplication or misattribution of biological transcripts which can in turn greatly affect cell classification and tissue organization. Such duplication or misattribution can result in errors in interpretation of disease etiology and mechanism.

[0032] Element 406 represents an image of the target object after the distorted image shown in element 404 has been compensated for optical distortion in accordance with the techniques disclosed herein. As shown, the modified image does not include any duplicated features, thus providing a more accurate representation of the target object.

[0033] FIG. 5 is a flow chart of an example process 500 for measurement and compensation for optical distortion according to the techniques disclosed herein. The process 500 can be implemented by the system 100 of FIG. 1 discussed in the preceding examples.

[0034] The process 500 begins by capturing an image of a target object via an imaging instrument, at step 502. As discussed above, the imaging instrument may be a microscope such as an epifluorescence wi defield microscope and the target object may be a high contrast imaging target consisting of fluorescent diffraction-limited polystyrene beads. After an initial image of the target has been captured, process 500 proceeds to move the target object a known distance via an actuator, at step 504. The actuator may be a linear actuator having a high-resolution encoder. In an example, the actuator includes a two-axis stage that can move the target object in two directions such as the horizontal and vertical directions. The actuator may move the target object with high precision such as the distance the target object travels is highly accurate.10-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT

[0035] After the target object has been moved, another image of the target object is captured via the imagining instrument, at step 506. The process of moving the target object a known distance via the actuator and capturing another image of the target object after the target object has been moved is then repeated a predetermined number of times until a plurality of images of the target object have been captured, at step 508. In some implementations, the predetermined number of times is three times, and the plurality of images includes the initial image, a second image captured after the target object is moved a known distance in the horizontal direction and a third image captured after the target object is moved a know n distance in the vertical direction.

[0036] Once a desired number of images have been captured, a correspondence is established between a feature of the first image and a corresponding feature of each of the other images, via a distortion measurement engine, to identity’ corresponding features in consecutive images, at step 510. This may be achieved via one or more of feature matching, spot localization and nonlinear least square optimization techniques. Next, the process 500 proceeds to measuring a distance between each two identified corresponding features via the distortion measurement engine, at step 512.

[0037] The measured distance between each two identified corresponding features is then utilized to measure parameters for optical distortion of the imaging instrument, at step 514. Once the parameters for optical distortion have been measured, they can be used for future optical distortion compensation in one or more images captured by the imaging instrument. Thus, the measured parameter for optical distortion is utilized to modify the first image to compensate for optical distortion, at step 516. The modified image is then provided as an output, at step 518.

[0038] The detailed examples of systems, devices, and techniques described in connection with FIGS. 1-7 are presented herein for illustration of the disclosure and its benefits. Such examples of use should not be construed to be limitations on the logical process embodiments of the disclosure, nor should variations of user interface methods from those described herein be considered outside the scope of the present disclosure. It is understood that references to displaying or presenting an item (such as. but not limited to, presenting an image on a display device, presenting audio via one or more loudspeakers, and / or vibrating a device) include issuing instructions, commands, and / or signals causing, or reasonably expected to cause, a device or system to display or present the item. In some embodiments, various features described in FIGS. 1-7 are implemented in respective modules, which may also be referred to as, and / or include, logic, components, units, and / or11-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT mechanisms. Modules may constitute either software modules (for example, code embodied on a machine-readable medium) or hardware modules.

[0039] In some examples, a hardware module may be implemented mechanically, electronically, or with any suitable combination thereof. For example, a hardware module may include dedicated circuitry or logic that is configured to perform certain operations. For example, a hardware module may include a special-purpose processor, such as a field- programmable gate array (FPGA) or an Application Specific Integrated Circuit (ASIC). A hardware module may also include programmable logic or circuitry that is temporarily configured by software to perform certain operations and may include a portion of machine- readable medium data and / or instructions for such configuration. For example, a hardware module may include software encompassed within a programmable processor configured to execute a set of software instructions. It will be appreciated that the decision to implement a hardware module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (for example, configured by software) may be driven bycost, time, support, and engineering considerations.

[0040] Accordingly, the phrase “hardware module” should be understood to encompass a tangible entity capable of performing certain operations and may be configured or arranged in a certain physical manner, be that an entity7that is physically constructed, permanently configured (for example, hardwired), and / or temporarily configured (for example, programmed) to operate in a certain manner or to perform certain operations described herein. As used herein, “hardware-implemented module” refers to a hardware module. Considering examples in which hardware modules are temporarily configured (for example, programmed), each of the hardware modules need not be configured or instantiated at anyone instance in time. For example, where a hardware module includes a programmable processor configured by software to become a special-purpose processor, the programmable processor may be configured as respectively different special-purpose processors (for example, including different hardware modules) at different times. Software may accordingly configure a processor or processors, for example, to constitute a particular hardware module at one instance of time and to constitute a different hardware module at a different instance of time. A hardware module implemented using one or more processors may be referred to as being “processor implemented” or “computer implemented.”

[0041] Hardware modules can provide information to, and receive information from, other hardware modules. Accordingly, the described hardware modules may be regarded as being communicatively coupled. Where multiple hardware modules exist12-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT contemporaneously, communications may be achieved through signal transmission (for example, over appropriate circuits and buses) between or among two or more of the hardware modules. In embodiments in which multiple hardware modules are configured or instantiated at different times, communications between such hardware modules may be achieved, for example, through the storage and retrieval of information in memory devices to which the multiple hardware modules have access. For example, one hardware module may perform an operation and store the output in a memory device, and another hardware module may then access the memory device to retrieve and process the stored output.

[0042] In some examples, at least some of the operations of a method may be performed by one or more processors or processor-implemented modules. Moreover, the one or more processors may also operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS). For example, at least some of the operations may be performed by, and / or among, multiple computers (as examples of machines including processors), with these operations being accessible via a network (for example, the Internet) and / or via one or more software interfaces (for example, an application program interface (API)). The performance of certain of the operations may be distributed among the processors, not only residing within a single machine, but deployed across several machines. Processors or processor-implemented modules may be in a single geographic location (for example, within a home or office environment, or a server farm), or may be distributed across multiple geographic locations.

[0043] FIG. 6 is a block diagram 600 illustrating an example software architecture 602, various portions of which may be used in conjunction with various hardware architectures herein described, which may implement any of the above-described features. FIG. 6 is a nonlimiting example of a software architecture, and it will be appreciated that many other architectures may be implemented to facilitate the functionality described herein. The software architecture 602 may execute on hardware such as a machine 700 of FIG. 7 that includes, among other things, processors 710, memory' 730, and input / output (I / O) components 750. A representative hardware layer 604 is illustrated and can represent, for example, the machine 700 of FIG. 7. The representative hardware layer 604 includes a processing unit 606 and associated executable instructions 608. The executable instructions 608 represent executable instructions of the software architecture 602, including implementation of the methods, modules and so forth described herein. The hardware layer 604 also includes a memor / storage 610, which also includes the executable instructions 608 and accompanying data. The hardware layer 604 may also include other hardware modules13-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT612. Instructions 608 held by processing unit 606 may be portions of instructions 608 held by the memory / storage 610.

[0044] The example software architecture 602 may be conceptualized as layers, each providing various functionality. For example, the software architecture 602 may include layers and components such as an operating system (OS) 614, libraries 616, frameworks / middleware 618, applications 620, and a presentation layer 644. Operationally, the applications 620 and / or other components within the layers may invoke API calls 624 to other layers and receive corresponding results 626. The layers illustrated are representative in nature and other software architectures may include additional or different layers. For example, some mobile or special purpose operating systems may not provide the frameworks / middleware 618.

[0045] The OS 614 may manage hardware resources and provide common services. The OS 614 may include, for example, a kernel 628, services 630, and drivers 632. The kernel 628 may act as an abstraction layer between the hardware layer 604 and other software layers. For example, the kernel 628 may be responsible for memory management, processor management (for example, scheduling), component management, networking, security settings, and so on. The services 630 may provide other common services for the other software layers. The drivers 632 may be responsible for controlling or interfacing with the underlying hardware layer 604. For instance, the drivers 632 may include display drivers, camera drivers, memory / storage drivers, peripheral device drivers (for example, via Universal Serial Bus (USB)), network and / or wireless communication drivers, audio drivers, and so forth depending on the hardware and / or software configuration.

[0046] The libraries 616 may provide a common infrastructure that may be used by the applications 620 and / or other components and / or layers. The libraries 616 typically provide functionality for use by other software modules to perform tasks, rather than interacting directly with the OS 614. The libraries 616 may include system libraries 634 (for example, C standard library) that may provide functions such as memory allocation, string manipulation, file operations. In addition, the libraries 616 may include API libraries 636 such as media libraries (for example, supporting presentation and manipulation of image, sound, and / or video data formats), graphics libraries (for example, an OpenGL library for rendering 2D and 3D graphics on a display), database libraries (for example, SQLite or other relational database functions), and web libraries (for example, WebKit that may provide web browsing functionality). The libraries 616 may also include a wide variety of other libraries 638 to provide many functions for applications 620 and other software modules.14-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT

[0047] The frameworks / middleware 618 provide a higher-level common infrastructure that may be used by the applications 620 and / or other software modules. For example, the frameworks / middleware 618 may provide various graphic user interface (GUI) functions, high-level resource management, or high-level location services. The frameworks / middleware 618 may provide a broad spectrum of other APIs for applications 620 and / or other software modules.

[0048] The applications 620 include built-in applications 640 and / or third-party applications 642. Examples of built-in applications 640 may include, but are not limited to, a contacts application, a browser application, a location application, a media application, a messaging application, and / or a game application. Third-party applications 642 may include any applications developed by an entity other than the vendor of the particular platform. The applications 620 may use functions available via OS 614, libraries 616, frameworks / middleware 618, and presentation layer 644 to create user interfaces to interact with users.

[0049] Some software architectures use virtual machines, as illustrated by a virtual machine 648. The virtual machine 648 provides an execution environment where applications / modules can execute as if they were executing on a hardware machine (such as the machine 700 of FIG. 7, for example). The virtual machine 648 may be hosted by a host OS (for example, OS 614) or hypervisor, and may have a virtual machine monitor 646 which manages operation of the virtual machine 648 and interoperation with the host operating system. A software architecture, which may be different from software architecture 602 outside of the virtual machine, executes within the virtual machine 648 such as an OS 650, libraries 652, frameworks 654. applications 656, and / or a presentation layer 658.

[0050] FIG. 7 is a block diagram illustrating components of an example machine 700 configured to read instructions from a machine-readable medium (for example, a machine- readable storage medium) and perform any of the features described herein. The example machine 700 is in a form of a computer system, within which instructions 716 (for example, in the form of software components) for causing the machine 700 to perform any of the features descnbed herein may be executed. As such, the instructions 716 may be used to implement modules or components described herein. The instructions 716 cause unprogrammed and / or unconfigured machine 700 to operate as a particular machine configured to carry out the described features. The machine 700 may be configured to operate as a standalone device or may be coupled (for example, networked) to other machines. In a networked deployment, the machine 700 may operate in the capacity of a15-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT server machine or a client machine in a server-client network environment, or as a node in a peer-to-peer or distributed network environment. Machine 700 may be embodied as, for example, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a gaming and / or entertainment system, a smart phone, a mobile device, a wearable device (for example, a smart watch), and an Internet of Things (loT) device. Further, although only a single machine 700 is illustrated, the term ‘"machine” includes a collection of machines that individually or jointly execute the instructions 716.

[0051] The machine 700 may include processors 710, memory' 730, and I / O components 750, which may be communicatively coupled via, for example, a bus 702. The bus 702 may include multiple buses coupling various elements of machine 700 via various bus technologies and protocols. In an example, the processors 710 (including, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, or a suitable combination thereof) may include one or more processors 712a to 712n that may execute the instructions 716 and process data. In some examples, one or more processors 710 may execute instructions provided or identified by one or more other processors 710. The term ’‘processor” includes a multicore processor including cores that may execute instructions contemporaneously. Although FIG. 7 shows multiple processors, the machine 700 may include a single processor with a single core, a single processor with multiple cores (for example, a multicore processor), multiple processors each with a single core, multiple processors each with multiple cores, or any combination thereof. In some examples, the machine 700 may include multiple processors distributed among multiple machines.

[0052] The memory / storage 730 may include a main memory 732, a static memory’ 734, or other memory, and a storage unit 736, both accessible to the processors 710 such as via the bus 702. The storage unit 736 and memory 732, 734 store instructions 716 embodying any one or more of the functions described herein. The memory / storage 730 may also store temporary, intermediate, and / or long-term data for processors 710. The instructions 716 may also reside, completely or partially, within the memory 732, 734, within the storage unit 736, within at least one of the processors 710 (for example, within a command buffer or cache memory'), within memory' at least one of I / O components 750, or any suitable combination thereof, during execution thereof. Accordingly, the memory' 732. 734, the storage unit 736, memory in processors 710, and memory in I / O components 750 are examples of machine- readable media.16-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT

[0053] As used herein, “machine-readable medium"’ refers to a device able to temporarily or permanently store instructions and data that cause machine 700 to operate in a specific fashion, and may include, but is not limited to, random-access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical storage media, magnetic storage media and devices, cache memory, network-accessible or cloud storage, other types of storage and / or any suitable combination thereof. The term “machine-readable medium” applies to a single medium, or combination of multiple media, used to store instructions (for example, instructions 716) for execution by a machine 700 such that the instructions, when executed by one or more processors 710 of the machine 700, cause the machine 700 to perform and one or more of the features described herein. Accordingly, a “machine-readable medium” may refer to a single storage device, as well as “cloud-based” storage systems or storage networks that include multiple storage apparatus or devices. The term “machine- readable medium” excludes signals per se.

[0054] The I / O components 750 may include a wide variety of hardware components adapted to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I / O components 750 included in a particular machine will depend on the type and / or function of the machine. For example, mobile devices such as mobile phones may include a touch input device, whereas a headless server or loT device may not include such a touch input device. The particular examples of I / O components illustrated in FIG. 7 are in no way limiting, and other types of components may be included in machine 700. The grouping of I / O components 750 are merely for simplifying this discussion, and the grouping is in no way limiting. In various examples, the I / O components 750 may include user output components 752 and user input components 754. User output components 752 may include, for example, display components for displaying information (for example, a liquid crystal display (LCD) or a projector), acoustic components (for example, speakers), haptic components (for example, a vibratory motor or force-feedback device), and / or other signal generators. User input components 754 may include, for example, alphanumeric input components (for example, a keyboard or a touch screen), pointing components (for example, a mouse device, a touchpad, or another pointing instrument), and / or tactile input components (for example, a physical button or a touch screen that provides location and / or force of touches or touch gestures) configured for receiving various user inputs, such as user commands and / or selections.

[0055] In some examples, the I / O components 750 may include biometric components 756, motion components 758, environmental components 760, and / or position components17-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT762, among a wide array of other physical sensor components. The biometric components 756 may include, for example, components to detect body expressions (for example, facial expressions, vocal expressions, hand or body gestures, or eye tracking), measure biosignals (for example, heart rate or brain waves), and identify a person (for example, via voice-, retina-, fingerprint-, and / or facial-based identification). The motion components 758 may include, for example, acceleration sensors (for example, an accelerometer) and rotation sensors (for example, a gyroscope). The environmental components 760 may include, for example, illumination sensors, temperature sensors, humidity sensors, pressure sensors (for example, a barometer), acoustic sensors (for example, a microphone used to detect ambient noise), proximity sensors (for example, infrared sensing of nearby objects), and / or other components that may provide indications, measurements, or signals corresponding to a surrounding physical environment. The position components 762 may include, for example, location sensors (for example, a Global Position System (GPS) receiver), altitude sensors (for example, an air pressure sensor from which altitude may be derived), and / or orientation sensors (for example, magnetometers).

[0056] The I / O components 750 may include communication components 764. implementing a wide variety of technologies operable to couple the machine 700 to network(s) 770 and / or device(s) 780 via respective communicative couplings 772 and 782. The communication components 764 may include one or more network interface components or other suitable devices to interface with the network(s) 770. The communication components 764 may include, for example, components adapted to provide wired communication, wireless communication, cellular communication, Near Field Communication (NFC), Bluetooth communication, Wi-Fi, and / or communication via other modalities. The device(s) 780 may include other machines or various peripheral devices (for example, coupled via USB).

[0057] In some examples, the communication components 764 may detect identifiers or include components adapted to detect identifiers. For example, the communication components 764 may include Radio Frequency Identification (RFID) tag readers, NFC detectors, optical sensors (for example, one- or multi-dimensional bar codes, or other optical codes), and / or acoustic detectors (for example, microphones to identify tagged audio signals). In some examples, location information may be determined based on information from the communication components 764, such as, but not limited to, geo-location via Internet Protocol (IP) address, location via Wi-Fi, cellular. NFC, Bluetooth, or other wireless station identification and / or signal triangulation.18-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT

[0058] FIGS. 8-10 show successful results from the distortion measurement procedures disclosed herein.

[0059] Specifically, FIG. 8 shows results indicating consistent measurements for an individual instrument. In this case, the instrument used is a CosMx-0247 for a fiducial (FID) slide. As can be seen from the highlighted readings in FIG. 8, a very7low coefficient of variation (CV) is obtained for average Ko and Ki.

[0060] FIG. 9 shows comparable results obtained from internal instruments. Again, a low CV is obtained for consistent Ko and Ki, as shown in the highlighted regions of FIG. 9. The upper portion of FIG. 9 shows the average of all tested internal instruments. The middle portion of FIG. 9 shows the average for production instruments. Finally, the lower portion of FIG. 9 shows the instrument results summary.

[0061] FIG 10 shows nearly identical calibration results obtained from tissue slides and fiducial slides. This can be seen by comparing the highlighted results for the Beta04 FID slide and the Beta04 tissue slide in FIG. 10.

[0062] In the preceding detailed description, numerous specific details are set forth by way of examples in order to provide a thorough understanding of the relevant teachings. However, it should be apparent that the present teachings may be practiced without such details. In other instances, well known methods, procedures, components, and / or circuitry’ have been described at a relatively high-level, without detail, in order to avoid unnecessarily obscuring aspects of the present teachings.

[0063] While various embodiments have been described, the description is intended to be exemplary7, rather than limiting, and it is understood that many more embodiments and implementations are possible that are within the scope of the embodiments. Although many possible combinations of features are shown in the accompanying figures and discussed in this detailed description, many other combinations of the disclosed features are possible.Any feature of any7embodiment may be used in combination w7ith or substituted for any other feature or element in any other embodiment unless specifically restricted. Therefore, it will be understood that any of the features shown and / or discussed in the present disclosure may be implemented together in any suitable combination. Accordingly, the embodiments are not to be restricted except in light of the attached claims and their equivalents. Also, various modifications and changes may be made within the scope of the attached claims.

[0064] While the foregoing has described what are considered to be the best mode and / or other examples, it is understood that various modifications may be made therein and that the subject matter disclosed herein may be implemented in various forms and examples, and that19-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT the teachings may be applied in numerous applications, only some of which have been described herein. It is intended by the following claims to claim any and all applications, modifications and variations that fall within the true scope of the present teachings.

[0065] Unless otherwise stated, all measurements, values, ratings, positions, magnitudes, sizes, and other specifications that are set forth in this specification, including in the claims that follow, are approximate, not exact. They are intended to have a reasonable range that is consistent with the functions to which they relate and with what is customary in the art to which they pertain.

[0066] The scope of protection is limited solely by the claims that now follow. That scope is intended and should be interpreted to be as broad as is consistent with the ordinary' meaning of the language that is used in the claims when interpreted in light of this specification and the prosecution history that follows and to encompass all structural and functional equivalents. Notwithstanding, none of the claims are intended to embrace subject matter that fails to satisfy the requirement of Sections 101, 102, or 103 of the Patent Act, nor should they be interpreted in such a way. Any unintended embracement of such subject matter is hereby disclaimed.

[0067] Except as stated immediately above, nothing that has been stated or illustrated is intended or should be interpreted to cause a dedication of any component, step, feature, object, benefit, advantage, or equivalent to the public, regardless of whether it is or is not recited in the claims.

[0068] It will be understood that the terms and expressions used herein have the ordinary meaning as is accorded to such terms and expressions with respect to their corresponding respective areas of inquiry and study except where specific meanings have otherwise been set forth herein. Relational terms such as first and second and the like may be used solely to distinguish one entity or action from another without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprises,” “comprising,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by “a” or “an” does not, without further constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element. Furthermore, subsequent limitations referring back to “said element” or “the element” performing certain functions signifies that “said element” or “the element” alone or in combination with additional20-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT identical elements in the process, method, article, or apparatus are capable of performing all of the recited functions.

[0069] The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various examples for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed example. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.21-Bruker Confidential-

Claims

PATENT NovoTecliIP Docket No. 240613A-002PCTCLAIMS:

1. A system for measurement and compensation of optical distortion comprising: an imaging instrument; an actuator; a processor; and a memory storing executable instructions that, when executed, cause the processor alone or in combination with other processors to perform operations of: capturing a first image of a target object via the imaging instrument; moving the target object a known distance, via the actuator; capturing a second image of the target object via the imaging instrument, after the target object has been moved the known distance; repeating a process of moving the target object the known distance via the actuator and capturing at least one additional image of the target object after the target object has been moved a predetermined number of times until a plurality of images, including the first image and other images, including the second image and the at least one additional image, of the target object have been captured; establishing correspondence betw een a feature of the first image and a corresponding feature of each of the other images, via a distortion measurement engine to identify corresponding features in consecutive images; measuring pixel coordinates of each pair of identified corresponding features via the distortion measurement engine; utilizing a displacement vector between each pair of identified corresponding features to measure parameters for optical distortion of the imaging instrument: utilizing the measured parameters for optical distortion to modify the first image to compensate for optical distortion; and providing the modified image as an output of the imaging instrument.

2. The system of claim 1, wherein the target object is a biomedical object.

3. The system of claim 1, wherein the target object is a high contrast object.

4. The system of claim 1, wherein the imaging instrument is a microscope.22-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT5. The system of claim 1, wherein the actuator is a linear actuator having a high-resolution encoder.

6. The system of claim 5, wherein the actuator includes a two-axis stage that can move the target object in orthogonal directions.

7. The system of claim 1, wherein a two-axis stage is operated via an encoder feedback loop which is controlled using an on-board controller.

8. The system of claim 7, wherein the predetermined number of times is three and the plurality of images includes the first image, a second image captured after the target object is moved the known distance in a horizontal direction and a third image captured after the target object is moved the known distance in a vertical direction.

9. The system of claim 1, wherein the distortion measurement engine utilizes one or more of feature matching, spot localization and nonlinear least square optimization techniques to measure the displacement vector between each pair of identified corresponding features.

10. The system of claim 1, wherein utilizing the measured parameter for optical distortion to modify the first image to compensate for optical distortion comprises applying an image warping technique to the first image.

11. The system of claim 1 , wherein each of the plurality' of images are modified to compensate for optical distortion and the modified plurality of images are used to generate an output image.

12. A method for measurement and compensation of optical distortion in spatial biology- implemented in a system, the method comprising: capturing a first image of a target object via an imaging instrument; moving the target object a known distance, via an actuator; capturing a second image of the target object, via the imaging instrument, after the target object has been moved via the actuator; repeating a process of moving the target object the known distance via the actuator and capturing at least one additional image of the target object after the target23-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT object has been moved a predetermined number of times until a plurality of images, including the first image and other images, including the second image and the at least one additional image, of the target object have been captured; establishing correspondence between a feature of the first image and a corresponding feature of each of other images, via a distortion measurement engine to identify corresponding features in consecutive images; measuring pixel coordinates of each pair of identified corresponding features via the distortion measurement engine; utilizing a displacement vector between each pair of identified corresponding features to measure parameters for optical distortion of the imaging instrument: utilizing the measured parameters for optical distortion to modify the first image to compensate for optical distortion; and providing the modified image as an output.

13. The method of claim 12, wherein the imaging instrument is an epifluorescence wi defield microscope.

14. The method of claim 12, wherein the target object comprises fluorescent diffractionlimited polystyrene beads.

15. The method of claim 14, wherein the fluorescent diffraction-limited polystyrene beads are impregnated with a plurality of fluorescent dyes.

16. The method of claim 14, wherein the fluorescent diffraction-limited polystyrene beads are deposited randomly on a microscope slide at a known density to generate the target object.

17. The method of claim 12, wherein the target object is a fluorescently labelled object with intensity patterns.

18. The method of claim 17, wherein the fluorescently labelled object is a tissue sample, a cell pellet array, or a tissue microarray.24-Bruker Confidential-PATENT NovoTecliIP Docket No. 240613A-002PCT19. The method of claim 12, further comprising storing the parameters for optical distortion of the imaging instrument in a calibration data store for use in future calibration of images captured by the imaging instrument.

20. Anon-transitory computer readable medium on which are stored instructions that, when executed, cause a programmable device to perform functions of: accessing a first image of a target object captured via an imaging instrument; accessing other images of the target object captured after the target object is moved a known distance, via an actuator; establishing a correspondence betw een a feature of the first image and a corresponding feature of each of the other images, via a distortion measurement engine to identify corresponding features in consecutive images; measuring pixel coordinates of each pair of identified corresponding features via the distortion measurement engine; utilizing a displacement vector between each pair of identified corresponding features to measure parameters for optical distortion of the imaging instrument; utilizing the measured parameter for optical distortion to modify the first image to compensate for optical distortion; and providing the modified image as an output.21 . The non-transitory computer readable medium of claim 20, wherein the actuator is a linear actuator having a high-resolution encoder.

22. The non-transitory computer readable medium of claim 20. wherein the imaging instrument is a widelield microscope.25-Bruker Confidential-

Images