Systems and methods for imaging a sample with reduced sample motion artifacts

Abstract

Systems and methods of identifying and / or reducing or eliminating sample motion artifacts are disclosed. Sample motion artifacts can be reduced or eliminated using a scan pattern that reduces the difference in acquisition time between acquisition of the peripheral pixels in adjacent tiles compared to a conventional raster scan, thereby reducing or eliminating discontinuities that would otherwise occur at tile boundaries in the image. In some embodiments, the intensity of test images acquired using a relatively small test scan pattern or test points acquired at different times can be compared to determine whether sample motion has occurred. In some embodiments, the intensity of adjacent pixels at a tile boundary are compared. In some embodiments, the intensity of one or more single pixels is monitored over time to determine whether sample motion has occurred over a period of time. In some embodiments, a flattening or reshaping tool can be used during imaging to suppress sample motion.

Description

[0001] Priority application

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 975,742, filed on February 12, 2020, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure generally relates to systems, methods, and tools for imaging samples (e.g., biological tissues). In some embodiments, the systems, methods, and / or tools are used to reduce or eliminate artifacts from sample motion. Background Technology

[0004] Microscopic examination of biological tissue samples often involves extensive tissue preparation steps, including tissue fixation, paraffin embedding, sectioning to a few micrometers thick, and mounting on a glass slide. Therefore, the thin tissue sections in the resulting “slide” are completely immobile and further maintained by the rigidity of the slide. Thus, even though slide imaging takes time and involves moving the slide around to take images at various locations to reconstruct a larger mosaic, with or without splicing, there are never any problems related to unwanted movement of the tissue sample during imaging. Some systems allow “slide-free” tissue microscopy of samples. These tissue samples can even be unfixed and unthawed (e.g., fresh tissue samples removed during surgery). In some techniques that do not use such fixation procedures, the sample may move during imaging (e.g., relax) for example, due to its size and / or weight. Sample movement can be noticeable during image acquisition. Therefore, sample movement during imaging can lead to undesirable imaging artifacts.

[0005] An imaging system used for acquiring images can be a continuous imaging system. For example, an imaging system might acquire an image of one patch before acquiring an image of another. Imaging a moving sample with such an imaging system may produce images containing sample motion artifacts, which will manifest as discontinuities between patches. Summary of the Invention

[0006] While small sample motion artifacts in an image can be annoying to an image observer, larger sample motion artifacts can interfere with (e.g., a physician or computer algorithm) the interpretation of the image. For example, a fresh breast lump placed on a flat imaging window of an imaging system may take several minutes to fully relax and stop moving (on a microscopic scale). At this time frame, sample motion may initially be large (thus producing large sample motion artifacts) and eventually small (thus producing small sample motion artifacts), eventually becoming imperceptible relative to the resolution of the image being acquired.

[0007] This disclosure particularly provides systems and methods for reducing or eliminating image artifacts that might otherwise be present in an image due to sample motion occurring during imaging. The system may include and / or the method may utilize an array of micro-optical elements (e.g., one or more of refractive lenses, Fresnel zone plates, reflecting objectives, and gradient-index (GRIN) lenses). Images acquired by such systems may include pixel patches, each patch corresponding to a micro-optical element in the array of micro-optical elements, and each pixel corresponding to the position of the micro-optical element in the scan pattern during image acquisition. The imaging system used for image acquisition may be a parallel imaging system. For example, the imaging system may acquire pixels from all patches simultaneously during imaging. For example, the objective may be a micro-optical element in an array of micro-optical elements. Imaging a moving sample with such an imaging system may also produce images containing sample motion artifacts, which, for example, will manifest as discontinuities between patches when using conventional raster scanning.

[0008] Sample motion artifacts can be mitigated by balancing image acquisition time and scan resolution (e.g., where the minimum acceptable resolution is determined by the sample features to be imaged). Sample motion artifacts can also be reduced or eliminated by using scan patterns that reduce the acquisition time difference between acquiring peripheral pixels in adjacent tiles, compared to conventional raster scanning. For example, a spiral scan pattern (inward or outward spiral) can be used to achieve a smaller time difference between adjacent pixels in adjacent tiles across an image. The reduced time difference can reduce or eliminate discontinuities that would otherwise appear at tile boundaries in the image (e.g., along one or two dimensions). In some embodiments, the intensity of test images acquired using a relatively small test scan pattern or test points acquired at different times (e.g., periodically) can be compared to determine whether sample motion has occurred. In some embodiments, the acquired images are analyzed to determine whether sample motion has occurred during imaging by comparing adjacent pixels at tile boundaries. For example, sample motion is determined to have occurred when the pixel intensity difference across tile boundaries exceeds a threshold (e.g., determined by internal pixels in one or more tiles). In some implementations, the intensity of one or more individual pixels (e.g., individual pixels in each of one or more patches) is monitored over time to determine whether sample motion has occurred over a period of time. In some implementations, flattening or reshaping tools may be used during imaging to suppress sample motion.

[0009] In some aspects, a method of imaging a sample using an array of micro-optical elements (e.g., to reduce sample motion artifacts) includes scanning the array of micro-optical elements along a scan pattern defined by a sequential position array to generate an image of the sample. The sequential position array may be an M×N array, where M and N are each not less than 10, and optionally, M ≥ N. In some embodiments, the image comprises pixel patches. Each patch may correspond to a corresponding micro-optical element in the array, and each pixel corresponds to a position in the sequential position array. In some embodiments, for each pair of a first pixel in a first patch and a second pixel adjacent to the first pixel, where the second pixel is located in a patch in the patch that is different from the first patch in the patch, the time difference between acquiring the first pixel and acquiring the second pixel is less than 30% (e.g., less than 10%, less than 5%, or less than 1%) of the total time required to scan the array of micro-optical elements along each position in the scan pattern.

[0010] In some embodiments, the method includes, during the scanning process, (i) providing illumination light to the sample via micro-optics, and (ii) collecting corresponding back-emitted light from the sample using micro-optics, and subsequently receiving the back-emitted light at a detector. In some embodiments, the method includes generating an image of the sample by a processor of a computing device based on the corresponding back-emitted light received at the detector.

[0011] In some implementations, the scanning pattern has a size corresponding to the size of a unit cell of a micro-optical element in an array of micro-optical elements.

[0012] In some implementations, for each pair of a first pixel in a first block of the image and a second pixel adjacent to the first pixel, where the second pixel is located in a block of the image that is different from the first block, the time difference between acquiring the first pixel and acquiring the second pixel is less than (MN-2M+1)dt, where dt is the time step of the scan, M and N are each not less than 5, and M≥N (e.g., the positional difference between the position corresponding to the first pixel and the position corresponding to the second pixel is less than (MN-2M+1)). In some implementations, the time difference does not exceed (3M-3)dt. In some implementations, the time difference does not exceed (2M-1)dt.

[0013] In some embodiments, the sequential position array includes a start position and a final position, wherein the final position is a distance in the length dimension no more than two-thirds of the length of the scanned pattern and in the width dimension no more than two-thirds of the width of the scanned pattern from the start position. In some embodiments, the final position is no more than half the length in the length dimension and no more than half the width in the width dimension from the start position. In some embodiments, the width dimension is perpendicular to the length dimension. In some embodiments, a set of positions in the periphery of the scanned pattern includes at least one-third (e.g., at least one-half or at least three-quarters) of all positions in the periphery, wherein the positions in the set are consecutive sequential positions in the scanned pattern.

[0014] In some embodiments, the sequential position array forms a spiral (e.g., an inward spiral or an outward spiral). In some embodiments, after each of the multiple directional changes in a series of sequential positions, the micro-optical element array moves to a certain number of positions in the series of sequential positions before another directional change occurs, the number being (i) always not less than or (ii) always not more than the number of positions moved since the immediately preceding directional change in the multiple directional changes.

[0015] In some embodiments, the series of sequential positions comprises a series of sequential position rows, wherein each sequential row in the series that is not the first or last sequential row is spatially separated from its temporally adjacent row in the series by at least one other row in the series. In some embodiments, the scan pattern is unidirectional (e.g., such that sequentially adjacent rows in the series of sequential rows are scanned in the same direction). In some embodiments, the scan pattern is bidirectional (e.g., such that sequentially adjacent rows in the series of sequential rows are scanned in different directions). In some embodiments, the starting position of the scan pattern is in an inner sequential position row of the series of sequential position rows, and the final position of the scan pattern is in an outer sequential position row of the series of sequential position rows. In some embodiments, each row in the series of sequential position rows is no closer to an inner sequential position row than the immediately preceding row in the series of sequential position rows.

[0016] In some embodiments, the starting position of the scan pattern is in an outer sequential position row, and the final position of the scan pattern is in an inner sequential position row. In some embodiments, each row in the series of sequential position rows is no further away from an inner sequential position row than its immediately preceding row. In some embodiments, each sequential row in the series that is not the first or last sequential row is spatially separated from its temporally adjacent row in the series by at least one other row in the series.

[0017] In some implementations, the sequential position array is a regular array (e.g., a square array).

[0018] In some aspects, a method of imaging a sample [e.g., a biological sample (e.g., a removed tissue sample) (e.g., a stained biological sample)] using an array of micro-optical elements (e.g., to reduce sample motion artifacts) includes scanning the array of micro-optical elements along a scan pattern defined by an array of sequential positions to generate an image of the sample. A set of peripheral positions in the periphery of the scan pattern may include at least one-third (e.g., at least one-half or at least three-quarters) of all positions in the periphery. The peripheral positions in the set may be consecutive sequential positions in the scan pattern.

[0019] In some aspects, a method for imaging a sample [e.g., a biological sample (e.g., a removed tissue sample) (e.g., a stained biological sample)] using an array of micro-optical elements (e.g., to reduce sample motion artifacts) includes scanning the array of micro-optical elements along a scanning pattern defined by an array of sequential positions to generate an image. The series of sequential positions may include a starting position and a final position. The final position may be a distance in the length dimension not exceeding two-thirds of the length of the scanning pattern from the starting position. The final position may also be a distance in the width dimension not exceeding two-thirds of the width of the scanning pattern (e.g., and in the length dimension not exceeding two-thirds of the length of the scanning pattern).

[0020] In some aspects, a method of imaging a sample [e.g., a biological sample (e.g., a removed tissue sample) (e.g., a stained biological sample)] using a micro-optical array (e.g., to reduce sample motion artifacts) includes scanning the micro-optical array along a series of sequential positions defining a scanning pattern to generate an image of the sample (e.g., based on back-emitted light collected from the sample during said scanning). In some embodiments, after each of a plurality of directional changes in the series of sequential positions, the micro-optical array moves to a certain number of positions in the series of sequential positions before another directional change occurs, said number (i) always not less than or (ii) always not more than the number of positions moved since the immediately preceding directional change in the plurality of directional changes.

[0021] In some aspects, a method of imaging a sample [e.g., a biological sample (e.g., a removed tissue sample) (e.g., a stained biological sample)] using an array of micro-optical elements (e.g., to reduce sample motion artifacts) includes scanning the array of micro-optical elements along a scanning pattern defined by a sequential position array to generate an image of the sample, wherein the sequential position array is an M x N array. The image may include pixel patches. Each patch may correspond to a unit cell of a corresponding micro-optical element in the array. Each pixel may correspond to a position in the sequential position array. In some embodiments, for each pair of a first pixel in a first patch and a second pixel adjacent to the first pixel, the second pixel is located in a patch in the patch that is different from the first patch in the patch, and the positional difference between the position corresponding to the first pixel and the position corresponding to the second pixel is less than (MN - 2M + 1), where M and N are each not less than 5 and M ≥ N. In some embodiments, M = N and the positional difference does not exceed (3M - 3). In some embodiments, M = N and the positional difference does not exceed (2M - 1).

[0022] In some aspects, a method for imaging a sample [e.g., a biological sample (e.g., a removed tissue sample) (e.g., a stained biological sample)] using a micro-optical element array (e.g., to reduce sample motion artifacts) includes scanning the micro-optical element array along a scan pattern defined by a sequential position array to generate an image of the sample. The sequential position array may be an M×N array of sequential positions consisting of peripheral positions and internal positions within the peripheral positions. In some embodiments, the average sequential position of the peripheral positions in the sequential position array is less than MN / 2 (e.g., where the average time for acquiring pixels corresponding to the peripheral positions during scanning is less than (MN / 2)dt, and dt is the time step at each corresponding position in the sequential position array during scanning). In some embodiments, the average sequential position is less than 0.6*(MN / 2). In some embodiments, the average sequential position is less than 0.1*(MN / 2).

[0023] In some aspects, a method for detecting whether sample motion has occurred during image acquisition includes receiving, by a processor of a computing device, an image comprising pixel patches, each of which corresponds to an area scanned by micro-optics in an array of micro-optics during image acquisition. In some embodiments, the method includes the processor determining, at least in part, whether sample motion (e.g., and the amount of sample motion) has occurred during imaging based on at least one pair of adjacent peripheral pixels in the image. In some embodiments, each of the at least one pair of adjacent peripheral pixels includes a first peripheral pixel of a pixel in a first patch of the patch and a second peripheral pixel of a pixel in a second patch adjacent to the first patch, the second peripheral pixel being adjacent to the first peripheral pixel in the image.

[0024] In some embodiments, the method includes the processor determining, at least partially, whether sample motion has occurred during imaging based on the intensity of pixels in at least one pair of adjacent peripheral pixels. In some embodiments, the at least one pair of adjacent peripheral pixels includes a plurality of first pixels (e.g., each pixel) in a first tile edge and a plurality of second pixels in a second tile edge, the first tile edge being adjacent to the second tile edge in the image. In some embodiments, the method includes the processor determining, at least partially, whether sample motion has occurred during imaging based on the average intensity of the plurality of first pixels and the average intensity of the plurality of second pixels. In some embodiments, the method includes the processor determining, at least partially, whether sample motion has occurred during imaging based on the sum of the intensities of the plurality of first pixels and the sum of the intensities of the plurality of second pixels. In some embodiments, the at least one pair of adjacent peripheral pixels includes pixels from more than two pairs of adjacent tiles [e.g., at least half of the tiles (e.g., each of the tiles)].

[0025] In some implementations, the method includes determining intensity statistics (e.g., mean, median, pattern, variance, standard deviation) by a processor based on the intensity of internal pixels among the pixels of one or more of the tiles. The method may include determining an intensity difference threshold by a processor based at least in part on the intensity statistics. The method may include determining, by a processor, whether sample motion has occurred during imaging based at least on a comparison between (i) at least one pair of adjacent peripheral pixels and (ii) the intensity difference threshold.

[0026] In some embodiments, each of the inner pixels of the pixel is adjacent to at least one other pixel of the inner pixels of the pixel in the image. In some embodiments, the method includes a processor determining, based on determining that the intensity difference between pixels in each of at least one pair of adjacent peripheral pixels exceeds an intensity difference threshold, whether sample motion has occurred during imaging (e.g., sample motion has occurred) (e.g., and the amount of sample motion).

[0027] In some embodiments, the at least one pair of adjacent peripheral pixels includes a plurality of first pixels (e.g., each pixel) in a first patch edge and a plurality of second pixels in a second patch edge, the first patch edge being adjacent to the second patch edge in the image, and the processor determining whether sample motion has occurred during imaging includes comparing an intensity difference based on the intensity of the plurality of first pixels and the intensity of the plurality of second pixels with an intensity difference threshold. In some embodiments, the intensity difference is based on the average intensity of the plurality of first pixels and the average intensity of the plurality of second pixels. In some embodiments, the intensity difference is based on the sum of the intensity of the plurality of first pixels and the sum of the intensity of the plurality of second pixels.

[0028] In some implementations, the method includes applying an intensity difference threshold to an image by a processor to generate a thresholded image, and displaying the thresholded image via one or more graphical user interfaces (e.g., by the processor).

[0029] In some embodiments, the method includes the processor determining, at least in part, that sample motion has occurred during imaging based on the at least one pair of adjacent peripheral pixels (e.g., the intensity of pixels in the at least one pair of adjacent peripheral pixels). In some embodiments, the method includes automatically notifying (e.g., in one or more graphical user interfaces) a user that sample motion has occurred during imaging (e.g., the amount of sample motion). In some embodiments, the method includes the processor automatically acquiring a second image of the sample after determining that sample motion has occurred during imaging. In some embodiments, acquiring the second image includes scanning an array of micro-optical elements along a scanning pattern (e.g., where the size of the scanning pattern corresponds to the size of a unit cell of the micro-optical elements in the array).

[0030] In some embodiments, the method includes, before determining whether sample motion has occurred during imaging, normalizing the image by a processor based on the characteristic (e.g., average) intensity of patches in the image (e.g., thereby reducing the influence of the Gaussian distribution of light intensity provided by the micro-optical array at the time of image acquisition). In some embodiments, the method includes, before determining whether sample motion has occurred during imaging (e.g., and after determining an intensity difference threshold), applying one or more morphological operators to the image by a processor. In some embodiments, the one or more morphological operators include erosion and immediately following dilation. In some embodiments, the erosion is one-dimensional and the dilation is one-dimensional (e.g., applied perpendicular to the slow scan direction).

[0031] In some implementations, the method includes displaying the amount of sample movement to a user in a graphical user interface by a processor.

[0032] In some embodiments, pixels correspond to positions in a scanning pattern along which the array of micro-optical elements is scanned to acquire an image. In some embodiments, each patch includes data corresponding to back-emitted light received from the sample via a corresponding micro-optical element in the array of micro-optical elements during image acquisition.

[0033] In some aspects, a method for detecting whether sample motion has occurred during image acquisition includes receiving an image comprising pixel patches by a processor of a computing device. Each of the patches may correspond to an area scanned by micro-optics in an array of micro-optics during image acquisition. The method may include generating a difference image from the image by the processor based on single-pixel shifts (e.g., in a horizontal or vertical direction). The method may include determining, at least in part, by the processor whether sample motion (e.g., and the amount of sample motion) has occurred during imaging based on the difference image.

[0034] In some embodiments, the method includes determining whether sample motion has occurred during imaging, at least in part, based on whether the intensity of pixels in the difference image exceeds an intensity threshold. In some embodiments, the intensity threshold is based on the average intensity of pixels (e.g., all pixels) (e.g., internal pixels) in the difference image. In some embodiments, the method includes normalizing the difference image before determining whether sample motion has occurred. In some embodiments, the method includes having one or more morphological operators applied to the image by a processor before determining whether sample motion has occurred during imaging (e.g., after normalization).

[0035] In some aspects, a system for detecting whether sample motion has occurred during image acquisition may include a processor and a non-transitory computer-readable medium storing instructions, wherein the instructions, when executed by the processor, cause the processor to: receive an image comprising pixel patches, each of the patches corresponding to an area scanned by micro-optics in an array of micro-optics during image acquisition. The instructions, when executed, may cause the processor to determine, at least in part, whether sample motion has occurred during imaging (e.g., and the amount of sample motion) based on at least one pair of adjacent peripheral pixels in the image. In some embodiments, each of the at least one pair of adjacent peripheral pixels includes a first peripheral pixel of a pixel in a first patch of the patch and a second peripheral pixel of a pixel in a second patch adjacent to the first patch, the second peripheral pixel being adjacent to the first peripheral pixel in the image. In some embodiments, the system includes an array of micro-optics.

[0036] In some aspects, an imaging system for detecting whether sample motion has occurred during image acquisition includes a processor and a non-transitory computer-readable medium storing instructions, wherein the instructions, when executed by the processor, cause the processor to: receive an image comprising pixel patches, each of which corresponds to an area scanned by micro-optics in an array of micro-optics during image acquisition. The instructions, when executed, can cause the processor to generate a difference image from the image based on single-pixel shifts (e.g., in a horizontal or vertical direction). The instructions, when executed, can cause the processor to determine, at least in part, whether sample motion has occurred during imaging (e.g., and the amount of sample motion) based on the difference image. In some embodiments, the system includes an array of micro-optics.

[0037] In some aspects, methods for determining whether a sample [e.g., a biological sample (e.g., a removed tissue sample) (e.g., a stained biological sample)] has moved (e.g., has become self-stabilized prior to imaging) include acquiring a first test image of the sample, partly by scanning an array of micro-optical elements according to a first test scan pattern (e.g., automatically). The first test scan pattern may have an area smaller than the area of ​​a unit cell of a micro-optical element in the array of micro-optical elements; in some embodiments, the method includes acquiring a second test image of the sample, partly by scanning the array according to a second test scan pattern after a delay period (e.g., and without manipulating the sample) (e.g., automatically). The second test scan pattern may correspond in size to the first test scan pattern. In some embodiments, the method includes determining, at least partially by a processor of a computing device, whether sample movement has occurred between acquiring the first test image and acquiring the second test image by comparing the second test image with the first test image.

[0038] In some embodiments, comparing the second test image with the first test image includes determining a stability index S(t2-t1). In some embodiments, comparing the second test image with the first test image includes comparing corresponding pairs of pixels from the first test image and the second test image.

[0039] In some implementations, the determination of whether sample movement has occurred is based at least in part on whether the sample movement rate does not exceed a predetermined sample movement rate threshold. In some implementations, the determination of whether sample movement has occurred is based at least in part on whether the sample movement amount does not exceed a predetermined sample movement threshold.

[0040] In some embodiments, the method includes determining that the sample movement rate does not exceed a predetermined sample movement rate threshold. In some embodiments, the method includes subsequently acquiring the entire image (e.g., automatically, e.g., without user input) by scanning the array of micro-optical elements according to a scanning pattern. The scanning pattern may have an area corresponding to the area of ​​a unit cell. In some embodiments, positions in the first test scanning pattern and positions in the second scan pattern are each spatially denser than positions in the scan pattern used to acquire the entire image. In some embodiments, the resolution of the first test image and the resolution of the second test image are each higher than the resolution of the entire image. In some embodiments, the method includes acquiring the entire test image of the sample between acquiring the first test image and acquiring the second test image. The entire test image may be acquired partially by scanning the array of micro-optical elements according to a scanning pattern having an area corresponding to the area of ​​a unit cell of the micro-optical elements in the array. In some implementations, the entire test image is acquired (i) relatively quickly, (ii) at a relatively low resolution, or (iii) relatively quickly and at a relatively low resolution, and the entire image is acquired (i) relatively slowly, (ii) at a relatively high resolution, or (iii) relatively slowly and at a relatively high resolution, respectively.

[0041] In some implementations, the determination of whether sample motion has occurred is based at least in part on whether the sample motion rate does not exceed a predetermined sample motion rate threshold and whether the predetermined sample motion rate threshold does not exceed 1.5 times the pixel size of the entire image (e.g., image resolution) divided by the acquisition time of the entire image. In some implementations, the predetermined sample motion rate threshold is the pixel size of the entire image (e.g., image resolution) divided by the acquisition time of the entire image. In some implementations, the delay period corresponds to the acquisition time of the entire image. In some implementations, the delay period corresponds to no more than 50% of the acquisition time of the entire image (e.g., no more than 25% or no more than 10%).

[0042] In some embodiments, the area of ​​the first test scan pattern and the area of ​​the second test scan pattern are each not less than one-thousandth (e.g., not less than one-hundredth) and not more than one-quarter (e.g., not more than one-hundredth) of the area of ​​the unit cell. In some embodiments, the area of ​​the first test scan pattern and the area of ​​the second test scan pattern are each not less than one-hundredth and not more than one-tenth of the area of ​​the unit cell. In some embodiments, the area of ​​the first test scan pattern and the area of ​​the second test scan pattern are each not less than one-thousandth and not more than one-hundredth of the area of ​​the unit cell.

[0043] In some embodiments, each position in the first test scan pattern corresponds to a corresponding position in the second test scan pattern (e.g., the second test scan pattern is the first test scan pattern). In some embodiments, comparing the second test image with the first test image includes determining an intensity difference (e.g., normalized intensity) between a portion of the first test image and a spatially corresponding portion of the second test image (e.g., an average intensity difference). In some embodiments, determining the intensity difference includes directly comparing pixels of the first test image with pixels of the second test image (e.g., including directly comparing a subset of pixels of the first test image with a subset of pixels of the second test image). In some embodiments, comparing the second test image with the first test image includes applying image correlation techniques to determine a displacement from the first test image to the second test image.

[0044] In some implementations, the delay period is at least 2 seconds and no more than 60 seconds. In some implementations, the delay period is at least 2 seconds (e.g., at least 5 seconds) and no more than 30 seconds.

[0045] In some embodiments, the method includes, after a second delay period, acquiring a third test image of the sample, partly by scanning an array according to a third test scan pattern (e.g., automatically), wherein the third test scan pattern corresponds in size to the first test scan pattern. The method may include determining whether sample motion has occurred, at least partly by comparing the third test image with the second test image. In some embodiments, the second delay period is equal to the delay period. In some embodiments, the method includes determining that sample motion has occurred, wherein the third test image is acquired after the second delay period, following the determination that sample motion has occurred.

[0046] The delay period can be the time spent resetting to the beginning of the scan pattern and starting the scan again, or the time spent performing another scan (e.g., in the case of comparing or otherwise analyzing every other scan).

[0047] In some implementations, the method includes determining that the sample motion rate does not exceed a predetermined sample motion rate threshold, and optionally, subsequently (e.g., via a graphical user interface, such as a pop-up notification) (e.g., automatically) notifying the user that the sample has become self-stabilized.

[0048] In some embodiments, the first test scan pattern and the second test scan pattern are each one-dimensional scan patterns. In some embodiments, the size of the first test scan pattern and / or the size of the second test scan pattern is smaller than the size of a unit cell (e.g., optionally and / or the resolution of the first test image and / or the second test image is smaller than the resolution of the subsequently acquired entire image). In some embodiments, the first test scan pattern and / or the second test scan pattern correspond to a fast scan axis (e.g., for the subsequently acquired entire image). In some embodiments, the first test scan pattern and the second test scan pattern are each two-dimensional scan patterns.

[0049] In some aspects, an imaging system for determining whether a sample [e.g., a biological sample (e.g., a removed tissue sample) (e.g., a stained biological sample)] has moved (e.g., has self-stabilized before imaging) includes a processor and a non-transitory computer-readable medium storing instructions, wherein the instructions, when executed by the processor, cause the processor to: acquire a first test image of the sample, partially by scanning an array of micro-optical elements according to a first test scan pattern (e.g., automatically). The first test scan pattern may have an area smaller than the area of ​​a unit cell of a micro-optical element in the array of micro-optical elements. The instructions, when executed, cause the processor to acquire a second test image of the sample, after a delay period (e.g., and without manipulating the sample), partially by scanning the array according to a second test scan pattern. The second test scan pattern may correspond in size to the first test scan pattern. The instructions, when executed, cause the processor to determine, at least partially, whether sample movement has occurred between acquiring the first test image and acquiring the second test image by comparing the second test image with the first test image. In some embodiments, the system includes an array of micro-optical elements.

[0050] In some aspects, methods for determining whether a sample [e.g., a biological sample (e.g., a removed tissue sample) (e.g., a stained biological sample)] has moved (e.g., self-stabilized before imaging) includes monitoring the intensity of a single pixel corresponding to a micro-optical element in the array by a processor of a computing device while the array of micro-optical elements is held in a fixed position. The intensity may be based on the amount of light reflected from the sample and received by a detector through the micro-optical elements. The method may include determining, by the processor, whether sample movement has occurred based on fluctuations in the intensity of a single pixel.

[0051] In some implementations, determining whether sample motion has occurred is based on the intensity difference between the minimum and maximum intensity of a single pixel (e.g., each patch) over a period of time. In some implementations, determining whether sample motion has occurred is based on the cumulative absolute difference of the intensity of a single pixel over a period of time (e.g., between all recorded consecutive values) (e.g., where the cumulative absolute difference of intensity has been averaged, for example, using a moving average filter).

[0052] In some implementations, determining whether sample motion has occurred is based at least in part on determining whether the intensity of a single pixel fluctuates within a certain time frame by no more than a threshold amount (e.g., a predetermined threshold amount). In some implementations, the threshold amount does not exceed 20%. In some implementations, the threshold amount does not exceed 10%.

[0053] In some embodiments, the method includes acquiring a sample image after determining that the intensity of a single pixel has fluctuated no more than a threshold amount over the specified time period (e.g., automatically, for example, without user input). In some embodiments, the threshold amount is a predetermined threshold amount, and the method includes pre-determining the threshold amount based on one or more of the following: (i) the resolution of the image to be acquired before monitoring begins (e.g., a selected resolution) and (ii) one or more characteristics of the sample. In some embodiments, acquiring the sample image includes scanning an array of micro-optical elements according to a scanning pattern, wherein the scanning pattern has an area corresponding to the area of ​​a unit cell of the micro-optical element.

[0054] In some embodiments, the method includes acquiring sampled test images simultaneously with monitoring the intensity of a single pixel by acquiring test images between discrete measurements of the intensity of the single pixel. In some embodiments, the method includes acquiring sampled test images during monitoring the intensity of a single pixel, wherein at least one discrete measurement of the intensity of the single pixel is canceled or delayed to complete the acquisition of the test image. In some embodiments, acquiring the test image includes scanning an array of micro-optical elements according to a scanning pattern. The scanning pattern may have an area corresponding to the area of ​​a unit cell of a micro-optical element. In some embodiments, the method includes acquiring sampled images after determining that the intensity of a single pixel has fluctuated no more than a threshold amount over a period of time (e.g., automatically, for example, without user input). Test images may be acquired (i) relatively quickly, (ii) at a relatively low resolution, or (iii) relatively quickly and at a relatively low resolution, and sampled images may be acquired (i) relatively slowly, (ii) at a relatively high resolution, or (iii) relatively slowly and at a relatively high resolution, respectively.

[0055] In some implementations, the method includes notifying the user that the sample has self-stabilized after determining that the intensity of a single pixel has fluctuated no more than a threshold amount over a period of time (e.g., via a graphical user interface, such as a pop-up notification) (e.g., automatically).

[0056] In some embodiments, the method includes monitoring the intensity of a corresponding single pixel of one of the plurality of micro-optical elements for each of the plurality of micro-optical elements while the array is held in a fixed position. The intensity of the corresponding single pixel may be based on the amount of light reflected from the sample and received by a detector through said one of the plurality of micro-optical elements. The method may include determining whether sample movement has occurred based at least in part on the corresponding single pixel of each of the plurality of micro-optical elements.

[0057] In some embodiments, the method includes acquiring sampled images after determining that the maximum fluctuation of the intensity of a corresponding single pixel in each of a plurality of micro-optical elements does not exceed a threshold amount over a period of time (e.g., automatically). In some embodiments, the method includes acquiring sampled images after determining that the difference between the maximum and minimum fluctuations of the intensity of a corresponding single pixel in each of the plurality of micro-optical elements does not exceed a threshold amount over a period of time (e.g., automatically). In some embodiments, the method includes acquiring sampled images after determining that the cumulative absolute difference of the intensity of a corresponding single pixel in each of the plurality of micro-optical elements does not exceed a threshold amount over a period of time (e.g., after normalization, for example, using a moving average filter) (e.g., automatically).

[0058] In some embodiments, the method includes acquiring sampled images after determining that the intensity of a corresponding single pixel of each of a plurality of micro-optical elements has fluctuated no more than a threshold amount over a period of time (e.g., automatically). In some embodiments, the method includes acquiring sampled images after determining that the average intensity of a corresponding single pixel of each of a plurality of micro-optical elements has fluctuated no more than a threshold amount over a period of time (e.g., automatically). In some embodiments, the method includes acquiring sampled images after determining that the average fluctuation of the intensity of a corresponding single pixel of each of a plurality of micro-optical elements has not exceeded a threshold amount over a period of time (e.g., automatically). In some embodiments, the plurality of micro-optical elements comprises at least one-quarter (e.g., at least one-half) of micro-optical elements in an array. In some embodiments, the plurality of micro-optical elements comprises each micro-optical element in an array.

[0059] In some implementations, the time period is at least 2 seconds and no more than 90 seconds. In some implementations, the time period is at least 5 seconds and no more than 30 seconds.

[0060] In some implementations, monitoring the intensity of a single pixel includes receiving first back-emitted light from a sample via a micro-optical element on a detector during a first time period. The intensity of the single pixel at the first time period can be automatically determined based on the received first back-emitted light. Monitoring the intensity of a single pixel may also include receiving second back-emitted light from the sample via a micro-optical element on a detector during a second time period (e.g., after a delay period from the receipt of the first back-emitted light) of length equal to the first time period. The intensity of the single pixel at the second time period can be automatically determined based on the received second back-emitted light.

[0061] In some implementations, determining whether sample motion has occurred includes at least comparing the intensity of a single pixel at a first time point with the intensity of a single pixel at a second time point.

[0062] In some implementations, the detector is a CCD or CMOS camera.

[0063] In some aspects, imaging systems for determining whether a sample [e.g., a biological sample (e.g., a removed tissue sample) (e.g., a stained biological sample)] has moved (e.g., self-stabilized before imaging) include a processor and a non-transitory computer-readable medium storing instructions, wherein the instructions, when executed by the processor, cause the processor to: monitor the intensity of a single pixel corresponding to a micro-optical element in the array while the array of micro-optical elements is held in a fixed position. The intensity may be based on the amount of light reflected from the sample and received by a detector through the micro-optical elements. The instructions, when executed, cause the processor to determine whether sample movement has occurred based on fluctuations in the intensity of the single pixel. In some embodiments, the system includes an array of micro-optical elements.

[0064] In some aspects, a sample leveling tool for leveling biological samples [e.g., biological samples (e.g., excised tissue samples) (e.g., stained biological samples)] includes a support member with a shape defining a channel, and optionally, a removable leveling member including a retaining lip sized and shaped to rest on the support member. The removable leveling member may include an insertable portion sized and shaped to insert into the channel.

[0065] In some embodiments, the insertable portion has a shape defining one or more holes (e.g., square or round holes), each hole being sized to accommodate a fork protruding from the sample (e.g., each hole is no larger than 1 cm, for example, no larger than 5 mm). In some embodiments, the support member has a shape defining one or more holes, each hole being sized to accommodate a fork protruding from the sample (e.g., each hole is no larger than 1 cm, for example, no larger than 5 mm).

[0066] In some embodiments, the tool includes one or more removable weights, the size and shape of which are designed to be placed on the top surface of the insertable portion (e.g., the size and shape of which are designed such that at least one of the one or more holes remains uncovered when placed on the top surface of the insertable portion).

[0067] In some implementations, when the lip is held resting on the support member, the bottom surface of the flattening member is positioned no more than 1 mm (e.g., no more than 500 μm) above the bottom of the support member.

[0068] In some embodiments, the support member has an annular cross-section. In some embodiments, the support member has a rectangular cross-section.

[0069] In some respects, a sample leveling tool for leveling biological samples [e.g., biological samples (e.g., excised tissue samples) (e.g., stained biological samples)] includes a flat top portion and optionally one or more wings that extend downward from the top portion (e.g., such that when the tool levels the sample, the one or more wings partially cover the sample).

[0070] In some embodiments, the one or more wings are a single wing. In some embodiments, the tool has rotational symmetry. In some embodiments, the one or more wings include one or more weight supports. In some embodiments, the one or more weight supports are a plurality of weight supports evenly spaced around the periphery of the tool. In some embodiments, the one or more weight supports extend horizontally and parallel to the flat top portion.

[0071] In some embodiments, the tool includes one or more removable weights, each removable weight being sized and shaped to be mounted on at least one of the one or more weight supports. In some embodiments, each of the one or more wings has a shape defining one or more holes (e.g., square or round holes), each hole being sized to accommodate a fork protruding from the sample (e.g., each hole is no larger than 1 cm, e.g., no larger than 5 mm).

[0072] In some embodiments, the top portion has a shape defining one or more holes (e.g., square or circular holes), each hole being sized to accommodate a fork protruding from the sample (e.g., each hole is no larger than 1 cm, e.g., no larger than 5 mm). In some embodiments, the one or more wings are attached to the top surface of the top portion such that the top portion and the one or more wings define a recess. In some embodiments, the tool includes one or more removable weights, the size and shape of which are designed to be at least partially positioned within the recess on the top surface of the top portion (e.g., the size and shape of which are designed such that at least one of the one or more holes remains uncovered when positioned on the top surface of the top portion).

[0073] In some embodiments, the tool is made of injection-molded plastic. In some embodiments, the tool comprises metal.

[0074] In some aspects, a method for imaging a stable sample [e.g., a biological sample (e.g., a removed tissue sample) (e.g., a stained biological sample)] using an imaging system includes: placing the sample on a transparent imaging window so that a user can access the sample during imaging; reshaping the sample with a reshaping tool; resting a portion of the reshaping tool holder on an upper working surface of the imaging system so that the sample remains reshaped during imaging; and imaging the sample while it remains reshaped.

[0075] In some aspects, a method for imaging a sample [e.g., a biological sample (e.g., a removed tissue sample) (e.g., a stained biological sample)] using an array of micro-optical elements (e.g., to reduce sample motion artifacts) includes selecting an imaging time and imaging resolution based on one or more features of the sample to be resolved (e.g., cells in the sample, cell nuclei, or cell organelles). The imaging time can be from 5 to 30 seconds, and optionally, the imaging resolution can be from 10 µm to 2 µm. The method may also include acquiring images of the sample, in part, by scanning the array of micro-optical elements according to a scanning pattern (e.g., automatically). The scanning pattern may have an area corresponding to the area of ​​a unit cell of the micro-optical elements in the array. In some embodiments, the imaging time is selected based on the imaging resolution. In some embodiments, the imaging resolution is selected based on the imaging time. In some embodiments, the imaging time does not exceed 10 seconds and the imaging resolution is 5 μm or better. In some embodiments, the method includes further selecting the imaging time and imaging resolution based at least in part on the material of the sample. In some implementations, the ratio Q of imaging time (t) to image resolution (r) is not less than 1 and does not exceed 5 (1≤Q≤5) (e.g., 2≤Q≤5).

[0076] In some aspects, a method for imaging a sample using an imaging system including an array of micro-optical elements includes: acquiring partial image data over a period of time (e.g., automatically); determining changes in the partial image data over the period of time; determining, based on the changes in the partial image data, that no more than a predetermined amount of sample movement has occurred during the period of time; and acquiring, after determining that no more than a predetermined amount of sample movement has occurred during the period of time (e.g., automatically), a full image of the sample, wherein the full image corresponds to a larger region of the sample than the region corresponding to the partial image data.

[0077] In some embodiments, the partial image data includes intensity data corresponding to the back-emitted light collected from the sample. In some embodiments, acquiring the partial image data includes (e.g., automatically) acquiring one or more test images of the sample by scanning a micro-optical element array according to a first test scan pattern, wherein the first test scan pattern has an area smaller than the area of ​​a unit cell of a micro-optical element in the micro-optical element array. In some embodiments, the one or more test images include two or more test images acquired at different times, and determining the change of the partial image data over a period of time includes comparing the two or more test images (e.g., by comparing the intensity data therein). In some embodiments, the intensity data corresponds to the intensity data of one or more single pixels (e.g., only one single pixel) (e.g., a single pixel from each of multiple patches) (e.g., an isolated single pixel within one or more patches). In some embodiments, determining whether sample motion has occurred during the period includes determining whether a motion index or stability index exceeds a predetermined threshold. In some embodiments, once the imaging system has determined that no more than a predetermined amount of sample motion has occurred during the period, the imaging system automatically acquires the entire image.

[0078] In some embodiments, the method includes presenting to a user via a graphical user interface provided by the imaging system an indicator (e.g., graphic, text, and / or sound) indicating that no more than a predetermined amount of sample motion has occurred during the time period. In some embodiments, the indicator is a graphic (e.g., color and / or symbol) or text (e.g., value and / or measure). In some embodiments, the indicator represents all samples (e.g., based on a single scalar representing all samples) (e.g., derived from data of multiple tiles, each tile corresponding to a different micro-optical element in the array). In some embodiments, the indicator is updated in real time. In some embodiments, the entire image is automatically acquired after the indicator is presented in the graphical user interface or after receiving start imaging input from the user. In some embodiments, determining that no more than a predetermined amount of sample motion has occurred is based on partial image data from multiple tiles, each tile corresponding to a different micro-optical element in the array. In some embodiments, determining that no more than a predetermined amount of sample motion has occurred is based on the average of partial image data from multiple tiles, each tile corresponding to a different micro-optical element in the array. In some implementations, the entire image is acquired using a ratio Q (1≤Q≤5)(e.g., 2≤Q≤5) of imaging time (t) to image resolution (r) that is not less than 1 and not more than 5.

[0079] In some aspects, a method for detecting whether sample motion has occurred during image acquisition includes: receiving, by a processor of a computing device, an image comprising pixel patches, each of which corresponds to an area scanned by micro-optical elements in an array of micro-optical elements during image acquisition; and determining, by the processor, whether sample motion has occurred during imaging (e.g., and the amount of sample motion) based at least in part on a motion index (Mαβ) (e.g., where Mαβ is the amount of sample motion). αβ > 2.75 indicates that sample movement has occurred.

[0080] In some aspects, a method for detecting whether sample motion has occurred during image acquisition includes: receiving, by a processor of a computing device, an image comprising pixel patches, each of the patches corresponding to an area scanned by micro-optical elements in an array of micro-optical elements during image acquisition; and having the processor, at least in part, base its detection on a motion index (M). YZ ) to determine whether sample motion has occurred during imaging (e.g., and the amount of sample motion) (e.g., where in M YZ > 2.75 indicates that sample movement has occurred.

[0081] In some aspects, a method for detecting whether sample motion has occurred during image acquisition includes: receiving, by a processor of a computing device, an image comprising pixel patches, each of the patches corresponding to an area scanned by micro-optical elements in an array of micro-optical elements during image acquisition; and determining, by the processor, whether sample motion (e.g., the amount of sample motion) has occurred during imaging based at least in part on a comparison of the intensity difference between adjacent pixels in a first pair of adjacent pixels in an adjacent row and the intensity difference between adjacent pixels in a second pair of adjacent pixels in the same row. In some embodiments, one of the pixels in the first pair is one of the pixels in the second pair.

[0082] In some aspects, a method for determining whether a sample [e.g., a biological sample (e.g., a removed tissue sample) (e.g., a stained biological sample)] has moved (e.g., has become self-stabilized prior to imaging) includes: (e.g., automatically) acquiring a first test image of the sample (e.g., partially by scanning an array of micro-optical elements according to a first test scan pattern, wherein the first test scan pattern has an area smaller than the area of ​​a unit cell of a micro-optical element in the array of micro-optical elements); after a delay period (e.g., and without manipulating the sample), (e.g., automatically) acquiring a second test image of the sample (e.g., partially by scanning the array according to a second test scan pattern, wherein the second test scan pattern corresponds in size to the first test scan pattern); and determining, at least in part, by a processor of a computing device, whether sample movement has occurred by determining a stabilization index S(t2-t1).

[0083] In some aspects, a method for determining whether a sample [e.g., a biological sample (e.g., a removed tissue sample) (e.g., a stained biological sample)] has moved (e.g., has become self-stabilized prior to imaging) includes: (e.g., automatically) acquiring a first test image of the sample (e.g., partially by scanning an array of micro-optical elements according to a first test scan pattern, wherein the first test scan pattern has an area smaller than the area of ​​a unit cell of a micro-optical element in the array of micro-optical elements); after a delay period (e.g., and without manipulating the sample), (e.g., automatically) acquiring a second test image of the sample (e.g., partially by scanning an array according to a second test scan pattern, wherein the second test scan pattern corresponds in size to the first test scan pattern); and determining, at least partially by a processor of a computing device, whether sample movement has occurred by comparing corresponding pairs of pixels from the first test image and the second test image.

[0084] Any two or more features described in this specification (including in this Summary of the Invention section) may be combined to form an implementation not specifically (e.g., explicitly) described in this specification.

[0085] definition

[0086] To facilitate understanding of this disclosure, specific terms used herein are defined below. Additional definitions for these and other terms may be set forth throughout the specification.

[0087] In this application, unless otherwise stated, the use of "or" means "and / or". As used in this application, the term "comprise" and variations thereof, such as "comprising" and "comprises", are not intended to exclude other additives, components, integers, or steps. As used in this application, the terms "about" and "approximately" are used as equivalents. Any numbers used in this application with or without about / approximately are intended to cover any normal fluctuations known to those skilled in the art. In some embodiments, the terms "about" or "approximately" refer to a range of values ​​falling within 10%, 9%, 8%, 7%, 6%, 5%, 4%, 3%, 2%, 1%, or less of the stated reference value in either direction (greater or less), unless otherwise stated or otherwise apparent from the context (except where such numbers would exceed 100% of the possible values).

[0088] Image: As used herein, the term "image" includes, for example, any visual representation such as a two-dimensional or three-dimensional image of excised tissue (or other sample), such as a photograph, video frame, streaming video, and any electronic, digital, or mathematical analogue of a photograph, video frame, or streaming video. In some embodiments, any system or device described herein includes a display for displaying an image or any other result produced by a processor. In some embodiments, any method described herein includes steps for displaying an image or any other result produced by the method. In some embodiments, any system or device described herein outputs an image to a remote receiving device [e.g., a cloud server, remote monitor, or hospital information system (e.g., a Picture Archiving and Communication System (PACS))] or to an external storage device that can be connected to said system or device. In some embodiments, images are generated using fluorescence imaging systems, cold light imaging systems, and / or reflectance imaging systems. In some embodiments, the image is a two-dimensional (2D) image. In some embodiments, the image is a three-dimensional (3D) image. In some embodiments, the image is a reconstructed image. In some embodiments, the image is a confocal image. An image (e.g., a 3D image) can be a single image or a set of images. In some implementations, the presence of one or more sample motion artifacts in an image (e.g., the entire image or a test image) reflects whether sample motion has occurred. These one or more sample motion artifacts may be detectable by image processing performed by the imaging system. In some implementations, determining the presence of one or more sample motion artifacts determines (e.g., decides) whether sample motion has occurred.

[0089] User: As used herein, a user is anyone who uses the imaging system disclosed herein. A user can be, for example, but not limited to, a surgeon, surgical staff (e.g., a nurse or medical personnel in the operating room), laboratory technician, scientist, or pathologist. It should be understood that while the action is described as being performed by a surgeon, in some embodiments, a user who is not a surgeon performs an equivalent function. Attached Figure Description

[0090] The patent or application documents contain at least one color drawing. The patent office will provide a color copy of the drawing from the publication of this patent or patent application upon request and payment of the necessary fees.

[0091] The accompanying drawings are provided for illustrative purposes only and are not intended to be limiting. The foregoing and other objects, aspects, features, and advantages of this disclosure will become more apparent and better understood by referring to the following description taken in conjunction with the accompanying drawings, in which:

[0092] Figure 1A and Figure 1B This is a plan view illustrating an illustrative rectangular optical chip including a microlens array disposed in a square dot matrix according to an illustrative embodiment of the present disclosure;

[0093] Figure 1C It is based on the illustrative implementation scheme of this disclosure. Figure 1A and Figure 1B A cross-section of a portion of the optical chip is shown in the image.

[0094] Figure 2A This is a schematic diagram of an illustrative imaging system for illuminating a tissue sample according to an illustrative embodiment of the present disclosure;

[0095] Figure 2B The illustrative embodiment of this disclosure illustrates the detector's detection of reflected light from a sample. Figure 2A A schematic diagram illustrating the imaging system;

[0096] Figure 3 This is a schematic diagram illustrating a conventional bidirectional scanning pattern for an objective lens (e.g., a micro-optical element in a micro-optical element array) according to an illustrative embodiment of the present disclosure; and

[0097] Figure 4 These are images acquired by an imaging system, in which no significant sample movement occurred during the acquisition process;

[0098] Figure 5 It is an image acquired by an imaging system, in which some sample movement occurred during the acquisition process;

[0099] Figure 6It is an image acquired by an imaging system, in which a relatively large amount of sample movement occurs during the acquisition process;

[0100] Figure 7 This is a graph showing a representative optimization of the method for mitigating sample motion artifacts according to an illustrative embodiment of the present disclosure between image resolution and image time;

[0101] Figure 8 An outward spiral scanning pattern according to an illustrative embodiment of this disclosure is shown;

[0102] Figure 9 An inward spiral scanning pattern according to an illustrative embodiment of this disclosure is shown;

[0103] Figure 10 An inward bidirectional alternating row scan pattern according to an illustrative embodiment of the present disclosure is illustrated, wherein the first row is the top row;

[0104] Figure 11 An outward bidirectional alternating row scanning pattern according to an illustrative embodiment of this disclosure is illustrated, wherein the first row is the middle row;

[0105] Figure 12 This is a comparison of a conventional raster scan pattern (top) with an outward spiral scan pattern (middle) and an inward spiral scan pattern (bottom) according to an illustrative embodiment of this disclosure;

[0106] Figure 13 The relative acquisition times of various points in a conventional bidirectional raster scanning pattern according to an illustrative embodiment of this disclosure are shown.

[0107] Figure 14 The relative acquisition times of various points in a conventional unidirectional raster scanning pattern according to an illustrative embodiment of this disclosure are shown.

[0108] Figure 15 The relative acquisition times of each point in the outward spiral scanning pattern according to the illustrative embodiment of this disclosure are shown;

[0109] Figure 16 The relative acquisition times of each point in an inward spiral scan pattern according to an illustrative embodiment of this disclosure are shown.

[0110] Figure 17 The illustration depicts the relative acquisition times of various points in an outward bidirectional alternating line scan pattern starting from the middle row, according to an illustrative embodiment of this disclosure;

[0111] Figure 18 The illustration depicts the relative acquisition times of various points in an inward bidirectional alternating row scan pattern starting from the bottom row, according to an illustrative embodiment of this disclosure;

[0112] Figure 19 This is a process diagram of a method for determining whether sample motion occurs during image acquisition according to an illustrative embodiment of the present disclosure;

[0113] Figure 20 The illustration depicts pixels adjacent to surrounding pixels (i.e., along the edge of a tile) according to an illustrative embodiment of the present disclosure, including pixels in adjacent tiles;

[0114] Figure 21A -E illustrates a method based on an illustrative embodiment of the present disclosure for determining whether sample motion has occurred during imaging by comparing the peripheral pixels of a patch with the internal pixels of the patch;

[0115] Figure 22A -B illustrates a comparison between a test scan pattern according to an illustrative embodiment of the present disclosure and a position in a scan pattern used to acquire the entire image;

[0116] Figure 23 This is a process diagram of a method for determining whether a sample has been moved according to an illustrative embodiment of this disclosure;

[0117] Figure 24A -B is a process diagram of a method for determining whether a sample has been moved according to an illustrative embodiment of this disclosure;

[0118] Figure 25 Images are from an imaging system according to an illustrative embodiment of this disclosure;

[0119] Figure 26 This is a process diagram of a method for reshaping a sample and imaging the reshaping sample according to an illustrative embodiment of this disclosure;

[0120] Figure 27 This is a diagram of a fixture according to an illustrative embodiment of the present disclosure;

[0121] Figure 28A -C is a diagram of a thread pattern that can be used on the clamping member of a fixture according to an illustrative embodiment of the present disclosure;

[0122] Figure 29 This is a diagram of a fixture according to an illustrative embodiment of the present disclosure;

[0123] Figure 30 This is a diagram of a reshaping tool based on an illustrative embodiment of this disclosure;

[0124] Figure 31 This is a diagram of a fixture according to an illustrative embodiment of the present disclosure;

[0125] Figure 32This is a schematic diagram of a sample leveling tool according to an illustrative embodiment of this disclosure;

[0126] Figure 33 This is a schematic diagram of a sample leveling tool according to an illustrative embodiment of this disclosure;

[0127] Figure 34 This is a diagram of a fixture according to an illustrative embodiment of the present disclosure;

[0128] Figure 35 This is a schematic diagram of a sample leveling tool according to an illustrative embodiment of this disclosure;

[0129] Figure 36 It is a cross-section of a reshaping tool with a concave clamping member according to an illustrative embodiment of the present disclosure;

[0130] Figure 37 It is a cross-section of a reshaping tool having orifices with different cross-sections according to an illustrative embodiment of this disclosure;

[0131] Figure 38 This is a schematic diagram of a reshaping tool with a concave clamping member according to an illustrative embodiment of the present disclosure;

[0132] Figure 39 This is a block diagram of an exemplary network environment used in the methods and systems described herein, according to an illustrative embodiment of this disclosure;

[0133] Figure 40 This is a block diagram of an exemplary computing device and an exemplary mobile computing device used in the illustrative embodiments of this disclosure;

[0134] Figure 41 The illustration depicts a method for determining whether sample motion has occurred during imaging based on comparing the intensity differences between multiple pairs of adjacent pixels located on both sides of a tile boundary, according to an illustrative embodiment of the present disclosure.

[0135] Figure 42A -D presents exemplary images acquired by an imaging system according to an illustrative embodiment of the present disclosure, wherein different levels of sample motion occurred during acquisition;

[0136] Figure 43 An outward unidirectional alternating row scanning pattern according to an illustrative embodiment of this disclosure is shown, wherein the first row is the middle row;

[0137] Figure 44 An inward unidirectional alternating row scan pattern according to an illustrative embodiment of the present disclosure is illustrated, wherein the first row is the top row;

[0138] Figure 45A method for determining whether a sample has become self-stabilized based on a comparison of the intensities of corresponding pairs of pixels belonging to test scans acquired at different times, according to an illustrative embodiment of the present disclosure, is shown.

[0139] Figure 46 The illustration depicts a method, according to an illustrative embodiment of the present disclosure, for determining whether a sample has become self-stabilized by comparing the intensities of corresponding pairs of pixels belonging to test scans acquired at different times with the intensities of multiple pairs of adjacent pixels belonging to the same test scan.

[0140] Figure 47A -B illustrates an exemplary representation of the stability index and the corresponding motion index according to an illustrative embodiment of this disclosure;

[0141] Figure 48 This is a process diagram of a method for determining whether a sample has been moved according to an illustrative embodiment of this disclosure; and

[0142] Figure 49 This is a process diagram for determining whether a sample has been moved, based on an illustrative embodiment of this disclosure. Detailed Implementation

[0143] The systems, apparatus, methods, and processes disclosed herein are intended to cover variations and adaptations made using information from the embodiments described herein. Adaptations and / or modifications to the systems, apparatus, methods, and processes described herein can be performed by those skilled in the art.

[0144] Throughout this specification, where articles, apparatuses, and systems are described as having, containing, or including specific components, or where processes and methods are described as having, containing, or including specific method steps, it is contemplated that there are articles, apparatuses, and systems according to certain embodiments of this disclosure that substantially consist of or comprise the described components, and that there are processes and methods according to certain embodiments of this disclosure that substantially consist of or comprise the described processing steps.

[0145] It should be understood that the order of steps or the order in which a particular action is performed is not important, as long as operability is not compromised, or unless otherwise explicitly or implicitly stated. Furthermore, two or more steps or actions may be performed simultaneously.

[0146] The headings are provided for the convenience of the reader and are not intended to limit the subject matter claimed. As will be apparent to those skilled in the art, implementations or portions thereof from one heading may be used together with or applied to implementations or portions thereof from another heading.

[0147] This disclosure provides, in particular, systems and methods for reducing or eliminating image artifacts that might otherwise be present in an image due to sample movement occurring during imaging. The sample may be exposed to the user during imaging (e.g., in an open-top architecture), for example, placed on an open sample disk positioned over a transparent imaging window of the imaging system. Such samples may be able to move during imaging. For example, certain biological samples, such as excised tissue samples, may relax over a period of time during image acquisition. Imaging artifacts may occur, for example, when the imaging system uses a parallel scanning method, such as an imaging system comprising an array of micro-optical elements for imaging the sample. In this example, sample movement may cause discontinuities (e.g., noticeable lines) in the image at one or more boundaries between regions corresponding to different micro-optical elements in the array.

[0148] Examples of miniature optical element arrays and imaging systems

[0149] In some embodiments, the imaging system for imaging includes an array of micro-optical elements, which may include one or more of a refractive lens, a Fresnel zone plate, a reflecting objective, and a gradient refractive index (GRIN) lens. The array of micro-optical elements can be scanned during imaging, for example, by a scanning stage including an actuator, according to a scanning pattern. The scanning pattern may have a size corresponding to the size of a unit cell of the micro-optical elements in the array (e.g., a square of approximately equal size). In this way, each micro-optical element in the array can scan an area corresponding to its unit cell to produce an image of a size corresponding to the array of micro-optical elements (e.g., having the same order of magnitude). The scanning pattern may include a series of sequential positions (e.g., arranged in an array, such as a regular array) that are moved sequentially during imaging. The array of sequential positions defining the scanning pattern can generally be an M x N array, where M = N or M ≠ N. Illumination light can be provided to the sample at a subset (e.g., all) of the sequential positions in a series (e.g., an array) of micro-optical elements. For example, when the imaging system is a fluorescence microscope such as a confocal microscope, a miniature optical array can be used to collect back-emitted light from the sample at a subset (e.g., all) of sequential locations in a series (e.g., an array).

[0150] In some embodiments, the imaging system is housed in the operating room and used during surgical procedures (e.g., diagnostic procedures or treatment for a diagnosed disease). In some embodiments, the system is used and / or the method is performed intraoperatively.

[0151] A micro-optical element array can be disposed on the surface of an optical chip. For example, micro-optical elements can be disposed on the surface of a substrate of the optical chip. In some embodiments, the optical chip includes a micro-optical element array attached to a holder (e.g., not disposed on a substrate) around the periphery of the array. Typically, the outer periphery of the optical chip can have any shape. In some embodiments, the optical chip is rectangular (e.g., square or non-square). For example, in some embodiments, the micro-optical element array is integral with the substrate of the optical chip. The micro-optical element array can be non-integrated, but rather attached to the substrate of the optical chip. The array of micro-optical elements may include at least 25,000 microlenses (e.g., having a radius of curvature (ROC) between 200 μm and 300 μm). Absorption and / or reflection layers (e.g., serving as apertures) may be provided on the optical chip between the micro-optical elements in the array. The optical chip may be made of fused silica. The micro-optical elements may be arranged in a regular array on the optical chip (e.g., a square lattice). In some embodiments, the array of micro-optical elements has a spacing of 100 μm to 500 μm (e.g., 200 μm to 300 μm). In some embodiments, the optical chip has an irregular array of micro-optical elements, for example, with different spacing in the x and y directions. In some embodiments, the optical chip has a high numerical aperture for high-resolution imaging and more effective background suppression.

[0152] In some implementations, the micro-optical element array is not a part of an optical chip. For example, in some implementations, the micro-optical element array is an array of discrete objectives, which are mounted, for example, in fixed relative positions (e.g., mounted to each other or mounted to a physical support).

[0153] In some embodiments, the micro-optical element array is a regular array, and the spacing between the micro-optical elements in the array in a first direction is equal to the spacing between the micro-optical elements in the array in a second direction perpendicular to the first direction. For example, the micro-optical elements may be arranged in a square lattice. In some embodiments, each micro-optical element in the micro-optical element array has at least one convex surface. For example, each micro-optical element may be a plano-convex lens or a biconvex lens. The convex surface of each micro-optical element may have a shape obtained by rotating a conical section (e.g., having a radius of curvature between 200 μm and 300 μm). In some embodiments, each micro-optical element in the micro-optical element array focuses light onto a region (point) smaller than the spacing (e.g., the array spacing). In some embodiments, the micro-optical elements in the micro-optical element array are collectively focused onto a common focal plane. For example, each element in the micro-optical element array may be focused onto a single point on the common focal plane.

[0154] Figure 1A and Figure 1B Two views of an illustrative optical chip 100 are schematically illustrated. The optical chip includes a micro-optical element array 102 and can be used in the systems disclosed herein and / or to perform the methods disclosed herein. Figure 1A A plan view of the entire optical chip 100 is shown (in Figure 1A Individual micro-optical elements and optional reflective / absorbent layers are not shown. Optical chip 100 has a rectangular cross-section having dimensions W and L (i.e., where W ≠ L). In some embodiments, W = L. Optical chip 100 has high parallelism, wherein the edges of optical chip 100 have a parallelism better than about ±0.250 mrad (e.g., no more than or about ±0.125 mrad). Figure 1B A portion of an optical chip 100, including a portion of a micro-optical element array 102, is shown. The micro-optical element array disposed on the surface of the optical chip 100 may include at least 1,000 micro-optical elements, at least 5,000 micro-optical elements, at least 10,000 micro-optical elements, at least 20,000 micro-optical elements, at least 30,000 micro-optical elements, at least 50,000 micro-optical elements, at least 60,000 micro-optical elements, or at least 100,000 micro-optical elements. The micro-optical element array 102 is highly parallel to the edge of the optical chip 100. The array 102 has a parallelism relative to the edge of the optical chip better than about ±0.250 mrad (e.g., no more than or about ±0.125 mrad). The array 102 is a regular array. In some embodiments, the micro-optical element array is irregular. Dashed box 112a shows an example of a unit cell of micro-optical elements in the array 102. Dashed box 112b shows an example of a unit cell of micro-optical elements in array 102 drawn with an origin different from that of dashed box 112a. Generally, the choice of the origin is arbitrary. The crosshairs in each micro-optical element of array 102 indicate the corresponding center of the micro-optical element.

[0155] Figure 1CA cross-sectional view of a portion of an illustrative optical chip 100 is shown. The optical chip 100 includes a substrate 106 and an array of micro-optical elements. Each micro-optical element 102 is a convex microlens. The convex microlens 102 is integral with the substrate 106, such that the substrate 106 and the microlens 102 together are a continuous material. For example, they can be formed simultaneously during manufacturing. As shown, the thickness (H) of the optical chip 100 can be taken as the distance between the top of the micro-optical element and the opposite surface of the substrate. The thickness of the optical chip can be less than 2.0 mm (e.g., less than 1.5 mm or about 1.5 mm). The optical chip can have a total thickness variation and / or total flatness deviation of less than 20 μm (e.g., less than 15 μm, less than 10 μm, or less than 5 μm). The optical chip 100 is coated with a chromium reflective layer 104. The reflective layer 104 is disposed in the inter-lens region between the micro-optical elements 102. It should be understood that, as Figure 1A and Figure 1B As shown, the reflective layer disposed in the inter-lens region may extend partially onto one or more lenses near the periphery of the lenses. If the reflective layer 104 extends partially onto the micro-optical element near the periphery of the micro-optical element, the diameter 110 of the micro-optical element is larger than the reflective layer aperture 108 formed by the reflective layer 104.

[0156] Figure 2A This is a schematic diagram of an illustrative imaging system 200, illustrating the behavior of the optics of the illustrative system during irradiation of a tissue sample. The imaging system 200 may include the features stated herein and / or may be used to perform the methods disclosed herein. Figure 2B This is an illustrative imaging system 200, showing the detector's detection of back-emitted light from a sample. Now refer to... Figure 2AA laser 218, providing light with wavelengths between 450 nm and 490 nm, provides an illumination beam to a focusing lens 216. The illumination beam passes through the focusing lens 216 and a first aperture 214, and is then guided by a dichroic mirror 214. The dichroic mirror reflects the illumination beam onto a collimating lens 202. The illumination beam is collimated by the collimating lens 202, and the collimated illumination beam propagates to an optical chip 222. The optical chip includes an array of micro-optical elements. The micro-optical elements in the array can be refractive lenses, Fresnel zone plates, reflecting objectives, GRIN lenses, or microlenses. In some embodiments, the optical chip includes an array of refractive microlenses. The micro-optical elements focus the light from the collimated illumination beam onto the sample through an imaging window. In this case, the sample 228 is placed on a disposable sample holder 226, which is directly mounted to the imaging window 224. In some embodiments, the sample is placed on the imaging window (e.g., on a sample tray) during imaging (e.g., without contact with the imaging window). In some implementations, there is no sample holder 226 and the sample is mounted directly on the transparent imaging window during imaging. Using a sample tray reduces or eliminates the need to clean (e.g., disinfect) the transparent imaging window when changing samples. Figure 25 A sample disk 2504 is shown mounted on a transparent imaging window 2502, with a sample 2520 disposed within the transparent imaging window, as an example of an imaging system 2500 that can be used with and / or with the sample disk 2502. Imaging system 200 can be modified or designed similarly.

[0157] Refer again Figure 2A Optical chip 222 is connected to a support of scanning stage 220. During imaging, scanning stage 220 moves optical chip 222 along a scanning pattern using a controller and actuator connected to the support. Each micro-optical element of optical chip 222 produces a close focus (e.g., a small spot of light) of light from a collimated illumination beam on or in the sample during imaging on or in a common focusing (imaging) plane on or in the sample.

[0158] Figure 2B This is a schematic diagram of an illustrative imaging system 200, showing that in Figure 2AThe behavior of the optics shown in the diagram during detection is as follows: Light from a collimated illumination beam focused onto a sample 228 by an array of micro-optical elements in optical chip 222 produces light (e.g., fluorescence or cold light) in sample 228, which is then reflected back toward optical chip 222 through imaging window 224. The reflected light is then collected by micro-optical elements in the array in optical chip 222 and directed toward detector 212. The reflected light passes through dichroic mirror 204 because it is within the transmission band of the mirror. The reflected light then passes through second aperture 206 and is collimated by imaging lens 208. The collimated reflected light passes through emission filter 210 and then onto detector 212. Detector 212 is a CMOS camera that includes an array of detector elements (e.g., pixels in a camera), each of which receives the reflected light from the micro-optical elements in the array of optical elements in optical chip 222. An opaque housing may be positioned around the optical path of the reverse-emitted light passing through the filter 210 to block ambient (e.g., stray) light from incident on the detector 212.

[0159] The imaging system can be used for operating room imaging of fresh tissue removed during surgical procedures (e.g., cancer surgery). In some embodiments, the imaging system is operable to image a portion of a sample in less than 10 minutes (e.g., less than 5 minutes, less than 3 minutes, or less than 2 minutes). In some embodiments, the system is operable to image a portion of a sample in less than 2 minutes (e.g., less than 90 seconds or less than 1 minute). In some embodiments, the portion of the sample has a depth of at least 10 cm. 2 (For example, at least 12 cm) 2 At least 15 cm 2 Or at least 17 cm 2 The area of ​​the sample. In some embodiments, the volume of the sample does not exceed 10 cm x 10 cm x 10 cm, and the system is configured to image the entire outer surface of the sample in an imaging time of no more than 45 minutes (e.g., no more than 30 minutes).

[0160] Imaging systems that can be used (e.g., implemented) according to certain embodiments of this disclosure are discussed in U.S. Patent Nos. 10,094,784 and 10,539,776, each of which is incorporated herein by reference in its entirety. Sample disks that can be used in certain embodiments of this disclosure are discussed in U.S. Patent Application No. 16 / 146,518, filed September 28, 2018, the disclosure of which is incorporated herein by reference in its entirety. Samples may be stained prior to imaging. For example, samples may be stained using a staining solution disclosed in U.S. Provisional Patent Application No. 16 / 806,555, filed March 2, 2020, the disclosure of which is incorporated herein by reference in its entirety.

[0161] Sources of imaging artifacts in parallel imaging systems

[0162] The imaging system used to acquire images can be a parallel imaging system. For example, the imaging system can acquire pixels from all patches simultaneously during imaging. For example, the objective lens can be a miniature optical element in a miniature optical element array. Imaging a moving sample with such an imaging system may also produce images containing sample motion artifacts, which, for example, will manifest as discontinuities between patches when using conventional raster scanning.

[0163] Figure 3 A conventional bidirectional grating scanning pattern is shown, defined by a sequential position array used with a single objective or parallel imaging system. Figure 3 The scanning pattern shown corresponds to the physical pattern formed during scanning by individual micro-optical elements in a micro-optical element array of a parallel imaging system. Therefore, each micro-optical element will follow the same scanning pattern, but at a different physical location. Due to the nature of the grating scanning pattern, there are fast and slow scanning axes. Figure 3 The scanning pattern is a bidirectional raster scan pattern with an M x N scan step size, equidistant from the fast scan axis and the slow scan axis. Assuming each step takes time dt, completing one line scan along the fast scan axis will take M x dt time, regardless of the orientation. Two patches along the fast scan axis (in...) Figure 3 In the context of the map, the time difference between two horizontally adjacent tiles is approximately M x dt. However, completing a full scan of the scanned region will take M x N x dt. Therefore, the time difference between two tiles along the slow axis is approximately M x N x dt (in the context of the map). Figure 3 In the image (two vertically adjacent tiles), the time difference between horizontal edge rows is significantly greater than the time difference between vertical edges.

[0164] For bidirectional raster scanning used in parallel imaging systems, such as Figure 3 As shown, even when the sample does move slightly during imaging, if M x dt is sufficiently small compared to the time taken for the sample to move a distance equivalent to or greater than one scan step, then there may be almost no discontinuity (sample motion artifact) on the patch along the fast scan axis. For example, M=125 and dt=2 ms are sufficiently small compared to the time taken for a sample of a removed breast mass to move a distance of approximately 2 micrometers. For example, although Figures 4 to 6 The images show artifacts of varying degrees from sample motion along the slow axis, but no sample motion artifacts were observed along the fast scan axis (i.e., between any two horizontally adjacent patches). Figures 4 to 6 An image is shown that is approximately 4 mm x 2 mm in size and includes an array of 16 x 8 tiles (e.g., formed by scanning an array of 16 x 8 micro-optical elements).

[0165] Still referencing Figure 3 M x N x dt is more likely to be relatively large compared to the time taken to move a sample a distance equivalent to or greater than one scan step. For example, if M = 125 and dt = 2 ms, the imaging system can complete imaging in approximately 31.25 seconds. In such an imaging setup, if the sample moves significantly less than 10 micrometers in approximately 30 seconds due to relaxation or motion caused by its own weight, no discontinuity can be observed along the slow scan axis between adjacent patches (i.e., between any two vertically adjacent patches) (see [reference]). Figure 4 In such an imaging setup, if the sample moves approximately 10 micrometers over about 30 seconds due to relaxation or motion caused by its own weight, moderate discontinuities can be observed along the slow scan axis at the boundary between two adjacent patches (see [link to image]). Figure 5 In such an imaging setup, if the sample moves approximately 100 micrometers over about 30 seconds due to relaxation or motion caused by its own weight, a significant discontinuity can be observed along the slow scan axis at the boundary between two adjacent patches (see [link to image]). Figure 6 Using different Figure 3 Different scanning patterns of conventional bidirectional grating scanning patterns can reduce or eliminate [the problem]. Figure 5 and Figure 6 The discontinuities observed in the image can be seen even when the sample moves considerably during imaging (see the “Scan Patterns” section below).

[0166] Optimized imaging time and resolution

[0167] Generally, the longer an imaging system takes to image a sample, the more sensitive the system will be to sample motion artifacts, because the sample will move more (in absolute terms) during a longer imaging acquisition time compared to a shorter acquisition time. Generally, the finer the resolution used in an imaging system, the more visible sample motion artifacts will appear (e.g., the sharper the image will appear). During imaging, sample motion at distances smaller than the resolution chosen by the system for a particular image will be imperceptible. For a given resolution, the shorter the imaging time, and therefore the shorter the amount of sample motion, the fewer visible motion artifacts will be. For a given imaging time, and therefore for a given amount of sample motion, the better the image resolution, the more visible motion artifacts will be. In practice, imaging systems may not have access to any combination of resolution and imaging time. For a typical point scan imaging system, the relationship between resolution and imaging time is quadratic: doubling the resolution increases the imaging time by a factor of four. Therefore, certain combinations of imaging time and resolution are more suitable for preventing the observation of sample motion artifacts in the image. For example, when imaging tissue samples (e.g., fresh tissue samples removed during surgery), an imaging time of 30 seconds and an image resolution of 2 micrometers will often result in sample motion artifacts in images based on typical imaging techniques. Conversely, an imaging time of 5 seconds and an image resolution of 10 micrometers will result in almost no sample motion artifacts in the images. There is no clear boundary between combinations of imaging times and resolutions that are prone to producing sample motion artifacts and those that are not, but the combinations mentioned above generally represent the limits of acceptable combinations for certain types of samples.

[0168] Of course, different samples (e.g., samples of different properties, samples made of different tissues, or samples of different shapes) may have different motion characteristics. For example, flexible (or so-called “soft”) tissue (e.g., a breast lump) may have a longer feature relaxation time than rigid (or so-called “hard”) tissue (e.g., bone). The optimal optimization between imaging time and resolution will depend in particular on the application (e.g., which features are desired to be seen in the image), the sample type (e.g., material, size, and / or weight), and the characteristics of the imaging system (e.g., the characteristics of the array of miniature optical elements).

[0169] In some implementations, a "sweet spot" exists between image resolutions of 2 μm and 10 μm, allowing imaging of cells and nuclei within a sample while permitting operation with imaging times below 30 seconds. For applications where 10 μm resolution is sufficient, an imaging system with an imaging time of 5 seconds will, in many cases, exhibit no sample motion artifacts. For applications requiring higher resolution, reducing the imaging time to below 10 seconds and achieving images with resolutions of 5 μm or higher represents a good trade-off between image information content and sample motion artifacts. Images with a resolution of 2.5 μm and an imaging time of approximately 19 seconds will exhibit significantly fewer motion artifacts compared to images with a resolution of 2 µm and an imaging time of 30 seconds. Images with a resolution of approximately 4 µm and an imaging time of approximately 7.5 seconds will be several times (e.g., at least 4 times) better in terms of motion artifact magnitude compared to images with a resolution of 2 µm and an imaging time of 30 seconds. See the graph visually illustrating the optimization between imaging resolution and imaging time. Figure 7 .

[0170] This disclosure recognizes that in some embodiments of imaging using an imaging system (e.g., a parallel imaging system employing, for example, an array of micro-optical elements), reducing the ratio (Q = t / r) of imaging time (in seconds) to image resolution (r) (in µm) reduces the number and magnitude of motion artifacts in the image. When imaging fresh tissue samples (e.g., fresh breast tissue samples removed during surgery), a ratio Q < 1 provides a reasonable guarantee that the image will rarely contain motion artifacts that a user can detect, and that the observed motion artifacts will not materially affect the user's understanding of the image (for this purpose, such artifacts are referred to as "minor" motion artifacts). A ratio 1 < Q < 2 may result in the presence of some minor motion artifacts, and rarely in motion artifacts that a skilled user would consider to complicate the understanding of the image (for this purpose, such artifacts are referred to as "major" motion artifacts). A Q ratio of 2 < Q < 5 may result in numerous minor motion artifacts, some major motion artifacts, and very few motion artifacts that could adversely affect image understanding (e.g., making the image difficult to interpret accurately) (for this purpose, such artifacts are referred to as “severe” motion artifacts). A Q ratio of 5 < Q < 20 may result in numerous minor and major motion artifacts, as well as some severe motion artifacts. A summary of these conditions is provided in Table 1. While a lower Q ratio is better in terms of artifacts, it also corresponds to a longer imaging time, which may be longer than acceptable for a given application. Therefore, in some embodiments, imaging is performed by an imaging system (e.g., comprising an array of miniature optical elements) under the condition of 1 ≤ Q ≤ 5 (e.g., 2 ≤ Q ≤ 5). In some embodiments, imaging is performed by an imaging system under the condition of Q ≤ 5 (e.g., Q ≤ 2 or Q ≤ 1). In some embodiments, test images are acquired by an imaging system under the condition of 5 ≤ Q (e.g., 10 ≤ Q).

[0171] In some embodiments, a method of imaging a sample using an array of micro-optical elements includes selecting the imaging time and imaging resolution based at least in part on one or more features of the sample being resolved (e.g., cells, cell nuclei, or organelles in the sample). In some embodiments, the imaging time is 5 to 30 seconds and the imaging resolution is 10 µm to 2 µm. Subsequently, images of the sample can be acquired (e.g., automatically) by scanning the array of micro-optical elements according to a scanning pattern. In some embodiments, the scanning pattern has an area corresponding to the area of ​​a unit cell of the micro-optical elements in the array. The sample may be a biological sample (e.g., a removed tissue sample) (e.g., a stained biological sample). In some embodiments, the imaging time is selected based on the imaging resolution. In some embodiments, the imaging resolution is selected based on the imaging time. In some embodiments, the imaging time does not exceed 10 seconds and the imaging resolution is 5 μm or better. In some embodiments, the imaging time and imaging resolution are further selected based at least in part on the material of the sample.

[0172] Scanned pattern

[0173] Conventional raster scanning (whether bidirectional or as in...) Figure 3 Is it unidirectional (e.g., as in...)? Figure 14 The middle section has a fast scan axis and a slow scan axis (e.g., such as...). Figure 3 (Note: This is a Chinese character indicating a time step size). Regardless of whether the time step for each scan point is constant (e.g., denoted as "dt"), data acquisition for the last row position in a conventional raster scan occurs very late relative to the total image acquisition time. For example, as... Figure 3 As shown, for an MxN array, the final position in the scan pattern is acquired at (MxN)dt, the first position in the row is acquired at (M(N-1)+1)dt, the first position in the pattern is acquired at dt, and the last position in the first row is acquired at Mxdt. Therefore, a time difference of (M(N-2)+1)dt occurs between acquiring the first row and starting to acquire the last row. For parallel scanning systems (e.g., including arrays of micro-optics), where each objective (e.g., a microlens) scans simultaneously according to the scan pattern, adjacent to each other in the slow scan axis direction (in... Figure 3 In the case of vertically adjacent tiles, there will be significant temporal discontinuities between the surrounding (or "boundary") rows of adjacent tiles. If sample movement occurs, especially when the sample movement is large relative to the scan pattern size and image acquisition time, significant discontinuities may appear in the images at the intersections of adjacent tiles (e.g., as shown in the image). Figure 5 or Figure 6As shown in the diagram, this can make reading images (e.g., for diagnostic purposes) difficult. Even adjacent tiles along the fast scan axis (in...) Figure 3 Such artifacts can also occur when there are only minor artifacts (or none at all) at intersections (where they are horizontally adjacent), because the acquisition time difference between adjacent pixels (their position in the scan pattern within adjacent tiles) can be significantly smaller (in the case of horizontally adjacent pixels). Figure 3 In the case of (M-1)dt, for higher resolution scans with more scan locations in the scan pattern used, image artifacts may be more pronounced (unless the acquisition time step is reduced accordingly, which is possible).

[0174] By using different scanning patterns (e.g., reducing the average position of peripheral locations and / or scanning peripheral locations earlier) to scan parallel objectives (e.g., arrays of miniature optics), the acquisition time difference between peripheral rows or columns in adjacent patches of an image can be reduced. This reduction can decrease or eliminate noticeable image artifacts caused by sample motion (e.g., to the user). For a constant time step (typically used to improve intensity uniformity across an image), each pair of sequential locations in the scanning pattern will have a constant time difference dt. However, by tending to acquire all adjacent (and / or next nearest neighbor) locations closer together, sample motion can be effectively 'hidden' within each patch. Slight discontinuities (e.g., intensity variations) may become less noticeable.

[0175] exist Figures 8 to 11 The paper provides some examples of scan patterns that can be used to reduce discontinuities at tile boundaries by reducing the acquisition time difference. Figure 8 and Figure 9 Two versions of the "spiral" scan pattern are shown, in which the peripheral position is acquired first or last, depending on the outward spiral used. Figure 8 ) or inward spiral ( Figure 9 Spirals with offset start or end points can also be used. Compared to conventional raster scanning, the start and end positions in each spiral are closer together. In some implementations, the start and end (or “final”) positions of the scan pattern are separated by no more than two-thirds of the length (e.g., no more than half) and no more than two-thirds of the width of the scan pattern (e.g., no more than half). Sometimes, sample motion decreases over time (e.g., when caused by relaxation mechanisms). In at least some of these cases, outward spiral scan patterns (e.g., starting at the center of the scan pattern array) are particularly suitable for reducing or eliminating discontinuities in the acquired image because when the sample motion rate decreases more than at the beginning of image acquisition, it scans a greater distance at the edges at the end.

[0176] An alternative type of scan pattern that can be used to reduce acquisition time lag is the "alternating row" method, an example of which is shown in... Figure 10 , Figure 11 , Figure 43 and Figure 44 The explanation is as follows. Figure 10 and Figure 11 The diagram illustrates a bidirectional alternating line scanning pattern. Figure 43 and Figure 44 This illustrates a unidirectional alternating line scan pattern. The starting line can be located somewhere inside (e.g., in...). Figure 11 or Figure 43 In the middle, where the middle line is the starting line) or at the edge (such as in Figure 10 or Figure 44 In the text, the top row is the starting row. Figure 10 and Figure 44 In the text, rows alternate starting from the top row, then the bottom row, then the second-top row, then the second-bottom row, and so on. Figure 11 and Figure 43 In this process, rows alternate starting from the middle row, then the adjacent row below, then the adjacent row above, then the next adjacent row below, then the next adjacent row above, and so on. Therefore, in some implementations, each row that is not the first or last row is spatially separated from its temporally adjacent row by at least one other row. For example, in... Figure 10 and Figure 44 In the example, the second row collected is the bottom row in space, and the third row collected is the second row from the top in space (and is therefore spatially separated by the fourth to tenth rows collected). As another example, in Figure 11 and Figure 43 In this configuration, the second and third acquisition rows are spatially separated by the first acquisition row. In some embodiments, the starting position of the scan pattern is in the inner row, and the final position is in the outer row, and optionally, each consecutive row is no closer to the inner row than the immediately preceding row. Sometimes, sample movement decreases over time (e.g., when caused by a relaxation mechanism). In at least some of these cases where the time to travel from one scan row to the next depends on the distance between the two rows, outward "alternating row" scan patterns (e.g., starting in the inner row somewhere) are particularly suitable for reducing or eliminating discontinuities in the acquired image because when the sample movement rate decreases more than at the start of image acquisition, it scans a greater distance at the edges at the end. In some embodiments, the starting position of the scan pattern is in the outer row, and the final position is in the inner row, and optionally, each consecutive row is no further away from the inner row than the immediately preceding row.

[0177] By bringing more peripheral locations within a scan pattern closer together in the positional sequence that constitutes the scan pattern, acquisition time discontinuities between adjacent patches can be reduced. In some embodiments, at least one-third (e.g., at least one-half, or at least three-quarters) of the locations within the periphery of the scan pattern are a consecutive set of locations within the scan pattern (e.g., such as...). Figures 8 to 11 (As shown in the diagram). Width and length can be considered as vertical (e.g., if the scan pattern is a rectangular array of positions). This contrasts with conventional raster scan patterns, where at most a quarter (quarter plus one) of the positions around the perimeter of the scan pattern are a set of consecutive positions (see...). Figure 3 ).

[0178] Figure 12 A more detailed analysis of the acquisition time difference between adjacent tiles in inward and outward spiral patterns is shown compared to conventional bidirectional raster scanning. On the left, labels for tiles adjacent to the central tile are provided for comparison purposes. On the right, each point in each scan pattern is labeled with its acquisition time relative to the scan start time, where dt is the acquisition time step. (The starting point is labeled t=0, but data (e.g., light) acquisition will elapse over time dt before moving to the second position. Therefore, the labels can be equivalently rewritten where the starting and final positions of each scan pattern are acquired at (MN)dt instead of (MN-1)dt, but the time difference remains unaffected. See also...) Figures 13 to 16 , where i=1 instead of i=0 at the starting position. ) gives the characteristic time difference between tiles at different boundaries. Because a larger acquisition time difference (all other things being equal) generally leads to more significant (e.g., noticeable and / or larger) artifacts, and for illustrative purposes only, the characteristic time difference is considered to be the maximum acquisition time difference (scan position) between adjacent pixels of two different tiles (e.g., miniature optical elements in an array, respectively) belonging to either side of the tile boundary.

[0179] In the case of a conventional bidirectional raster scan pattern, the feature time difference between adjacent positions (or pixels) in the fast scan direction (boundaries 2 and 4 in this example) is (M-1)dt, while the feature time difference in the slow scan direction (boundaries 1 and 3) is (MN-1)dt. For inward and outward spiral scan patterns, the feature time difference at boundary 1 and boundary 3 is (2M+N-3)dt, and the feature time difference at boundary 2 and boundary 4 is (M+2N-5)dt. Therefore, for specific boundaries (boundaries 1 and 3, where the order of magnitude is M instead of MN), the feature time difference of the spiral scan pattern is significantly smaller than that of the conventional bidirectional raster scan pattern, while for other boundaries (boundaries 2 and 4, where the order of magnitude is 3M (assuming N≈M) instead of M), it is only slightly larger. Therefore, on average, it has a smaller impact on the entire perimeter. (Note that the nomenclature for "fast scan axis" and "slow scan axis" does not apply to spiral patterns.) For inward bidirectional alternating line scan patterns and outward bidirectional alternating line scan patterns, the characteristic time difference at boundary 1 and boundary 3 is (2M-1)dt, and the characteristic time difference at boundary 2 and boundary 4 is (M-1)dt. Therefore, the characteristic time difference of the bidirectional alternating line scan pattern is significantly smaller than the characteristic time difference along the slow scan axis of the conventional bidirectional raster scan pattern (for boundary 1 and boundary 3, where the order of magnitude is M instead of MN), and is equivalent along the fast scan axis (for boundary 2 and boundary 4). For inward unidirectional alternating line scan patterns and outward unidirectional alternating line scan patterns, the characteristic time difference at boundary 1 and boundary 3 is Mdt, and the characteristic time difference at boundary 2 and boundary 4 is (M-1)dt. Therefore, the characteristic time difference of a unidirectional alternating line scanning pattern is significantly smaller than that of a conventional bidirectional raster scanning pattern along the slow scan axis (for boundaries 1 and 3, where the order of magnitude is M instead of MN), and is equivalent along the fast scan axis (for boundaries 2 and 4). See also Figures 17 to 18 The reference image is marked with corner points. It is noteworthy that in the alternating row example, no boundary has a longer feature time difference than the corresponding boundary in a regular raster scan.

[0180] Still referencing Figure 12Compared to the corresponding slow-scan axis boundaries (boundary 1 and boundary 3) of conventional raster scanning, the characteristic time difference of the spiral scanning pattern is not only smaller, but also smaller than the minimum time difference between adjacent positions (or pixels) along the slow-scan axis boundary. In raster scanning, the minimum time difference between adjacent pixels along the slow-scan axis boundary is (MN-M)dt or (MN-2M+1)dt, depending on whether N is odd or even. Therefore, assuming M≥N and M is sufficiently large (e.g., >5), the minimum possible time difference of conventional raster scanning along the slow-scan axis boundary is (MN-2M+1)dt, while the characteristic time difference of the spiral scanning pattern along the equivalent boundary (taken as the maximum value) is (2M+N-3)dt. Obviously, for sufficiently large M and N (e.g., M≥N, M>5), the latter is always smaller than the former, and for relatively large M and N (e.g., M≥N, M>50), it is more than an order of magnitude smaller. (For M < 5, the image resolution is often too low to be practically usable.) Alternating line example ( Figures 10 to 11 as well as Figures 17 to 18 The corresponding feature time difference is 2M-1, which is even smaller than the feature time difference of the spiral scan pattern example.

[0181] Table 2 provides a summary of the maximum acquisition time difference for various examples of scan patterns at various exemplary relative sizes, and compares it with bidirectional raster scanning. Table 2 also includes the maximum time difference (for the slow scan axis or equivalent boundary) as a percentage of the total scan time of the pattern (in addition to being expressed as the difference in scan point positions within the corresponding scan pattern). The maximum time difference for conventional raster scan patterns is on the order of MN, while the maximum time difference for spiral and alternating line scan patterns is roughly on the order of M or N. Therefore, the percentage of the maximum acquisition time difference of the spiral and alternating line examples in the total scan time decreases with increasing relative scan pattern size (increasing number of positions), while the percentage of the maximum acquisition time difference of the conventional raster scan in the total scan time actually increases (slightly) with increasing relative scan pattern size. For example, for a relatively large array of scan positions in a scan pattern that can be used to acquire the entire image of a sample (e.g., M = N = 125 or 250), the maximum time difference for a conventional scan pattern is >99% of the total scan time, while for a spiral scan pattern it is <2%, and for an alternating line scan pattern it is <1%. Therefore, when using spiral or alternating line scanning patterns, image artifacts from sample motion are significantly reduced or even eliminated (e.g., become barely noticeable). For smaller arrays in the scanning pattern (e.g., M=N=25), which can be used to acquire test scanning patterns (e.g., as used in the "Tests to Determine if a Sample Has Been Self-Stabilized" section below), the maximum time difference for spiral or alternating lines is still at least 8 times smaller than that for conventional raster scanning. The maximum time difference for spiral scanning patterns is less than 30% of the total scan time, and the maximum time difference for alternating line scanning patterns is less than 20% of the total scan time.

[0182]

[0183] In bidirectional raster scanning, the maximum time difference between adjacent pixels within the same patch (e.g., pixels not located on either side of the patch boundary) is (2M-1)dt. Unidirectional scanning patterns (e.g., unidirectional raster scanning patterns or unidirectional alternating line scanning patterns) may be more practical to implement than bidirectional scanning patterns (e.g., bidirectional raster scanning patterns or bidirectional alternating line scanning patterns) because step points are always acquired while traveling in the same direction. In unidirectional raster scanning, assuming line return takes time dt, the maximum time difference between adjacent pixels within the same patch (e.g., pixels not located on either side of the patch boundary) is (M-1)dt. In practice, line return may take more than dt to complete, but is still on the order of dt (e.g., <10dt). However, even considering the actual line return time of unidirectional raster scanning, for bidirectional and unidirectional raster scanning used in parallel imaging systems, there is substantially no discontinuity (sample motion artifact) between adjacent pixels within the same patch, even when the sample does move slightly during imaging, as... Figure 4As shown in the figure. For example, M=125 and dt=2 ms are sufficiently small compared to the time taken to move a breast mass sample of about 2 micrometers. In helical scanning, the maximum time difference between adjacent pixels within the same patch (i.e., pixels not located on either side of the patch boundary) is (2M+2N-5)dt. In unidirectional and bidirectional alternating line scanning, the maximum time difference between adjacent pixels within the same patch (i.e., pixels not located on either side of the patch boundary) is (2M)dt, which is smaller than helical scanning and comparable to bidirectional raster scanning. Furthermore, for bidirectional and unidirectional alternating line scanning used in parallel imaging systems, there is almost no discontinuity (sample motion artifact) between adjacent pixels within the same patch, even if the sample does move slightly during imaging.

[0184] The low time difference between the acquisition of adjacent pixels in adjacent patches, achieved through, for example, spiral and alternating row scanning patterns, is reflected in various properties of the scanning pattern. In some embodiments, the time difference between the acquisition of each pair of adjacent pixels in adjacent patches is less than (MN-2M+1)dt, where dt is the time step used for scanning. Alternatively, in some embodiments, the position difference in the sequential position array defining the scanning pattern is no greater than (MN-2M+1). In some embodiments, the time difference does not exceed (3M-3)dt (e.g., and M=N). In some embodiments, the time difference does not exceed (2M-1)dt (e.g., and M=N). In some embodiments, the array of micro-optical elements moves to a certain number of positions in a series of sequential positions during scanning and after a change of orientation, the number (i) always not less than or (ii) always not more than the number of positions moved to since the immediately preceding change of orientation (e.g., spiraling outwards or inwards, respectively). In some embodiments, the average sequential position of all peripheral positions in the scanning pattern is less than MN / 2. That is, in some embodiments, there are more positions in the periphery of the scanned pattern that appear before the middle position than after the middle position. In some embodiments, the average sequence position is less than 0.6*(MN / 2). In some embodiments, the average sequence position is less than 0.1*(MN / 2).

[0185] Detecting sample motion in images

[0186] Advantageously, the imaging system detects sample movement based on the acquired image during imaging. Such a system can inform the user of the presence of sample motion during or immediately after imaging. The imaging system can also inform the user of the size of the area affected by sample motion artifacts (e.g., absolute or relative to the sample surface area or relative to the entire image area). The imaging system can also output a visual representation of where it has detected sample motion artifacts. For example, the imaging system can display a thumbnail of the entire image, indicating the detected sample motion artifacts (e.g., color-coded on top of the image thumbnail). With this information, a user who wants to obtain an image with no or substantially no sample motion artifacts can then try imaging the sample again—actively or inactively attempting to further stabilize the sample and prevent unwanted movement of the sample during imaging.

[0187] As previously described, a common sample motion artifact in parallel imaging systems is a discontinuity at the boundaries of patches in an image (where each patch corresponds to a corresponding objective in an array of objectives, e.g., a miniature optical element array). Discontinuities are often visible because the intensity difference between adjacent peripheral pixels of different patches is much larger than the intensity variation between adjacent internal pixels within a patch. Depending on the sample characteristics and the features being imaged (e.g., organelles of a tissue sample, which may be differently stained), the intensity variation between adjacent internal pixels of a patch in an image may differ in images of different samples, or be captured at different times or using different imaging settings. However, regardless of the specific range of intensity variation between adjacent internal pixels within a patch of an image, the intensity variation between adjacent peripheral pixels in the adjacent patch where the artifact resides will likely be even greater. Therefore, it is possible to determine, at least in part, whether sample motion has occurred during imaging based on adjacent peripheral pixel pairs (from adjacent patches). In some implementations, the intensity difference between at least one pair of adjacent peripheral pixels is compared with a threshold determined based on the intensity variation of internal pixels of a patch to determine whether sample motion has occurred during imaging, wherein one of the at least one pair of adjacent peripheral pixels is from a first patch and the other is from a second adjacent patch.

[0188] In some implementation schemes, and in reference to Figure 19 and Figure 20One method involves determining whether sample motion has occurred during imaging based on an image comprising pixel patches. Each patch may correspond to a micro-optical element in an array of micro-optical elements. In step 1902 of method 1900, a processor of a computing device receives an image comprising pixel patches. Each of the patches corresponds to a micro-optical element in an array of micro-optical elements. In some embodiments, the size of each patch corresponds to a unit cell of a micro-optical element in the array that has been scanned during imaging. In optional step 1904, the image is normalized. For example, in some embodiments, the illumination beam provided by the array of micro-optical elements has a Gaussian distribution, which may cause intensity variations between patches, such that image normalization reduces the effect of these variations.

[0189] In step 1906, the processor determines intensity statistics (e.g., difference, average difference, mean, median, pattern, standard deviation, variation) of the internal pixels in one or more tiles. For example, the intensity statistics could be the average intensity of multiple (e.g., all) pixels in the tile. As another example, the intensity statistics could be, for example, the average intensity difference between an internal pixel and each of its neighboring pixels, calculated over all internal pixels in the tile. Figure 20 In the expression, pixel (i, j-1) is an internal pixel; pixel (i, j) is adjacent to pixel (i, j-1). Figure 20 Each pixel shown corresponds to the position of the micro-optical element during image acquisition, which moves the micro-optical element along the scanning pattern. In step 1908, the processor determines an intensity difference threshold based at least in part on the intensity statistics. In this way, the intensity difference threshold reflects the typical variation in intensity between adjacent internal pixels in a patch of the image. Then, the intensity difference between adjacent internal pixels is expected to decrease below the threshold, and adjacent peripheral pixels in adjacent patches where sample motion artifacts (e.g., discontinuities) occur (e.g., ...). Figure 20 The intensity difference between pixels (i,j+1) and (i,j) in the image will exceed the threshold. If there are no artifacts between the edges of adjacent adjacent tiles, then it will be expected that the intensity difference between adjacent surrounding pixels (e.g., ...) across those edges will exceed the threshold. Figure 20 The intensity difference between pixels (i,j+1) and (i,j) will not exceed the threshold.

[0190] In step 1914, the processor determines the adjacent surrounding pixels along the edge of the tile (e.g., Figure 20The intensity difference between pixels (i, j+1) and (i, j) is determined to exceed a threshold to determine whether sample motion has occurred. In some embodiments, only a pair of adjacent pixels with an intensity difference greater than the threshold can be used to determine that sample motion has occurred. In some embodiments, multiple pairs of adjacent pixels are used.

[0191] In some implementations, method 1900 includes optional step 1910, wherein the processor sums or averages the intensity differences of adjacent peripheral pixel pairs or the intensity of peripheral pixels of a pair of tile edges, and then differs the sum or average before comparing it with an intensity difference threshold. Figure 20 In the diagram, pixels (i-1,j), (i,j), and (i+1,j) are peripheral pixels along the edge of patch B, and pixel (i,j+1) is a peripheral pixel along the edge of patch A, which is adjacent to the edge of patch B.

[0192] In some implementations, method 1900 includes an optional step 1912, in which one or more morphological operators are applied to the image, followed by a comparison of the intensity difference between adjacent pixels along the edges of adjacent patches with an intensity difference threshold. For example, one-dimensional operators, such as erosion and / or dilation operators, may be applied. In some implementations, applying an erosion operator followed by a dilation operator amplifies the intensity difference between adjacent pixels in adjacent patches of the image, making their intensity difference larger than the intensity difference between adjacent internal pixels in a patch of the image. One-dimensional morphological operators may be applied along a slow scan direction (e.g., along a direction with strong sample motion artifacts). The intensity difference threshold may be determined after the application of the morphological operators.

[0193] Method 1900 may include one or both of optional steps 1910 and optional steps 1912 applied in any order.

[0194] In some implementations, the determination of whether sample motion has occurred during imaging (e.g., the amount of sample motion) is based at least in part on at least one pair of adjacent peripheral pixels in the image. Each of the at least one pair of adjacent peripheral pixels may include a first peripheral pixel in a first patch and an adjacent second peripheral pixel in a second patch adjacent to the first patch (e.g., the intensity of the pixels in the at least one pair of adjacent peripheral pixels). In some implementations, the determination of whether sample motion has occurred during imaging is based at least in part on the average or sum of the intensities of a plurality of first pixels in the edge of the first patch and the average or sum of the intensities of a plurality of second pixels in the edge of the second patch adjacent to the edge of the first patch. For example, refer to Figure 20The difference between the average intensity of pixels (i-1,j), (i,j), and (i+1,j) and the average intensity of pixels (i-1,j+1), (i,j+1), and (i+1,j+1) can be compared with an intensity difference threshold to determine whether sample motion has occurred. As another example, refer again... Figure 20 The difference between the sum of the intensities of pixels (i-1,j), (i,j), and (i+1,j) and the sum of the intensities of all pixels can be compared with an intensity difference threshold to determine whether sample motion has occurred. This can be determined using surrounding pixels from multiple pairs of adjacent patches in the image (e.g., each patch). For example, Figure 20 Only tiles A and B are shown, but the intensity difference of pixels from tiles horizontally adjacent to tile A and tiles horizontally adjacent to tile B can be used in combination with pixels from tiles A and B.

[0195] In some implementations, after sample motion is determined to have occurred, the user is automatically notified (e.g., in one or more graphical user interfaces) that sample motion (e.g., the amount of sample motion) has occurred during imaging. In some implementations, a second image of the sample is automatically acquired after sample motion is determined to have occurred. For example, a first image of the sample may be obtained, method 1900 may be performed, and if it is determined that sample motion occurred during the imaging of the first image, a second image may be automatically acquired. In some implementations, an intensity difference threshold is applied to the image, and the resulting thresholded image is then displayed to the user. In this way, the user can make a rapid qualitative assessment of whether sample motion has occurred and / or how much sample motion has occurred.

[0196] The amount of sample motion can be estimated based on the absolute intensity difference between adjacent peripheral pixel pairs at the edges of adjacent tiles. For example, the intensity difference can be used to determine the displacement vector corresponding to the amount of sample motion. In some implementations, the amount of sample motion is displayed to the user (e.g., in a graphical user interface).

[0197] Figure 21A -E illustrates an alternative method for determining whether sample motion has occurred during imaging based on neighboring pixel pairs at the edges of adjacent tiles in an image. Figure 21A The normalized image with sample motion artifacts is shown. Figure 21B The difference images are shown, which have been based on images from... Figure 21A The difference image is generated by horizontally shifting a single pixel in the image. Figure 21C It shows in Figure 21B The image shows a one-dimensional curve of the intensity on the difference image. Figure 21CThe dashed lines in the diagram indicate an intensity threshold based on the intensity of internal pixels (clusters), which can be used to determine whether sample motion has occurred during imaging. For example, Figure 21C The spikes (pixel intensity) in the graph are caused by the large intensity difference between adjacent pixels at the edges of adjacent vertical tiles, which exceeds the intensity threshold. Figure 21D The image shown is a difference image after a one-dimensional (along the vertical direction) erosion operator, and then a one-dimensional dilation operator has been applied to the difference image. Figure 21E It shows in Figure 21D A one-dimensional curve of intensity on the modified difference image. Figure 21E The dashed lines in the diagram illustrate the different intensity thresholds determined after the morphological operators have been applied. (As shown in...) Figure 21D Qualitatively see and in Figure 21E Quantitatively, it can be observed that sample motion artifacts (corresponding to...) Figure 21E The spikes in the image become more pronounced after applying the morphological operator. Images without sample motion artifacts show a more pronounced spike in the one-dimensional intensity plot (e.g., similar to...). Figure 21C and Figure 21E The image will not have obvious spikes. In some implementations, the intensity threshold is based on the average intensity of pixels (e.g., all pixels) (e.g., internal pixels) in the difference image.

[0198] Figure 41 This illustration depicts a method for determining whether sample motion has occurred during imaging by comparing the intensities of adjacent pixel pairs at the edges of adjacent tiles in an image with the intensities of adjacent pixel pairs within the same tile. In some embodiments, the intensity comparison may involve calculating the intensity difference between adjacent pixels across tile edges and comparing that intensity difference with the intensity difference between adjacent pixels within the tile. For example, adjacent pixel pairs at tile edges might belong to pixel line Z of tile α and pixel line A of tile β, and adjacent pixel pairs within the same tile might belong to pixel lines Y and Z of tile α. This method can use one or more pairs of pixels at the edges of adjacent tiles in the image and one or more pairs of pixels within the same tile. For example, all pairs of pixels ZA at the edges of adjacent tiles in the image can be used, and all pairs of pixels YZ within tile α can be used. Let I α (Y1) is the intensity value of pixel Y1 in patch α, and I α (Z1) is the intensity value of pixel Z1 in patch α, and so on. The mean absolute intensity difference Δext along the boundary between patch α and patch β is... αβ The calculation can be performed as follows:

[0199]

[0200] Similarly, the average absolute intensity difference Δint of 1 pixel located along the horizontal segment far from the boundary between tile α and tile β. α The calculation can be performed as follows:

[0201]

[0202] Furthermore, the motion index M of the boundary between patch α and patch β can be calculated. αβ Calculated as Δext αβ With Δint α The ratio between:

[0203]

[0204] In the example above, the motion index M is close to 1. αβ The intensity difference between adjacent pixel pairs across tile boundaries was not significantly different from the intensity difference between adjacent pixel pairs within a tile. For some imaging parameters, a motion index M below 1.5 was observed when no significant motion of the sample occurred during imaging. αβ (as in) Figure 42A (In the middle), and the resulting image can be seamless, or only show motion artifacts that are imperceptible or barely perceptible to the user during image interpretation. When relatively small motion of the sample has occurred during imaging and results in minute motion artifacts in the image, a motion index M between 1.5 and 2.75 was observed. αβ (as in) Figure 42B In the middle section, the minor motion artifacts, such as discontinuities across tile boundaries, do not substantially affect the ease of image understanding for the user. When relatively large motion of the sample has occurred during imaging, resulting in significant motion artifacts in the image, a motion index M between 2.75 and 4 was observed. αβ (as in) Figure 42C (In the context of the image) The significant motion artifacts mentioned above may annoy users, but do not substantially affect image interpretation. A motion index M greater than 4 was observed when relatively large motion of the sample occurred during imaging, resulting in severe motion artifacts in the image. αβ (as in) Figure 42D In some implementations, severe motion artifacts can adversely affect image understanding (e.g., parts of a sample may be too heavily distorted in the image, or missing from the image). If M... αβ >2.75 (e.g., M) αβ >2), then (for example, automatically) it is determined that sample movement has occurred. The arrows in 42A-D indicate the horizontal position of the tile boundary in the figure.

[0205] In some implementations, motion indices can be calculated for adjacent pixel pairs belonging to the same patch. For example, refer to... Figure 41The similarity motion index of adjacent pixel pairs can be calculated, where the first pixel in a pair belongs to pixel row Y, and the second pixel in a pair belongs to pixel row Z. For some scanning patterns (e.g., raster scan patterns, spiral scan patterns, alternating row scan patterns), the time difference between two adjacent pixels in pixel row Y can be significantly smaller than the time difference between a pixel in pixel row Y and an adjacent pixel in pixel row Z, and the latter may be large enough to produce some motion artifacts. Let I(Y1) be the intensity value of pixel Y1, and I(Z1) be the intensity value of pixel Z1 in the same patch, and so on. For example, the motion index M YZ The calculation can be performed as follows:

[0206]

[0207] in

[0208]

[0209] and

[0210]

[0211] In some implementations, if M YZ >2.75 (e.g., M) YZ If >2), then (e.g., automatically) it is determined that sample motion has occurred. In some embodiments, the method for detecting sample motion in an image can be used after image acquisition has been completed. In some embodiments, the method for detecting sample motion in an image can also be used during image acquisition. Optionally, the system can notify the user whenever sample motion above a predetermined level is detected during image acquisition (e.g., determined using a motion index). Optionally, the system can also automatically terminate image acquisition whenever sample motion above a predetermined level is detected during image acquisition (e.g., determined using a motion index). Optionally, the system can also automatically terminate the current image acquisition and automatically restart image acquisition whenever sample motion above a predetermined level is detected during image acquisition (e.g., determined using a motion index). Based on the automatic output of the motion index received by the user (e.g., presented graphically or in text form), the user can manually determine whether to reacquire the image or continue interpreting the image.

[0212] Determine if the sample has been tested for self-stability.

[0213] To reduce the impact of sample motion in high-resolution images, it is sufficient to leave the sample undisturbed on a transparent imaging window for a period of time, allowing the sample to self-stabilize on the imaging window. Starting the imaging process only after the sample has reached a sufficiently stable level (e.g., when its motion rate during the time spent acquiring the image becomes less than the image resolution) can significantly reduce or even completely eliminate sample motion artifacts. However, because sample motion affecting image quality is at a microscale (e.g., ~10 µm–100 µm over tens of seconds), it is not always perceptible to an unhelpful user. Furthermore, the self-stabilization period varies from sample to sample, and generally takes longer for larger samples than for smaller ones. Additionally, some sample motion may be acceptable for some applications, while the same amount of motion may be unacceptable for others. Therefore, a convenient test to qualitatively, semi-quantitatively, or quantitatively determine that sample motion has not exceeded a predetermined (e.g., pre-selected) threshold is desirable. For example, the predetermined threshold can be determined empirically or by calculation for a given sample type (e.g., material and / or size). In some implementations, in order to detect when a sample has been sufficiently stable for a desired application, the imaging system may initiate a stability test scan process that compares the localization of scanned samples across predefined (e.g., preselected) time intervals.

[0214] Any test scan of the sample should take less time (e.g., smaller in size) than a full scan that will be run to image the sample once it has self-stabilized. In some embodiments, the time interval and other parameters of the self-stabilizing test scans may be selected to detect the rate of motion corresponding to approximately one image pixel in the imaging time. In some embodiments, a higher resolution test scan may be used to ensure that if there is no noticeable sample motion in the test scan, there will be no noticeable sample motion in the full scan. By using a significantly smaller test scan size (e.g., no more than 50% or 25% of the full scan area) (e.g., no more than 50% or 25% of the area of ​​a unit cell of a micro-optical element in an array of micro-optical elements scanned during imaging), a higher resolution test scan may still take less time to run than a full scan, even for a constant time step. Alternatively or additionally, a faster time step may be used. In some embodiments, when the stability test scan process detects no difference in sample position between consecutive intervals (e.g., by comparing consecutive test scans), an imaging process that will produce a full image without sample motion artifacts may be initiated.

[0215] In some application contexts under high time pressure (e.g., when performing intraoperative imaging), samples may take a relatively long time to stabilize to a point where the motion rate corresponds to approximately one image pixel in the imaging time. In such application contexts, it may be preferable to initiate image acquisition even if the sample is not yet fully stabilized, and if the resulting image will contain some degree of motion artifacts and their distribution. Small to moderate motion artifacts and / or motion artifacts located in some regions of the sample may be tolerable, as long as they do not impair image interpretation and therefore do not impair clinical outcomes. However, widespread motion artifacts and / or critically significant motion artifacts that could adversely affect image interpretation and clinical outcomes must be avoided. Therefore, in such application contexts, it is necessary to monitor the stabilization state of the sample to determine a good time to initiate image acquisition, e.g., when sample motion has decreased to a level that should not adversely affect image interpretation and therefore clinical outcomes. In some implementations, when the stability test scan process detects no difference in sample position between consecutive intervals (e.g., by comparing consecutive test scans), an imaging process that will produce the entire image can be initiated, where motion artifacts (if any) should not adversely affect the interpretation of the image and therefore the clinical outcome.

[0216] Stability testing scans can be performed at the same resolution as subsequent images and at time intervals corresponding to the imaging time. However, such stability testing does not allow users to save time compared to simply acquiring images and detecting the presence of sample motion, for example, using the methods described above. To save time, stability testing scans can be performed at the same resolution as subsequent images but on a smaller scan area completed in a shorter time. Alternatively, stability testing scans can be performed at a higher resolution than subsequent images but on a smaller scan area completed in a shorter time. For parallel imaging systems, the scan area can be significantly smaller than the area between adjacent micro-optical elements (e.g., unit cells of micro-optical elements), thus allowing for significant savings in scanning time. By comparing stability testing scans acquired at different times (e.g., consecutive stability testing scans), it can be determined whether the sample is still in motion (e.g., still relaxed).

[0217] Figure 22A-B illustrates a comparison between an exemplary test scan pattern 2204 and a scan pattern (“full scan pattern”) used to acquire an entire image comprising a series of sequential positions 2206. Each box 2202 indicates an area to be scanned by a corresponding micro-optical element in the array of micro-optical elements. Thus, the three boxes 2202 shown correspond to a corresponding unit cell of each of the three micro-optical elements in the array of micro-optical elements. The test scan pattern 2204 has a higher resolution than the full scan pattern, as indicated by the relative density of positions in the corresponding scan pattern (where span 2208 indicates the resolution of the full scan pattern). Figure 22A In this context, the test scan pattern 2204 has an area not exceeding approximately 10% of the area of ​​the corresponding unit cell, but test scan patterns 2204 of other sizes can be used. Figure 22B In this design, the test scan pattern 2204 has the same resolution as the full scan pattern, as indicated by the relative density of positions in the respective scan pattern (where span 2208 indicates the resolution of the full scan pattern). It is desirable, but not absolutely necessary, that the combination of resolution and step time used to scan the test scan pattern 2204 results in a faster acquisition time than the combination of resolution and step time used in the full scan pattern. It is generally undesirable to use a lower resolution for the test scan pattern, as a lower resolution test scan pattern may obscure sample motion that might be apparent in the higher resolution full scan pattern. In some embodiments, the test scan pattern and the full scan pattern use the same step time. To determine whether sample motion is occurring or has occurred, the micro-optical array can be scanned multiple times according to the test scan pattern 2204 to acquire multiple images, each scanned at a different time, and the images can then be compared. The test scan pattern 2204 is shown as a bidirectional raster scan, but other scan patterns can be used (e.g., see “Scan Patterns” as described above).

[0218] exist Figure 22A The test scan pattern 2204 shown extends along two dimensions. Other test scan patterns can be obtained by scanning along a single dimension, for example, as... Figure 22BThe test scan pattern 2204 is shown in the figure. In imaging systems that include a fast scan axis and a slow scan axis, it is advantageous, from a temporal perspective, to generate a one-dimensional test scan pattern by scanning only along the fast scan axis. For example, a test scan pattern can be obtained by scanning a single line along the fast scan axis while keeping the slow scan axis in the same position. In some embodiments, the one-dimensional test scan pattern and the two-dimensional full scan pattern use the same resolution and step time. In some such embodiments, if the full scan pattern consists of M×N scan steps at equal distances along the fast and slow scan axes, the one-dimensional test scan pattern will be N times faster. This temporal gain is even greater for larger N (e.g., N=125 or N=250). In some embodiments, a one-dimensional test scan pattern extends along the same length as the fast axis of the full scan pattern. In some embodiments, the one-dimensional scan pattern extends only over a portion of the range of the full scan pattern. Therefore, the temporal gain can be even greater.

[0219] One-dimensional test scans can be acquired continuously, even at high frequencies, because their scan times are short, and they are compared with each other to detect sample motion. One-dimensional test scans can be compared with the immediately preceding or succeeding one-dimensional test scan, or with another one-dimensional test scan that is time-separated by one or more other one-dimensional test scans. The longer the time difference between the compared one-dimensional test scans, the higher their sensitivity in detecting sample motion.

[0220] Figure 45 This diagram illustrates a method for determining whether a sample has become self-stabilized based on comparing the intensities of corresponding pairs of pixels belonging to test scans acquired at different times. For example, the average absolute intensity difference between pixels in pixel line A belonging to two different scan patterns can be compared to a predetermined (e.g., predefined) threshold or the intensity of a pixel. Let I(A1,t1) be the intensity value of pixel A1 at time t1, I(A1,t2) be the intensity value of pixel A1 at time t2, and so on. The average absolute intensity difference Δ(t2-t1) between corresponding pixels in pixel line A belonging to two different scan patterns is:

[0221]

[0222] In some implementations, Δ(t2-t1) can be compared to a predetermined threshold for the intensity variation of internal pixels in a patch in a reference image based on the same tissue sample type (e.g., breast mass excision). In some implementations, a relative stability index can be obtained by dividing Δ(t2-t1) by the sum of the intensities of pixels in line A at time t2 (or t1).

[0223]

[0224] S(t2-t1) can be compared with a predetermined (e.g., predefined) threshold to determine whether sample motion is occurring or has occurred. For example, the predetermined (e.g., predefined) threshold can be determined empirically or by calculation for a given sample type (e.g., material and / or size).

[0225] Figure 46 A method is illustrated for determining whether a sample has become self-stabilized by comparing the intensities of corresponding pairs of pixels belonging to test scans acquired at different times with the intensities of multiple pairs of adjacent pixels belonging to the same test scan. In some embodiments, the intensity comparison may involve calculating the intensity difference between corresponding pixels belonging to two different test scans and comparing the intensity difference with the intensity difference between adjacent pixels within the same test scan. For example, the average absolute intensity difference between pixels of pixel line A belonging to two different scan patterns may be compared with the average absolute intensity difference between adjacent pixels of pixel lines A and B belonging to the same test scan. Let I(A1, t1) be the intensity value of pixel A1 at time t1, I(A1, t2) be the intensity value of pixel A1 at time t2, and so on. The average absolute intensity difference Δ(t2-t1) between corresponding pixels of pixel line A belonging to two different scan patterns is:

[0226]

[0227] The average absolute intensity difference between adjacent pixels of pixel lines A and B belonging to the same test scan is:

[0228]

[0229] And the stability exponent S(t2-t1) is:

[0230]

[0231] In the example above, a stability index S of 1 indicates that the intensity difference between two test scans is equal to the intensity difference between adjacent pixel pairs within the patch. If the time difference between the two test scans corresponds to the imaging time of the full-image scan, and if the resolution of the test scans is the same as that of the full-image scan, then this indicates that the image acquired after the last test scan will not exhibit motion artifacts caused by sample motion greater than one pixel. It will now be apparent to those skilled in the art that similar implementations can be used (e.g., implementations using more than two rows of pixels to calculate the mean absolute intensity difference between adjacent pixels and / or to calculate the stability index).

[0232] In some implementations, the time interval between the test scans to be compared needs to be carefully selected. If the time difference between the compared test scans is too small, small motions of the sample may not be perceived on that timescale, but will still result in visible motion artifacts in the entire image acquired later. Conversely, if the time difference is too large, sample motion that has already occurred early in the observation period will be interpreted as the sample still being in motion, even if it may have stabilized by then, resulting in wasted time. A good trade-off results in an ideal time difference between the test scans to be compared of approximately 1 to 10 seconds (e.g., 2 to 8 seconds, 3 to 7 seconds). Of course, test scans can be acquired at a higher frequency, but compared with another test scan that is separated by this ideal time difference. This will result in a higher refresh rate.

[0233] The parameters of a test scan can make one or more patches in the test scan insufficiently sensitive to sample motion (e.g., if there is insufficient spatial frequency modulation and / or contrast organization in the patch region of the test scan). Therefore, it may be advantageous to consider regions composed of multiple patches when assessing whether sample motion has occurred or is occurring. For example, a unique stability index can be calculated for each region composed of multiple patches (e.g., the stability index of each patch in a region can be averaged to give the stability index of that region). These regions can be constructed by isotropic merging (e.g., 1x2 patches, 3x4 patches, 6x8 patches, 1x12 patches) and isotropic merging (e.g., grouping 2x2 patches, 3x3 patches, 4x4 patches, 6x6 patches, 8x8 patches, 16x16 patches). Since sample motion is sometimes confined to relatively small areas, combining too many patches together in a given area can be counterproductive, especially when the patches are positioned relatively far apart from each other. A good compromise can be achieved for areas with at least 2 tiles but no more than 16 tiles and a total of 4 to 256 tiles (e.g., 2x2, 3x3, 4x4, 6x6, 8x8, 9x9, 12x12, 16x16, 3x4, 6x8, 9x12).

[0234] In some implementations, methods for monitoring the intensity of pixels (e.g., single pixels) include determining a measure (e.g., statistics) for each of multiple patches or merged patches in an image. Differences or statistics can then be determined over time to determine whether sample movement is occurring or has occurred over a period of time. For example, the intensity of each patch can be averaged over a period of time for each of multiple time points. Sample movement that has occurred can then be determined based on the fluctuation of the average of each of the multiple patches over time (e.g., where the fluctuation exceeds a threshold over a period of time). As another example, changes in the minimum or maximum average intensity of adjacent patches, or changes in the difference between the minimum and maximum average intensities, or differences between the minimum and maximum average intensities, can all be used as a basis for determining whether a sample is moving or has moved (e.g., whether sample movement has occurred). (Those skilled in the art will appreciate that, unlike the average intensity of patches or patches, such measures can also be used for single pixels. For example, changes in the minimum or maximum intensity of a single pixel, or changes in the difference between the minimum and maximum intensities of pixels, or differences in the minimum or maximum intensity of adjacent single pixels can all be used as a basis.)

[0235] The stability index calculated above can be compared with a predetermined (e.g., predefined) threshold to reveal the presence or absence of significant sample motion. For example, the predetermined (e.g., predefined) threshold can be determined empirically or by calculation for a given sample type (e.g., material and / or size). Given its similarity to the motion index presented above, the stability index can also be used as a predictive indicator of the magnitude of sample motion to be observed in the image to be acquired. Therefore, the stability index can be expressed as a motion index to reveal sample motion that will result in motion artifacts of varying magnitudes corresponding to the motion index presented above. Figure 47A -B is the color map representation of sample motion. Figure 47A It is a representation of the stability index calculated from the test scan, where a stability index is calculated for the entire 8x8 patch. As the sample movement increases, the color map changes from blue to green to yellow to red. Figure 47B From the use of Figure 47A The motion index is calculated from the entire image acquired immediately after the last test scan. There is no merging, therefore it represents a motion index for each patch. As the magnitude of motion artifacts in the samples increases, the color map changes from blue to green to yellow to red. In the example above, blue represents the motion index M. αβ Below 1.5 (no motion artifacts or minute motion artifacts that the user won't even notice during image interpretation); green represents the motion index M between 1.5 and 2.75. αβ(Minor motion artifacts that will not affect the user's comfort in image interpretation); yellow indicates a motion index M between 2.75 and 4. αβ (Significant motion artifacts that interfere with image interpretation); and red indicates a motion index M greater than 4. αβ (This may adversely affect image interpretation due to severe motion artifacts).

[0236] Figure 23 and Figure 48 This is a process diagram of method 2300 for determining whether a sample has been moved. In step 2302, a first test image is acquired using a first test scan pattern. In step 2304, a second test image is acquired using a second test scan pattern after a first delay period. The second test scan pattern may be the same as or different from the first test scan pattern (e.g., the first test scan pattern may be an inward spiral, and the second test scan pattern may be an outward spiral). The second test scan pattern may have a size corresponding to or different from the size of the first test scan pattern. No manipulation of the sample (e.g., by the user) occurs between acquiring the first test image and acquiring the second test image; however, natural sample movement (e.g., due to relaxation) may occur between acquiring the first and second test images. The first delay period may be at least 2 seconds (e.g., at least 5 seconds) and no more than 60 seconds (e.g., no more than 30 seconds); the first delay period may be selected based on sample characteristics (e.g., sample size or sample material). The two test scans may also be acquired one after the other without a delay period. In this case, the time difference between the scan points in the two test scans corresponds to the duration of the test scan. In optional step 2306, a third test image is acquired using a third test scan pattern after the second delay period. The third test scan pattern may be the same as or different from one or more of the first and / or second test scan patterns. The second delay period may be the same as or different from the first delay period (e.g., it may be selected based on the amount or rate of sample motion determined by comparing the first and second test images, such that step 2306 occurs after step 2308). The sample stability monitoring tool can continuously acquire test scans, with or without a delay period between each test scan, and determine sample motion by comparing one test scan with one or more test scans, which are either immediately preceding / next test scans or test scans temporally separated by one or more other test scans.

[0237] The delay period can be the time spent resetting to the beginning of the scan pattern and starting the scan again, or the time spent performing another scan (e.g., in the case of comparing or otherwise analyzing every other scan).

[0238] In step 2308, the sample motion rate is determined at least in part based on a comparison of the first test image with the second test image. In some embodiments, the first test image and / or the second test image is also compared with a third test image (from step 2306). In some embodiments, the sample motion rate is determined based on the comparison, and if the sample motion rate exceeds a predetermined sample motion rate threshold, it is determined that sample motion has occurred. In some embodiments, the amount of sample motion is determined and compared with a sample motion threshold.

[0239] In optional step 2310, the user is (e.g., automatically) notified whether sample motion has occurred (e.g., the rate or amount of sample motion between acquiring the first and second test images). The system may notify the user of the steady-state of the sample to support the user in deciding when to optimally initiate image acquisition. In some embodiments, the user may be notified via a single event that is automatically triggered when the sample motion meets predetermined rules (e.g., when the sample motion has become sufficiently small to produce visible motion artifacts across the entire image to be acquired). In some embodiments, the current state of the sample motion is continuously communicated to the user via a continuously updated indicator (e.g., graphic or text), which may be reduced to a single scalar of the entire sample (e.g., a color or symbol in the case of a graphic, or a value (e.g., a measure) in the case of text). In some embodiments, the current state of the sample motion is continuously communicated to the user via a continuously updated array of indicators that locally represent the state of the sample motion (e.g., displayed as a color-coded micromap of the sample).

[0240] In optional step 2312, if no sample motion occurs (e.g., if no sample motion rate threshold is exceeded), the entire image is acquired (e.g., automatically).

[0241] In the additional optional step 2314, the entire image is acquired following an explicit request from the user. In application contexts under high time pressure, users may want to be authorized to initiate acquisition at any time they deem appropriate (e.g., continuous notifications based on the current state of sample motion). See also Figure 48 This shows an additional process flow for method 2300.

[0242] Test images can be compared in any manner suitable for determining sample motion. The specific comparison performed may depend on sample characteristics, such as what materials are present in the sample or which features in the sample will be imaged (e.g., cells, nuclei, or other organelles). In one example of comparison, the average intensity of each patch in a first test image is compared to the average intensity of the corresponding patch in a second test image. A comparison between two test images may involve comparing only a portion of one test image with the corresponding portion of the other test image (e.g., comparing a subset of all patches). In some embodiments, comparing test images includes determining an intensity difference (e.g., normalized intensity) between a portion of the first test image and the spatially corresponding portion of the second test image (e.g., an average intensity difference). In some embodiments, determining the intensity difference includes directly comparing pixels of the first test image with pixels of the second test image (e.g., directly comparing a subset of pixels of the first test image with a subset of pixels of the second test image). Comparing test images may include using image correlation techniques (e.g., using image mapping or image matching algorithms). In some embodiments, image correlation techniques determine a displacement vector based on the intensity offset between two test images that can be used to determine the rate or amount of sample motion.

[0243] Test scan patterns can generally be of any size. For example, extremely small test scan patterns may make test images difficult to compare due to the specific image correlation techniques used to compare them. Relatively large test scan patterns may not save time compared to simply performing the entire image acquisition. In some embodiments, the area of ​​the test scan pattern is not less than one-thousandth (e.g., not less than one-hundredth) and not more than one-quarter (e.g., not more than one-tenth or one-twentieth, not more than one-hundredth) of the area of ​​a unit cell of a micro-optical element in the micro-optical element array. In some embodiments, the area of ​​the test scan pattern is not more than one-hundredth of the area of ​​a unit cell of a micro-optical element in the micro-optical element array. Using no more than one-hundredth of the unit cell area provides sufficient area for a sufficiently representative test scan while minimizing acquisition and comparison time, thereby speeding up the overall imaging process. In some embodiments, each position in the first test scan pattern corresponds to a corresponding position in the second test scan pattern (e.g., the second test scan pattern is the first test scan pattern).

[0244] Whether sample motion has occurred can be determined directly or indirectly based on the sample motion rate occurring between the acquisition of test images. Typically, sample motion artifacts are eliminated when the sample moves less than the spatial resolution of the image, for example, within the time taken to scan a scan pattern. Therefore, in some embodiments, determining whether sample motion has occurred (e.g., whether the sample has self-stabilized) involves determining whether the sample motion rate exceeds a predetermined sample motion rate based on a comparison of test images acquired by scanning according to a test scan pattern. For example, a sample motion rate threshold can be predetermined based on the acquisition time used to acquire the entire image (e.g., a 30 s or 60 s scan time for the full scan pattern). In some embodiments, the predetermined sample motion rate threshold does not exceed 1.5 times the pixel size (e.g., image resolution) of the entire image to be acquired, divided by the acquisition time of the entire image. For example, the pixel size used for the entire image (which also corresponds to the spatial step size in the full scan pattern) could be approximately 100 µm, and the acquisition time used for the entire image could be 25 s, such that the predetermined sample motion rate threshold does not exceed 6 µm / s. In some implementations, the predetermined sample motion rate threshold is the pixel size (e.g., image resolution) of the entire image to be acquired divided by the acquisition time of the entire image. In some implementations, a sample is said to be self-stabilized once its motion within the image acquisition time is less than the resolution of the acquired image, so as to produce an image in which there are no visible motion artifacts. For example, the pixel size to be used for the entire image (which may also correspond to the spatial step size in the full scan pattern) may be approximately 2 µm, and the acquisition time to be used for the entire image may be 40 s, such that the predetermined sample motion rate threshold does not exceed 0.05 µm / s. As another example, the pixel size to be used for the entire image (which may also correspond to the spatial step size in the full scan pattern) may be approximately 10 µm, and the acquisition time to be used for the entire image may be 2.5 s, such that the predetermined sample motion rate threshold does not exceed 4 µm / s. The delay period may correspond to the acquisition time of the entire image to be acquired (e.g., not longer than 10% of the acquisition time, and optionally, not shorter than 10% of the acquisition time). Such a delay period can improve the accuracy of assessing whether a sample has been sufficiently self-stabilized before acquiring the entire image. However, such a delay will not allow users to save time compared to simply acquiring an image and detecting the presence of sample motion using, for example, the methods described above. The delay period may correspond only to a portion of the acquisition time of the entire image to be acquired (e.g., no more than 50%, no more than 25%, no more than 10%).

[0245] More than two test images can be compared (e.g., compared to each other as pairs) to determine whether sample motion has occurred and / or to determine the sample motion rate. The delay period between test images can be constant or variable. For example, if it is determined that sample motion occurred at a relatively high rate between the first two test scan patterns, a third test image can be acquired after a longer delay period to avoid acquiring a test image when it is highly likely that sample motion is still occurring at an undesirably high rate (e.g., self-stabilization may not have occurred yet). The subsequent delay period between acquiring test images can be selected based on the sample motion rate determined from comparisons with previous (e.g., immediately preceding) test images. For example, the delay period before acquiring the third test image can be selected (e.g., automatically) based on the sample motion rate determined from comparisons with the first and second test images.

[0246] An alternative approach that may be more time-efficient is to monitor the intensity of only one pixel (or several pixels within each patch, or one pixel per few patches, or several pixels per few patches) relative to time within each patch. For parallel imaging systems, such as those comprising arrays of miniature optics, monitoring a single pixel within each patch does not even require any movement of the optics (or the sample). The temporal intensity fluctuations of samples that have moved significantly (e.g., compared to image resolution and / or imaging rate) will be greater than those of samples that have not moved significantly (e.g., compared to image resolution and / or imaging rate). A threshold amount can be set based on, for example, the typical intensity variation between adjacent pixels in the image, below which the intensity fluctuation of a single pixel will indicate no sample movement (e.g., compared to image resolution and / or imaging rate). The typical intensity variation can be known and / or determined based on image parameters (e.g., resolution) and / or sample characteristics. The threshold amount can be predetermined or determined during monitoring as, for example, a percentage of the intensity fluctuations during an initial period.

[0247] Figure 24A -B and Figure 49This is a process diagram of method 2400 for determining whether a sample has moved. In step 2402, while the micro-optical element is held in a fixed position, individual pixels corresponding to the micro-optical elements in the array of micro-optical elements are monitored. The intensity of the individual pixel is based on the amount of back-emitted light received by the detector and collected by the corresponding micro-optical element. In step 2404, it is determined whether sample movement has occurred, which in this example is determined at least in part based on whether the intensity fluctuation of the individual pixel does not exceed a threshold amount over a period of time. In some embodiments, multiple individual pixels (e.g., each individual pixel corresponds to a corresponding micro-optical element in the array of micro-optical elements, such as where the corresponding micro-optical element is at least one-quarter, at least one-half, or all of the micro-optical elements in the array) are monitored simultaneously to determine whether sample movement has occurred. Determining whether sample movement has occurred may be based at least in part on the fluctuation of each corresponding individual pixel not exceeding a threshold amount; the average intensity fluctuation of the corresponding individual pixel not exceeding a threshold amount; or the average intensity fluctuation of the corresponding individual pixel not exceeding a threshold amount. The period of time may correspond to the acquisition time of the entire image to be acquired. In optional step 2406, an image of the sample is acquired after determining that the intensity fluctuation of a single pixel does not exceed a threshold amount within the stated time period (e.g., automatically). In optional step 2408, based on the determination in step 2404, the user is notified (e.g., via a graphical user interface, such as a pop-up notification) whether sample motion has occurred. The system may notify the user of the sample's steady-state to support the user in deciding when to optimally initiate image acquisition. In some embodiments, the user may be notified via a single event that is automatically triggered when the sample motion meets predetermined rules (e.g., when the sample motion has become sufficiently small to produce visible motion artifacts across the entire image to be acquired, for example, this is determined by a motion index). In some embodiments, the current state of sample motion is continuously communicated to the user via a continuously updated indicator (e.g., a graphical or textual indicator), which may be reduced to a single scalar of the entire sample (e.g., a single color or symbol in the case of graphics, or a single value (e.g., a measure) in the case of text). In some implementations, the current state of the sample motion is continuously communicated to the user via a continuously updated array of indicators that locally represent the state of the sample motion (e.g., displayed as a color-coded micromap of the sample).

[0248] In some implementations of method 2400, such as Figure 24BAs shown, in step 2410, intensity is used to determine whether the sample has locally moved beyond a threshold amount over a period of time. In step 2412, the user is informed that the sample has moved beyond the threshold amount. In step 2414, an image is acquired following an explicit request from the user. In application contexts under high time pressure, users may want to be authorized to initiate acquisition at any time they deem appropriate (e.g., continuous notifications based on the current state of sample motion).

[0249] Figure 49 Additional illustrative process flow for method 2400 is shown.

[0250] In some implementations, an image of a sample is acquired after it has been determined that the intensity of a single pixel has fluctuated no more than a threshold amount over a period of time (e.g., automatically, for example, without user input). In some implementations, the threshold amount is a predetermined (e.g., predefined) threshold amount, and the method includes pre-determining the threshold amount based on the resolution (e.g., a selected resolution) of the image to be acquired before monitoring begins. In some implementations, the threshold amount is a predetermined (e.g., predefined) threshold amount, and the method includes pre-determining the threshold amount based on one or more characteristics of the sample. In some implementations, the threshold amount does not exceed 20% or 10%. Generally, as sample movement slows down or stops, intensity fluctuations will decrease because there is generally no significant intensity discontinuity between adjacent pixels, and pixel drift due to sample movement will slow down. In some implementations, using an absolute threshold amount of no more than 20% or 10% may be sufficient to reduce or eliminate noticeable sample motion artifacts from subsequently acquired images. In some implementations, the period of time is at least 2 s and no more than 90 s. In some implementations, the period of time is at least 5 s and no more than 30 s.

[0251] Monitoring the intensity of a single pixel may include discrete measurements of back-emitted light received in individual short intervals. For example, the intensity at a first time point may be based on back-emitted light received at a detector (e.g., a CCD or CMOS camera) via a micro-optical element in a first short interval (e.g., less than one millisecond), and the intensity at a second time point may be based on back-emitted light received at the detector via the micro-optical element in a second short interval, the second short interval being equal in duration to the first short interval. A delay period (e.g., at least 1 s and no more than 60 s) may exist between the first and second short intervals. Determining whether a sample has moved may include processing (e.g., comparing) the intensity at the first time point with the intensity at the second time point. In some implementations, the delay period needs to be carefully selected. If the delay period is too small, small movements of the sample may not be perceived at this timescale, but will still result in visible motion artifacts in the entire image acquired later. On the other hand, if the delay period is too large, sample movement that has already occurred early in the observation period will lead to the assumption that the sample is still moving, even if it may have stabilized during this period, resulting in wasted time. In some implementations, a good trade-off between the two results in a delay period between 2 and 30 seconds. The intensity fluctuation over time can be based on discrete measurements of intensity taken at a set of times during the monitoring period. Of course, the intensity of a single pixel could be recorded at a higher frequency, but compared to the intensity of that same pixel at another time separated by this delay period. This would result in a higher refresh rate.

[0252] Intensity fluctuations can be simply calculated by taking the absolute value of the intensity difference between pixels at two points separated by a delay period. This method only provides sparse sampling and may therefore be insensitive to intensity fluctuations that have already occurred between the two sampling points (e.g., the intensity may have changed and returned to more or less the same value). Intensity fluctuations can be calculated more sensitively by recording pixel intensity at multiple points and taking the intensity difference between the maximum and minimum values ​​recorded over a period of time. Such intensity fluctuation measures can also be normalized by dividing them by the time elapsed between the maximum and minimum values. Intensity fluctuations can also be calculated more sensitively by recording pixel intensity at multiple points and taking the cumulative absolute intensity difference between all consecutive values ​​recorded over a period of time. Such intensity fluctuation measures can be normalized by dividing them by the delay period in which they were calculated. The advantage of this method is that it is more sensitive to sample motion that causes pixel intensity to change non-monotonically over time. However, its disadvantage is that it is also more sensitive to noise in the intensity signal. Therefore, it may be necessary to smooth the intensity signal, for example, by using a moving average filter, before calculating intensity fluctuations in this way. For example, for intensity values ​​recorded consecutively at intervals of approximately 1 to 5 milliseconds, averaging at least 25 values ​​(e.g., using a moving window filter) would be desirable.

[0253] The delay period can be the time spent resetting to the beginning of the scan pattern and starting the scan again, or the time spent performing another scan (e.g., in the case of comparing or otherwise analyzing every other scan).

[0254] When monitoring the intensity of a single pixel in each patch, depending on the nature of the sample, it is relatively possible that there is insufficient spatial frequency modulation and / or contrast organization in the patch region of the test scan to provide adequate sensitivity to sample motion. Therefore, it may be advantageous to consider regions composed of multiple patches when assessing whether sample motion has occurred or is occurring. For example, a unique measure of intensity fluctuation can be calculated for regions composed of multiple patches (e.g., the intensity fluctuation in each patch of a region can be averaged to give the average intensity fluctuation of that region). These regions can be constructed by isotropic merging (e.g., grouping 2x2, 3x3, 4x4, 6x6, 8x8, and 16x16 patches) of isotropic merging (e.g., 1x2, 3x4, 6x8, and 1x12 patches). Since sample motion is sometimes confined to relatively small areas, combining too many patches together in a given area can be counterproductive, especially when the patches are positioned relatively far apart from each other. A good compromise can be achieved for areas with at least 2 tiles but no more than 16 tiles and a total of 4 to 256 tiles (e.g., 2x2, 3x3, 4x4, 6x6, 8x8, 9x9, 12x12, 16x16, 3x4, 6x8, 9x12).

[0255] Imaging workflows can be designed to allow users to perform one or more useful tasks during a sample self-stabilization period. For example, low-resolution, rapid test images can be acquired during the sample self-stabilization period to ensure sample positioning allows scanning of desired areas of the sample (e.g., the sample surface). In some embodiments, the imaging system will prioritize acquiring test images over monitoring single pixels (e.g., during method 2400) or small test image acquisition (e.g., during method 2300), which will resume after the test images are acquired. In some embodiments, test images of the sample are acquired simultaneously with the intensity of monitoring single pixels (e.g., by acquiring test images between discrete measurements of the intensity of single pixels). In some embodiments, intensity measurements or test image acquisition are interrupted or canceled by low-resolution test image acquisition. In some embodiments, a complete (e.g., low-resolution) test image of the sample is acquired between the acquisition of a first test image and a second test image. In some embodiments, acquiring (e.g., low-resolution) test images includes scanning an array of micro-optical elements according to a scanning pattern having an area corresponding to the area of ​​a unit cell of the micro-optical elements in the array. In some implementations, test images of the sample are acquired (i) relatively quickly, (ii) at a relatively low resolution, or (iii) relatively quickly and at a relatively low resolution, and then the acquired images of the sample are acquired (i) relatively slowly, (ii) at a relatively high resolution, or (iii) relatively slowly and at a relatively high resolution, respectively.

[0256] Flattening tools, reshaping tools and their usage

[0257] Another approach to reducing sample motion artifacts is to act on the sample itself to prevent it from moving during imaging. For that purpose, reshaping tools (e.g., clamps) are useful. While clamps (e.g., tweezers) are typically used for hand-held manipulation of objects, they have proven very practical for holding samples in place and preventing movement during imaging. Generally, heavier clamps provide better sample stabilization and prevent unwanted movement. Clamps with locking mechanisms are also preferred for stabilizing the sample and preventing unwanted movement. Physical manipulation, including flattening (described further later), can be particularly useful for large samples that may have long self-stabilization times.

[0258] In some embodiments, if samples are placed on a surface with a low coefficient of friction, the forces exerted on the grips by the samples will cause them to slide. Increasing the coefficient of friction between the tweezers and the surface they are on helps stabilize the samples and prevent unwanted movement. This can be achieved, for example, by placing a rubber or silicone pad between the grips and the imaging system surface, as described in more detail below. Alternatively or additionally, the material of the upper working surface of the imaging system surface and / or the material of the grips in contact with the imaging system surface can be used to provide a high coefficient of friction. For example, in some embodiments, stainless steel tweezers with rubber-coated finger rings can be used.

[0259] The shape of the clamp jaws can also affect its efficiency in stabilizing (e.g., reshaping) the sample and preventing unwanted movement. In some implementations, one of three methods is used to stabilize (e.g., reshaping) the sample: (i) applying a force from above to flatten the sample and accelerate self-stabilizing movement; (ii) holding the sample from the side; or (iii) a combination of (i) and (ii). The advantage of method (i) is that it maximizes the surface area of ​​the tissue in contact with the imaging window, and thus maximizes the imaging of the tissue (e.g., it reduces "valleys" in the tissue). Method (ii) reduces the direction (space) in which the sample can move (because the force is compensated), and thus reduces the self-stabilizing movement time. Method (ii) also has the advantage of enabling imaging of sample surfaces on which the sample does not remain alone.

[0260] In some implementations, the sample is reshaped during imaging, for example to reduce or eliminate sample motion during imaging. Figure 25 An imaging system 2500 is shown on which a sample can be stabilized during imaging. The imaging system 2500 includes a transparent imaging window 2502 on which a sample 2520 is placed during imaging. During imaging, illumination light is provided to the sample 2520, and reflected light is collected from the sample 2520. A user can approach the sample 2520 during imaging. The sample 2520 is placed directly on a sample tray 2504, which is placed directly on the transparent imaging window 2502. The imaging system 2500 includes an upper working surface 2506 comprising a high-friction material (e.g., rubber or silicone). The high-friction material may be removable (e.g., as a pad) or non-removable. The sample 2520 is reshaped using a clamp 2510 comprising two clamping members 2512. After the user releases the clamp 2510, the high-friction material holds the clamp 2510 in the desired position (e.g., preventing the arm of the clamp 2510 from extending or retracting), thereby reshaping the sample 2520, which can remain reshaped throughout the imaging process.

[0261] Figure 26This is a process flow diagram illustrating an example of a method for stabilizing a sample [e.g., a biological sample (e.g., a removed tissue sample) (e.g., a stained biological sample)] during imaging. For example, an imaging system 2500 may be used to perform the method. In step 2602, the sample is positioned on a transparent imaging window, allowing the user to access the sample during imaging. In step 2604, the sample is reshaped using a clamp, for example, by squeezing the clamp into a partial closure. In step 2606, a portion of the clamp rests on the upper working surface of the imaging system, allowing the sample to remain reshaped during subsequent imaging. For example, a high-friction material in the upper working surface maintains the clamp's position after it is rested (e.g., as shown in the image). Figure 25 (In the imaging system 2500). In some embodiments, the sample remains reshaped even if it undergoes slight movement (e.g., sinking) after the fixture is placed. In step 2608, the sample is imaged while being reshaped. The fixture can be tweezers or forceps.

[0262] Figures 27 to 31 An example of a fixture or reshaping tool that can be used to reshape a sample during imaging is shown. Figure 27 A clamp 2700 is shown, comprising an actuating element 2702, a lock 2704, a hinge 2706 (e.g., a high-friction hinge), and a clamping member 2712. The clamping member 2712 has a length 2713a and a width 2713b. In some embodiments, the height and length of the clamping member are at least 50% of the height and length of the sample used with it, respectively. A larger clamping member provides better support for the reshaped sample during imaging. In some embodiments, the clamping member has ribbed surfaces to improve friction with the sample. Each clamping member 2712 includes an aperture 2714 that allows the sample to be reshaped substantially without deformation (e.g., compression), thereby minimizing sample slack that could lead to sample motion artifacts. The lock 2704 can be used to maintain a specific shape of the sample. Alternatively or additionally, the hinge 2706 may be a high-friction hinge that resists movement (e.g., undesirable opening), thereby improving sample stability during imaging and potentially reducing sample motion artifacts. Figure 28A -C illustrates the horizontal ribs ( Figure 28A ), vertical ribs ( Figure 28B ) and hash rib ( Figure 28C ). Figure 29 An example of a clamp 2900 is shown, which includes an actuating element 2902, a hinge 2906, and two parallel clamping members 2912 for holding the sample 2902. Figure 30An example of a reshaping tool 3000 is shown, comprising three corner pieces 3002a-c that are reshaping a sample 3020 on a sample tray 3030. The corner pieces 3002a-c have flat surfaces that contact the sample 3020, but angled or curved (e.g., circular) surfaces may also be used. The corner pieces 3002a-c can maintain their position on the sample tray 3030 due to one or more of, for example, weight, friction, attraction, and magnetism. Figure 31 A swingable gripper 3100 is shown, comprising three swingable (e.g., flexible) arms 3104a-c and a manipulating element 3102. The manipulating element can be used, for example, to dispose of the gripper during sample positioning and / or reshaping.

[0263] In some implementations, a flattening tool is provided on the sample during imaging, for example, to reduce or eliminate sample movement during imaging. Figures 32 to 35 Examples of usable leveling tools are shown. One or more features from one of the examples of leveling tools are contemplated to be available or suitable for other examples of leveling tools. In some embodiments, the leveling tool is made of injection-molded plastic. In some embodiments, the leveling tool comprises metal (e.g., made of metal). In some embodiments, the leveling tool is sterilizable.

[0264] Figure 32 This is a cross-section of an example of a leveling tool 3200. The leveling tool 3200 includes a support member 3204 and a removable leveling member 3202. The support member 3204 has a shape defining a channel. A sample 3220 is located within the support member 3204 during imaging. The sample 3220 can be partially reshaped by the support member 3204. The removable leveling member 3202 includes a retaining lip 3206 and an insertable portion 3208. The retaining lip 3206 is sized and shaped to rest on the support member 3204, while the insertable portion 3208 is inserted into the channel defined by the support member 3204. The support member 3204 may include one or more pieces. The leveling tool 3200 includes a removable weight 3210, which can be placed on the leveling member 3202 during imaging (e.g., on the top side of the insertable portion 3208) to further level the sample 3220. The removable weight may be made of, for example, metal and / or plastic. The sample 3220 and support member 3204 are mounted on the sample tray 3230. In some embodiments, the sample 3220 and support member 3204 are mounted directly on the transparent imaging window during imaging. The support member 3204 has an annular cross-section (when perpendicular to the...). Figure 32(When the cross-section shown is taken from the cross-section). In some embodiments, the support member has a rectangular cross-section. When the retaining lip 3206 of the leveling member 3202 rests on the support member 3204, the leveling member 3202 is positioned no more than 1 mm above the bottom of the support member 3204.

[0265] Figure 33 This is a cross-section of an example of a leveling tool 3300. Leveling tool 3300 includes features similar to those in leveling tool 3200. For example, leveling tool 3300 includes a support member 3304, a removable leveling member 3302 including a retaining lip 3306 and an insertable portion 3308, and a removable weight 3310. Leveling tool 3300 includes additional features not present in leveling tool 3200. Specifically, the removable leveling member 3302 has a shape defining a plurality of through holes 3303, the size and shape of which are designed to accommodate a fork 3340 (e.g., a square or round hole). For example, each hole may have a size not exceeding 1 cm (e.g., not exceeding 5 mm). In some embodiments, the fork 3340 is used to mark samples (e.g., sample orientation or location) and may additionally or alternatively serve as a reference marker. For example, one or more forks may be placed inside the patient before surgery to identify a portion of the tissue to be removed (e.g., the tip of the fork may be located near a cancerous area). A removable, flat member with one or more holes can accommodate forks, thereby reducing or eliminating the need to remove forks before imaging, which could otherwise be time-consuming and / or damaging to the tissue sample. Multiple holes allow the sample to be oriented in different ways while leaving the forks in approximately their original position and / or accommodating multiple forks. Although not in Figure 33 As shown, but the support member 3304 may additionally or alternatively have a shape defining one or more holes, the size and shape of which are designed to accommodate a fork. The removable weight 3310 does not cover at least one of the holes in the removable leveling member 3302. The leveling tool 3300 is leveling the sample 3320 placed on the sample tray 3330.

[0266] Figure 34This is a cross-section of an example of a leveling tool 3400. The leveling tool 3400 includes a flat top portion 3408 and one or more wings 3405 extending downward from the top portion 3408. In the leveling tool 3400, the one or more wings 3405 are single annular wings. In some embodiments, the leveling tool has rotational symmetry (e.g., triple, quadruple, quintuple, or infinitely sequential symmetry). The one or more wings 3405 extend above the top surface of the flat top portion 3408, thereby defining a recess in which a removable weight 3410 is held in place. The size and shape of the removable weight 3410 are designed to be at least partially (e.g., completely) housed in the recess. The one or more wings 3405 partially cover a sample 3420 disposed on a sample tray 3430. The leveling tool 3400 has a shape defining one or more holes 3403, the size and shape of which are designed to receive a fork 3440. One or more wings 3405 are shaped to define some of the holes in one or more holes 3403, and the flat top portion 3408 is shaped to define other holes in one or more holes 3403. In some embodiments, the wings have a shape without holes. In some embodiments, the flat top portion has a shape without holes. In some embodiments, when one or more removable weights are placed (e.g., on the top surface of the flat top portion), one or more holes (e.g., through the flat top portion) remain uncovered.

[0267] Figure 35 This is a cross-section of an example of a leveling tool 3500. The leveling tool 3500 has some features similar to those of the leveling tool 3400. The leveling tool 3500 further includes one or more weight supports 3507. The inclusion of one or more weight supports 3307 extending from one or more wings 3505 allows a removable weight 3510 to be placed around the periphery of the leveling tool 3500, which helps to distribute the added weight more evenly onto the sample 3520. The sample 3520 is positioned on a transparent imaging window 3530. The one or more weight supports 3507 extend horizontally and parallel to the flat top portion 3508.

[0268] The reshaping tool may include a recessed clamping member and / or a clamping member with an orifice. Figure 36 The image shows a cross-section of a reshaping tool 3600, which includes a concave clamping member 3612 that holds a convex sample 3620 for reshaping on a sample disk 3630. Figure 37This is a cross-section of a reshaping tool 3700, which includes a clamping member 3712 having an orifice for holding a convex sample 3720 (the convex sample protruding through the orifice). Each of the clamping members has a different cross-sectional periphery of the orifice (the left member is elliptical and the right member is square). Figure 38 A reshaping tool 3800 is shown, comprising an actuating element 3802, a high-friction hinge 3706, and recessed clamping members 3812, each of which further includes multiple orifices. The high-friction hinge and / or lock (e.g., as...) Figure 27 The reshaping tool (illustrated in 2700) can be particularly useful for preventing unwanted sample movement during imaging (e.g., caused by forces exerted on the reshaping tool by the reshaping sample). Considering the typical shape of a particular tissue sample (e.g., a breast mass excision sample), concave clamping members and clamping members with orifices can be particularly useful for helping to hold the sample in a stable position (preventing movement) without substantially deforming the sample (e.g., constraining or compressing it). Deforming or constraining the sample can often lead to motion artifacts because the sample has pathways to relax.

[0269] Computer system and network implementation methods

[0270] The foregoing description of illustrative embodiments of the systems and methods disclosed herein refers to computations performed locally by a computing device. However, computations performed over a network are also contemplated. Figure 39 An illustrative network environment 3900 is shown for use in the methods and systems described herein. In a brief overview, reference is now made to... Figure 39 A block diagram illustrating an illustrative cloud computing environment 3900 is shown and described. The cloud computing environment 3900 may include one or more resource providers 3902a, 3902b, 3902c (collectively referred to as 3902). Each resource provider 3902 may include computing resources. In some implementations, computing resources may include any hardware and / or software for processing data. For example, computing resources may include hardware and / or software capable of executing algorithms, computer programs, and / or computer applications. In some implementations, the illustrative computing resources may include application servers and / or databases with storage and retrieval capabilities. Each resource provider 3902 may connect to any other resource provider 3902 in the cloud computing environment 3900. In some implementations, resource providers 3902 may be connected via a computer network 3908. Each resource provider 3902 may connect to one or more computing devices 3904a, 3904b, 3904c (collectively referred to as 3904) via the computer network 3908.

[0271] The cloud computing environment 3900 may include a resource manager 3906. The resource manager 3906 can be connected to resource providers 3902 and computing devices 3904 via a computer network 3908. In some implementations, the resource manager 3906 may facilitate one or more resource providers 3902 to provide computing resources to one or more computing devices 3904. The resource manager 3906 may receive requests for computing resources from a particular computing device 3904. The resource manager 3906 may identify one or more resource providers 3902 capable of providing the computing resources requested by the computing device 3904. The resource manager 3906 may select a resource provider 3902 to provide the computing resources. The resource manager 3906 may facilitate a connection between the resource provider 3902 and the particular computing device 3904. In some implementations, the resource manager 3906 may establish a connection between the particular resource provider 3902 and the particular computing device 3904. In some implementations, the resource manager 3906 may redirect a particular computing device 3904 to a particular resource provider 3902 that has the requested computing resources.

[0272] Figure 40 Examples of computing devices 4000 and mobile computing devices 4050 that can be used in the methods and systems described herein are shown. Computing device 4000 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Mobile computing device 4050 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are intended to be illustrative only and not to be limiting.

[0273] The computing device 4000 includes a processor 4002, a memory 4004, a storage device 4006, a high-speed interface 4008 connected to the memory 4004 and a plurality of high-speed expansion ports 4010, and a low-speed interface 4012 connected to a low-speed expansion port 4014 and the storage device 4006. Each of the processor 4002, memory 4004, storage device 4006, high-speed interface 4008, high-speed expansion port 4010, and low-speed interface 4012 is interconnected using various buses and may be mounted on a common motherboard or otherwise mounted as appropriate. The processor 4002 can process instructions for execution within the computing device 4000, including instructions stored in the memory 4004 or storage device 4006, to display graphical information of a GUI on an external input / output device, such as a display 4016 coupled to the high-speed interface 4008. In other implementations, multiple processors and / or multiple buses may be used together with multiple memories and various types of memory as appropriate. Furthermore, multiple computing devices can be connected, each providing a portion of the necessary operation (e.g., as a server group, a set of blade servers, or a multiprocessor system). Therefore, as used herein, when multiple functions are described as being performed by a “processor,” this covers implementations where the multiple functions are performed by any number of processors (e.g., one or more processors) of any number of computing devices (e.g., one or more computing devices). Additionally, when a function is described as being performed by a “processor,” this covers implementations where said function is performed by any number of processors (e.g., one or more processors) of any number of computing devices (e.g., one or more computing devices) (e.g., in a distributed computing system).

[0274] Memory 4004 stores information within computing device 4000. In some implementations, memory 4004 is a volatile memory cell. In some implementations, memory 4004 is a non-volatile memory cell. Memory 4004 can also be another form of computer-readable medium, such as a magnetic disk or optical disk.

[0275] Storage device 4006 provides large-capacity storage for computing device 4000. In some implementations, storage device 4006 may be or contain computer-readable media, such as floppy disk devices, hard disk devices, optical disk devices, magnetic tape devices, flash memory or other similar solid-state storage devices, or device arrays, including those in a storage area network or other configuration. Instructions may be stored in an information carrier. When executed by one or more processing devices (such as processor 4002), the instructions perform one or more methods, such as those described above. Instructions may also be stored by one or more storage devices, such as computer or machine-readable media (such as memory 4004, storage device 4006, or memory on processor 4002).

[0276] High-speed interface 4008 manages bandwidth-intensive operations of computing device 4000, while low-speed interface 4012 manages lower bandwidth-intensive operations. This allocation of functions is merely an example. In some implementations, high-speed interface 4008 is coupled to memory 4004, display 4016 (e.g., via a graphics processor or accelerator), and high-speed expansion port 4010 that accepts various expansion cards (not shown). In this implementation, low-speed interface 4012 is coupled to storage device 4006 and low-speed expansion port 4014. Various communication ports (such as USB, Bluetooth) may be included. ® The low-speed expansion port 4014 (such as Ethernet, wireless Ethernet) can be coupled to one or more input / output devices, such as keyboards, pointing devices, scanners, or networking devices (such as switches or routers), through a network adapter.

[0277] The computing device 4000 can be implemented in many different forms, as shown in the figures. For example, it can be implemented as a standard server 4020 or multiple times in a group of such servers. Alternatively, the computing device can be implemented in a personal computer, such as a laptop computer 4022. The computing device can also be implemented as part of a rack-mount server system 4024. Alternatively, components from the computing device 4000 can be combined with other components in a mobile device (not shown), such as a mobile computing device 4050. Each of such devices may include one or more of the computing device 4000 and the mobile computing device 4050, and the entire system may consist of multiple computing devices communicating with each other.

[0278] Mobile computing device 4050 includes a processor 4052, memory 4064, input / output devices (such as a display 4054), a communication interface 4066, and a transceiver 4068, as well as other components. Mobile computing device 4050 may also include storage devices, such as microdrives or other devices, to provide additional storage. Each of the processor 4052, memory 4064, display 4054, communication interface 4066, and transceiver 4068 is interconnected using various buses, and some of these components may be mounted on a common motherboard or otherwise mounted as appropriate.

[0279] Processor 4052 can execute instructions within mobile computing device 4050, including instructions stored in memory 4064. Processor 4052 can be implemented as a chipset comprising individual and multiple analog and digital processors. Processor 4052 can provide, for example, coordination of other components of mobile computing device 4050, such as control of user interfaces, application execution performed by mobile computing device 4050, and wireless communication performed by mobile computing device 4050.

[0280] Processor 4052 can communicate with the user via control interface 4058 and display interface 4056 coupled to display 4054. Display 4054 can be, for example, a TFT (Thin Film Transistor Liquid Crystal Display) or OLED (Organic Light Emitting Diode) display, or other suitable display technologies. Display interface 4056 may include suitable circuitry for driving display 4054 to present graphics and other information to the user. Control interface 4058 can receive commands from the user and translate the commands for submission to processor 4052. Additionally, external interface 4062 can provide communication with processor 4052 to enable near-field communication between mobile computing device 4050 and other devices. External interface 4062 may provide, in some implementations, wired communication, or in others, wireless communication, and multiple interfaces may be used.

[0281] Memory 4064 stores information within the mobile computing device 4050. Memory 4064 may be implemented as one or more of a computer-readable medium, a volatile memory cell, or a non-volatile memory cell. Extended memory 4074 may also be provided and connected to the mobile computing device 4050 via an extended interface 4072, which may include, for example, a SIMM (Single In-line Memory Module) card interface. Extended memory 4074 may provide additional storage space for the mobile computing device 4050, or it may store applications or other information for the mobile computing device 4050. Specifically, extended memory 4074 may include instructions for performing or supplementing the above processes, and may also include security information. Therefore, for example, extended memory 4074 may be provided as a security module of the mobile computing device 4050 and may be programmed with instructions that allow secure use of the mobile computing device 4050. Furthermore, secure applications and additional information, such as placing identification information on the SIMM card in an unbreakable manner, may be provided via a SIMM card.

[0282] The memory may include, for example, flash memory and / or NVRAM (non-volatile random access memory), as discussed below. In some implementations, instructions are stored in an information carrier, and the instructions, when executed by one or more processing devices (such as processor 4052), perform one or more methods, such as those described above. Instructions may also be stored by one or more storage devices, such as one or more computer or machine-readable media (such as memory 4064, extended memory 4074, or memory on processor 4052). In some implementations, instructions may be received in a propagated signal, for example, via transceiver 4068 or external interface 4062.

[0283] Mobile computing device 4050 can wirelessly communicate via communication interface 4066, which may include digital signal processing circuitry if necessary. Communication interface 4066 can provide communication under various modes or protocols, such as GSM voice calls (Global System for Mobile Communications), SMS (Short Message Service), EMS (Enhanced Messaging Service) or MMS (Multimedia Messaging Service), CDMA (Code Division Multiple Access), TDMA (Time Division Multiple Access), PDC (Personal Digital Cellular), WCDMA (Wideband Code Division Multiple Access), CDMA2000, or GPRS (General Packet Radio Service), and others. For example, such communication can be performed using radio frequency via transceiver 4068. Alternatively, it can be performed using, for example, Bluetooth. ®Short-range communication can be achieved via Wi-Fi™ or other such transceivers (not shown). Additionally, the GPS (Global Positioning System) receiver module 4070 can provide additional navigation and location-related wireless data to the mobile computing device 4050, which can be used by applications running on the mobile computing device 4050 as appropriate.

[0284] The mobile computing device 4050 can also use an audio codec 4060 for audible communication, which receives spoken information from a user and converts the spoken information into usable digital information. The audio codec 4060 can also generate audible sound for the user, such as through a speaker in a handset of the mobile computing device 4050. This sound may include sound from a voice telephone call, recorded sound (e.g., voice messages, music files, etc.), and may also include sound generated by applications running on the mobile computing device 4050.

[0285] The mobile computing device 4050 can be implemented in a variety of different forms, as shown in the figure. For example, the mobile computing device can be implemented as a cellular phone 4080. It can also be implemented as part of a smartphone 4082, a personal digital assistant, or other similar mobile device.

[0286] Various implementations of the systems and technologies described herein can be implemented in digital electronic circuits, integrated circuits, specially designed ASICs (Application-Specific Integrated Circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs capable of being executed and / or interpreted on a programmable system comprising: at least one programmable processor, which may be dedicated or general-purpose, coupled to receive and transmit data and instructions from and to a storage system, at least one input device, and at least one output device.

[0287] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented in high-level programming languages ​​and / or object-oriented programming languages, and / or in assembly language / machine language. As used herein, the terms machine-readable medium and computer-readable medium refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term machine-readable signal refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0288] To enable interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) by which the user can provide input to the computer. Other types of devices can also be used to enable interaction with the user; for example, feedback provided to the user can be any form of perceptual feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the ability to receive input from the user in any form, including sound, voice, or tactile input.

[0289] The systems and technologies described herein can be implemented in computing systems, including: back-end components (e.g., as data servers); or middleware components (e.g., application servers); or front-end components (e.g., client computers with graphical user interfaces or web browsers, through which users can interact with the implementations of the systems and technologies described herein); or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected via any form of digital data communication or digital data communication medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0290] A computing system may include clients and servers. Clients and computers are generally located far apart and typically interact through communication networks. The relationship between clients and servers is maintained by computer programs running on the respective computers and having a client-server relationship with each other.

[0291] The various embodiments described in this invention may be used in combination with one or more other embodiments, unless technically incompatible.

[0292] Some embodiments of this disclosure have been described above. However, it is explicitly stated that this disclosure is not limited to those embodiments, but rather that additions and modifications to the content expressly described herein are also included within the scope of this disclosure. Furthermore, it will be understood that the features of the various embodiments described herein are not mutually exclusive and can exist in various combinations and arrangements without departing from the spirit and scope of this disclosure, even if such combinations or arrangements are not explicitly stated. Specific implementations of the systems and methods for imaging samples have been described, and it will now be apparent to those skilled in the art that other implementations incorporating the concepts of this disclosure can be used. Therefore, the claimed invention should not be limited to the specific described embodiments, but should be limited only by the spirit and scope of the appended claims.

Claims

1. A method for imaging a sample using a micro-optical element array, the method comprising: The array of micro-optical elements is scanned along a scanning pattern defined by a sequential position array to generate an image of the sample, wherein the sequential position array is an M×N array, where M and N are each not less than 10, and M≥N; The image comprises pixel patches, each patch corresponding to a corresponding micro-optical element in the array, and each pixel corresponding to a position in the sequential position array. Wherein, for each pair of a first pixel in a first block of the image and a second pixel adjacent to the first pixel, the second pixel is located in a block of the image that is different from the first block of the image, and the position difference between the position corresponding to the first pixel and the position corresponding to the second pixel is less than (MN-2M+1).

2. The method according to claim 1, wherein the position difference between the position corresponding to the first pixel and the position corresponding to the second pixel in the scan position sequence of the scan pattern does not exceed (3M-3).

3. The method according to claim 1, wherein the position difference between the position corresponding to the first pixel and the position corresponding to the second pixel in the scan position sequence of the scan pattern does not exceed (2M-1).

4. The method according to claim 1, wherein for each pair of adjacent pixels belonging to the same patch, the position difference between the position corresponding to the first pixel and the position corresponding to the second pixel in the scan position sequence of the scan pattern does not exceed (2M+2N-5).

5. The method of claim 1, wherein for each pair of adjacent pixels belonging to the same patch, the position difference between the position corresponding to the first pixel and the position corresponding to the second pixel in the scan position sequence of the scan pattern does not exceed (2M).

6. The method of claim 1, wherein for each pair of a first pixel in a first block of the block and a second pixel adjacent to the first pixel, the second pixel is located in a block of the block that is different from the first block of the block, and the position difference between the position corresponding to the first pixel and the position corresponding to the second pixel is less than 30% of the total number of scan positions in the scan pattern.

7. The method of claim 1, wherein for each pair of adjacent pixels belonging to the same patch, the position difference between the position corresponding to the first pixel and the position corresponding to the second pixel is less than 30% of the total number of scan positions in the scan pattern.

8. The method of claim 1, wherein the sequential position array forms a spiral.

9. The method of claim 1, wherein the series of sequential positions comprises a series of sequential position rows, wherein each of the sequential rows in the series is time-separated from its spatially adjacent row in the series by no more than two rows in the series.

10. The method of claim 9, wherein the starting position of the scan pattern is in an inner sequential position row of the sequential position rows, and the final position of the scan pattern is in an outer sequential position row of the sequential position rows.

11. The method of claim 10, wherein each row in the series of sequential position rows is not closer to the innermost sequential position row in the series of sequential position rows than the row immediately preceding it.

12. The method of claim 9, wherein the starting position of the scan pattern is in a sequential position row outside the sequential position row, and the final position of the scan pattern is in a sequential position row inside the sequential position row.

13. The method of claim 12, wherein each row in the series of sequential position rows is no further from the innermost sequential position row than the row immediately preceding it in the series of sequential position rows.

14. The method of claim 1, wherein the sequential position array is a regular array.

15. The method of claim 1, wherein during the scanning, (i) illuminating the sample through the micro-optical element, and (ii) collecting a corresponding back-emitted light from the sample with the micro-optical element, and subsequently receiving the corresponding back-emitted light at a detector.

16. The method of claim 15, the method comprising generating the image of the sample by a processor of a computing device based on the corresponding back-emitted light received at the detector.

17. The method of claim 1, wherein the scanning pattern has a size corresponding to the size of a unit cell of a micro-optical element in the micro-optical element array.

18. The method of claim 9, wherein the scanning pattern is unidirectional.

19. The method of claim 9, wherein the scanning pattern is bidirectional.

Images