Speckle-based image distortion correction for laser scanning microscopy

Abstract

To provide methods and systems that make use of naturally occurring speckles from an illuminating laser to determine how much distortion in a scanning direction has occurred and accordingly modifying the image to correct the distortion.SOLUTION: A method of correcting distortion of an image includes: analyzing, by a processor, an image segment of the image to identify a speckle artifact, the image segment being obtained from a scanning imaging device; determining, by the processor, an aspect ratio of a shape of the speckle artifact; determining, by the processor, a correction factor for the shape of the speckle artifact based on the aspect ratio; and adjusting, by the processor, a dimension of the image segment based on the correction factor.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] This application is based on and claims the benefit of and priority to U.S. Provisional Patent Application No. 62 / 797,548, filed January 28, 2019, the entire disclosure of which is incorporated herein by reference for all purposes.

[0002] This invention was made with government support under Grant No. R01DK091923 awarded by the National Institutes of Health. The government has certain rights in this invention. [Background technology]

[0003] Image distortion caused by non-uniform laser scanning can be a challenge in certain laser scanning imaging devices, such as those used in catheter endoscopy and capsule endoscopy. For example, rotating or scanning components have a certain amount of friction, which can vary during rotation depending on, for example, the orientation of the imaging device or the waveform of the scan. Because this friction is generally unknown and unpredictable, distortions due to such friction can persist in the generated image data, such as geometric distortion of the sample. Summary of the Invention

[0004] Disclosed herein are methods and systems for correcting distortion in images from an imaging probe, including techniques for measuring and correcting image distortion based on the shape and / or size of speckles or other features appearing in the image.

[0005] In various embodiments, the methods and systems may include one or more of the following:

[0006] 1) The scanning speed is determined based on the shape and / or size of the speckles.

[0007] 2) Calculating the speckle shape and / or size based on the autocovariance of the image, for example after filtering the image with a high-pass filter.

[0008] 3) Calibration is performed to adjust for the non-linear relationship between the speed of imaging probe movement (eg, rotational or linear movement) versus speckle shape and / or size.

[0009] 4) Aligning the multiple distortion-compensated lines by minimizing cross-correlation.

[0010] These correction techniques can be employed in any imaging device with a rotational and / or linear scanning component, and because they can be implemented in software to measure distortion, they can make products incorporating embodiments of the methods and systems disclosed herein simpler, more cost-effective, and more robust in their results.

[0011] In one embodiment, the present invention encompasses a method for correcting image distortion, the method comprising the steps of: analyzing, by a processor, image segments of an image obtained from the scanning imager to identify speckle artifacts; determining, by a processor, an aspect ratio of a shape of the speckle artifact; determining, with a processor, a correction factor for the shape of the speckle artifact based on the aspect ratio; Adjusting, by the processor, the dimensions of the image segment based on the correction factor.

[0012] In another embodiment, the invention encompasses an apparatus for correcting image distortion, the apparatus comprising: a processor; a memory in communication with the processor and storing a set of instructions; It is equipped with A set of instructions, when executed by a processor, causes the processor to: analyzing image segments of images obtained from the scanning imaging device to identify speckle artifacts; Calculating the aspect ratio of the shape of the speckle artifact; determining a correction factor for the shape of the speckle artifact based on the aspect ratio; The dimensions of the image segments are adjusted based on the correction factor.

[0013] In yet another embodiment, the invention encompasses a distortion correction method, the distortion correction method comprising: Providing an image; Dividing the image into a plurality of substantially parallel strips, each strip extending in a first direction; Dividing each strip into a plurality of sub-strips along a second direction substantially perpendicular to the first direction; analyzing each of the plurality of sub-strips to locate at least one locally bright feature; adjusting at least a portion of each sub-strip to cause at least one locally bright feature to approximate a predetermined shape, thereby creating a corrected sub-strip; reconstructing the corrected partial strips into a correction strip; reconstructing the plurality of correction strips into a corrected image; It has.

[0014] In yet another embodiment, the invention encompasses an apparatus for correcting image distortion, the apparatus comprising: a scanning imager including a coherent light source and configured to acquire an image of the sample; a processor in communication with the scanning imager; a memory in communication with the processor and storing a set of instructions; It is equipped with A set of instructions, when executed by a processor, causes the processor to: analyzing image segments of the image to identify speckle artifacts; Calculating the aspect ratio of the shape of the speckle artifact; determining a correction factor for the shape of the speckle artifact based on the aspect ratio; The dimensions of the image segments are adjusted based on the correction factor.

[0015] A more complete understanding of the various objects, features, and advantages of the presently disclosed invention can be obtained by considering the following detailed description of the presently disclosed invention in conjunction with the following drawings, in which like reference numerals represent like elements and in which: [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 illustrates an example system that can be used to collect data for use with embodiments of the methods and systems disclosed herein. [Figure 2] 1A and 1B show examples of speckle before (left panel, inset) and after (right panel, inset) distortion compensation according to embodiments disclosed herein. [Figure 3] Figure 1 shows the relationship found between image-based strain estimates based on speckle aspect ratio (x-axis) and actual strain calculated by measuring the aspect ratio of spherical microbeads (y-axis) (numbers on each axis are unitless ratios). Various curve-fitting techniques are shown superimposed on the data as part of the process for calibrating the relationship between strain determined using speckle in the image and actual strain. [Figure 4] 4 is a graph showing the analysis results of FIG. 3 with the results of a bisquare curve fitting procedure superimposed on the data. [Figure 5] 4 is a graph showing the analysis results of FIG. 3 with the results of a conventional curve fitting procedure (based on the mathematical formulas of the Bisquare curve fitting procedure of FIG. 4) superimposed on the data. [Figure 6] FIG. 10 shows how one sub-strip (red square area in the top image) is adjusted to form a corrected sub-strip (bottom image). [Figure 7] FIG. 10 illustrates how the distortion correction described herein can be used to create smoother, more regular images by applying compensation in the axial scan direction. [Figure 8] 10 is a flowchart illustrating an example of an operation sequence of a distortion correction algorithm according to the present invention. [Figure 9] FIG. 1 illustrates a computer system for implementing embodiments of the methods and systems disclosed herein. [Figure 10] FIG. 2 illustrates an example process for correcting image distortion in accordance with some embodiments of the presently disclosed invention. [Figure 11] FIG. 2 illustrates an example process for distortion correction according to some embodiments of the presently disclosed invention. DETAILED DESCRIPTION OF THE INVENTION

[0017] Some embodiments of the invention disclosed herein provide mechanisms (which may include systems, methods, and media) for image distortion correction.

[0018] While laser speckle is sometimes considered an unwanted artifact, advantageous embodiments disclosed herein use this laser speckle spot, which is assumed to be substantially circular, to identify the amount and orientation of any distortions introduced into the image based on the degree of deviation of the speckle from a substantially circular shape. In embodiments, the features used to compensate for distortions in the disclosed procedures can be readily distinguished using the disclosed automated procedures from features with particularly localized brightness, such as laser speckle. Although these features are referred to herein as "laser speckle," they may originate from different sources, but are useful for identifying and correcting image distortions.

[0019] In many imaging applications (e.g., capsule endoscopy using an illumination laser), undesirable distortion artifacts can occur. For example, in a 360° rotation capsule endoscopy application, the motors that rotate the illumination laser and / or the image sensor can stick and / or slip. In such cases, the rotation is not smooth, resulting in distortion in the captured image. This is because image reconstruction from scan data is performed under the assumption that the data points of the scanned object are equally spaced. That is, a sequence of data points is collected while the imaging probe is rotated and / or moved through the sample. In the case of a helical scan, obtained by passing the probe through an intraluminal sample while simultaneously rotating and moving the probe, an image of the sample is generated by assembling the sequence of data points into a two-dimensional or three-dimensional representation based on the assumption that the probe is rotating and moving at a uniform speed. However, friction in the device (e.g., wear on the motor itself) can cause distortion. Friction (friction between the rotating optical fiber and the sheath that houses it, or any of a variety of other possible sources of friction) can cause sticking or slippage, which results in non-uniform motion, which can cause distortions in the images generated from the data stream.

[0020] The disclosed distortion correction method and system thus utilizes the naturally occurring speckle from the illuminating laser to determine how much distortion has occurred in the scan direction, and modifies the image accordingly to account for and compensate for that distortion.

[0021] FIG. 1 illustrates an example system 100 that can be used to collect data for use with embodiments of the methods and systems disclosed herein. System 100 includes a catheter 110 within which an optical probe 120 is disposed, comprising a reflector 130 (e.g., a ball lens, as shown, or a prism or mirror) at the end of a waveguide 140 (e.g., an optical fiber). A light source 150 provides illumination to waveguide 140, which transmits the light to reflector 130 of optical probe 120, which transmits beam 122 to a sample 160 (e.g., a luminal tissue such as a blood vessel or a portion of the digestive system). Light from sample 160 returns along this path to an image acquisition and processing system 170, which collects and processes the returned light to form an image or other data. Light source 150 is a coherent light source, such as a laser, which generates speckle artifacts that are used in the procedures of the present invention to perform image correction.

[0022] The optical probe 120 can be rotated 124 and / or translated 126 during data collection to collect data from the entire circumferential region of the sample 160 (by rotation 124) and along the axial length of the sample 160 (by translation 126). The waveguide 140 can be rotated 124 and / or translated 126 within the sheath 142 before reaching the catheter 110. To facilitate rotation and translation of the optical probe 120, a rotation / translation device 180 is attached to the proximal end of the catheter (which can also include a rotating photocoupler). The rotation / translation device 180 can include a linear motor for translating the optical probe 120 and a motor (e.g., a hollow-core motor) for rotationally translating the optical probe 120. In some embodiments, instead of the catheter 110, the optical probe 120 can be housed within a capsule device that the patient swallows, particularly for probing the digestive system. In some embodiments, particularly capsule-based embodiments, a motor can be located at the distal end adjacent to the reflector 130 to rotate the reflector 130. In some embodiments, the optical probe 120 can rotate at a speed of 1-30 Hz, 1-100 Hz, 1-1000 Hz, or 1-10,000 Hz, and can move in a straight line at a speed of 1-10 mm / min.

[0023] In various embodiments, factors that can cause image distortion include sticking or slippage of the waveguide 140 within the sheath 142 or catheter 110 and / or sticking or slippage of components in the rotary / translatory device 180, such as a rotary or linear / translatory motor. In certain embodiments, the correction procedures disclosed herein can be performed after the data is collected, e.g., by processing stored data at a later date. In other embodiments, the correction procedures can be performed simultaneously with data collection, e.g., by processing the data as it was collected and storing a corrected version of the data. In certain embodiments, the correction procedures can be performed at substantially the same speed as the data is collected, such that the processing is effectively real-time.

[0024] FIG. 2 shows example speckle before (left panel, inset) and after (right panel, inset) distortion compensation according to embodiments disclosed herein. In some embodiments, the entire image (shown uncorrected in the left panel of FIG. 2) is first distorted into multiple small speckles. The image may be divided into strips. For example, in some embodiments, the image may be divided into strips having a first dimension (e.g., the horizontal dimension in FIG. 2 ) and a second dimension substantially orthogonal to the first dimension, where the first dimension may be in the range of 300-500 pixels (400 pixels in particular embodiments), and the second dimension may be determined by a dimension of the original image, e.g., the “width” of the original image, where the width of the image (e.g., the vertical dimension in FIG. 2 ) may correspond to a direction of movement (e.g., a direction of rotation), particularly data collected from one revolution of a rotating imaging probe through a sample lumen. In this case, the image is a flattened representation of a tubular dataset. FIG. 2 assumes a 360° rotation that “spirals” downward through the lumen, e.g., generating a helical dataset by rotating the imaging probe while advancing it linearly, where the left side of each panel in FIG. 2 corresponds to one end of the lumen. (See the vertical dashed lines in the left panel of FIG. 2, which show how an image can be divided into a possible set of image strips.) In certain embodiments, the number and size of strips in an image can be set relative to the total height of the strip, thereby, for example, dividing each image into multiple strips of substantially equal size.

[0025] Each strip can then be subdivided lengthwise (e.g., along the vertical dimension in FIG. 2) into multiple partial strips or "frames." For example, each partial strip can be substantially square, e.g., in the range of 300-500 pixels, or in more specialized cases, a "long" partial strip of 400 pixels. The number and size of the partial strips within a strip can be set relative to the total length of the strip, thereby, for example, dividing each strip into multiple partial strips of substantially equal size.

[0026] After forming the sub-strips, each sub-strip is analyzed for areas of high or low illumination, searching for, for example, "bright spots" or other features indicative of areas of laser speckle. In certain embodiments, an image filtering procedure is used to automatically identify speckle or "bright spots." In one particular embodiment, a high-pass filtering technique may be used, for example, to subtract an original image from a filtered copy of the original image that has reduced or removed "noise" (e.g., using an adaptive filter such as a Wiener filter) to produce a resultant image in which the only feature or dominant feature of the image is speckle or "bright spots."

[0027] After one or more "bright spots" or other features are identified in the partial strip, the shape of each "bright spot" or feature is analyzed. In certain embodiments (see below) in which the size of the partial strip or frame is selected so that the partial strip or frame contains at most one speckle or "bright spot" or other feature, an automatic procedure can be used to recognize the boundaries of such features. For example, this can be done by mathematically determining the x- and / or y-dimensions of all features based on their full width at half maximum (FWHM). If distortion is present, the ratio of the FWHM of the horizontal (or x-) and vertical (or y-) dimensions of the feature (i.e., the aspect ratio) will indicate this distortion. For example, an aspect ratio of approximately 1 indicates little or no distortion, whereas an aspect ratio value greater than or less than 1 indicates the presence of distortion in the image (e.g., stretching or shrinking in a particular direction).

[0028] In some embodiments, a procedure can be used to identify feature aspect ratios based on calculating the autocovariance of image data (or calculating the autocovariance of processed image data that has enhanced speckles, "bright spots," or other features, as described above). This procedure can include: Feature width values ​​in two orthogonal directions can be returned. For example, the SpeckleSize_() function in the code shown below uses autocovariance to process substrips and returns feature width values ​​in two orthogonal directions that can account for multiple speckles or other features in a given substrip. If there is more than one such feature in a substrip, the returned feature width value in each direction will take into account the dimensions in each direction of all two or more of these features, effectively providing an average of these dimensions. While some embodiments determine feature width by calculating the full width at half maximum (FWHM), in the embodiment shown in the code below, Gaussian values ​​of 0.5 and 1 / e 2The feature width is determined using the width where the Gaussian function falls to a value of . This determination of the feature width provides a better understanding of the size of the speckle, similar to what would be expected for a good fit of the normalized data, even when the amplitude of the Gaussian function is not 1 (i.e., the Gaussian function is not normalized so that the amplitude is equal to 1.0), but this procedure can produce unexpected trends.

[0029] After determining the degree of distortion from the aspect ratio, the partial strips are actually stretched or contracted to bring each speckle, "bright spot," or other feature closer to a substantially circular contour (i.e., "undistorted"), thereby creating a corrected partial strip or frame. This analysis and distortion process is repeated for each partial strip of a given strip, after which the corrected partial strips are reassembled (e.g., "stitched") in their original order to form the corrected strip.

[0030] In some embodiments, the aspect ratio may be further adjusted before using the value to undo the distortion in the partial strip or frame. For example, the "measured" aspect ratio may be adjusted to obtain an "actual" aspect ratio, which may then be applied to the partial strip or frame to provide an adjustment appropriate to compensate for the distortion. This adjustment of the measured aspect ratio may be performed using any one of the curve-fitting formulas obtained by the procedures shown in Figures 3-5 and the accompanying text (see below), or any other suitable adjustment.

[0031] In one embodiment, the dimensions of the strips and partial strips can be selected so that each partial strip contains, on average, at most one speckle or "bright spot" or other feature. Adjusting the dimensions of the strips and partial strips to include at most one speckle or other feature allows adjustment of each partial strip based on one speckle or other feature rather than two or more features in a partial strip. However, in other embodiments, the dimensions of the strips and partial strips can be selected to include, on average, two, three, four, or more speckles, "bright spots," or other features. In some embodiments, the size of the partial strips or frames is selected so that when a partial strip or frame contains multiple (e.g., 2, 3, or more) speckles, "bright spots," or other features, the features are close enough together that the amount of distortion that occurs to these features is equal or substantially equal, so that all correction factors applied to the partial strips or frames have a substantially equal effect on constraining all features to a near-circular outline. If the partial strips or frames are excessively large, they may contain distorted areas as well as other undistorted areas, which will affect the determination of the correction factors and the degree to which the image distortion is adequately compensated for. In some embodiments, the partial strips may be processed so that they are square or nearly square, with a substantially equal number of pixels in each dimension.

[0032] In some embodiments, the motion-based distortion may be relatively localized and may be, for example, This can occur due to a slight delay in motion when friction is applied to the lobe, or a slight acceleration in motion when friction is removed from the imaging probe (this is sometimes referred to as "intra-rotation NURD"; see below). Due to the local nature of the distortion, the amount of correction may need to be different for two adjacent partial strips, and a distorted partial strip may be adjacent to an undistorted partial strip that does not require any correction. Furthermore, distortion in a rotating imaging probe may occur primarily in the direction of rotation (which corresponds to the vertical dimension in FIG. 2 ). Therefore, in some embodiments, adjustments (e.g., changes in aspect ratio) of specific partial strips identified based on speckle, "bright spots," or other features can be made by changing (lengthening or shortening) only the dimension of the partial strip in the direction of imaging probe movement (e.g., the vertical dimension in FIG. 2 ), while keeping the dimension perpendicular to that direction fixed. An advantage of this approach is that the adjusted partial strips can be recombined into a single strip of uniform width.

[0033] In certain embodiments, motion-based distortion and / or correction of image data to compensate for such distortion may result in data discontinuities between adjacent image strips (this is sometimes referred to as "rotational NURD" - see below). Various procedures may be applied to properly align and recombine the image strips, including the following procedure, which involves measuring the cross-correlation of adjacent strips (which is described, for example, with reference to the flowchart in FIG. 8):

[0034] Each strip is similarly split, analyzed / distorted, and reintegrated, and the multiple correction strips are reintegrated in their original order to form a corrected image. An example of a corrected image is shown in the right panel of Figure 2. The corrected image can then be further utilized for various purposes.

[0035] Adjacent sub-strips of a strip and / or adjacent strips of an image can be cross-correlated in any desired manner to align the edges of portions of the image (e.g., individual laser speckles) and thereby assist in accurately integrating the correction sub-strips and / or correction strips. As a simple example, if one correction strip shows a substantially semicircular artifact with a known curvature about its circumference and an adjacent correction strip shows an opposing substantially semicircular artifact with a very similar or identical circumference, the two correction strips can be presumed to be parts of a single substantially circular artifact and "matched" because the semicircular artifacts "match" or the circumferences of the semicircular artifacts are otherwise aligned. The correction image can be provided for further processing and / or analysis in any suitable form, including various forms that are user-perceptible. Corrected images generated using the techniques disclosed herein can be communicated and / or presented to a user (e.g., a researcher, operator, clinician, etc.) and / or stored (e.g., as part of a medical record or research database associated with the subject).

[0036] To calibrate the distortion correction procedure, we collected and analyzed image sequences of fluorescent beads of different sizes (Figure 3). Using this procedure, we confirmed that naturally occurring speckle is an effective means of identifying and correcting image distortions and adjusting any deviations in the corrections determined using speckle to align with distortions identified using a reference standard, such as microbeads. The actual distortion in the image was determined based on a measurement of the aspect ratio of each bead, where a bead is considered to have a spherical structure if the aspect ratio is 1. Therefore, any deviation of the bead aspect ratio from 1 is considered to indicate the presence of actual distortion in the image. We also performed an image-based distortion estimation based on the speckle aspect ratio, again assuming an aspect ratio of 1. Thus, Figure 3 3 shows the relationship found between image-based distortion estimates based on speckle aspect ratio and the actual distortion calculated by measuring the aspect ratio of spherical microbeads (both axes are unitless). This relationship was used to assist in the calibration of software for one example distortion correction task.

[0037] In Figure 3, each plotted circle represents an image or portion of an image for which aspect ratios were determined using both microbeads and speckles. The calculated aspect ratio for each image is shown on the x-axis for speckle-based aspect ratios and on the y-axis for microbead-based aspect ratios. Because we determined that the aspect ratios (and thus estimated distortions) produced using the two procedures did not perfectly match, we used multiple functions to identify a mathematical relationship between the results obtained using the two procedures. A simple linear fit of the data produced a line with a slope of 1.1 in Figure 3, whereas the various curves shown in the figure were generated by other fitting procedures. In addition to linear fitting, other procedures include least absolute value (LAR), Bisquare (see Figure 4), and Robust Off (which stands for "Turning off any robust approximation"; i.e., a standard least-squares polynomial fit, as opposed to LAR or Bisquare fitting). Additionally, a conventional curve-fitting formula was determined based on the Bisquare formula ("Conventional"; see Figure 5).

[0038] FIG. 4 shows the results of curve fitting the data set using the Bisquare technique with the following parameters:

[0039] f(x)=p1×x 3 +p2×x 2 +p3×x+p4

[0040] where the coefficients (with 95% confidence limits) are:

[0041] p1=07736(0.4771,1.07)

[0042] p2=-1.865(-2.779,-0.951)

[0043] p3=2.739(1.896,3.581)

[0044] p4=-0.427(-0.6545,-0.1994)

[0045] FIG. 5 shows the results of curve fitting the data set using conventional techniques with the following parameters:

[0046] f(x)=p1×x 3 +p2×x 2 +p3×x+p4

[0047] where the coefficients (with 95% confidence limits) are:

[0048] p1=1.619(1.303,1.934)

[0049] p2=-3.786(-4.759,-2.813)

[0050] p3=4.088(3.191,4.985)

[0051] p4=-0.7122(-0.9544,-0.47)

[0052] See Figure 4 for further details.

[0053] The test sample used to generate the images in Figures 6 and 7 contained a set of dry microspheres, which were attached to the inside wall of a vial and imaged using a capsule-type imager. The microspheres were 50.2 μm ± 0.3 μm in size (4K-50, Thermo Scientific 3K / 4K Series Particle Counter Standard). The microspheres were large enough to see the circular shape of each microsphere in the image, yet small enough to fit within a single field of view measuring 200 μm × 200 μm; however, microspheres of other sizes, such as less than 100 μm or more than 10 μm, can also be used. The image in Figure 7 was collected at a reduced magnification and therefore represents a larger field of view than Figure 6.

[0054] FIG. 6 illustrates how one sub-strip (the area within the dashed box on the left side of the top panel of FIG. 6 ) is adjusted to form a corrected sub-strip (the lower panel of FIG. 6 ). The top panel of FIG. 6 shows an image strip, and the left portion of the top panel (enclosed in the dashed box and labeled “Before Correction”) is the sub-strip that is processed using the procedure disclosed herein to identify the degree of distortion and determine a correction factor to undo the distortion. The bottom panel of FIG. 6 shows the sub-strip after its aspect ratio has been adjusted based on the correction factor determined using the procedure disclosed herein. While the compensation performed in this example is horizontal stretching only, and thus stretching / compensation in the scan direction is used for correction in this example, any combination of image stretching or shrinking in any direction may be useful in creating a corrected sub-strip for a particular application environment. Also of note is the similar pattern of light and dark areas in the “before” and “after” regions within the red outline.

[0055] FIG. 7 illustrates how the distortion correction described herein can be used to create smoother, more regular images not only in the rotational scan direction but also in a secondary (e.g., axial) scan direction. The left panel shows a test or phantom image before distortion correction is applied, and the right panel shows the same image after distortion correction. The vertical dimension in each panel is the rotational scan direction, and the horizontal dimension in each panel is the linear scan direction. As can be seen in the left panel image before distortion correction, many features in the image are misaligned horizontally, whereas in the distortion-corrected image in the right panel, the features are more closely aligned from left to right. While this sample is a phantom used for demonstration purposes, one skilled in the art can understand how the distortion correction disclosed herein can be used to assist in more accurate and more useful imaging using laser speckle-assisted processes (e.g., imaging the interior of a patient's body lumen to detect and / or monitor lesions).

[0056] 8 is a flow chart illustrating a method 800 for distortion correction according to an embodiment of the present invention. Step 810 may involve loading data on memory 812, followed by dividing the data through each rotation 814, followed by initializing a variable i=1 816. Method 800 continues at 818 until i>number of rotations, at which point method 800 proceeds to end 820.

[0057] The next step in method 800 may include converting 822 the data segment into a plurality of rotated image strips. Then, method 800 may include dividing 824 each image strip into a plurality of frames or sub-strips. A variable j may be initialized 826, setting j=1. Analysis of each sub-strip or frame continues until j>number of frames 828. Method 800 may perform the following steps for each sub-strip or frame: filtering the strips of each frame; Step 830 extracts a speckle image, step 832 measures the aspect ratio of the speckle autocovariance, step 834 calculates a correction factor from the aspect ratio, and step 836 adjusts the size of the image in the rotational direction using the correction factor. After analyzing each sub-strip or frame, variable j is incremented to j=j+1 838, and control is returned to step 828 to determine whether j>the number of frames. These steps of compensating for the distortion of each sub-strip or each frame correct rotational NURD (rotational non-uniform distortion).

[0058] If all frames for a particular image strip have been analyzed (j > number of frames in step 828), then rotational NURD, i.e., adjustments to correct for distortion between adjacent image strips, are performed. The first step may involve measuring 840 the cross-correlation between the current image strip and the previous image strip, the next step may involve calculating 842 the phase delay value that maximizes the cross-correlation, and the next step may involve aligning 844 the current strip with the previous strip using this phase delay. Once this is complete, the variable i may be incremented so that i = i + 1 846, and control returns to step 818 to determine whether i > number of rotations. If i is greater than the number of rotations, then the method 800 reaches the end 820.

[0059] In some embodiments, commercially available data analysis software can be used to implement the procedures of the present disclosure; one particular embodiment of such code for the Matlab software package is provided below.

[0060] %Speckle-based distortion correction

[0061] data_ND = [];

[0062] for s = 1:hori / vert

[0063] ind = 1+(s-1)*vert:s*vert;

[0064] img = data1(:,ind);

[0065] min_int = mean(min(img));

[0066] max_int = mean(max(img));

[0067] avg_int = mean(mean(img));

[0068] med_int = median(median(img));

[0069] imgf = img - wiener2(img,

[0022] ); % Wiener filter

[0070] [size_x, size_y] = size(imgf);

[0071] imgf = imresize(imgf, 2);

[0072] [HFWHM, VFWHM] = SpeckleSize_gray(imgf);

[0073] mNURD = VFWHM / HFWHM;

[0074] dNURD = mNURD + 0.8109...

[0075] -(-451.6 * exp(-0.0007511 * avg_int)...

[0076] + 0.9899 * exp(-3.833e - 06 * avg_int));

[0077] rNURD = 1.619 * dNURD^3 - 3.786 * dNURD^2 + 4.088 * dNURD - 0.7122; % Conventional

[0078] rNURD = max([rNURD 1 / vert]); %prevent negative coefficients

[0079] img_ND = imresize(data1(:,ind),[size_x round(size_y*rNURD)]);

[0080] data_ND = [data_ND,img_ND];

[0081] logdata = [int2str(i),' ',...

[0082] int2str(s),' ',...

[0083] int2str(min_int),' ',...

[0084] int2str(max_int),' ',...

[0085] int2str(avg_int),' ',...

[0086] int2str(med_int),' ',...

[0087] num2str(HFWHM),' ',...

[0088] num2str(VFWHM),' ',...

[0089] '\r\n'];

[0090] fprintf(logID,logdata);

[0091] %Write to TIF file

[0092] maxint = mean(max(img)) / 2;

[0093] minint = mean(min(img));

[0094] img = uint16((img-minint) / maxint*(2^16));

[0095] maxint = mean(max(img_ND)) / 2;

[0096] minint = mean(min(img_ND));

[0097] img_ND = uint16((img_ND-minint) / maxint*(2^16));

[0098] index = ceil(i / Num_steps);

[0099] fname = strcat(bead_folder,c,num2str(index),'-',...

[0100] num2str(s),' (',num2str(VFWHM / HFWHM),')');

[0101] imwrite(img,strcat(fname,'.tif'),'tif');

[0102] fname = strcat(nurd_folder,c,num2str(index),'-',...

[0103] num2str(s),' (',num2str(VFWHM / HFWHM),')_ND');

[0104] imwrite(img_ND,strcat(fname,'.tif'),'tif');

[0105] end

[0106] data1 = data_ND;

[0107] if size(data1,2) < hori / 2

[0108] data1 = data0;

[0109] end

[0110] % background compensation

[0111] hori_cur = size(data1,2);

[0112] hori_max = max(hori_max,hori_cur);

[0113] dummy = 0; data1_min = min(data1)';

[0114] data1_temp = [data1_min(hori_cur-dummy+1:hori_cur);

[0115] data1_min;data1_min(1:dummy)];

[0116] ylower = smooth(data1_temp,200)';

[0117] ylower = ylower(dummy+1:hori_cur+dummy);

[0118] comp_BG = repmat(mean(ylower). / ylower,vert,1);

[0119] data1 = data1.*comp_BG;

[0120] % Image realignment based on cross-correlation

[0121] data1 = circshift(data1,[0 preDiff]);

[0122] strip_prev = smooth(mean(data0)-mean(mean(data0)),2000);

[0123] strip_curr = smooth(mean(data1)-mean(mean(data1)),2000);

[0124] [acor,lag] = xcorr(strip_prev,strip_curr); length_acor = length(acor);

[0125] wfunc = normpdf(1:length_acor,(length_acor+1) / 2,(length_acor+1) / 2 / 16)';

[0126] acor = acor.*wfunc; [~,I] = max(acor); cofactor = (cofactor*(i-1)+I) / i;

[0127] lagDiff = lag(I)*(i>1); preDiff = lagDiff;

[0128] data1 = circshift(data1,[0 lagDiff]);

[0129] data0 = data1;

[0130] The following is the Matlab code for the SpeckleSize_gray() function.

[0131] function [HFWHM,VFWHM]=SpeckleSize_gray(SpeckleImg)

[0132] M = size(SpeckleImg,1); %area height

[0133] N = size(SpeckleImg,2); %area width

[0134] SpeckleImg = double(SpeckleImg);

[0135] E = SpeckleImg(1,:); %Create an array with row 1 values

[0136] s = size(xcov(E)); %Find the size of the autocovariance array

[0137] D = zeros(s); %Create an empty array of size s

[0138] D = double(D); %Cast the value to double type

[0139] for i = 1:M

[0140] C = SpeckleImg(i,:);

[0141] D = imadd(D,xcov(C,'coeff')); %Add the xcov arrays of all rows and store them in D

[0142] end

[0143] H0 = D / max(D); %normalize the finished horizontal result H

[0144] sizeH = size(H0,2);

[0145] H = H0 (floor(sizeH / 2)-5 : floor(sizeH / 2)+5);

[0146] E1 = SpeckleImg(:,1); %Create an array with values ​​in column 1

[0147] s1 = size(xcov(E1)); %Find the size of the autocovariance array

[0148] D1 = zeros(s1); %Create an empty array of size s1

[0149] D1 = double(D1); %Cast the value to double type

[0150] for j = 1:N

[0151] C1 = SpeckleImg(:,j);

[0152] if max(C1)>0

[0153] D1 = D1 + xcov(C1,'coeff'); %Add the xcov arrays of all rows and store them in D1

[0154] end

[0155] end

[0156] V0 = D1 / max(D1); %normalize the finished vertical result V

[0157] sizeV = size(V0);

[0158] V= V0(floor(sizeV / 2)-5 : floor(sizeV / 2)+5);

[0159] %H and V are fitted to a Gaussian and speckle size is extracted from this fit

[0160] helper1 = 1:size(H,2);

[0161] helper2 = 1:size(V);

[0162] gauss1 = fittype('gauss1'); %Set up Gaussian curve fitting

[0163] excludeLowH = excludedata(helper1',H','range',[.1,1]); %Remove noise from outside the speckle

[0164] excludeLowV = excludedata(helper2',V,'range',[.1,1]);

[0165] optionsH = fitoptions(gauss1);

[0166] optionsV = fitoptions(gauss1);

[0167] optionsH.Exclude = excludeLowH;

[0168] optionsV.Exclude = excludeLowV;

[0169] [HFit, HFitStats] = fit(helper1',H',gauss1, optionsH);

[0170] [VFit, VFitStats] = fit(helper2',V,gauss1, optionsV);

[0171] HFWHM = (2*(HFit.c1)*sqrt(-log(.5 / (HFit.a1)))); %FWHM value( Full width when fit=.5)

[0172] VFWHM = (2*(VFit.c1)*sqrt(-log(.5 / (VFit.a1))));

[0173] HeSquared = ((HFit.c1)*sqrt(-log((.1353353) / (HFit.a1)))); %1 / e 2 Value (current (Total width when fit = .135...)

[0174] VeSquared = ((VFit.c1)*sqrt(-log((.1353353) / (VFit.a1))));

[0175] end

[0176] 9 is a schematic block diagram illustrating an example computer system 900 with hardware components capable of implementing embodiments of the systems and methods for identifying and correcting distortions disclosed herein, and the system 900 may include various systems and subsystems. The system 900 may be a personal computer, laptop computer, workstation, computer system, appliance, application specific integrated circuit (ASIC), server, server blade center, server farm, etc., and may be distributed across one or more computing devices.

[0177] The system 900 may include a system bus 902, a processing unit 904, a system memory 906, storage devices 908 and 910, a communication interface 912 (e.g., a network interface, etc.), a communication link 914, a display 916 (e.g., a video screen, etc.) or other output device, and input devices 918 (e.g., a keyboard, mouse, touch screen, touch pad, etc.). The system bus 902 may be in communication with the processing unit 904 and the system memory 906. The additional storage devices 908 and 910 may include various non-transitory storage devices, such as hard disk drives, servers, standalone databases, or other non-volatile memory, and may be in communication with the system bus 902. The system bus 902 interconnects the processing unit 904, the storage devices 906, 908, and 910, the communication interface 912, the display 916, and the input device 918. In some examples, the system bus 902 may also be connected to one or more additional ports, such as a universal serial bus (USB) port, an Ethernet port, or other communication mechanism / connection.

[0178] The processing unit 904 may be a computing device and may include an application specific integrated circuit (ASIC) or other processor or microprocessor. The processing unit 904 executes an instruction set to perform the operations of the example embodiments disclosed herein. The processing unit may include a processing core.

[0179] Additional storage devices 906, 908, and 910 may store data, programs, instructions, dataset queries in textual or compiled form, as well as any other information that may be required for the computer to operate. Memories 906, 908, and 910 may be embodied as computer-readable media (embedded or removable), such as memory cards, disk drives, compact discs (CDs), or servers accessible over a network. In particular cases, memories 906, 908, and 910 may include text, images, video, and / or audio, some of which may be available in a human-understandable format.

[0180] Additionally or alternatively, the system 900 can access external data or query sources via a communication interface 912 that can communicate with the system bus 902 and a communication link 914 .

[0181] In operation, system 900 can be used to implement one or more portions of the distortion correction of the present invention. Computer-executable logic for implementing portions of the distortion correction resides, in particular cases, in one or more of system memory 906 and storage devices 908 and 910. Processing device 904 executes one or more computer-executable instructions from system memory 906 and storage devices 908 and 910. As used herein, the term "computer-readable medium" refers to one or more media that participate in providing instructions to processing device 904 for execution.

[0182] Thus, it is understood that the computer-readable medium is a non-transitory medium and may include multiple distributed media operably connected to the processing device, for example, via one or more of a local bus or network connections.

[0183] In some embodiments, any suitable computer-readable medium may be used to store instructions for performing the functions and / or processes described herein. For example, in some embodiments, the computer-readable medium may be a transitory or non-transitory medium. For example, a non-transitory computer-readable medium may be a magnetic medium (e.g., a hard disk, a floppy disk, etc.), an optical disk (e.g., a compact disk, a digital video disk, a Blu-ray disc, etc.), a semiconductor medium (e.g., RAM, Transient computer-readable media may include media such as flash memory, electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), any suitable medium in which no form of permanence is lost or lost during transfer, and / or any suitable tangible medium. As another example, transitory computer-readable media may include signals on a network, wires, conductors, optical fibers, circuits, or any suitable medium in which no form of permanence is lost or lost during transfer, or any suitable intangible medium.

[0184] It should be noted that the term "mechanism" as used herein may include hardware, software, firmware, or any combination thereof.

[0185] 10 illustrates an example process 1000 for correcting image distortion according to some embodiments of the presently disclosed invention. As shown in FIG. 10, process 1000 can analyze an image segment of an image acquired from a scanning imaging device to identify speckle artifacts at 1002. At 1004, process 1000 can determine an aspect ratio of the speckle artifact shape. At 1006, process 1000 can determine a correction factor for the speckle artifact shape based on the aspect ratio. Finally, process 1000 can adjust the dimensions of the image segment based on the correction factor at 1008.

[0186] FIG. 11 illustrates an example process 1100 for distortion correction according to some embodiments of the presently disclosed invention. As shown in FIG. 11, at 1102, process 1100 can provide an image. At 1104, process 1100 can divide the image into a plurality of substantially parallel strips, each extending in a first direction. At 1106, process 1100 can divide each strip into a plurality of sub-strips along a second direction substantially perpendicular to the first direction. At 1108, process 1100 can analyze each of the plurality of sub-strips to locate at least one locally bright feature. At 1110, process 1100 can distort at least a portion of each sub-strip to approximate local maximum / minimum features to a predetermined shape, thereby creating a corrected sub-strip. At 1112, process 1100 can reassemble the plurality of corrected sub-strips into a single corrected strip. Finally, at 1114, process 1100 can reconstruct the plurality of corrected sub-strips into a single corrected strip. The trip can be reconstructed into a corrected image.

[0187] It should be understood that the above steps of Figures 10 and 11 are not limited to the order or sequence shown and described in the figures, but may be performed or carried out in any order or sequence, and some of the above steps of the processes of Figures 10 and 11 may be performed or carried out substantially simultaneously, where appropriate, or in parallel to reduce delays or processing times.

[0188] The singular forms "a" and "an" and "that" used here may differ from one another depending on the context. Furthermore, the terms "comprises" and / or "comprises" as used herein may specify the presence of the structures, steps, operations, elements, and / or components to which the terms refer, but do not exclude the presence or addition of one or more other structures, steps, operations, elements, components, and / or groups thereof.

[0189] As used herein, the term "and / or" or the like may include any and all combinations of one or more of the listed items to which the term pertains.

[0190] When an element is referred to as being "on," "attached to," "connected to," "coupled to," or "contacting" another element, it is understood that it may be directly on, attached to, connected to, coupled to, or in direct contact with the other element, or that there may be intervening elements. In contrast, when an element is referred to as being "directly on," "directly attached to," "directly connected to," "directly coupled to," or "directly in contact with," there are no intervening elements.

[0191] For ease of explanation, spatial terms such as "below," "below," "lower," "above," and the like may be used herein to describe an element or feature shown in each figure, or the relationship of one element or feature to another element or feature. It should be understood that such spatial terms can include other orientations of the device in use or operation in addition to the orientation shown in the figures. For example, if the device in the figures were inverted, an element described as "below" or "below" another element or feature would now be oriented "above" that other element or feature.

[0192] The features described herein may be included in, constitute, or essentially constitute the present invention in any combination.

[0193] Thus, while the invention has been described above with reference to particular embodiments and examples, the invention is not necessarily limited thereto, and numerous other embodiments, examples, uses, modifications, and derivations from these embodiments, examples, and uses are intended to be included within the scope of the appended claims.

Claims

1. 1. A method for correcting image distortion, comprising: analyzing, with a processor, image segments of the image obtained from the scanning imager to identify locally bright features including at least one of laser speckle or a reference object; determining, by the processor, an aspect ratio of the shape of the locally bright feature; determining, with the processor, a correction factor for the shape of the locally bright feature based on the aspect ratio; adjusting, by the processor, dimensions of the image segment based on the correction factor; A method comprising:

2. the reference object comprises at least one of a microbead or a microsphere; The method of claim 1.

3. the image is obtained from at least one of a rotary scanning device or a linear scanning device; The method of claim 1.

4. the image segments are derived by dividing the image into a plurality of strips; each of the plurality of strips is subdivided into a plurality of sub-strips; the image segment includes one of the plurality of sub-strips; The method of claim 3.

5. the image was generated by rotationally scanning a sample; each of the plurality of strips corresponds to data collected in a rotational scan direction of the sample; The method of claim 4.

6. adjusting a dimension of each of the plurality of partial strips based on a respective correction factor determined for each of the plurality of partial strips; reconstructing each of the adjusted partial strips into a plurality of correction strips; reconstructing the plurality of correction strips into a corrected image; further comprising The method of claim 4.

7. Reconstructing the plurality of correction strips into a corrected image includes: determining a cross-correlation between two adjacent ones of the corrected partial strips; calculating a maximum cross-correlation of the cross-correlations; aligning the two adjacent strips based on the calculated maximum cross-correlation; further comprising The method of claim 6.

8. Analyzing the image segment to identify the locally bright features comprises: extracting a speckle image of the image segment based on filtering the image segment; and The step of determining an aspect ratio of the shape of the locally bright feature comprises: determining the aspect ratio based on the autocovariance of the locally bright features in the speckle image; further comprising The method of claim 1.

9. the locally bright features are caused by at least partially coherent light generated by a laser; The method of claim 1.

10. the locally bright features at least partially comprise laser speckle; 10. The method of claim 9.

11. 1. An apparatus for correcting image distortion, comprising: a processor; a memory in communication with the processor and storing a set of instructions; It is equipped with The set of instructions, when executed by the processor, causes the processor to: analyzing image segments of the image obtained from the scanning imager to identify locally bright features comprising at least one of laser speckle or a reference object; determining an aspect ratio of the shape of the locally bright feature; determining a correction factor for the shape of the locally bright feature based on the aspect ratio; adjusting the dimensions of the image segment based on the correction factor; An apparatus characterized in that

12. the reference object comprises at least one of a microbead or a microsphere; 12. The device of claim 11.

13. the image from which the image segments are derived is generated by at least one of a rotational scan or a linear scan of a sample; 12. The device of claim 11.

14. the image segments are derived by dividing the image into a plurality of strips; each of the plurality of strips is subdivided into a plurality of sub-strips; the image segment includes one of the plurality of sub-strips; 14. The apparatus of claim 13.

15. the image was generated by rotationally scanning the sample; each of the plurality of strips corresponds to data collected in a rotational scan direction of the sample; 15. The apparatus of claim 14.

16. The instructions further cause the processor to: adjusting a dimension of each of the plurality of partial strips based on the correction coefficients determined for each of the plurality of partial strips; reconstructing each of the adjusted partial strips into a plurality of correction strips; reconstructing the plurality of correction strips into a corrected image; 15. The apparatus of claim 14.

17. The instructions further cause the processor, when reconstructing the plurality of correction strips into a corrected image, to: determining a cross-correlation between two adjacent strips of the corrected plurality of partial strips; calculating a maximum cross-correlation of the cross-correlations; aligning the two adjacent strips based on the calculated maximum cross-correlation; 17. The apparatus of claim 16.

18. The instructions further cause the processor, when analyzing the image segment to identify the locally bright features, to: extracting a speckle image of the image segment based on filtering the image segment; The instructions further cause the processor, in determining the aspect ratio of the shape of the locally bright feature, to: measuring the aspect ratio based on the autocovariance of the locally bright features in the speckle image; 12. The device of claim 11.

19. the locally bright features are caused by at least partially coherent light generated by a laser; 12. The device of claim 11.

20. the locally bright features at least partially comprise laser speckle; 20. The apparatus of claim 19.

Images