Image atlas system and method

Abstract

In some embodiments, a process and system for generating a user interface for classifying a sample image of a cell is provided, the process comprising receiving a sample image of a sample particle from a biological sample and selecting a reference image of a reference particle that depicts the biological sample. The reference images can be sorted based on similarity, and the reference images can be selected based on the order. The first selected reference image can be aligned with the sample image and extended so that the adjacent edges of the reference image and the sample image are the same. The extended image can be dynamically filled. The sample image and the extended reference image can be displayed in the user interface.

Description

Technical Field

[0001] The present disclosure generally relates to the analysis of images for clinical applications. More specifically, but not by way of limitation, the present disclosure relates to the analysis of images of cells and particles collected for clinical diagnostics such as hematology and urinalysis and the normalization of the display of said images to allow a user to perform clinical analysis without interference based on irrelevant differences between images. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] The nature and advantages of the various examples may be further understood by reference to the following drawings.

[0003] Figure 1 A block diagram of an exemplary system that may be used to perform image analysis, normalization, and display is shown in accordance with an embodiment.

[0004] Figure 2 A block diagram of an exemplary computer system that may be used to perform image analysis, normalization, and display is shown in accordance with an embodiment.

[0005] Figure 3 A flow chart illustrating an example of a process for performing image analysis, normalization, and display according to an embodiment is shown.

[0006] Figure 4 A flow chart illustrating an example of a process for determining an order of images based on similarity, according to an embodiment.

[0007] Figure 5 A flow chart illustrating an example of a process for determining similarity between images according to an embodiment.

[0008] Figure 6 A flowchart illustrating an example of a process for aligning images according to an embodiment is shown.

[0009] Figure 7 A flowchart illustrating an example of a process for dynamically filling the background of an image, according to an embodiment.

[0010] Figure 8 A flow chart illustrating an example of a process for determining a background color of an image, according to an embodiment.

[0011] Figures 9 to 14 An example of an image displayed according to an embodiment is shown.

[0012] Figures 15 to 21 An example of a user interface according to an embodiment is shown. DETAILED DESCRIPTION

[0013] The analysis of cells or particles from living organisms (e.g., humans, animals, and plants) can be used as a medical diagnostic tool for identifying diseases and cell defects and healthy cells. Capturing cells or particles for analysis can be accomplished by, for example, collecting particles from a fluid (i.e., a biological sample) from a living organism. For example, cells contained in a blood sample from a person can be analyzed to determine whether these cells are healthy or have some problems that can be diagnosed. For example, a blood sample from a patient cared for by a doctor can be evaluated using a hematology system that is equipped with multiple light angle detection parameters, such as the system described in patent application No. 14 / 775,448 entitled "HEMATOLOGY SYSTEMS AND METHODS" filed on September 11, 2015 and published as US 2016-0041083 A1 on February 11, 2016. For another example, a urine sample can be evaluated. In order to analyze cells or particles, it may be beneficial to image cells and compare cells with other images of cells with known characteristics. Although some analysis and comparison can be performed by the automated system, not all cell images are clear enough, have known problems, or are similar enough to cell images with known automation problems to work normally or effectively. In this case, the medical diagnostician can perform manual comparison. Optionally, the following user interface can be provided, which provides an image of the sample cell to be diagnosed or classified, and a reference image of the cell with known characteristics. Even if the cells in the reference image and the sample image may have the same features or characteristics or are cells of the same type, the sample image for classification may also have differences with the reference image, which may interfere with the comparison results. Differences such as image size, orientation, and the gap between the displayed images are examples of differences that may interfere with the cell image comparison results. The embodiments described herein can compensate for those interferences and provide an optimized user interface to compare sample images and reference images using an image atlas.

[0014] Unless otherwise expressly stated, "particles" or "particles" referred to in this disclosure will be understood to encompass any discrete or shaped object dispersed in a fluid. As used herein, "particles" may include all measurable and detectable (e.g., by images and / or other measurable parameters) components in a biological fluid. Particles have any material, any shape, and any size. Particles may include cells. Examples of particles include, but are not limited to, cells, including blood cells, embryonic cells, epithelial cells, stem cells, tumor cells, or bacteria, parasites, or fragments of any of the foregoing or other fragments in a biological fluid. Blood cells may be any blood cells, including any normal or abnormal, mature or immature cells that may be present in a biological fluid, such as red blood cells ("RBC"), white blood cells ("WBC"), platelets ("PLT"), and other cells. These members also include immature or abnormal cells. Immature WBCs may include metamyelocytes, myelocytes, promyelocytes, and primitive cells. In addition to mature RBCs, members of RBCs may include nucleated RBCs ("NTRC") and reticulocytes. PLT can include "giant" PLT and PLT group. Platelets, reticulocytes, nucleated RBC and WBC, including neutrophils, lymphocytes, monocytes, eosinophils, basophils and immature WBC, including primitive cells, promyelocytes, myelocytes or metamyelocytes are counted and analyzed as particles. Throughout the specification, images are described as images of cells or particles. Although referred to as cells in many cases, these images can be images of any particles. "Image" refers to the visual impression of something produced by an optical device (such as, a lens, a mirror, a camera or a microscope).

[0015] Exemplary urine particles can include urine deposition particles. Exemplary urine deposition particles can include red blood cells (RBC), heterosexual red blood cells, white blood cells (WBC), neutrophils, lymphocytes, phagocytes, eosinophils, basophils, squamous epithelial cells, transitional epithelial cells, decoy cells, renal tubular epithelial cells, casts, crystals, bacteria, yeast, parasites, oval fat bodies, fat droplets, sperm, mucus, trichomonas, cell mass and cell debris. Exemplary cells can include red blood cells, white blood cells and epithelial cells. Exemplary casts can include non-cytochrome casts, unclassified casts (e.g., granular casts). Exemplary non-cellular casts can include, for example, wax casts, wide casts, fat casts and transistors. Exemplary cell casts can include, for example, RBC casts, WBC casts and cell casts. Exemplary crystals can include, for example, calcium oxalate, tripolyphosphate, calcium phosphate, uric acid, calcium carbonate, leucine, cystine, tyrosine and amorphous crystals. Exemplary non-squamous epithelial cells may include, for example, renal epithelial cells and transitional epithelial cells. Exemplary yeasts may include, for example, budding yeasts and yeasts with pseudohyphae. Exemplary urine sediment particles may also include RBC masses, fat, oval fat bodies, and trichomonas. The system and method may be used, for example, to characterize particles in biological fluids, such as detecting and quantifying red blood cells (RBC), heteromorphic red blood cells, white blood cells (WBC), neutrophils, lymphocytes, phagocytes, eosinophils, basophils, squamous epithelial cells, transitional epithelial cells, decoy cells, renal tubular epithelial cells, casts, crystals, bacteria, yeasts, parasites, oval fat bodies, fat droplets, sperm, mucus, trichomonas, cell masses, and cell debris, classification and subclassification, counting and analysis.

[0016] In the following description, for the purpose of explanation, specific details are set forth in order to provide a thorough understanding of the embodiments of the present invention. However, it is apparent that various embodiments can be practiced without these specific details. These drawings and descriptions are not intended to be limiting.

[0017] The systems depicted in some of the figures may be provided in various configurations. Optionally, the system may be configured as a distributed system, wherein one or more components of the system are distributed over one or more networks in a cloud computing system. All features of the described systems apply to the described methods with necessary modifications, and vice versa.

[0018] Figure 1 A block diagram of an exemplary system 100 that can be used to perform image analysis, normalization, and display is shown. System 100 can include a network 105 through which various components can communicate, including database 120, database 125, server 110, computing device 115, and analyzer 130.

[0019] The analyzer 130 may collect images of living biological particles and cells, for example, by a body fluid system that captures body fluid cell images, as described in detail in patent application Ser. No. 14 / 775,448, filed Sep. 11, 2015, entitled “HEMATOLOGY SYSTEMS AND METHODS.” The analyzer 130 may collect the images and store them as sample cell images in the database 120. Reference images may be collected by the analyzer 130 and / or by other capture methods for comparison and stored in the database 125.

[0020] Database 120 may store sample images for analysis. Database 125 may store image atlases (i.e., a database of reference images) for comparison during analysis of sample images stored in database 120. Optionally, database 120 and database 125 may be a single database (e.g., sample images are stored in a different table than reference images). Databases 120 and 125 may be any suitable databases, including, for example, SQL database, Database or spreadsheet.

[0021] The network 105 can be any suitable number or type of networks or links, including but not limited to a dial-up network, a local area network (LAN), a wide area network (WAN), a public switched telephone network (PSTN), a cellular network, a WiFi network, the Internet, an intranet, or any combination of hardwired and / or wireless communication links. The network 105 may include one or more subnetworks. In some examples, two or more components of the system (e.g., computing device 115, server 110, databases 120 and 125, analyzer 130, or any combination of these components) can be connected to the network 105 and communicate via the network.

[0022] The computing device 115 may be a Figure 2 The computing system 200 described herein. The computing device 115 can be a system capable of performing image analysis, normalization, and display as discussed in more detail herein. The computing device 115 can be a system used by a user, and although depicted as a desktop computer, can be any user device, including a laptop or wireless device (e.g., a smartphone, tablet computer, personal data assistant (PDA), e-reader, or smart watch).

[0023] Optionally, the server 110 may be a Figure 2The computing system 200 described herein. The server 110 is capable of performing image analysis and normalization, and because the server sometimes does not include its own display device, the resulting image is sent to the computing device 115 for display, for example, in a user interface. Optionally, the server 110 can be any suitable type of server. The server can be, for example, in a server farm or rack server. For example, the computing device 115 and the server 110 can be in a cloud computing environment.

[0024] Figure 2 A block diagram of an exemplary computer system 200 that can be used to perform image analysis, normalization, and display is shown. Computing device 200 can be or include, for example, a laptop computer, a desktop computer, a tablet computer, an e-reader, a smart phone or mobile device, a smart watch, a personal data assistant (PDA), or other electronic device. Computing device 200 can be, for example, computing device 115 or server 110.

[0025] The computing device 200 may include a processor 240 connected to other hardware via a bus 205. The memory 210, which may include any suitable tangible (and non-transitory) computer-readable medium (such as RAM, ROM, EEPROM, etc.), may embody program components (e.g., instructions 215) that configure the operation of the computing device 200. In some examples, the computing device 200 may include an input / output ("I / O") interface component 225 (e.g., for connecting to a display 245, a keyboard, or a mouse) and an additional storage device 230.

[0026] Computing device 200 may include network component 220. Network component 220 may represent one or more of any components that facilitate network connectivity. In some examples, network component 220 may facilitate wireless connectivity and include a wireless interface, such as IEEE 802.11, Bluetooth, or a radio interface for accessing a cellular telephone network (e.g., a transceiver / antenna for accessing a CDMA, GSM, UMTS, or other mobile communications network). In other examples, network component 220 may be wired and may include an interface, such as Ethernet, USB, or IEEE 1394.

[0027] although Figure 2 A single computing device 200 with a single processor 240 is depicted, but the system may include any number of computing devices 200 and any number of processors 240. For example, multiple computing devices 200 or multiple processors 240 may be distributed over a wired or wireless network (e.g., a wide area network, a local area network, or the Internet), such as network 105. Multiple computing devices 200 or multiple processors 240 may perform any steps of the present disclosure individually or in coordination with each other. For example, processor 240 and / or computing device 200 may be, for example, Figure 1computing device 115 and / or server 110 .

[0028] Figure 3 A flow chart showing an example of a process 300 for performing image analysis, normalization, and display is shown. The process 300 may be used, for example, in Figure 1 More specifically, Figure 2 is implemented as program code on the computer system 200.

[0029] Process 300 is performed in, for example, Figure 2 Starting at 305 on the computer system 200 of the present invention. At 310, the computer system 200 may receive a sample image of a sample particle (e.g., a cell) from a living organism. As briefly discussed above, cells and particles of a living organism may be imaged or photographed using an imaging system (e.g., analyzer 130) that uses a biological sample (e.g., a urine or blood sample) collected from the organism. The image may be magnified and depict an image of a single cell or particle before or after capture, such as Fig. 9 The sample image 905 can be displayed on the computing device 115 (shown in more detail as Figure 2 The computer system 200) is passed Figure 1 The network 105 from e.g. Figure 1 The database 120 receives sample images.

[0030] At 315, computer system 200 may select a plurality of reference images from a database of reference images. The database of reference images may be, for example, Figure 1 The database 125 may contain any number of reference images. The reference images may be populated by a similar capture method as described above for the reference sample images. The reference images may be, for example, Fig. 9 Reference images 910. The reference images can be populated by a library of reference images stored in the database 125 selected by the user. Optionally, the images selected by the user can be from a library of reference images previously captured. Optionally, the images selected by the user can include images that are sample images added to the reference image database by the user. Optionally, the images selected by the user can use file names based on randomly generated globally unique identifiers (GUIDs) to ensure uniqueness and portability between systems. The reference images can be stored in any suitable format (e.g., *.png, *.gif, *.jpg). The reference images can be stored as they are captured, meaning that the images can have, for example, different file sizes, pixel heights, pixel widths, and / or magnifications of imaged cells.

[0031] The reference image can be a cell or particle containing known features, or a known cell or particle of a certain type. For example, a whole blood sample typically includes three major classes of blood cells, including erythrocytes, leukocytes, and thrombocytes. Each class can be further divided into subclasses of members. For example, subclasses of the five major types of white blood cells (WBC) have different shapes and functions. White blood cells can include neutrophils, lymphocytes, monocytes, eosinophils, and basophils. There are also subclasses of red blood cell types. The appearance of particles in a sample may vary depending on pathological conditions, cell maturity, and other reasons. Red blood cell subclasses can include reticulocytes and nucleated red blood cells. For another example, the imaged cell can be a sickle cell. The reference image can include annotations that can identify the cell class within the reference image, or labels that identify cell features or characteristics within the reference image. For example, annotations made to an image of a cell known to be a sickle cell can identify the cell as a sickle cell. For another example, annotations made to an image of a cell known to be a red blood cell can identify the cell as a red blood cell. Some reference images may include multiple labels that identify multiple features or characteristics of cells. For example, a cell may be annotated as a red blood cell and have an additional label that identifies the cell as a reticulocyte. Cells or particles may be labeled as, for example, neutrophils, lymphocytes, monocytes, eosinophils, basophils, platelets, reticulocytes, nucleated red blood cells (RBC), primitive cells, promyelocytes, myelocytes, metamyelocytes, red blood cells (RBC), cells, bacteria, granules, cell clusters or cell fragments or components. Optionally, the computer system 200 may select multiple reference images from a database of reference images by selecting, for example, all reference images with specific labels and / or annotations. For example, the computer system 200 may select all reference images annotated as red blood cells from a database. For another example, the computer system 200 may select all reference images with white blood cell annotations from a database. Optionally, the computer system 200 may allow a user to select a label or annotation for selecting a reference image. Optionally, the computer system 200 may select all reference images in a reference image database. Optionally, the computer system 200 may allow the user to select a desired reference image from a reference image database. Optionally, the computer system 200 may allow the user to select an additional label as a filter condition and, for example, select all images classified as white blood cells and further labeled as neutrophils.

[0032] Labeling or annotating images can provide an effective method for rapid retrieval of images. For a large number of reference images with potentially many different subtypes, it may be beneficial to provide subsets that target one or more subtypes based on changes in physical characteristics or changes in disease stage of the same class type (classification). This can be accomplished through hierarchical labeling, such as labeling within an annotation that identifies the class of the imaged cell. For example, a reference image may include an annotation that the cell is a red blood cell ("RBC") (i.e., an annotation of RBC that the imaged cell belongs to the RBC class) and a label that marks the reference image as having a particular characteristic, and, for example, a nucleated red blood cell may have an RBC annotation that identifies it as nucleated.

[0033] At 320, the computer system 200 can determine an order for the plurality of reference images based on the similarity of each reference image to the sample image. This can save a significant amount of time in sorting the reference images so that the most similar images are displayed first. As previously described, the sample image can be displayed in a user interface along with the reference images for comparison to identify characteristics, features, or types of cells imaged in the sample image. Displaying the most similar images first can limit the amount of time spent searching for reference images that match the sample image to classify the sample image as the same type of cell in the reference image or to identify that the sample image has the same characteristics or features as the reference image. The computer system 200 can determine the order by, for example, comparing the sample image to each selected reference image, such as the reference image. Figure 4 and Figure 5 Describe in more detail.

[0034] At 325, computer system 200 may select a first reference image from the plurality of reference images based on the determined order. After computer system 200 determines the order of the reference images such that the first reference image is the image determined to be most similar, the second reference image is the image determined to be the second most similar, and so on, computer system 200 may select the first reference image for performing analysis, normalization, and display. After performing the remaining steps of process 300 on the first reference image, computer system 200 may select the second reference image for performing analysis, normalization, and display in the remaining steps of process 300, and continue to repeat these steps on each of the plurality of reference images until all selected reference images have been analyzed, normalized, and displayed according to process 300.

[0035] At 330, the computer system 200 may align the first reference image with the sample image. The alignment of the first reference image with the sample image may be used, for example, to display the image to a user through a user interface. The alignment may include aligning adjacent edges of the images and centering the images relative to each other for display in the user interface. Figure 6An example of one type of alignment that can be performed is described in more detail. Optionally, the size of adjacent edges of the sample image and the reference image are different. In this case, the smaller image can be expanded to match the size of the larger image. For example, optionally, the smaller image can be scaled to expand to the same size as the larger image. Optionally, the image can be superimposed on an image block that has adjacent edges of the same size as the larger image. Optionally, each image can be placed to be displayed in a cell of a grid, and the cells of the grid can have a matching background color that appears in the display to adjust the size of the smaller image to the size of the larger image.

[0036] Optionally, the number of reference images for display in the user interface can be between 1 and 8. Any number of reference images can be aligned and extended, and the reference images can be aligned and extended in order based on images sharing adjacent edges. See Reference Fig.11 For more details on the optional image display layout, see the description of i,j is an m×n grid (where i∈[0,m-1],j∈[0,n-1], and width is W i,j , width is H i,j ) is the reference image of the i-th row and j-th column in the grid. The height RH of the i-th row of the display grid can be calculated as follows i and the width of the jth column CW j To align and expand:

[0037]

[0038]

[0039] Depending on the relative size of the sample image and the list of reference images used for display, the final row height or column width for a particular row / column in the display grid may result in the sample image or reference image needing to be expanded. Optionally, image background padding can be applied to any images whose dimensions differ from the calculated grid cell width or height. And the image itself can be positioned in the center of the relevant grid cell without scaling or resizing to maintain 100% fidelity to the original image.

[0040] At 335, the computer system 200 can dynamically fill the background of the aligned first reference image. During the alignment step, the background of the first reference image can be extended to ensure that the displayed image has adjacent edges of the same size as the sample image. The extended background can be dynamically filled with a color that is substantially the same as the existing background of the reference image to ensure uniformity and eliminate distractions with gaps between images containing different background colors. The colors of any image can include, for example, red, green, blue, and alpha channels. The background can be filled by determining a color that matches the background of the reference image and filling the extended background with the determined color, such as the reference image. Figure 7 Described in more detail. Optionally, the background is filled by superimposing a reference image on a block filled with the determined background color. During display, the block can be, for example, a cell grid. It should be noted at this point that two different types of cells are discussed in this application. The cell image (i.e., the reference image or sample image) is an image of a somatic cell of a living organism. The cells of the grid are different types of cells, such as the cells in the table. Throughout the application, when talking about cells, unless it is specifically stated that cells of the grid are being discussed, it is meant to be the somatic cell that has been imaged.

[0041] Optionally, the alignment of the sample image and the reference image may result in adjusting the size of the sample image to extend the background (e.g., because the sample image is smaller than the reference image). In this case, the background of the sample image may be dynamically filled in the same manner as described above with reference to dynamically filling the background of the first reference image.

[0042] At 340 , the computer system 200 may display the sample image and the dynamically populated and aligned first reference image in a user interface. Figures 9 to 18 An example of a user interface display of a sample image and one or more reference images is shown. Optionally, the user interface can be displayed in a display device associated with the computer system 200. For example, the computing device 115 (which can be any user device with a monitor that can display a user interface, including a desktop, laptop, smart phone, tablet computer, or any other suitable computing device 115 including a display monitor) can be the computer system 200. Optionally, the computer system 200 can be the server 110. In such embodiments, the server 110 may not include a specific display monitor, so the server 110 can send information for display (e.g., reference images and sample images and associated computing layout and color information) to the computing device 115 for display to the user in the user interface.

[0043] Process 300 may end at 345. As described above, optionally, steps 325, 330, and 335 may be performed for the first reference image, and repeated for each reference image until each selected reference image is aligned and dynamically populated. Optionally, in each alignment and dynamic population of the reference images, the display step may be performed individually for each reference image, may be performed simultaneously for all selected reference images (only once after all selected reference images are aligned and dynamically populated), or may be performed at once for a certain number of reference images (e.g., such as in Fig.12 8 reference images are shown at a time along with the sample image).

[0044] Figure 4 4 is a flowchart showing an example of a process 400 for determining an order of images based on similarity. The process 400 may be used as Figure 3 The process 400 may be performed as part of step 320 of FIG. Figure 1 More specifically, Figure 2 is implemented as program code on the computer system 200.

[0045] At 405, the computer system 200 can compare the sample image to each of the plurality of reference images. For example, the comparison can be performed by generating a feature vector for each reference image and the sample image and calculating a distance metric between each reference image and the sample image, such as the reference image. Figure 5 Describe in more detail.

[0046] At 410, computer system 200 can determine the similarity of each reference image to the sample image. Figure 5 The distance metric computed as discussed above is used to make this determination.

[0047] Figure 5 1 is a flow chart showing an example of a process 500 for determining similarity between images. The process 500 may be used as Figure 4 The process 500 may be performed as part of step 405 of the embodiment of the present invention. Figure 1 More specifically, Figure 2 is implemented as program code on the computer system 200.

[0048] At 505, the computer system 200 can generate a feature vector for each reference image and the sample image in the plurality of reference images. The feature vector can include information about a plurality of features of each image. For example, the feature vector can identify block size, particle size, particle shape, average pixel intensity, and / or image average grayscale value. Block size can be the size of the total image. The image size can be scaled, as described above. The background can also be expanded, as described above. Block size is the size of the image before any of these changes occur. Particle size can be the size of a sample cell or reference cell in the image. Optionally, the cell size can appear larger or smaller within the image, for example, as Fig. 9 As shown. The particle size of the sample cell in the sample image 905 is smaller than the particle size of the reference cell in the reference image 910. The particle size may appear larger or smaller within the image based on, for example, the magnification used during image capture. The particle shape may be the shape of the sample cell or the reference cell in the image. The average pixel intensity may be calculated by selecting a plurality of pixels from the image, determining the intensity of each selected pixel, and calculating the average intensity of the selected pixels. The average gray value of the image may be calculated by selecting a plurality of pixels from the image, determining the gray value of each pixel, and calculating the average of the plurality of calculated gray values. Other features may be identified and used in a feature vector for each image.

[0049] At 510, the computer system 200 may calculate a distance metric between each of the plurality of reference images and the sample image. For example, the computer system 200 may calculate a distance metric between each reference image and the sample image using a feature vector. The smaller the distance metric value, the more similar the reference image is to the sample image. The distance metric value may then be used to sort the reference images in order from most similar to least similar to the sample image.

[0050] Optionally, a distance metric may be defined as follows:

[0051]

[0052] In this distance metric definition, the i-th reference image feature vector With f i,k The kth component of , where k∈[0,N-1] and N is the dimension of the feature vector. Candidate features can be based on, for example, image size, particle size in the image, particle shape in the image, average pixel intensity in the image, or average grayscale value of particles / blocks. The smaller the distance metric value, the more similar the i-th (reference) image and the j-th (sample) image are.

[0053] Figure 6 600 for aligning images. Figure 3 The process 600 may be performed as part of step 330 of FIG. Figure 1 More specifically, Figure 2 is implemented as program code on the computer system 200.

[0054] At 605, the computer system 200 may determine a first display position within a first cell of the grid for displaying the sample image in the user interface. For example, the image may be displayed in the user interface in the grid such that the sample image is centered and the reference image is adjacent to the sample image. For example, Fig. 9 A sample image 905 is shown adjacent to a reference image 910. Similarly, as another example, Fig.11 A sample image 1105 is shown adjacent to two reference images 1110 and 1115. For another example, Fig.12 A sample image 1205 is shown adjacent to reference images 1210 , 1215 , 1220 , and 1225 . Fig.12 Also shown are sample images that share corners with reference images 1230, 1235, 1240, and 1245, each of which is adjacent to another reference image. Optionally, the sample image is placed in the center of the grid so that it is adjacent to as many reference images as possible, such as Fig.12 shown.

[0055] At 610, the computer system 200 may determine a second display position adjacent to the first display position within a second cell of the grid, the second display position being used to display the first reference image in the user interface. Examples of sample images adjacent to the reference image may be found at Figures 9 to 14 For example, the sample image 905 may be adjacent to the reference image 910, such as Fig. 9 shown.

[0056] At 615, computer system 200 may align the centroid of the first reference image within the second display location with the centroid of the sample image within the first display location. For example, the center of the sample image may be determined and positioned so that it is located at the center of the first display location, which may be the center of a first cell of a grid (e.g., Fig. 9 The computer system 200 may also determine the center of the first reference image and place it so that it is located at the center of a second display location, which may be the center of a second cell of the grid (e.g., Fig. 9 Due to the definition of the grid, the center of the first display position and the center of the second display position adjacent to each other in the grid have a common center of the adjacent edges (e.g. Fig. 9Optionally, other methods may be used such that the centers of each image may be aligned by selecting a display position for each of the sample image and the first reference image that results in the centroids of each image being aligned.

[0057] Figure 7 A flow chart showing an example of a process 700 for dynamically filling the background of an image. The process 700 may be used as Figure 3 The process 700 may be performed as part of step 335 of FIG. Figure 1 More specifically, Figure 2 is implemented as program code on the computer system 200.

[0058] At 705, computer system 200 may extend the background of the first reference image within the second cell to fill the second cell. Optionally, the height of the first reference image may be, for example, less than the height of the sample image. For example, if the adjacent edges of the sample image and the first reference image are vertical edges, the adjacent edges of the first reference image and the sample image will be different. Similarly, for example, if the adjacent edges of the sample image and the first reference image are horizontal edges, and the width of the first reference image is less than the width of the sample image, the adjacent edges of the images will be different. The background of the first reference image may be extended to ensure that the adjacent edges of the first reference image and the sample image will be the same. Optionally, the size (height and / or width) of the sample image may be less than the size of the first reference image. In such embodiments, the sample image background may be extended.

[0059] At 710, the computer system 200 can determine the background color of the first reference image. The background color of the first reference image can be determined in a variety of ways. For example, the background color of the first reference image can be determined using Figure 8 The background color is determined by the process for determining the average background color described in detail in . Optionally, a standard color may be selected to be used as the background color of the reference images. For example, if all reference images have the same background color, the background color will always be the same and may be known.

[0060] At 715, computer system 200 may fill the expanded background of the first reference image with the background color. For example, the second cell of the grid may be filled with the determined background color so that when displayed in the user interface, the first reference image appears to be expanded without modifying the first reference image itself. Optionally, the first reference image may be processed to expand the image size and fill the expanded pixels with the determined background color.

[0061] Figure 8 8 is a flowchart showing an example of a process 800 for determining the background color of an image. The process 800 may be used as Figure 7 The process 800 may be performed as part of step 710 of FIG. Figure 1 More specifically, Figure 2 is implemented as program code on the computer system 200.

[0062] At 805, the computer system 200 may determine four corner pixels of the first reference image. A typical image has 4 corners, and a corner pixel in each corner may be used. For example, if the size of the image block is W×H pixels, the four corner pixels are: [0, 0], [W-1, 0], [0, H-1], [W-1, H-1]. Optionally, more or fewer pixels may be used. Optionally, any number of pixels selected from edge pixels may be used. Optionally, an edge pixel may be any pixel along an edge of an image. Optionally, an edge may be a pixel within a threshold distance from an absolute edge of the image (e.g., a distance of three pixels from the absolute edge of the image).

[0063] At 810, the computer system 200 may calculate an average color value for the four corner pixels. The average color value may be determined by first determining the color value of each of the four corner pixels (or if other edge pixels are used, identifying each pixel to determine the background color). Once the color value of each of the four corner pixels is determined, the computer system 200 may calculate the average color value for the four corner pixels using the color value from each of the four corner pixels. The average value for each color component (e.g., red, green, blue, and alpha) may be calculated. Optionally, any selected pixel may be used to calculate the average color value for the background. The average color value may then be used as the background color for the image. Determining the average color value may help reduce noise by filling the reference image background with a color that substantially matches the entire background of the image. Figure 12 to Figure 14 An example of an alignment of a sample image and eight reference images and a dynamically populated grid is shown.

[0064] Figures 9 to 14 An example of an image displayed according to an embodiment is shown. Fig. 9 A portion of a user interface 900 is shown that can be used to compare a sample image 905 of a sample particle 930 with a reference image 910 of a reference particle 935. The user interface 900 can include a class indicator 915, a forward navigation button 925, and a backward navigation button 920.

[0065] Class indicator 915 may be an indication of an annotation associated with reference image 910. Fig. 9 As shown, reference image 910 may have an annotation "HYAL" associated with it. HYAL may mean, for example, that reference image 910 has been classified as a hyaline cast. Fig. 9As shown, the user can use, for example, a drop-down selection list to select a classification annotation or feature label. The options included in the selection may include, for example, the list provided in Table 1. The list provided in Table 1 includes different cell types or features of cells that can be identified. Once the user selects the class indicator 915, a reference image can be selected, such as, for example, a reference image. Figure 3 Step 315 is explained in detail.

[0066]

[0067]

[0068] Table 1

[0069] like Fig. 9 As shown, there is no gap between images 905 and 910 shown in user interface 900 to eliminate distractions associated with having different colors between the images. Figure 3 The sample image 905 is obtained as described in step 310 of receiving the sample image. Figure 3 The reference image 910 is selected as described in step 315 of selecting a reference image.

[0070] The forward navigation button 925, when selected by the user, may cause the reference image 910 to be replaced by another reference image in the user interface display. Optionally, the forward navigation button 925 may select the next reference image in the ordered list determined by similarity, such as the reference image 910. Figure 3 Optionally, forward navigation button 925 may select the next reference image selected by the computer system, for example in random order, from the reference image group that includes the annotation or label indicated by class indicator 915 .

[0071] The back navigation button 920, when selected by the user, may cause the reference image 910 to be replaced by another reference image in the user interface display. Optionally, the back navigation button 920 may select a previous reference image in the ordered list determined by similarity, such as the reference image 910. Figure 3 As described in step 325 of . Optionally, a backward navigation button 920 can select a previously viewed reference image, whether from a sequential list or not.

[0072] Fig.10 A portion of user interface 1000 is shown, which is very similar to Fig. 9 This portion of the user interface 900. Fig.10 As shown, sample image 1005 is Fig. 9The sample image 1005 is the same image as the sample image 905. The sample image 1005 is an image of the sample particle 1030, and the reference image 910 is an image of the reference particle 1035. The user interface 1000 may also include a class indicator 1015, a forward navigation button 1025, and a backward navigation button 1020. The user interface 1000 may be composed of a new selection identified by the user. Fig. 9 For example, if a user is utilizing user interface 900 and uses class indicator 915 from Fig. 9 The class indicator 915 indicates that the HYAL selection is as indicated by Fig.10 The class indicator 1015 indicates a WBC. WBC may mean white blood cell. The resulting user interface 1000 may then display the reference image 1010 with the WBC annotation.

[0073] Class indicator 1015 can be used in the same manner as above. Fig. 9 The forward navigation button 1025 can be implemented in the same manner as described above with reference to the class indicator 915. Fig. 9 The backward navigation button 1020 can be implemented in the same manner as described above with reference to the forward navigation button 925. Fig. 9 The back navigation button 920 is implemented in the same manner as described above.

[0074] like Fig. 9 and Fig.10 As shown, the portion of user interface 900 is larger than the portion of user interface 1000. Fig. 9 and Fig.10 , it can be seen that sample image 905 has the same width as sample image 1005, but the height of sample image 905 is greater than the height of sample image 1005. Sample image 905 and sample image 1005 are both the same sample cells and are the same sample images. However, the size (height) of sample image 905 and sample image 1005 is different. This may be because reference image 910 is larger than reference image 1010. In this case, it can be as reference Figure 3 The height of the sample image 905 is expanded and dynamically padded as described in steps 330 and 335 to ensure that adjacent edges of the sample image 905 and the reference image 910 have the same size.

[0075] Fig.11 A portion of a user interface 1100 is shown that is similar to user interface 900 and user interface 1000. User interface 1100 includes a sample image 1105 of a sample particle 1135, reference images 1110 and 1130 of reference particles 1140 and 1145, a forward navigation button 1125, a backward navigation button 1120, and a class indicator 1115. Class indicator 1115 may be displayed in the same manner as described above with reference to Fig. 9 The forward navigation button 1125 can be implemented in the same manner as described above with reference to the class indicator 915. Fig. 9 The backward navigation button 1120 can be implemented in the same manner as described above with reference to the forward navigation button 925. Fig. 9 The back navigation button 920 is implemented in the same manner as described above.

[0076] You can refer to Figure 3 The sample image 1105 is obtained as described in step 310 of receiving the sample image. Figure 3 Reference images 1110 and 1130 are selected as described above for selecting a reference image at step 315. For demonstration purposes only, reference images 1110 and 1130 of reference particles 1140 and 1145 are the same image and are test particles. Optionally, for example, if the reference database has few reference images, the same reference image can be displayed to the user to enhance the user's ability to recognize similarities between sample images and reference images. Optionally, after selecting the forward or backward navigation button 1120 or 1125, the same reference image can be displayed multiple times in the same or different locations of the grid display. The multiple displays can be attributed to, for example, the number of reference images in the reference database and the desire to fill each cell of the display grid.

[0077] exist Fig.11 , it can be seen that two reference images 1110 and 1130 are displayed, each at a different adjacent edge of the sample image 1105. For example, the selection of the positions of the reference image 1110 and the reference image 1130 can be based on the similarity of the images. The more similar image can be displayed first at the position where the reference image 1110 is located, and then the second most similar image can be displayed at the position where the reference image 1130 is located.

[0078] For image layout modes involving the presentation of more than one reference image and primarily in one dimension, such as a "3×1" layout (e.g. Fig.19 As described above), the reference images can be sorted according to the image similarity measure in the X-axis direction. The following figure shows this:

[0079]

[0080] Similarly, for a "3×3" image layout pattern involving two dimensions, the reference images can be ordered based on the spiral coordinates defined by the image neighborhood. The following figure shows the ordering of the reference images ([0], [1], ..., [7]) to match the natural image neighborhood ordering:

[0081]

[0082] When the number of reference images is not divisible by the sliding window size (the sliding window size for an m×n image layout is m*n-1. For example, the sliding window size for a "3×3" image layout mode is 3*3-1=8), the selection is adjusted so that there are no empty grid cells when displaying the reference images. For example, regardless of the navigation direction (e.g., using the forward or backward navigation button to move forward or backward in the reference image list, such as the reference image list). Fig. 9 For example, if the image layout mode of "3×3" has nine (9) reference images, i.e., the sliding window size is 8, then there are two sliding windows available for display, where the first window uses the first 8 reference images, and the second window uses the second 8 reference images. Specifically, for a list of N reference images, for an m×n image layout, the count of the sliding windows is given by:

[0083]

[0084] Where m≠1 or n≠1 mod represents a mathematical modulo operation. And the i-th sliding window starting index (entering the list of N reference images) is given by:

[0085]

[0086] Where m≠1 or n≠1, i∈[0,K-1]

[0087] Fig.12 Shows something like Figures 9 to 11 1200. User interface 1200 displays eight reference images 1210, 1215, 1220, 1225, 1230, 1235, 1240, and 1245. Fig.12 Also included are a forward navigation button, a backward navigation button, and a class indicator.

[0088] Fig.13 Shows something like Figures 9 to 12 A portion of user interface 1300 of the user interface in FIG. User interface 1300 displays eight reference images (not numbered). Fig.13 Also included are a forward navigation button, a backward navigation button, and a class indicator.User interface 1300 is an example of a user interface by which a user may compare a sample image within a box located in the center of the image to eight reference images that surround the sample image.

[0089] Fig.14 Shows something like Figures 9 to 13 A portion of user interface 1400 of the user interface in FIG. User interface 1400 displays eight reference images (not numbered). Fig.14Also included are a forward navigation button, a backward navigation button, and a category indicator. User interface 1400 is a user interface that a user can use to navigate to a page located at Fig.14 An example of a user interface comparing a sample image within a box at the center of FIG. 8 with eight reference images that surround the sample image.

[0090] Figures 15 to 18 Additional examples of user interfaces are shown. Fig.15 A user interface 1500 is shown that includes a floating window 1505, which may be, for example, Fig.19 A portion of user interface 1900 is shown. Floating window 1505 may include a context menu 1510 that may be displayed when a user right-clicks within floating window 1505. The sample image displayed in floating window 1505 (following context menu 1510) may be selected, for example, by the user selecting an image from image selection portion 1515. Optional options available in the context menu may include, for example, changing the display layout, changing the reference image source, selecting one or more labels, changing the orientation of one or more images, and zooming in or out of one or more images.

[0091] Fig.16 A user interface 1600 is shown that includes a floating window 1605, which may be, for example, Fig. 20 16. A portion of user interface 2000 is shown. Floating window 1605 may include a context menu 1610 that may be displayed when a user right-clicks within floating window 1605. Sample image 1620 displayed in floating window 1605 may be selected, for example, by a user selecting an image from image selection portion 1615. In this case, sample image 1620 is selected by a user selecting image 1625 from image selection portion 1615.

[0092] Fig.17 A user interface 1700 is shown that includes a floating window 1705, which may be, for example, Fig.21 A portion of the user interface 2100 is shown. The floating window 1705 may include a class identifier 1710, which may allow a user to select a label or annotation to select a reference image, such as a reference image. Figure 3 The sample image 1720 displayed in the floating window 1705 may be selected, for example, by the user selecting an image from the image selection portion 1715. In this case, the sample image 1720 is selected by the user selecting an image 1725 from the image selection portion 1715.

[0093] Fig.18 A user interface 1800 is shown that includes a floating window 1805, which may be, for example, Fig.21A portion of user interface 2100 is shown. Floating window 1805 may include a context menu 1810 that may be displayed when a user right-clicks within floating window 1805.

[0094] Fig.19 A portion of a user interface 1900 is shown, which may be depicted as Fig.15 1505. When the user right-clicks in this portion of the user interface 1900, a context menu 1905 may be displayed. The context menu 1905 may change the display layout of the sample image and the reference image. The "3×1" selection is grayed out because the display has been set to a 3×1 grid. The 3×1 option is shown in Fig.11 Another optional option may include, for example, Fig. 9 and Fig.10 Another optional choice may include, for example, Figure 12 to Figure 14 “3×3” shown in .

[0095] Fig. 20 A portion of user interface 2000 is shown, which may be depicted as Fig.16 1605. When the user right-clicks in this portion of the user interface 2000, a context menu 2005 may be displayed. The context menu 2005 may allow the user to "select a label," which may be a selection of labels for identifying reference images for selection within the reference image database, and may further narrow the displayed images to not only the selected class but also the selected label. In this way, the user may customize multiple selections of classes and labels for filtering. Fig. 20 , it can be seen that optionally the user can select a classification of reference images to select from, which is shown by the class selection box 2010. In this case, the selected class is "HYST". In addition, tags can be selected to further narrow the selection based on other characteristics of particles that have been labeled for reference images in the reference database. The context menu 2005 can allow the user to select one or more tags to filter the displayed reference images. The selected tags are displayed at 2015. The displayed reference images can be selected from the reference image database based on whether the reference image is classified in the selected class and whether it has a selected tag associated with the reference image.

[0096] Fig.21 A portion of a user interface 2100 is shown, which may be depicted as Fig.181805. When the user right-clicks in this portion of the user interface 2100, a context menu 2105 may be displayed. The context menu 2105 may allow the user to change the image source. Selecting the image source may select a database from which to select a reference image. The options for selecting the image source may be, for example, to use a built-in source, or, for example, to use a user-selected reference image database. The user-selected reference image database may be generated, for example, as described above.

[0097] A computer or other processor with hardware, software and / or firmware can be used to perform each of the calculations or operations described herein. Each method step can be performed by a module, and the module may include any of a variety of digital and / or analog data processing hardware and / or software arranged to perform the method steps described herein. The module optionally includes data processing hardware, which is suitable for performing one or more of these steps due to having an appropriate machine programming code associated with it, and the modules for two or more steps (or parts of two or more steps) are integrated into a single processor board or divided into different processor boards using any of a variety of integrated and / or distributed processing architectures. These methods and systems will generally use tangible media that embodies machine-readable codes with instructions for performing the above method steps. Suitable tangible media may include memory (including volatile memory and / or non-volatile memory), storage media (such as magnetic recording on floppy disks, hard disks, tapes, etc.; optical storage such as magnetic recording on CDs, CD-R / W, CD-ROMs, DVDs, etc.; or any other digital or analog storage media), etc.

[0098] Different arrangements of the parts depicted in the drawings or described above, and parts and steps not shown or described are also feasible. Similarly, some characteristic structures and sub-combinations are available, and they can be adopted when they are independent of other characteristic structures and sub-combinations. Embodiments of the present invention have been described for illustrative and non-restrictive purposes, but alternative embodiments will be apparent to the reader of this patent. In some cases, method steps or operations can be performed or implemented in different orders, or operations can be added, deleted or modified. It should be understood that in some aspects of the present invention, a single component can be replaced with multiple components, and multiple components can be replaced with a single component to provide an element or structure or to perform a given one or more functions. Except for the situation that such replacement will not effectively practice certain embodiments of the present invention, such replacement is considered to be within the scope of the present invention.

[0099] It should be understood that the drawings and descriptions of the embodiments of the present invention have been simplified to illustrate the elements relevant to a clear understanding of the present invention. However, it will be appreciated by those of ordinary skill in the art that these elements and other elements may be required. However, since such elements are well known in the art, and since they are not conducive to a better understanding of the present invention, the discussion of such elements is not provided herein. It should be understood that the accompanying drawings are provided for schematic illustration, and the accompanying drawings are not provided as construction diagrams. The details omitted and modifications or alternative embodiments are within the scope of the knowledge of those of ordinary skill in the art.

[0100] It should be understood that in certain aspects of the present invention, a single component can be replaced with multiple components, and multiple components can be replaced with a single component to provide an element or structure or to perform a given one or more functions. Except for the situation that such replacement will not effectively practice certain embodiments of the present invention, such replacement is considered to be within the scope of the present invention.

[0101] The examples provided herein are intended to illustrate potential and specific implementations of the present invention. It should be understood that the examples are primarily intended to provide an illustration of the present invention to those skilled in the art. Without departing from the essence of the present invention, these figures or the operations described herein may be varied. For example, in some cases, the method steps or operations may be performed or implemented in different orders, or the operations may be added, deleted or modified.

[0102] Furthermore, whereas specific embodiments of the present invention have been described herein for the purpose of illustrating the present invention and not limiting the present invention, it should be understood by those skilled in the art that many changes may be made to the details, materials and arrangements of elements, steps, structures and / or parts within the principles and scope of the present invention without departing from the present invention as described in the claims.

[0103] All patents, patent publications, patent applications, journal articles, books, technical references, etc. discussed in this disclosure are incorporated herein by reference in their entirety for all purposes.

Claims

1. A medical device for assisting in analyzing biological sample particles, the medical device comprising: processor; as well as a memory device storing instructions executable by the processor to cause the processor to: receiving a sample image of sample particles from a biological sample; selecting a plurality of reference images from a database of reference images of biological samples, each reference image in the plurality of reference images being a reference image of a reference particle in a plurality of reference particles from one or more biological samples; determining an order of the plurality of reference images based on a similarity between each reference image and the sample image; selecting a first reference image from the plurality of reference images based on the determined order; aligning the first reference image with the sample image; Dynamically fill the background of the aligned first reference image; as well as displaying the sample image and the dynamically filled and aligned first reference image in a user interface, The instructions for aligning the first reference image with the sample image include instructions executable by the processor to enable the processor to perform the following operations: Determine a first display position, the first display position is used to display the sample image in the user interface, the first display position is located in a first cell of the cell grid; Determining a second display position adjacent to the first display position, the second display position being used to display the first reference image in the user interface, the second display position being located in a second cell of the cell grid; and The centroid of the first reference image in the second display position is aligned with the centroid of the sample image in the first display position.

2. The medical device according to claim 1, wherein: The instructions for determining the order of the plurality of reference images include instructions executable by the processor to cause the processor to: comparing the sample image to each of the plurality of reference images; and The similarity between each reference image and the sample image is determined.

3. The medical device according to claim 2, wherein: The instructions for comparing the sample image to each of the plurality of reference images include instructions executable by the processor to cause the processor to: generating a feature vector for each reference image in the plurality of reference images and the sample image; as well as A distance metric between each reference image in the plurality of reference images and the sample image is calculated.

4. The medical device according to claim 3, wherein: Features used to generate the feature vector for each reference image in the plurality of reference images and the sample image include at least one of the following: block size, particle size, particle shape, average pixel intensity, and image average grayscale value.

5. The medical device according to claim 1, 2 or 4, wherein: The instructions for dynamically filling the background of the aligned first reference image include instructions executable by the processor to cause the processor to: extending the background of the first reference image within the second cell to fill the second cell; determining a background color of the first reference image; and The extended background of the first reference image is filled with the background color.

6. The medical device according to claim 5, wherein: The instructions for determining a background color of the first reference image include instructions executable by the processor to cause the processor to: determining four corner pixels of the first reference image; and Calculate the average color value of the four corner pixels.

7. The medical device according to claim 1 or 2 or 4 or 6, wherein: The plurality of reference images are selected based on a label associated with each of the plurality of reference images.

8. A method comprising: receiving, at a computer system, a sample image of sample particles from a biological sample; selecting, by the computer system, a plurality of reference images from a database of reference images of biological samples, each reference image in the plurality of reference images being a reference image of a reference particle in a plurality of reference particles from one or more biological samples; determining, by the computer system, an order of the plurality of reference images based on a similarity between each reference image and the sample image; selecting, by the computer system, a first reference image from the plurality of reference images based on the determined order; aligning, by the computer system, the first reference image with the sample image; dynamically filling, by the computer system, a background of the aligned first reference image; as well as The computer system displays the sample image and the dynamically filled and aligned first reference image in a user interface, Wherein, aligning the first reference image with the sample image comprises: The computer system determines a first display position, the first display position being used to display the sample image in the user interface, the first display position being located in a first cell of a cell grid; The computer system determines a second display position adjacent to the first display position, the second display position being used to display the first reference image in the user interface, the second display position being located in a second cell of the cell grid; and The computer system aligns a centroid of the first reference image in the second display location with a centroid of the sample image in the first display location.

9. The method according to claim 8, wherein: Determining the order of the plurality of reference images comprises: comparing, by the computer system, the sample image to each of the plurality of reference images; and The computer system determines the similarity between each reference image and the sample image.

10. The method according to claim 9, wherein: Comparing the sample image with each of the plurality of reference images comprises: generating, by the computer system, a feature vector for each reference image in the plurality of reference images and the sample image; and A distance metric between each reference image in the plurality of reference images and the sample image is calculated by the computer system based on the feature vector generated for each image.

11. The method according to claim 10, wherein: Features used to generate the feature vector for each reference image in the plurality of reference images and the sample image include at least one of the following: block size, particle size, particle shape, average pixel intensity, and image average grayscale value.

12. The method according to any one of claims 8 to 11, wherein: Dynamically filling the background of the aligned first reference image includes: expanding, by the computer system, a background of the first reference image located within the second cell to fill the second cell; determining, by the computer system, a background color of the first reference image; and The extended background of the first reference image is filled with the background color by the computer system.

13. The method according to claim 12, wherein: Determining the background color of the first reference image includes: determining, by the computer system, four corner pixels of the first reference image; and An average color value of the four corner pixels is calculated by the computer system.

14. The method according to any one of claims 8 to 11, wherein: The plurality of reference images are selected based on a label associated with each of the plurality of reference images.

15. A computer readable memory device comprising program code that, when executed by a processor, causes the processor to: receiving a sample image of sample particles from a biological sample; selecting a plurality of reference images from a database of reference images of biological samples, each reference image in the plurality of reference images being a reference image of a reference particle in a plurality of reference particles from one or more biological samples; determining an order of the plurality of reference images based on a similarity between each reference image and the sample image; selecting a first reference image from the plurality of reference images based on the determined order; aligning the first reference image with the sample image; Dynamically fill the background of the aligned first reference image; as well as displaying the sample image and the dynamically filled and aligned first reference image in a user interface, The program code for aligning the first reference image with the sample image includes program code that, when executed by the processor, causes the processor to perform the following operations: Determine a first display position, the first display position is used to display the sample image in the user interface, the first display position is located in a first cell of the cell grid; Determining a second display position adjacent to the first display position, the second display position being used to display the first reference image in the user interface, the second display position being located in a second cell of the cell grid; and The centroid of the first reference image in the second display position is aligned with the centroid of the sample image in the first display position.

16. The computer readable memory device of claim 15, wherein: The program code for determining the order of the plurality of reference images includes program code that, when executed by the processor, causes the processor to: comparing the sample image to each of the plurality of reference images; and The similarity between each reference image and the sample image is determined.

17. The computer readable memory device of claim 16, wherein: The program code for comparing the sample image to each of the plurality of reference images includes program code that, when executed by the processor, causes the processor to: generating a feature vector for each reference image of the plurality of reference images and the sample image; and A distance metric between each reference image in the plurality of reference images and the sample image is calculated.

18. The computer readable memory device of claim 17, wherein: Features used to generate the feature vector for each reference image in the plurality of reference images and the sample image include at least one of the following: block size, particle size, particle shape, average pixel intensity, and image average grayscale value.

19. A computer readable memory device according to any one of claims 15 to 18, wherein: The program code for dynamically filling the background of the aligned first reference image includes program code that, when executed by the processor, causes the processor to: extending the background of the first reference image within the second cell to fill the second cell; determining a background color of the first reference image; and The extended background of the first reference image is filled with the background color.

20. The computer readable memory device of claim 19, wherein: The program code for determining the background color of the first reference image includes program code that, when executed by the processor, causes the processor to: determining four corner pixels of the first reference image; and Calculate the average color value of the four corner pixels.

21. A computer readable memory device according to any one of claims 15 to 18, wherein: The plurality of reference images are selected based on a label associated with each of the plurality of reference images.

Images