Systems and methods for using different types of sample container carriers in diagnostic laboratory systems

Abstract

Diagnostic laboratory systems provided herein employ a machine learning software model to identify locations of sample containers and empty container slots in different types of sample container carriers. The model training data is based on images of different sample container carrier types each having at least two sample containers and at least one empty container slot. The images are overlaid with an estimated grid of slots based on identified locations of the at least two sample containers in the image and at least one pre-determined grid parameter. Image patches are extracted from the images based on the estimated grid. Each image patch includes a sample container or an empty container slot upon which locations of sample containers and empty container slots can be identified in sample container carriers received in a diagnostic laboratory system. Systems and methods of training a model and operating a diagnostic laboratory system are disclosed.

Description

SYSTEMS AND METHODS FOR USING DIFFERENT TYPES OF SAMPLE CONTAINER CARRIERS IN DIAGNOSTIC LABORATORY SYSTEMSCROSS-REFERENCES TO RELATED APPLICATIONS

[0001] This application claims benefit under 35 USC § 119(e) of U.S. Provisional Patent Application No. 63 / 638,080, filed on April 24, 2024, and European Patent Application 24464006.6, filed on April 24, 2024, the disclosures of which are hereby incorporated by reference herein in their entirety.FIELD

[0002] This disclosure relates to systems and methods for using different types of sample container carriers in diagnostic laboratory systems.BACKGROUND

[0003] Diagnostic laboratory systems conduct clinical chemistry tests that identify analytes or other constituents in biological samples or specimens such as blood serum, blood plasma, urine, interstitial liquid, cerebrospinal liquids, and the like. The biological samples are collected in sample containers, such as sample tubes, and transported to a laboratory system that may be located in a laboratory.

[0004] After the sample containers are received at the laboratory system, the sample containers are loaded into one or more sample container carriers (e.g., tube carriers or tube racks). The sample container carriers are then loaded into a sample handler of the laboratory system. The sample handler is an input / output module of a laboratory system that enables the laboratory system to receive and discharge sample containers by way of the sample container carriers.

[0005] Sample handlers are typically configured to support only one or a few types of sample container carriers. If a laboratory system receives sample containers held in other types of sample container carriers, laboratory technicians will need to manually transfer the sample containers to a sample container carrier supported by the sample handler. This process can be time consuming and susceptible to human error. Accordingly, laboratory systems that are able to operate with different types of sample container carriers are sought.SUMMARY

[0006] According to a first aspect, a method of training a machine learning software model to identify locations of sample containers in different types of sample container carriers in a diagnostic laboratory system is provided. The method includes receiving a sample container carrier that includes a plurality of container slots, wherein each container slot is configured to receive a sample container, at least two sample containers are held in respective container slots, and at least one container slot is empty. The method also includes generating imagedata of the sample container carrier that includes the at least two sample containers and the at least one empty container slot. The method further includes: identifying locations of the at least two sample containers in the image data; estimating a grid of slots based on the identified locations of the at least two sample containers and at least one pre-determined grid parameter; and updating the model based on the estimated grid to train the model to identify locations of sample containers held in respective container slots and empty container slots of another sample container carrier.

[0007] In another aspect, a diagnostic laboratory system operative with different types of sample container carriers is provided. The diagnostic laboratory system includes: a sample handler configured to receive a sample container carrier; the sample container carrier comprising a plurality of container slots, each of the container slots configured to hold a sample container. The diagnostic laboratory system also includes a memory that has a machine learning software model stored therein, the model trained to identify locations of empty container slots and sample containers held in respective container slots in the sample container carrier, the model trained with training data based on images of different types of sample container carriers each having at least one empty container slot and at least two sample containers, the images each overlaid with an estimated grid of slots. The diagnostic laboratory system further includes a computer configured to execute one or more programs, the model, or both to: receive image data of the sample container carrier received in the sample handler; use the model to identify in the image data any sample containers held in the sample container carrier and to determine locations thereof; use the model to identify in the image data any empty container slots in the sample container carrier and to determine locations thereof; and direct a robot to move a sample container from the sample container carrier in response to identifying in the image data the sample container and to determining a location thereof.

[0008] In a further aspect, a method of operating a diagnostic laboratory system is provided. The method includes: receiving a sample container carrier that includes a plurality of container slots, each configured to hold a sample container therein; generating image data of the sample container carrier; employing a machine learning software model to identify in the image data any sample containers held in the sample container carrier and to determine locations thereof; employ the model to identify in the image data any empty container slots in the sample container carrier and to determine locations thereof; and direct a robot to move a sample container from the sample container carrier in response to identifying in the image data the sample container and to determining a location thereof; wherein the model was trained with training data based on images of different types of sample container carriers each having at least one empty container slot and at least two sample containers, the images each overlaid with an estimated grid of slots.

[0009] Still other aspects, features, and advantages of this disclosure may be readily apparent from the following description and illustration of a number of example embodiments, including the best mode contemplated for carrying out the disclosure. This disclosure may also be capable of other and different embodiments, and its several details may be modified in various respects, all without departing from the scope of the disclosure.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described below are provided for illustrative purposes and are not necessarily drawn to scale. Accordingly, the drawings and descriptions are to be regarded as illustrative in nature, and not as restrictive. The drawings are not intended to limit the scope of the disclosure in any way.

[0011] FIG. 1 illustrates a perspective view of a diagnostic laboratory system located in a laboratory according to one or more embodiments.

[0012] FIG. 2 illustrates a detailed view of the computer of FIG. 1 in communication with a sample handler of the diagnostic laboratory system according to one or more embodiments.

[0013] FIG. 3 illustrates a robot and a sample container carrier located in a sample handler of a diagnostic laboratory system according to one or more embodiments.

[0014] FIG. 4A illustrates a plan view of a first type of sample container carrier used in a sample handler of a diagnostic laboratory system having container slots in a first configuration according to one or more embodiments.

[0015] FIG. 4B illustrates a plan view of a second type of sample container carrier used in a sample handler of a diagnostic laboratory system having container slots in a second configuration according to one or more embodiments.

[0016] FIG. 5 illustrates a block diagram showing a method of identifying sample container carriers according to one or more embodiments.

[0017] FIG. 6A illustrates a top perspective view of a sample container carrier partially occupied with sample containers and viewed from a first viewpoint according to one or more embodiments.

[0018] FIG. 6B illustrates another top perspective view of the sample container carrier of FIG. 6A viewed from a second viewpoint according to one or more embodiments.

[0019] FIG. 7 illustrates a top perspective view of a sample container carrier image wherein a sample container identification model has bounded tops of sample containers with bounding boxes according to one or more embodiments.

[0020] FIG. 8A illustrates a top plan view of a carrier slot grid laid over an image of the sample container carrier shown in FIG. 7, wherein dashed lines indicate the carrier slot grid according to one or more embodiments.

[0021] FIG. 8B illustrates a top perspective view of the carrier slot grid laid over an image of the sample container carrier shown in FIG. 8A according to one or more embodiments.

[0022] FIG. 9A illustrates a top plan view of an image of empty and occupied container slots in a sample container carrier identified by a machine learning software model according to one or more embodiments.

[0023] FIG. 9B illustrates a top perspective view of an image of empty and occupied container slots in a sample container carrier identified by a machine learning software model according to one or more embodiments.

[0024] FIG. 10 illustrates a flowchart of a method of training a machine learning software model to identify locations of sample containers in different types of sample container carriers in a diagnostic laboratory system according to one or more embodiments.

[0025] FIG. 11 illustrates a flowchart of a method of operating a diagnostic laboratory system according to one or more embodiments.DETAILED DESCRIPTION

[0026] Independent of the grammatical term usage, individuals with male, female or other gender identities are included within the term.

[0027] Automated diagnostic laboratory systems perform analyses (e.g., tests) on various biological samples, such as blood, blood serum, urine, and other bodily fluids. The samples are collected from patients and placed into sample containers, such as sample tubes. The sample containers along with testing instructions are then transported to a laboratory. The testing instructions may indicate which tests are to be performed on the samples. A technician or computer executing appropriate software may determine which instruments in the laboratory are to perform each test on each of the samples per the instructions.

[0028] Sample containers are placed and held in a sample container carrier, which is sometimes referred to as a sample tray or rack. The sample container carrier is then placed into an input / output diagnostic instrument, such as a sample handler. Sample handlers may use vision systems to identify the presence of sample containers in the sample container carriers. Robots or the like in the sample handler may then remove the sample containers from the sample container carriers for transport to one or more other diagnostic instruments for testing of the samples contained therein.

[0029] To maintain efficient sample container handling, conventional sample handlers of diagnostic laboratory systems typically support only one or a few specific types of sample container carriers that are placed in designated areas within the sample handler via carrier tracks or drawers. That is, supported sample container carriers received and properly placed in a sample handler may have sample slot locations (relative to the placement of the sample container carrier within the sample handler) that are known to the sample handler. Using avision system, such sample handlers may determine which of the sample container slots have sample containers therein and, knowing the locations of the slots within which the sample containers are held, can then direct a robot to move its grippers (e.g., end effectors) to the known locations thereof to grasp and move sample containers in accordance with testing instructions. Thus, e.g., a first type of sample handler may be configured to receive and support only a first type of sample container carrier, and a second type of sample handler may be configured to receive and support only a second type of sample container carrier. More often than not, sample containers are transported to a diagnostic laboratory system in sample container carriers that are not supported by the system’s sample handler. Laboratory technicians, therefore, often need to manually transfer sample containers from unsupported sample container carriers to supported sample container carriers.

[0030] The diagnostic laboratory systems and methods disclosed herein overcome the problems described above by training and using a machine learning software model to identify empty and occupied container slots and their locations in different types of sample container carriers that may be arbitrarily placed within sample handlers. Different types of sample container carriers may have, e.g., 15 container slots arranged in a 3x5 configuration, 48 container slots arranged in a 4x12 configuration, or 55 container slots arranged in a 5x11 configuration. Other types of sample container carriers may have other configurations. The model may include one or more machine learning algorithms that are trained to identify and determine locations of container slots that are empty and container slots that are occupied within respective sample containers. Based on these identifications, a robot of a sample handler may then be directed precisely to particular sample containers located in container slots of the sample container carrier, regardless of the type (e.g., configuration) of the sample container carrier -- provided the model was trained on each type of sample container carrier to be used in the diagnostic laboratory system. Therefore, a plurality of different types of sample container carriers may be used in a single sample handler.

[0031] Training the machine learning software model includes receiving different types of sample container carriers at an imaging location, such as within a sample handler, wherein each sample container carrier can be imaged. Each sample container carrier used for training the model has some container slots occupied with a respective sample container and some empty container slots (i.e., not occupied with a respective sample container). In some embodiments, the sample container carrier used for training the model has at least two sample containers and at least one empty container slot. Image data of the sample container carrier includes the at least two sample containers and the at least one empty container slot. In some embodiments, image data may also include the perimeter of the sample container carrier. Image data may represent 3D images and / or may be consolidated from first image data generated by an imaging device (e.g., a digital camera or imaging sensor) at a first imagingviewpoint and second image data generated by the imaging device (if movable), or a second imaging device, at a second imaging viewpoint.

[0032] The at least two sample containers and their respective locations in the sample container carrier are then identified from the image data. The identified locations may be relative to a fixed reference coordinate system of the imaging device(s) and / or imaging processor that captured and / or generated the image data. Locations of container slots holding the at least two sample containers in the image data may next be determined in response to identifying the locations of the at least two sample containers. Based on the locations of the container slots holding the at least two sample containers and at least one pre-determined grid parameter, a grid of slots is estimated and overlaid on the sample container carrier image.

[0033] The at least one pre-determined grid parameter may be a slot-to-slot distance or may be calculated by the model from one or more other pre-determined grid parameters, which may include, e.g., a slot diameter dimension, a sample container diameter dimension, a sample container cap diameter dimension, a robot-gripper-fingers clearance dimension (i.e., the space needed to grasp a sample container without contacting or disturbing adjacent sample containers), and / or a carrier area dimension in the sample handler for receiving a sample container carrier.

[0034] The model may determine x and y coordinate (orthogonal) distances between the determined container slot locations of the at least two sample containers and, by applying the slot-to-slot distance (either provided as a pre-determined grid parameter or as calculated from one or more other pre-determined grid parameters), the number of empty container slots (by row and / or column) between the container slots locations of the at least two sample containers can be determined. If needed, the model may adjust the provided or calculated slot-to-slot distance if the determined number of empty container slots (by row and / or column) between the slots of the at least two sample containers is not a whole number - that is, the provided or calculated slot-to-slot distance does not result in a uniform interval between slots. Based further on the image data of the sample container carrier’s perimeter, or the grid parameter of the carrier area dimension in the sample handler for receiving a sample container carrier, a grid of slots may be estimated for the sample container carrier and overlaid on the image thereof.

[0035] The model may then extract image patches from the sample container carrier image based on the estimated grid of slots, wherein each image patch corresponds to a respective grid unit or box and represents either a container slot holding a sample container (based on image data of the at least two sample containers) or an empty container slot (based on image data excluding the at least two sample containers). Image patch data representing an empty container slot may include one or more slot parameters such as, e.g., slot shape, size, color, and / or inner appearance (e.g., some container slots have mechanisms for holdinga sample container in place) that may be used by the model in production mode for identifying container slots in other sample container carriers received in the sample handler. Updated with this image patch data, the model may be considered trained for that type of sample container carrier. This process may be repeated for other types of sample container carriers.

[0036] The trained machine learning software model may be employed in a diagnostic laboratory system and is operative for all sample container carriers included in the training. Upon a diagnostic laboratory system receiving a sample container carrier in, e.g., a sample handler (or other type of input / output diagnostic instrument of the system), the sample container carrier may be imaged in, e.g., 3D or from two or more viewpoints. The model may be employed to identify sample containers in the sample container carrier from the imaged data, identify container slots (both empty and occupied) and their locations in the sample container carrier from the imaged data (which may be done in parallel with the sample container identification), and associate the identified sample containers with their corresponding container slots and locations thereof. Based on the image patch data used to train the model, the model may identify container slots by identifying one or more slot parameters from the imaged data, such as, e.g., slot shape, size, color, inner appearance and distance between container slots. Container slots are further identified as empty or occupied with a sample container based on the image patch data. The model allows sample containers and empty container slots of various types of sample container carriers to be identified and sample container carriers to be arbitrarily placed within a sample handler, thus simplifying the design of sample handlers. The model or other software program (e.g., a robot controller) can then generate instructions to direct a robot to specific locations in sample container carriers to move and / or return sample containers therefrom and / or thereto. These and other systems, methods, and devices that enable various types of sample container carriers to be used in diagnostic laboratory systems are described in greater detail below in connection with FIGS. 1-11.

[0037] Reference is made to FIG. 1 , which is a perspective view of a laboratory 100. A diagnostic laboratory system 102 may be located within the laboratory 100. The laboratory system 102 may be configured to perform a plurality of analyses or tests on a plurality of different biological samples. For example, the tests may determine levels of constituents or chemicals present in biological samples, such as blood, urine, cerebral fluid, and other biological samples. In other embodiments, the laboratory system 102 may be configured to perform a plurality of different tests on a single biological sample type, such as blood serum. In other embodiments, the laboratory system 102 may be configured to perform a single type of test on a single biological sample type, such as blood serum.

[0038] The laboratory system 102 may include a plurality of diagnostic instruments 104 (a few labelled) that are configured to perform the same or different tests on the biologicalsamples. In some embodiments, the diagnostic instruments 104 may be interconnected by a transport system (e.g., transport system 216 - FIG. 2). The transport system may be configured to transport the biological samples between the diagnostic instruments 104 and / or other devices in the laboratory system 102, such as centrifuges and decappers. The configuration of the laboratory system 102 may be different than the configuration shown in FIG. 1. In some embodiments, the laboratory system 102 may only include a single one of the diagnostic instruments 104.

[0039] The laboratory system 102 may be coupled to a computer 120 that may be located within the laboratory 100 or external to the laboratory 100. In some embodiments, portions of the computer 120 may be located within the laboratory 100 and other portions of the computer 120 may be located external to the laboratory 100. The computer 120 may include a processor 122 and a memory 124, wherein the memory 124 stores programs 126 configured to be executed or run on the processor 122. In some embodiments, the memory 124 and / or the programs 126 may be located external to the computer 120. For example, the computer 120 may be connected to the Internet to access external data and the like. The programs 126 may operate the diagnostic instruments 104 and process data generated by the diagnostic instruments 104.

[0040] Additional reference is made to FIG. 2, which illustrates a more detailed embodiment of the computer 120 in communication with a sample handler 200. The computer 120 may include a plurality of programs 126 that may be run on the processor 122. One of the programs 126 may be a robot controller 204 that may be configured to generate instructions that cause a robot 206 to move to specific locations as described herein. For example, the instructions may cause the robot 206 to move within the sample handler 200. The instructions may also cause the robot 206 to move sample containers 210 between sample container carriers 212 and sample movers 214. The sample movers 214 may move the sample containers 210 between diagnostic instruments 104 (FIG. 1) by way of a transport system 216, which in the embodiment of FIG. 2 may be a track configured to move the sample movers 214.

[0041] An image processor 220 may be configured to receive image data (e.g., first image data 602 - FIG. 6A and second image data 604 - FIG. 6B) generated by one or more imaging devices (e.g., imaging device 324 - FIG. 3). In some embodiments, one or more portions of the image processor 220 may be implemented in the one or more imaging devices. In some embodiments, the imaging device 324 may be configured to capture three-dimensional images of the sample containers 210, the sample container carriers 212, and other items.

[0042] A sample container identification model 224 may analyze image data, such as image data processed by the image processor 220, to identify and / or locate the sample containers 210 and / or the sample container carriers 212. The sample container identification model 224 may be a machine learning software model that is trained to identify the samplecontainers 210 and / or the sample container carriers 212. In some embodiments, the sample container identification model 224 may be configured to identify and / or locate tops (e.g., tops 600 - FIG. 6A) or caps of the sample containers 210. A distance calculator 226 may be configured to calculate or otherwise measure distances between points in the sample handler 200, such as distances between sample containers 210. The sample container identification model 224 may identify two sample containers located in the sample handler 200 and the distance calculator 226 may calculate the distance between the two sample containers. In some embodiments, the distance calculator 226 may calculate distances between tops or caps of the sample containers 210 as described herein. The distances may be used in the training of a model to identify the individual sample container carriers.

[0043] A slot status model 230 may be configured to determine whether container slots 232 (a few labelled) are occupied with sample containers 210 or whether the container slots 232 are empty. The slot status model 230 may be a machine learning software model trained to determine whether the container slots 232 are occupied with sample containers 210 or whether the container slots 232 are empty. In some embodiments, a container slot detection model 234 may be configured to identify the container slots 232 in the sample container carriers 212. The container slot detection model 234 may be a machine learning software model trained to identify empty container slots. A sample container detection model 236 may be configured to identify sample containers 210 in the different types of the sample container carriers 212. In some embodiments, the container slot detection model 234 and / or the sample container detection model 236 may identify patterns or configurations of the container slots 232. Both the container slot detection model 234 and the sample container detection model 236 may be machine learning software models trained to identify sample containers 210, container slots 232 and locations thereof, and / or patterns or configurations of the container slots 232. The container slot detection model 234 also may be trained to identify parameters of the container slots 232 such as slot shape, size, color, inner appearance, and distance between container slots 232. These parameters may also be used to train the model to identify individual sample container carriers. In some embodiments, one or more of the sample container identification model 224, the distance calculator 226, the slot status model 230, the container slot detection model 234, and / or the sample container detection model 236 may be a single machine learning software model.

[0044] The computer 120 may be coupled to a workstation 240 that enables users to communicate with the computer 120. The workstation 240 may include a display 242 and a keyboard 244 and / or other input device(s). Data generated by the computer 120 may be displayed on the display 242. The user may input data to the computer 120 via the keyboard 244 and / or the other input device(s).

[0045] The computer 120 and / or the laboratory system 102 may be coupled to a laboratory information system (LIS) 250. In some embodiments, the LIS 250 may be a program and may be executed by the computer 120. The LIS 250 may receive data and / or instructions from a hospital information system (HIS) 252, which may be at least partially implemented in a program executed by the computer 120. Medical professionals may enter testing requirements for specific patients into the HIS 252. For example, a doctor may require that blood taken from a first patient be tested for a first chemical and blood taken from a second patient be tested for a second chemical. These testing requirements may be entered into the HIS 252. The testing requirements may then be transmitted to the LIS 250, which generates a testing plan to be run by the laboratory system 102, wherein specific ones of the diagnostic instruments 104 (FIG. 1) perform the tests.

[0046] The embodiment of the sample handler 200 is shown having three of the sample container carriers 212 received therein, which are referred to individually as a first container carrier 212A, a second container carrier 212B, and a third container carrier 212C. The sample container carriers 212 may be of different types, such as from different suppliers or manufacturers. Some of the container slots 232 may be occupied with sample containers 210. The occupied container slots are identified as having dark fill. The programs 126 may use machine learning to identify the patterns or configurations of the container slots 232 in the sample container carriers 212 and / or locations of occupied and / or empty container slots. A first sample container 210A is shown occupying a container slot in the third container carrier 212C and will be referenced in examples herein.

[0047] Additional reference is made to FIG. 3, which illustrates a front perspective view of an embodiment of the robot 206 grasping the first sample container 210A. The robot 206 may include a gripper 304 (e.g., an end effector) configured to grasp sample containers 210 and move the sample containers 210 throughout the sample handler 200, including into and out of a sample container carrier 212 (FIG. 2). In the embodiment of FIG. 3, the robot 206 is illustrated moving the first sample container 210A into and out of the third container carrier 212C. As described herein, the robot 206 may be configured to move the sample containers 210 into and out of all the sample container carriers 212 (FIG. 2) even if the sample container carriers 212 have different carrier slot configurations and parameters.

[0048] The robot 206 may include a plurality of gantries that enable the gripper 304 to move in an x-direction, a y-direction, and a z-direction. First gantries 310 may be configured to move the gripper 304 in the Y-direction. A second gantry 312 may be configured to move the gripper 304 in the X-direction. A third gantry 314 may be configured to move the gripper 304 in the Z-direction. The gantries may be controlled by motors (not shown) that may receive instructions generated by the robot controller 204 (FIG. 2). In order for the robot 206 to move the sample containers 210 into and out of the sample container carriers 212 properly, the robotcontroller 204 may need to be programmed with the exact locations of the container slots 232, which may be generated by the container slot detection model 234 and / or the sample container detection model 236. The ability of the container slot detection model 234 to identify the container slots 232 may enable the robot 206 to be positioned relative to the container slots 232 irrespective of the position or orientation of the sample container carriers 212. For example, the detected container slots may be used to train the sample container identification model 224.

[0049] The robot 206 may include an arm 320 to which the gripper 304 may be attached. In some embodiments, the arm 320 may be affixed to the third gantry 314. An imaging device 324 may be affixed to the arm 320. Thus, the robot 206 may be configured to move the imaging device 324 throughout the sample handler 200 to capture images of the third container carrier 212C and other components, such as the sample containers 210 (FIG. 2) and the container slots 232, from various viewpoints as described herein. Image data (e.g., first image data 602 - FIG. 6A) generated by the imaging device 324 may be initially processed by the image processor 220 (FIG. 2). In some embodiments, the imaging device 324 may be a three- dimensional (3D) camera. The 3D camera may be a stereo-type camera that generates at least two images. In other embodiments, the 3D camera may use time-of-flight from an artificial light source or laser technology, both using a single image.

[0050] As described above, different types of sample container carriers 212 may be used in the sample handler 200. Additional reference is made to FIG. 4A, which illustrates a top plan view of the third container carrier 212C, which may be a first type of sample container carrier. Additional reference is also made to FIG. 4B, which illustrates a top plan view of the second container carrier 212B, which may be a second type of sample container carrier. The third container carrier 212C includes a plurality of container slots 232 that may be arranged in an array or grid having a plurality of rows 406 (a few labelled) and a plurality of columns 408 (a few labelled). The rows 406 may be orthogonal to the columns 408. The container slots 232 filled with sample containers 210 (one labelled) are shown with crosshatch fill. A first pair of sample containers 210 may be located in first container slots extending in a first direction (e.g., the x-direction) and a second pair of sample containers 210 may be located in second container slots extending in a second direction (y-direction) that is orthogonal to the first direction.

[0051] Crosses 410 (a few labelled) shown in FIGS. 4A and 4B show centers of the container slots 232, which may be centers of the sample containers 210 when viewed from above (e.g., a plan view). In the embodiment of FIG. 4A, the rows 406 are evenly spaced with a distance 414 between the centers of the container slots 232 or tops of the sample containers 210 as measured in the y-direction. In the embodiment of FIG. 4A, the columns 408 are evenly spaced with a distance 416 between the centers of the container slots 232 or tops of thesample containers 210 in the x-direction. Referring to FIG. 4B, the second container carrier 212B has container slots 232 (a few labelled) that are spaced differently than the spacing of the container slots 232 in the third container carrier 2120. Furthermore, the second container carrier 212B has more rows and columns and thus more container slots 232 than the third container carrier 2120. The container slots 232 are arranged in rows 422 and columns 424 that may be orthogonal. The centers of the container slots 232 constituting the rows 422 are spaced a distance 428 from each other and the container slots 232 of the columns 424 are spaced a distance 430 from each other.

[0052] Each of the container slots 232 may include a spring device 434 (a few labelled) that are configured to force and hold the sample containers 210 in a direction F within the container slots 232. Therefore, the sample containers 210 will all be located in the lower left portions of the container slots 232 when viewed from the plan view of FIG. 4B. With all the sample containers 210 forced to the same positions within the container slots 232, the distances between centers of the sample containers 210 will be the same as the distances between the container slots 232.

[0053] In the embodiment of FIGS. 4A and 4B, the distance 414 between the rows 406 in the third container carrier 212C is greater than the distance 428 between the rows 422 in the second container carrier 212B. The distance 416 between the columns 408 is greater than the distance 430 between the columns 424. Other slot configurations may be employed (e.g., slots aligned along diagonals, concentric circles, or the like).

[0054] By knowing or learning the slot configuration of the container slots 232, the sample container identification model 224 (FIG. 2) may identify the type of sample container carrier. The robot controller 204 (FIG. 2) then is able to move the gripper 304 (FIG. 3) to exact locations of the container slots 232 and retrieve the sample containers 210 from the container slots 232 and place the sample containers 210 into the container slots 232. If the sample container identification model 224 and other programs are trained solely for sample container carriers with the container slot configuration of the second container carrier 212B, e.g., the robot 206 may not be able to retrieve or replace the sample containers 210 into or out of the container slots 232 of the third container carrier 212C when the third container carrier 212C is introduced into the sample handler 200.

[0055] Additional reference is made to FIG. 5, which is a block diagram illustrating a method 500 of characterizing and / or identifying sample container carriers and, more particularly, of locating empty and occupied container slots in a sample container carrier. The method 500 is divided into a training phase 504 and a test or production mode phase 506. In the following example, the third container carrier 212C is a new type of sample container carrier being introduced into the laboratory system 102 via the sample handler 200 (FIG. 2). The method 500 commences in block 510 by having the third container carrier 212C partiallyfilled with a plurality of sample containers and by using an imaging device, such as the imaging device 324 (FIG. 3), to capture images of the third container carrier 2120 and at least two sample containers therein from different viewpoints. In some embodiments, at least two images of the at least two sample containers are captured from two different viewpoints. In some embodiments, the at least two sample containers that are imaged are of a known type. In some embodiments, at least a pair of sample containers 210 may occupy container slots 232 in the x-direction and at least a pair of sample containers 210 may occupy container slots in the y-direction. The pairs may be located in container slots 232 that are next to one another. Thus, each of the pairs may be referred to as being nearest pairs. Calculating distances between the container slots 232 may be easier and more accurate using nearest pairs because the distances do not include empty slot containers.

[0056] In the embodiment of FIG. 4A, the rows 406 and the columns 408 may be located on orthogonal axes, wherein the rows 406 are located on a first axis A41 and the columns 408 are located on a second axis A42. The first axis A41 may be parallel to a horizontal edge of the third container carrier 212C and the second axis A42 may be parallel to a vertical edge of the third container carrier 212C. The above-described pairs may be located on these axes. A first pair may include a first container slot 232A and a second container slot 232B located on the first axis A41. A second pair may include the first container slot 232A and a third container slot 232C located on the second axis A42. Thus, only three of the container slots 232 may be required to form the two pairs of container slots. In another embodiment, when an axis is determined, the perpendicular distance from a container slot to the axis may be readily determined. For example, if the first axis A41 is determined, the distance between a fourth container slot 232D and the first axis A41 may be readily determined.

[0057] Additional reference is made to FIG. 6A, which illustrates a top perspective view of the third container carrier 212C and the sample containers 210 from a first viewpoint. FIG. 6B illustrates a top perspective view of the third container carrier 212C and the sample containers 210 from a second viewpoint. The first viewpoint and the second viewpoint may be known locations within the sample handler 200 (FIG. 2) or other component of the laboratory system 102 (FIG. 1). For example, the robot 206 (FIG. 3) may move the imaging device 324 to the first viewpoint and the second viewpoint. Images of the third container carrier 212C and the sample containers 210 may be captured from the first viewpoint and the second viewpoint. Capturing images generates image data that may be processed by the image processor 220 (FIG. 2). For example, first image data 602 may be generated from the first viewpoint and second image data 604 may be generated from the second viewpoint.

[0058] The images of the third container carrier 212C and the sample containers 210 may be captured using the imaging device 324 attached to the robot 206 (FIG. 3). By capturing images from two different viewpoints, the images may be processed to determine or calculatethe locations of the sample containers 210 as described herein. For example, the locations may be determined relative to the imaging device 324 using a parallax for pure translation between viewpoints or a triangulation algorithm for more generic cases between viewpoints.

[0059] In some embodiments, the imaging device 324 may be a 3D camera as described above. The robot 206 may move the imaging device 324 to a single location where image data is generated. In stereoscopic imaging, two images may be generated. In other 3D imaging, a single image with image data and other (e.g., distance) data may be generated. Such data is referred to collectively herein as image data.

[0060] Referring again to FIG. 5, the image data generated by the imaging device 324 (FIG. 3) may be processed to detect the sample containers 210 as shown by the sample container detection model 512. Detecting the sample containers may include determining locations of the sample containers 210. In some embodiments, the sample container identification model 224 (FIG. 2) may use machine learning to identify sample containers 210 captured in the images (e.g., the image data 602, 604). In some embodiments, the sample container identification model 224 may use machine learning to identify tops 600 of the sample containers 210. The tops 600 of the sample containers 210 may be caps that seal the sample containers 210. The sample container identification model 224 may include a YOLO (you only look once) algorithm that is trained to identify the sample containers 210 and / or the tops 600 of the sample containers 210. In other embodiments, the sample container identification model 224 may use machine learning such as a convolutional neural network (CNN) to identify the sample containers 210 and / or the tops 600 of the sample containers 210. Other models such as R-CNN, Fast R-CNN, Faster R-CNN may be employed.

[0061] The sample container identification model 224 may be configured to detect centers of the sample containers 210 regardless of the configuration of the container slots 232 of the sample container carrier in which the sample containers 210 are located. The centers of the container slots 232 when viewed from a plan view of the sample container carrier may be derived based on the locations of the sample containers 210 and / or the tops 600 of the sample containers 210. In some embodiments, the distances between the centers of the sample containers 210 or the centers of the tops 600 may be the same as the distances between the container slots 232, such as the distances 412, 414, 428, and 430 (FIGS. 4A-4B).

[0062] Additional reference is made to FIG. 7, which illustrates an embodiment of an image wherein the sample container identification model 224 has identified the tops 600 of the sample containers 210 and bounded the tops 600 with bounding boxes 700 (a few labelled). The sample container identification model 224 may generate, via machine learning, the bounding boxes 700. When the sample container identification model 224 has bounded the tops 600 of the sample containers 210, the sample container identification model 224 may identify centers of the tops 600 of the sample containers 210. For example, the samplecontainer identification model 224 may locate the centers of the bounding boxes 700, which may also be centers of the sample containers 210. The centers of the tops 600 may be the same locations as the centers of the container slots 232 or other predetermined points in the container slots 232 from which distances between the container slots 232 may be determined (e.g., calculated). For example, the distances 412, 414, 428, and 430 (FIGS. 4A-4B) may be determined.

[0063] In some embodiments, the locations of the sample containers 210, such as the centers of the sample containers 210 or the centers of the tops 600 may be determined in a fixed reference coordinate system based on the multi-view detection of the sample containers 210 and location information of the robot 206 (FIG. 3). The fixed reference coordinate system may be an x-y plane as shown by the x-axis and the y-axis. For example, the robot controller 204 (FIG. 2) may provide location information of the robot 206, and thus the imaging device 324, in an x, y, z coordinate plane. In the embodiments of FIGS. 6A and 6B, the distance calculator 226 (FIG. 2) may calculate angles to the first sample container 210A from the different viewpoints. In the embodiment of FIG. 6A, the first sample container 210A is located at an angle a relative to the x-axis, which is also relative to the imaging device 324. In the embodiment of FIG. 6B, the first sample container 210A is located at an angle p relative the x-axis, which is also relative to the imaging device 324. Other angles may be measured, such as to a second sample container 210B and a third sample container 210C. In order to use some parallax algorithms, the x-axis and the y-axis may be aligned with axes of the imaging device 324, which may be aligned with edges of the third sample container 212C. In these configurations, only the lateral displacement between two viewpoints and the angle changes of the same sample container seen from two images generated from the two viewpoints are needed to determine the physical location of the sample container.

[0064] Based on the locations of the sample containers 210 and / or the tops 600 of the sample containers relative to the imaging device 324, the distance calculator 226 may calculate the distances 414 between the rows 406 and the distances 416 between the columns 408. Several different algorithms may be used to determine the distances 414, 416 based on the locations of the sample containers 210 and / or the tops 600 in the image data generated from the first view (FIG. 6A) and the second view (FIG. 6B). For example, based on the angles a and p, the distance calculator 226 may use parallax algorithms to determine the location of the first sample container 210A. The locations of the second sample container 210B and the third sample container 210C may be determined in a similar manner. The distances between the container slots 232 in the sample containers 210A, 210B, and 210C may be determined or calculated to yield the distances 414 and 416 based on the locations of the sample containers 210. Many other algorithms may be used that determine the distances 414, 416 by analyzing the image data. For example, different photogrammetry analyses may be used toanalyze the image data and determine the distances 414, 416 based on the analyses. In some embodiments, triangulation may be used to determine the locations of the sample containers.

[0065] Additional reference is made to FIG. 8A, which illustrates a top plan view of a carrier slot grid 800 laid over an image of the third container carrier 2120, wherein the dashed lines define boundaries of the carrier slot grid 800. Referring to FIG 5, the container slot extraction model 514 may generate the carrier slot grid 800 using machine learning. In some embodiments, the container slot extraction model 514 may be implemented by the sample container identification model 224 (FIG. 2).

[0066] In some embodiments, the carrier slot grid 800 may be estimated based on one or more pre-determined grid parameters, such as, e.g., the distances between the detected sample container locations in the x-axis and the y-axis. In many embodiments, the carrier slot sizes are usually slightly greater than the diameters of the sample containers, so the space between the container slots 232 in each axis may be derived based on the distances between the sample containers in each axis. Once the carrier slot grid 800 has been determined, the locations of each of the container slots 232 may be determined. Image patches from the carrier slot grid 800 may be extracted for each of the container slots 232 and used to train at least the container slot detection model 234 or the sample container detection model 236.

[0067] The carrier slot grid 800 shown in FIG. 8A includes dashed lines 802 (a few labelled) extending in the x-direction and dashed lines 804 (a few labelled) extending in the y- direction that intersect to form a plurality of boxes 808 (a few labelled) bounded by the dashed lines 802, 804. The boxes 808 may be bounding boxes. Each of the boxes 808 may include one of the container slots 232. In some embodiments, the container slots 232 may be centered in the boxes 808.

[0068] FIG. 8B illustrates a top perspective view of an image of the third container carrier 212C with dashed lines 820 defining boundaries of the carrier slot grid 800. FIG. 8B shows that the carrier slot grid 800 may be implemented in a perspective view. The carrier slot grid 800 defines the configuration of the container slots 232, which may be unique to each different type of sample container carrier.

[0069] The boxes 808 are the basis for the extracted image patches in the image data 602, 604 (FIGS. 6A-6B). The machine learning may detect sample containers 210 and container slots 232 within the image patches and may be trained to identify the sample containers 210 and the container slots 232 within the third container carrier 212C by analyzing the image patches. In some embodiments, the machine learning may analyze each of the image patches (boxes 808) and identify sample containers 210 and / or container slots 232 in the image patches (boxes 808). The training of the sample container identification model 224 may be based on parameters of the container slots 232, such as slot shape, size, color, innerappearance, and distances between the container slots 232. The parameters may also include the presence of the spring devices 434 (FIG. 4).

[0070] Referring again to FIG. 5, the empty / non-empty slot detector training 516 may be performed as described herein. The status of the container slots 232 (empty or occupied) may be determined by the slot status model 230 (FIG. 2). The image data generated by the imaging device 324 (FIG. 3), particularly the extracted image data, may be analyzed to perform the empty / non-empty slot detector training 516 to further train a slot detection model 518 (FIG. 5). The training enables status identification of individual slots of the sample container carriers. When a new sample container carrier having a container slot configuration identical or similar to the slot configuration of the third container carrier 212C is introduced into the sample handler 200 (FIG. 2), the slot detection model 518 will be able to identify and / or locate the container slots 232 based on the extracted image data obtained from images of the third container carrier 212C that was used in training. For example, the slot detection model 518 may identify slots based on the parameters described herein. In some embodiments, the slot detection model 518 may determine whether the container slots 232 are occupied. In some embodiments, the slot detection model 518 may use YOLO machine learning. Other object detection algorithms such as R-CNN, Fast R-CNN, Faster R-CNN may be employed. The sample container identification model 224 may be trained to identify individual sample container carriers based on the identified slot parameters.

[0071] Referring again to FIG. 5, after the training phase 504 is completed, the slot detection model 518 is updated or trained and is operative for all sample container carriers or container slots included in the training phase 504. In this embodiment, the first container carrier 212A is identical or similar to the third container carrier 212C on which the slot detection model 518 was trained. Thus, the slot detection model 518 is trained to recognize the carrier slot configuration and, more particularly, the locations of empty and occupied container slots in the first container carrier 212A.

[0072] The test phase 506 commences with sample container carrier image capture 530. For example, the robot 206 (FIG. 3) may move the imaging device 324 (FIG. 3) to the first viewpoint as shown in FIG. 6A. The imaging device 324 may then capture an image of the first container carrier 212A (FIG. 2) from the first viewpoint. The robot 206 may then move the imaging device 324 to the second viewpoint as shown in FIG. 6B where the imaging device 324 may capture a second image of the first container carrier 212A from the second viewpoint. The first viewpoint and the second viewpoint used in the training phase 504 may be different than the viewpoints used in the test phase 506.

[0073] After the images are captured, both the container slot detection model 234 and the sample container detection model 236 may analyze the images (e.g., the first image data 602 and the second image data 604). The container slot detection model 234 detects the containerslots 232 in the sample container carriers 212 (the first container carrier 212A in this example) based on the slot detection model 518. The sample container detection model 236 detects the sample containers 210 in the container slots 232. The sample container detection model 236 may also detect whether the container slots 232 are vacant or occupied with sample containers 210. The three-dimensional (3D) positions of the detected slots and sample containers 210, such as the sample container centers, can be derived based on the multiple view detection and the position information of the robot 206 generated by the robot controller 204 as performed in the training phase 504.

[0074] Reference is made to FIGS. 9A and 9B, which illustrate images of empty and occupied container slots in the first container carrier 212A. FIG. 9A illustrates a top plan view image of the first container carrier 212A and FIG. 9B illustrates a top perspective view image of the first container carrier 212A. The empty container slots in FIGS. 9A and 9B (e.g., empty container slot 932A) are illustrated with dashed boxes and the occupied container slots (e.g., occupied container slot 932B) are illustrated with solid boxes. The container-slot association 540 (FIG. 5) may be performed by identifying pairs of container slot centers and sample container centers based on the distance between the pairs of container slot centers and / or sample container centers as described above. Based on this information, the locations of the sample containers in their corresponding container slots 232 are readily determined. The robot controller 204 may then generate instructions to move the gripper 304 (FIG. 3) to particular container slots 232 for removal of corresponding sample containers held therein in accordance with testing instructions.

[0075] Reference is now made to FIG. 10, which is a flowchart of a method 1000 of training a machine learning software model to identify locations of sample containers (e.g., sample containers 210) in different types of sample container carriers (e.g., sample container carriers 212) in a diagnostic laboratory system (e.g., laboratory system 102) in accordance with one or more embodiments. The method 1000 includes, in block 1002, receiving a sample container carrier (e.g., third container carrier 212C) that includes a plurality of container slots (e.g., container slots 232), wherein (1) each container slot is configured to receive a sample container (e.g., a sample container 210), (2) at least two sample containers (e.g., sample containers 210) are held in respective container slots, and (3) at least one container slot is empty (i.e., no sample container held therein). Note that the at least two sample containers do not need to be in adjacent container slots, and by not filling all the container slots 232 with sample containers 210, the model can be trained to distinguish between empty and occupied container slots (i.e., a sample container held therein).

[0076] The method 1000 includes, in block 1004, generating image data (e.g., image data 602) of the sample container carrier that includes the at least two sample containers (e.g.,sample containers 210) and the at least one empty container slot. For example, image data may be generated by imaging device 324 (FIGS. 6A-6B).

[0077] The method 1000 includes, in block 1006, identifying locations of the at least two sample containers in the image data. The locations of the at least two sample containers may be determined using parallax algorithms or other algorithms that analyze image data. These algorithms may be incorporated in the model.

[0078] The method 1000 includes, in block 1008, estimating a grid of slots based on the identified locations of the at least two sample containers and at least one pre-determined grid parameter. The grid of slots may be, e.g., carrier slot grid 800 (FIGS. 8A-8B), and the at least one pre-determined grid parameter may be one or more of, e.g., a slot diameter dimension, a sample container diameter dimension, a sample container cap diameter dimension, a robot- gripper-fingers clearance dimension, or a slot-to-slot distance.

[0079] The method 1000 includes, in block 1010, updating the model based on the estimated grid to train the model to identify locations of empty container slots and sample containers held in respective container slots of another sample container carrier. In some embodiments, the model is updated with image patch data representing image patches extracted from the generated image data of the sample container carrier based on the estimated slot grid. Each image patch corresponds to a respective grid unit or box and represents either a container slot holding a sample container (based on image data of the at least two sample containers) or an empty container slot (based on image data excluding the at least two sample containers).

[0080] Reference is now made to FIG. 11 , which illustrates a flowchart of a method 1100 of operating a diagnostic laboratory system (e.g., diagnostic laboratory system 102) in accordance with one or more embodiments. The method 1100 includes, in block 1102, receiving a sample container carrier (e.g., first container carrier 212A) that includes a plurality of container slots (e.g., container slots 232), each container slot configured to hold a sample container (e.g., a sample container 210) therein. Reference is made to FIG. 2, which shows the first container carrier 212A partially loaded with sample containers 210 and received in the sample handler 200.

[0081] The method 1100 includes, in block 1104, generating image data (e.g., first image data 602 and second image data 604) of the sample container carrier. For example, FIGS. 6A and 6B illustrate sample containers in the third container carrier 212C being imaged by imaging device 324, which outputs first and second image data 602 and 604.

[0082] The method 1100 includes, in block 1106, employing a machine learning software model to identify in the image data any sample containers held in the sample container carrier and to determine locations thereof. In some embodiments, the model was trained with training data based on images of different types of sample container carriers each having at least oneempty container slot and at least two sample containers, wherein the images are each overlaid with an estimated grid of slots. See, e.g., carrier slot grid 800 (FIGS. 8A-8B).

[0083] The method 1100 includes, in block 1108, employing the model to identify in the image data any empty container slots in the sample container carrier and to determine locations thereof.

[0084] The method 1100 includes, in block 1110, directing a robot (e.g., robot 206) to move a sample container from the sample container carrier in response to identifying in the image data the sample container and to determining a location thereof. For example, the robot controller 204 (FIG. 2) may generate instructions that direct the robot 206 to move sample container 210A from sample container carrier 212C to, e.g., a sample mover 214 received in the sample handler 200 based on the model’s identification of sample container 210A and determination of its location in sample container carrier 212C.NON-LIMITING ILLUSTRATIVE EMBODIMENTS

[0085] The following is a list of non-limiting illustrative embodiments disclosed herein.

[0086] Illustrative embodiment 1. A method of training a machine learning software model to identify locations of sample containers in different types of sample container carriers in a diagnostic laboratory system, the method comprising: receiving a sample container carrier that includes a plurality of container slots each configured to receive a sample container, wherein at least two sample containers are held in respective container slots and at least one container slot is empty; generating image data of the sample container carrier that includes the at least two sample containers and the at least one empty container slot; identifying locations of the at least two sample containers in the image data; estimating a grid of slots based on the identified locations of the at least two sample containers and at least one pre-determined grid parameter; and updating the model based on the grid to train the model to identify locations of empty container slots and sample containers held in respective container slots of another sample container carrier.

[0087] Illustrative embodiment 2. The method according to the preceding embodiment, wherein the updating the model comprises: extracting an image patch based on the grid for each container slot in the grid, wherein data from each image patch represents: a container slot holding a sample container based on extracted image patches of the container slots respectively holding the at least two sample containers, or an empty container slot based on extracted image patches of the grid excluding the image patches of the at least two sample containers; and updating the model with the image patch data.

[0088] Illustrative embodiment 3. The method according to one of the preceding embodiments, wherein the image patch data comprises at least one of a container slot shape, color, and inner appearance.

[0089] Illustrative embodiment 4. The method according to one of the preceding embodiments, wherein the at least one pre-determined grid parameter comprises a slot diameter dimension, a sample container diameter dimension, a sample container cap diameter dimension, a robot-gripper-fingers clearance dimension, or a slot-to-slot distance.

[0090] Illustrative embodiment 5. The method according to one of the preceding embodiments, further comprising, prior to the estimating, calculating a slot-to-slot dimension in response to the at least one pre-determined grid parameter comprising one or more of a slot diameter dimension, a sample container diameter dimension, a sample container cap diameter dimension, and a robot-gripper-fingers clearance dimension.

[0091] Illustrative embodiment 6. The method according to one of the preceding embodiments, wherein the identifying the locations of the at least two sample containers in the image data comprises identifying locations of tops of the at least two sample containers.

[0092] Illustrative embodiment 7. The method according to one of the preceding embodiments, wherein the generating image data comprises: generating first image data of the sample container carrier including the at least two sample containers and the at least one empty container slot from a first viewpoint; generating second image data of the sample container carrier including the at least two sample containers and at least one empty container slot from a second viewpoint; and consolidating the first and second image data to form the image data.

[0093] Illustrative embodiment 8. The method according to one of the preceding embodiments, wherein the at least two sample containers comprise a first pair of sample containers located in respective first container slots extending in a first direction and a second pair of sample containers located in respective second container slots extending in a second direction that is orthogonal to the first direction.

[0094] Illustrative embodiment 9. The method according to one of the preceding embodiments, wherein the generating image data comprises generating the image data using an imaging device configured to capture three-dimensional images.

[0095] Illustrative embodiment 10. The method according to one of the preceding embodiments, further comprising defining a bounding box about each of the at least two sample containers in the image data.

[0096] Illustrative embodiment 11. The method according to one of the preceding embodiments, wherein the estimating the grid of slots is based at least partially on locations of the bounding boxes, wherein each bounding box includes at least a portion of a container slot, and wherein the identifying also comprises identifying locations of the container slots associated with the bounding boxes.

[0097] Illustrative embodiment 12. The method according to one of the preceding embodiments, further comprising: providing an imaging device attached to a robot; moving theimaging device to a first viewpoint using the robot, wherein the generating the image data comprises generating first image data using the imaging device at the first viewpoint; and moving the imaging device to a second viewpoint using the robot, wherein the generating the image data further comprises generating second image data using the imaging device at the second viewpoint.

[0098] Illustrative embodiment 13. A diagnostic laboratory system operative with different types of sample container carriers, comprising: a sample handler configured to receive a sample container carrier, the sample container carrier comprising a plurality of container slots, each of the container slots configured to hold a sample container; a memory having a machine learning software model stored therein, the model trained to identify locations of empty container slots and sample containers held in respective container slots in the sample container carrier, the model trained with training data based on images of different types of sample container carriers each having at least one empty container slot and at least two sample containers, the images each overlaid with an estimated grid of slots; and a computer configured to execute one or more programs, the model, or both to: receive image data of the sample container carrier received in the sample handler; use the model to identify in the image data any sample containers held in the sample container carrier and to determine locations thereof; use the model to identify in the image data any empty container slots in the sample container carrier and to determine locations thereof; and direct a robot to move a sample container from the sample container carrier in response to identifying in the image data the sample container and to determining a location thereof.

[0099] Illustrative embodiment 14. The diagnostic laboratory system according to the preceding embodiment, wherein the training data includes data representing image patches extracted from the images of the different types of sample container carriers each overlaid with the estimated grid of slots, each image patch representing one empty container slot or one sample container.

[0100] Illustrative embodiment 15. The diagnostic laboratory system according to one of the preceding embodiments, wherein the computer further uses the model to identify in the image data any sample containers held in the sample container carrier and to determine locations thereof by: detecting any sample containers in the sample container carrier; identifying container slots in the sample container carrier and determining respective locations of the container slots; and associating each detected sample container with a corresponding container slot and respective location.

[0101] Illustrative embodiment 16. The diagnostic laboratory system according to one of the preceding embodiments, further comprising an imaging device configured to generate the image data.

[0102] Illustrative embodiment 17. The diagnostic laboratory system according to one of the preceding embodiments, further comprising a second robot configured to move the imaging device to a first viewpoint and to a second viewpoint to generate image data at the first viewpoint and the second viewpoint.

[0103] Illustrative embodiment 18. A method of operating a diagnostic laboratory system, the method comprising: receiving a sample container carrier that includes a plurality of container slots, each configured to hold a sample container therein; generating image data of the sample container carrier; employing a machine learning software model to identify in the image data any sample containers held in the sample container carrier and to determine locations thereof; employing the model to identify in the image data any empty container slots in the sample container carrier and to determine locations thereof; and directing a robot to move a sample container from the sample container carrier in response to identifying in the image data the sample container and to determining a location thereof; wherein: the model was trained with training data based on images of different types of sample container carriers each having at least one empty container slot and at least two sample containers, the images each overlaid with an estimated grid of slots.

[0104] Illustrative embodiment 19. The method according to one of the preceding embodiments, wherein the training data includes data representing image patches extracted from the images of the different types of sample container carriers each overlaid with the estimated grid of slots, each image patch based on the grid and representing one empty container slot or one sample container.

[0105] Illustrative embodiment 20. The method according to one of the preceding embodiments, further comprising: providing an imaging device attached to a second robot; moving the imaging device to a first viewpoint using the second robot, wherein the generating the image data comprises generating first image data using the imaging device at the first viewpoint; moving the imaging device to a second viewpoint using the second robot, wherein the generating the image data further comprises generating second image data using the imaging device at the second viewpoint; and consolidating, via an image processor, the first and second image data to form the image data.

[0106] While the disclosure is susceptible to various modifications and alternative forms, specific method and apparatus embodiments have been shown by way of example in the drawings and are described in detail herein. It should be understood, however, that the particular methods and apparatus disclosed herein are not intended to limit the disclosure.

Claims

CLAIMSWhat is claimed is:

1. A method of training a machine learning software model to identify locations of sample containers in different types of sample container carriers in a diagnostic laboratory system, the method comprising: receiving a sample container carrier that includes a plurality of container slots each configured to receive a sample container, wherein at least two sample containers are held in respective container slots and at least one container slot is empty; generating image data of the sample container carrier that includes the at least two sample containers and the at least one empty container slot; identifying locations of the at least two sample containers in the image data; estimating a grid of slots based on the identified locations of the at least two sample containers and at least one pre-determined grid parameter; and updating the model based on the grid to train the model to identify locations of empty container slots and sample containers held in respective container slots of another sample container carrier.

2. The method of claim 1 , wherein the updating the model comprises: extracting an image patch based on the grid for each container slot in the grid, wherein data from each image patch represents: a container slot holding a sample container based on extracted image patches of the container slots respectively holding the at least two sample containers, or an empty container slot based on extracted image patches of the grid excluding the image patches of the at least two sample containers; and updating the model with the image patch data.

3. The method of claim 2, wherein the image patch data comprises at least one of a container slot shape, color, and inner appearance.

4. The method of claim 1, wherein the at least one pre-determined grid parameter comprises a slot diameter dimension, a sample container diameter dimension, a sample container cap diameter dimension, a robot-gripper-fingers clearance dimension, or a slot-to- slot distance.

5. The method of claim 1 , further comprising, prior to the estimating, calculating a slot- to-slot dimension in response to the at least one pre-determined grid parameter comprisingone or more of a slot diameter dimension, a sample container diameter dimension, a sample container cap diameter dimension, and a robot-gripper-fingers clearance dimension.

6. The method of claim 1 , wherein the identifying the locations of the at least two sample containers in the image data comprises identifying locations of tops of the at least two sample containers.

7. The method of claim 1 , wherein the generating image data comprises: generating first image data of the sample container carrier including the at least two sample containers and the at least one empty container slot from a first viewpoint; generating second image data of the sample container carrier including the at least two sample containers and at least one empty container slot from a second viewpoint; and consolidating the first and second image data to form the image data.

8. The method of claim 1 , wherein the at least two sample containers comprise a first pair of sample containers located in respective first container slots extending in a first direction and a second pair of sample containers located in respective second container slots extending in a second direction that is orthogonal to the first direction.

9. The method of claim 1 , wherein the generating image data comprises generating the image data using an imaging device configured to capture three-dimensional images.

10. The method of claim 1 , further comprising defining a bounding box about each of the at least two sample containers in the image data.11 . The method of claim 10, wherein the estimating the grid of slots is based at least partially on locations of the bounding boxes, wherein each bounding box includes at least a portion of a container slot, and wherein the identifying also comprises identifying locations of the container slots associated with the bounding boxes.

12. The method of claim 1 , further comprising: providing an imaging device attached to a robot; moving the imaging device to a first viewpoint using the robot, wherein the generating the image data comprises generating first image data using the imaging device at the first viewpoint; andmoving the imaging device to a second viewpoint using the robot, wherein the generating the image data further comprises generating second image data using the imaging device at the second viewpoint.

13. A diagnostic laboratory system operative with different types of sample container carriers, comprising: a sample handler configured to receive a sample container carrier, the sample container carrier comprising a plurality of container slots, each of the container slots configured to hold a sample container; a memory having a machine learning software model stored therein, the model trained to identify locations of empty container slots and sample containers held in respective container slots in the sample container carrier, the model trained with training data based on images of different types of sample container carriers each having at least one empty container slot and at least two sample containers, the images each overlaid with an estimated grid of slots; and a computer configured to execute one or more programs, the model, or both to: receive image data of the sample container carrier received in the sample handler; use the model to identify in the image data any sample containers held in the sample container carrier and to determine locations thereof; use the model to identify in the image data any empty container slots in the sample container carrier and to determine locations thereof; and direct a robot to move a sample container from the sample container carrier in response to identifying in the image data the sample container and to determining a location thereof.

14. The diagnostic laboratory system of claim 13, wherein the training data includes data representing image patches extracted from the images of the different types of sample container carriers each overlaid with the estimated grid of slots, each image patch representing one empty container slot or one sample container.

15. The diagnostic laboratory system of claim 13, wherein the computer further uses the model to identify in the image data any sample containers held in the sample container carrier and to determine locations thereof by: detecting any sample containers in the sample container carrier; identifying container slots in the sample container carrier and determining respective locations of the container slots; andassociating each detected sample container with a corresponding container slot and respective location.

16. The diagnostic laboratory system of claim 13, further comprising an imaging device configured to generate the image data.

17. The diagnostic laboratory system of claim 16, further comprising a second robot configured to move the imaging device to a first viewpoint and to a second viewpoint to generate image data at the first viewpoint and the second viewpoint.

18. A method of operating a diagnostic laboratory system, the method comprising: receiving a sample container carrier that includes a plurality of container slots, each configured to hold a sample container therein; generating image data of the sample container carrier; employing a machine learning software model to identify in the image data any sample containers held in the sample container carrier and to determine locations thereof; employing the model to identify in the image data any empty container slots in the sample container carrier and to determine locations thereof; and directing a robot to move a sample container from the sample container carrier in response to identifying in the image data the sample container and to determining a location thereof; wherein: the model was trained with training data based on images of different types of sample container carriers each having at least one empty container slot and at least two sample containers, the images each overlaid with an estimated grid of slots.

19. The method of claim 18, wherein the training data includes data representing image patches extracted from the images of the different types of sample container carriers each overlaid with the estimated grid of slots, each image patch based on the grid and representing one empty container slot or one sample container.

20. The method of claim 18, further comprising: providing an imaging device attached to a second robot; moving the imaging device to a first viewpoint using the second robot, wherein the generating the image data comprises generating first image data using the imaging device at the first viewpoint;moving the imaging device to a second viewpoint using the second robot, wherein the generating the image data further comprises generating second image data using the imaging device at the second viewpoint; and consolidating, via an image processor, the first and second image data to form the image data.

Images