Sample handler for diagnostic laboratory analyzers and method of use

Abstract

A sample handler of a diagnostic laboratory system includes a plurality of holding positions configured to receive sample containers. An imaging device is movable within the sample handler and is configured to capture an image of the holding position and the sample container received therein. The controller is configured to generate instructions that cause the imaging device to move within the sample handler and capture an image. The classification algorithm is implemented in computer code and includes a training model configured to classify an object in the captured image. Other sample handlers and methods of handling sample containers are also disclosed.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 364,911, filed on May 18, 2022, entitled “SAMPLE HANDLERS OF DIAGNOSTICLABORATORY ANALYZERS AND METHODS OF USE,” the disclosure of which is hereby incorporated by reference in its entirety for all purposes. Technical Field

[0003] Embodiments of the present disclosure relate to a sample handler for a diagnostic laboratory analyzer and methods of using the sample handler. Background Art

[0004] Clinical diagnostic laboratory systems process patient samples, such as blood, urine, or body tissue, to test for various analytes. Samples are collected from patients and stored in sample containers, which are then transported to the laboratory that houses the diagnostic system. The laboratory system includes a sample processor that receives the sample containers. The sample containers are placed into trays, which are then loaded into the sample processor. Robots transfer the sample containers to and from carriers, which transport the sample containers between instruments and other components within the laboratory system.

[0005] In order to accurately approach and process the sample container, it is necessary to identify the position and type of the sample container in the tray so that the robot can locate and transport the sample container throughout the laboratory system. As the processing time of the laboratory system decreases, it is necessary to identify and locate the sample container more quickly. Therefore, a sample processing machine that can quickly identify and locate the sample container and a method for processing the sample container are needed. Summary of the invention

[0006] According to a first aspect, a sample processor of a diagnostic laboratory system is provided. The sample processor comprises: a plurality of holding positions configured to receive sample containers; an imaging device movable within the sample processor, the imaging device configured to capture images of the holding positions and generate image data representative of the images; a controller configured to generate instructions that cause the imaging device to move within the sample processor and cause the imaging device to capture images; and a classification algorithm implemented in computer code, the classification algorithm comprising a training model configured to classify objects in the images.

[0007] In another aspect, another sample processor of a diagnostic laboratory system is provided. The sample processor includes: a plurality of holding positions configured to receive sample containers; a robot capable of moving within the sample processor, the robot including a gripper configured to grip the sample container to move the sample container into and out of the holding positions; an imaging device fixed to the robot, the imaging device configured to capture an image of the sample container and generate image data representative of the image; a controller configured to generate instructions that cause the robot to move within the sample processor and capture images using the imaging device; and a classification algorithm implemented in computer code, the classification algorithm including a training model configured to recognize the sample container.

[0008] In another aspect, a method of operating a sample processor of a diagnostic laboratory system is provided. The method includes providing a plurality of holding positions within the sample processor, each of the plurality of holding positions being configured to receive a sample container; providing a robot having a gripper configured to grip the sample container and move the sample container into and out of the plurality of holding positions; transporting an imaging device within the sample processor; capturing an image of one or more of the sample containers; and classifying the image using a classification algorithm implemented in computer code, the classification algorithm comprising a trained model configured to recognize the sample container.

[0009] Other aspects, features and advantages of the present disclosure may be apparent from the following description and illustration of several exemplary embodiments including the best mode contemplated for implementing the present disclosure. The present disclosure may also have other and different embodiments, and its several details may be modified in various aspects, all without departing from the scope of the present disclosure. The present disclosure is intended to cover all modifications, equivalents and alternatives that fall within the scope of the claims and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described below are provided for illustrative purposes and are not necessarily drawn to scale. Therefore, the drawings and descriptions are to be regarded as illustrative rather than restrictive in nature. The drawings are not intended to limit the scope of the present disclosure in any way.

[0011] Figure 1 Illustrated is a block diagram of a diagnostic laboratory system including a sample handler according to one or more embodiments.

[0012] Figure 2 Illustrated is a top view of the interior of a sample processor of a diagnostic laboratory system in accordance with one or more embodiments.

[0013] Figure 3Illustrated is a perspective view of a robot in a sample handler of a diagnostic laboratory system coupled to a gantry configured to move the robot and attached imaging device along x, y, and z axes in accordance with one or more embodiments.

[0014] Figure 4 The present invention illustrates a method according to one or more embodiments. Figure 3 A side view of the robot wherein the imaging device operates to capture an image of a sample container.

[0015] Figure 5 A flow chart of a method of operating a robot in a sample handler of a diagnostic laboratory system is illustrated in accordance with one or more embodiments.

[0016] Figure 6 A flow chart of a method of identifying a sample container and operating a sample processor of a diagnostic laboratory system is illustrated in accordance with one or more embodiments.

[0017] Figure 7 An example of improperly gripping a sample container according to one or more embodiments is shown. Figure 3 Side view of the robot.

[0018] Figure 8 An example of improperly gripping a sample container according to one or more embodiments is shown. Figure 3 Another side view of the robot.

[0019] Fig. 9 Illustrated is a top view of a sample processor of a diagnostic laboratory system having two spill and misaligned trays in accordance with one or more embodiments.

[0020] Fig.10 A flow chart of a method of operating a sample processor of a diagnostic laboratory system is illustrated in accordance with one or more embodiments. DETAILED DESCRIPTION

[0021] The diagnostic laboratory system performs clinical chemistry reactions and / or assays to identify analytes or other components in biological samples such as serum, plasma, urine, interstitial fluid, cerebrospinal fluid, etc. These samples are collected in sample containers and then sent to the diagnostic laboratory system. The sample containers are then loaded into trays, which are then loaded into the sample processor of the laboratory system.

[0022] The robot within the sample handler is configured to grasp the sample container and transfer the sample container to a sample carrier that transports the sample container to a specific location, such as a specific processing or analytical instrument in a laboratory system. The robot or the robot's controller needs to know the location of the sample container in the tray in order to grasp the correct sample container. In addition, the laboratory system may need to determine the type of sample container stored in a specific location in the tray. For example, identification can determine whether the sample container is capped, uncapped, or a tube-top sample cup. Identification can also determine the manufacturer of the sample container and whether the sample container has any chemicals located therein that are used during testing.

[0023] Accurately identifying a sample container may be time consuming. For example, some sample processing machines include at least one fixed imaging device at a fixed position that captures an image of the sample container when the sample container is located in the sample processing machine. These fixed cameras may have a limited field of view and may not be able to capture enough images of the sample container to accurately identify the sample container. Some sample processing machines overcome some of these problems using multiple fixed cameras. However, multiple fixed cameras increase the cost of the sample processing machine and increase the processing resources of the sample processing machine.

[0024] The sample processing machine described herein includes an imaging device (e.g., a camera) that can move within the sample processing machine. In some embodiments, the imaging device is mounted to a robot that can move within the sample processing machine. In some embodiments, the robot can be configured to move sample containers within the sample processing machine. In other embodiments, another robot can be dedicated to moving the imaging device throughout the sample processing machine. When the imaging device moves throughout the sample processing machine, the imaging device can capture images of sample containers and other objects within the sample processing machine. These images can be used to identify, locate and / or classify sample containers and / or other objects.

[0025] The robot may include a gripper configured to grip the sample container. An imaging device may be secured to the gripper to provide a view of the sample container. In other embodiments, the imaging device may be secured to a side of the robot, which enables the imaging device to capture images of the sample container. In addition to identifying the classification of the sample container, the classification may also determine whether the robot has properly gripped the sample container. In some embodiments, the imaging device may be oriented to capture images in a downward direction, which enables the imaging device to capture the top or cap of the sample container. This orientation also enables the imaging device to capture images of spills and other objects in the sample processing machine. The classification described herein may identify spills and other objects.

[0026] refer to Figure 1-10 These and other sample processors and methods of processing sample containers in laboratory systems are described in more detail.

[0027] Reference now Figure 1 , which illustrates a block diagram of an embodiment of a diagnostic laboratory system 100. The laboratory system 100 may include a plurality of instruments 102 configured to process sample containers 104 (some labeled) and perform assays or tests on samples located in the sample containers 104. The laboratory system 100 may have a first instrument 102A and a second instrument 102B. Other embodiments of the laboratory system 100 may include more or fewer instruments.

[0028] The samples located in the sample containers 104 can be various biological specimens collected from individuals, such as patients being evaluated by medical professionals. These samples can be collected from patients and placed directly into the sample containers 104. The sample containers 104 can then be transported to a laboratory or facility that houses the laboratory system 100. As described in more detail below, the sample containers 104 can be loaded into a sample processing machine 106, which can be an instrument of the laboratory system 100. The sample containers 104 can be transferred from the sample processing machine 106 to a sample carrier 112 (some are labeled), which transports the sample containers 104 throughout the laboratory system 100, such as to the instrument 102, by means of a track 114.

[0029] The track 114 is configured to enable the sample carrier 112 to move throughout the laboratory system 100, including to and from the sample processor 106. For example, the track 114 may extend near or around at least some of the instruments 102 and the sample processor 106, such as Figure 1 The instrument 102 and the sample processing machine 106 may have a robot ( Figure 1 The track 114 may include a device (not shown) that transfers the sample container 104 to and from the sample carrier 112. The track 114 may include a plurality of segments 120 (some are labeled) that may be interconnected. The carrier 112 may move along the dashed lines 122, as shown in the segments 120. In some embodiments, some of the segments 120 may be integrated with one or more of the instruments 102.

[0030] Components of the laboratory system 100, such as the sample processor 106 and the instrument 102, may include or be coupled to a computer 130 configured to execute one or more programs that control the components of the laboratory system 100, including the sample processor 106. The computer 130 may be configured to communicate with the instrument 102, the sample processor 106, and other components of the laboratory system 100. The computer 130 may include a processor 132 configured to execute programs, including programs other than those described herein. The programs may be implemented in computer code.

[0031] The computer 130 may include or have access to a memory 134 that may store one or more programs and / or data described herein. The memory 134 and / or the programs stored therein may be referred to as non-transitory computer-readable media. These programs may be computer code that may be executed on or by the processor 132. The memory 134 may include a robotic controller 136 configured to generate instructions to control a robot and / or similar device in the instrument 102 and the sample processing machine 106. As described herein, the instructions generated by the robotic controller 136 may be in response to data, such as image data received from the sample processing machine 106.

[0032] The memory 134 may also store a classification algorithm 138 that is configured to identify and / or classify other items in the sample container 104 and / or the sample processing machine 106. In some embodiments, the classification algorithm 138 classifies objects in the image data. The classification algorithm 138 may include a training model, such as one or more neural networks. For example, the classification algorithm 138 may include a convolutional neural network (CNN) trained to recognize objects in the image data. The training model is implemented using artificial intelligence (AI). Therefore, the training model can learn to classify objects. It should be noted that the classification algorithm 138 is not a lookup table.

[0033] The computer 130 may be coupled to a workstation 139 that is configured to enable a user to interact with the laboratory system 100. The workstation 139 may include a display 140, a keyboard 142, and other peripheral devices (not shown). Data generated by the computer 130 may be displayed on the display 140. The data may include abnormal warnings detected by the classification algorithm 138. In addition, the user may input data into the computer 130 with the aid of the workstation 139. The data input by the user may be instructions for causing the robot controller 136 or the classification algorithm 138 to perform certain operations.

[0034] Now refer to Figure 2 , which illustrates a top view of the interior of a sample processing machine 106 according to one or more embodiments. The sample processing machine 106 is configured to capture images of sample containers 104 and move sample containers 104 between holding positions 210 (some are labeled) and sample carriers 112. Figure 2In some embodiments, the holding position 210 is located within a tray 214, as further described below. The sample processing machine 106 may include a plurality of slides 212 configured to hold a tray 214. In some embodiments, the sample processing machine 106 may include four slides 212, which are referred to as a first slide 212A, a second slide 212B, a third slide 212C, and a fourth slide 212D. The third slide 212C is shown as being partially removed from the sample processing machine 106, which may occur during replacement of a tray 214. Other embodiments of the sample processing machine 106 may include more than Figure 2 Fewer or more slides than shown.

[0035] Each slide 212 can be configured to hold one or more trays 214. Figure 2 In an embodiment of the present invention, the slide 212 may include a receiver 216 configured to receive a tray 214. Each tray 214 may include a plurality of holding positions 210, wherein each holding position 210 is configured to receive one of the sample containers 104. Figure 2 In the embodiment of the present invention, the trays can vary in size to include large trays having twenty-four holding positions 210 and small trays having eight holding positions 210. Other configurations of trays can include different numbers of holding positions 210.

[0036] In some embodiments, the sample handler 106 may include one or more slide sensors 220 configured to sense movement of one or more of the slides 212. The slide sensors 220 may generate signals indicative of slide movement, wherein the signals may be received and / or processed by the robotic controller 136 as described herein. Figure 2 In an embodiment of the present invention, the sample processing machine 106 includes four slide sensors 220, which are arranged so that each slide 212 is associated with one of the slide sensors 220. The first slide sensor 220A senses the movement of the first slide 212A, the second slide sensor 220B senses the movement of the second slide 212B, the third slide sensor 220C senses the movement of the third slide 212C, and the fourth slide sensor 220D senses the movement of the fourth slide 212D. The slide sensor 220 can use various technologies to sense the movement of the slide 212. In some embodiments, the slide sensor 220 can include a mechanical switch that switches when the slide 212 moves. The switching generates a signal indicating that the slide has moved. In other embodiments, the slide sensor 220 can generate an optical signal in response to the movement of the slide 212. In still other embodiments, the slide sensor 220 can be an imaging device that generates image data when the slide 212 moves.

[0037] The sample processing machine 106 includes an imaging device 226 that is movable throughout the sample processing machine 106. Figure 2 In some embodiments, the imaging device 226 is secured to a robot 228 that can move along an x-axis (e.g., in the x-direction) and a y-axis (e.g., in the y-direction) throughout the sample handler 106. In some embodiments, the imaging device 226 can be integrated with the robot 228. In some embodiments, the robot 228 can move along a z-axis (e.g., in the z-direction), i.e., into and out of the page. In other embodiments, the robot 228 can include one or more components that move the imaging device 226 in the z-direction (e.g., in the z-direction). Figure 2 In some embodiments, the robot 228 may receive commands from the robot controller 136 ( Figure 1 ) generated movement instructions. These instructions may be data indicating the x and y positions to which the robot 228 should move. In other embodiments, these instructions may be electrical signals that cause the robot 228 to move in the x and y directions. The robot controller 136 may generate instructions to move the robot 228 in response to one or more of the slide sensors 220 detecting movement of one or more of the slides 212. These instructions may cause the robot 228 to move while the imaging device 226 captures an image of the newly added sample container.

[0038] The imaging device 226 includes one or more cameras that capture images, wherein capturing an image generates image data representing the image. The image data can be transmitted to the computer 130 to be processed by the classification algorithm 138 as described herein. The one or more cameras are configured to capture images of the sample container 104 and / or other locations or objects in the sample processing machine 106. These images can be the top and / or side of the sample container 104. In some embodiments, the robot 228 can be a gripper robot that grips the sample container 104 and moves the sample container 104 between the holding position 210 and the sample carrier 112. As described herein, these images can be captured while the robot 228 grips the sample container 104.

[0039] Additional references Figure 3, which is a perspective view of an embodiment of a robot 228 coupled to a gantry 330 that is configured to move the robot 228 in the x-direction, the y-direction, and the z-direction. The gantry 330 may include two y slides 332 that enable the robot 228 to move in the y-direction, an x ​​slide 334 that enables the robot 228 to move in the x-direction, and a z slide 336 that enables the robot 228 to move in the z-direction. Movement in these three directions may be simultaneous and may be controlled by the robot controller 136. For example, the robot controller 136 may generate instructions that cause a motor (not shown) coupled to the gantry 330 to move the slides so as to move the robot 228 and the imaging device 226 to a predetermined position.

[0040] The robot 228 may include a gripper 340 (e.g., an end effector) configured to grip the sample container 304. The sample container 304 may be an example of the sample container 104. The robot 228 moves to a position above the holding position, and then moves in the z-direction to retrieve the sample container 304 from the holding position. The gripper 340 opens and the robot 228 moves downward in the z-direction so that the gripper 340 extends above the sample container 304. The gripper 340 closes to grip the sample container 304, and the robot 228 moves upward in the z-direction to extract the sample container 304 from the holding position. Figure 3 As shown in , the imaging device 226 can be secured to the robot 228. The imaging device 226 includes at least one camera configured to capture images, wherein the captured images are converted into image data for processing by the classification algorithm 138, for example.

[0041] Additional references Figure 4 , which is a side view of an embodiment of the robot 228 grasping the sample container 304 using the gripper 340 while the sample container 304 is being imaged by the imaging device 226. Figure 4 The imaging device 226 depicted in FIG may include a first camera 436 and a second camera 438. Other embodiments of the imaging device 226 may include a single camera or more than two cameras. The first camera 436 has a field of view 439 that extends at least partially along the y direction and may be configured to capture an image of a sample container (e.g., sample container 304) being grasped by the gripper 340. The illumination source 440 may illuminate an object in the field of view 439. In some embodiments, the spectrum and intensity of light emitted by the illumination source 440 may be controlled by the classification algorithm 138. In other embodiments, the robot controller 136 ( Figure 1 ) is configured to control at least one of the intensity of the illumination source 440 and the spectrum of light emitted by the illumination source 440.

[0042] The images captured by the first camera 436 may be analyzed by the classification algorithm 138 to determine characteristics of the sample container 304, the robot 228, and / or other components in the sample processing machine 106, as described herein. For example, the classification algorithm 138 may classify or identify the type of sample container 304. The classification algorithm 138 may also determine whether the sample container 304 is properly gripped by the gripper 340. In addition, the classification algorithm 138 may determine whether there are any anomalies in the sample processing machine 106, as described herein. Examples of anomalies include a spilled sample ( Figure 2 ), misplaced sample containers 104, improperly closed slides 212, and other problems.

[0043] Second camera 438 may have a field of view 442 extending in the z-direction and may capture images of tray 214, sample containers 104 located in tray 214, and other objects in sample handler 106. Illumination source 444 may illuminate objects in field of view 442. In some embodiments, the spectrum and intensity of light emitted by illumination source 444 may be controlled by classification algorithm 138. In other embodiments, robot controller 136 ( Figure 1 ) is configured to control at least one of the intensity of the illumination source 444 and the spectrum of light emitted by the illumination source 444. The field of view 442 enables the capture of an image of the top of the sample container 104, such as Figure 2 The captured image can be classified by the classification algorithm 138 ( Figure 1 ) analysis to classify or identify the sample container 104 and / or determine if there are any anomalies in the sample processing machine 106. In some embodiments, the imaging device 226 may have a single camera having a field of view that can capture at least a portion of the sample processing machine 106 and at least a portion of one of the trays 214.

[0044] In operation, a medical provider may order certain tests to be performed on a sample collected from a patient. The collected sample is placed in a sample container 104. The sample container 104 may be received in a laboratory or other facility where one or more trays 214 are located outside of the sample processing machine 106. A laboratory technician (e.g., a user) places the sample container 104 into a holding position 210 of a tray 214.

[0045] Now refer to Figure 5, which is a flow chart of a method 500 of operating a robot 228 and using an imaging device 226 to capture images according to one or more embodiments. When one or more holding positions 210 in one or more trays 214 are filled with sample containers 104, the tray is placed onto one of the slides 212 and the slide is inserted into the sample processing machine 106. Processing then proceeds to tray placement detection 502, where receipt of the slide in the sample processing machine 106 is detected. In this example, the third slide 212C may have the above-mentioned tray located thereon. When the third slide 212C slides into the sample processing machine 106, the third slide sensor 220C detects the movement of the third slide 212C and sends a signal to the computer 130. The robot controller 136 ( Figure 1 ) and / or the classification algorithm 138 may receive the signal.

[0046] In response to the signal, the robot controller 136 may generate instructions that cause the robot to move to one or more positions within the sample processing machine 106 so that the imaging device 226 can capture one or more images of the newly added sample container. Therefore, in some embodiments, the robot controller 136 is configured to generate instructions to move the imaging device 226 within the sample processing machine 106 in response to the signal and capture one or more images. These instructions may cause the robot 228 to move away from the third slide 212C in the z direction so that the imaging device 226 can capture wide-angle images of multiple newly added sample containers. The captured images may be analyzed at the image analysis 504. Based on the analysis, the computer 130 may determine which of the holding positions 210 contain sample containers.

[0047] As described in greater detail herein, the robotic controller 136 may move the imaging device 226 to a particular position relative to the third slide 212C according to the image control 506. For example, the robotic controller 136 may move the robot 228 to a holding position 210 containing sample containers 104 so that the imaging device 226 may capture images of the sample containers and the classification algorithm 138 may classify or identify the sample containers 104. Thus, the robotic controller 136 may generate instructions that cause the robot 228 to move to the holding position 210 within the sample processing machine 106 in response to identifying a sample container located in the holding position 210.

[0048] Based on the image analysis 504, the image control 508 may set the lighting via the lighting 510 to capture subsequent images at the image capture 512. In some embodiments, the intensity of the lighting may be adjusted based on the lighting 510. For example, if the image is dark, the image control 508 may instruct the lighting 510 to increase the intensity during one or more subsequent image captures. The image control 508 may also instruct the lighting 510 to set a specific lighting spectrum. The subsequently captured images may be analyzed by the image analysis 504, which may generate other image control and robot control instructions.

[0049] Several other embodiments of controlling the robot 228 and the imaging device 226 are described below. Note that in some embodiments, the imaging device 226 may be moved throughout the sample processing machine 106 via a transport system that is independent of the robot 228. Thus, in these embodiments, the imaging device 226 is not secured to the robot 228. In other embodiments, the imaging device 226 may be secured to a robot (not shown) that is dedicated to moving the imaging device 226 throughout the sample processing machine 106.

[0050] In some embodiments, one or more of the trays 214 may be dedicated to sample containers that require high priority, which may be referred to as stat. For example, a tray 214 with certain markings may be dedicated to stat sample containers, such as an imageable identification mark or a specific size (e.g., a small tray in the tray 214). In other embodiments, a tray loaded into a specific slide (e.g., the fourth slide 212D) may be designated as an stat sample container. The stat sample container may be placed in an stat queue for priority sorting by the sorting algorithm 136 as described herein.

[0051] One method of characterizing sample containers 104 newly loaded into a sample processing machine 106 is referred to as opportunistic scanning, which can minimize the impact of scanning on the cycle time of the sample processing machine 106. For example, opportunistic scanning can have a minimal impact on the ability of the robot 228 to transfer sample containers 104 in and out of the sample processing machine 106. In opportunistic scanning, the laboratory system 100 can process (e.g., image) sample containers 104 using a dual-queue first-in-first-out (FIFO) scanning method, wherein each sample container in the emergency queue has a higher priority than sample containers in the normal or non-emergency queue. Therefore, a newly added sample container can be time-sensitive (e.g., urgent) only if: (1) there are no sample containers in the emergency queue and a tray containing emergency sample containers has just been loaded; or (2) no sample containers of any type (emergency or normal) have been previously loaded. The opportunistic scanning algorithm can scan newly added sample containers and / or trays only when the sample processing machine 106 has no other tasks to perform or one of conditions (1) or conditions (2) is met.

[0052] If the holding positions 210 occupied by the sample containers 104 are known, the opportunistic scanning can be further optimized. The determination of which holding positions 210 are occupied can be accomplished by using a fixed wide field of view camera mounted at a remote vantage point to perform a quick and cursory scan of the newly inserted tray, or by positioning the imaging device 226 at a high position to obtain a large field of view. Based on the field of view of the imaging device 226 and the distribution of sample containers in the tray 214, the robot controller 136 ( Figure 1 ) can determine an optimal path for directing robot 228 to image sample containers 104 or other objects. A fixed wide field of view imaging device can be implemented in one or more of slide sensors 220. When one of slide sensors 220 detects the insertion or movement of a corresponding slide in slides 212, a quick and rough scan of the newly inserted tray can be performed as described above.

[0053] Another scanning method is referred to as an improved confidence scanning algorithm and can resolve inconsistent characterizations. For example, the classification algorithm 138 can determine that one or more of the sample containers 104 or other objects (e.g., spills) are characterized incorrectly or have low classification confidence. The algorithm can plan additional scanning paths using the imaging device 226 to capture additional images of the sample containers 104 with low classification confidence, which can be determined by the classification algorithm 138. These additional images can change the illumination intensity or spectrum, such as by lighting 510 ( Figure 1 ). In addition, additional images may be captured using different positions of the robot 228 and / or the imaging device 226. In other embodiments, the scanning speed of the imaging device 226 during image capture may be changed (e.g., slowed) to improve the robustness of the sample container characterization. The algorithm may be implemented using a closed-loop system triggered by another vision system that does not agree with the sample container characterization.

[0054] Additional references Figure 6 , which is a flow chart illustrating image analysis 504 in conjunction with classification algorithm 138. Image data may be received at operation block 602, where pre-processing, such as deblurring, gamma correction, and radial distortion correction, may be performed prior to further processing. The pre-processing performed at operation block 602 may be performed in conjunction with Figure 5 The image data may be captured using one or both of the first camera 436 and the second camera 438. Thus, the image may include the sample container 104 and / or the top of the sample container 304 gripped by the gripper 340.

[0055] Processing may continue to sample container localization and classification at operation block 604, where the image of the sample container 104 may be localized and classified. Localization may include surrounding the image of the sample container or other object with a virtual box (e.g., a bounding box) to isolate the sample container 104 and other objects for classification. Classification may be performed using a data-driven machine learning-based approach, such as a convolutional neural network (CNN). The CNN may be enhanced using YOLOv4 or other image recognition networks or models.

[0056] YOLOv4 is a real-time object detection model that works by breaking the object detection task into two pieces, using regression to identify object localization through bounding boxes, and using classification to determine the class of the object. Localization provides a bounding box for each detected sample container or object. Classification determines high-level characteristics of the sample container, such as whether a sample container is present in the holding position 210 of the tray 214. In addition to the classification confidence, high-level characteristics may also include determining whether the sample container 104 is capped, uncapped, or a tube top sample cup (TTSC).

[0057] Figure 2 An example of advanced features is illustrated in FIG. As shown, some holding positions 210 are displayed with circles, triangles, squares, or blanks. Circles may represent sample containers without caps, squares may represent sample containers with caps, and triangles may represent tube-top sample cups. Holding positions without circles, squares, or triangles represent empty holding positions.

[0058] Processing may continue to sample container tracking at operation block 606, where, for each newly detected sample container, the computer 130 (e.g., the robot controller 136 or the classification algorithm 138) may assign a new tracklet identification to each sample container. Alternatively, the computer 130 may attempt to associate the detected sample container with an existing tracklet established in a previous image based on the overlap area between the detected bounding box and the predicted bounding box established on the motion trajectory, the classification confidence, and other features derived from the appearance of the image of the sample container. In situations where a detection may be missed and this prevents tracking, a more complex data association algorithm, such as the Hungarian algorithm, may be utilized to ensure the robustness of the tracking.

[0059] When a trajectory segment contains enough observations collected across multiple images (e.g., frames), the classification algorithm 138 may begin to estimate more detailed characteristics per operation block 608. These characteristics include, but are not limited to, the height and diameter of the sample container, the color of the cap, the shape of the cap, and the barcode reading results when the barcode is in the field of view of the imaging device 226. Since the sample container 104 does not change its position within the tray 214, each trajectory segment may be mapped to a virtual tray position while maintaining a relative position with respect to other trajectory segments per operation block 610. Using the positioning information and motion profile obtained by the robot controller 136 in operation block 612, each trajectory segment may be associated with its physical position in the tray 214.

[0060] In some embodiments, the processing in the sample processing machine 106 may be able to utilize the sample container characterization information and the image information to implement other operations of the sample processing machine 106. For example, the imaging device 226 is movable and can monitor the first camera 436 in the field of view 439 ( Figure 4 ) and / or each sample container in the field of view 442 of the second camera 438. In this case, when the gripper 340 ( Figure 4 ) interacts with the sample container 104, the image data can be processed by the computer 130 to verify the pick and place operation of the sample container 104.

[0061] Additional references Figure 7 , which illustrates that the robot 228 is not properly grasping the sample container 304. Figure 7 As shown in FIG. 4 , the sample container 304 is skewed relative to the gripper 340. The imaging device 226, such as the first camera 436, captures an image of the sample container 304 after being illuminated by the illumination source 440. The image data may be obtained as Figure 6 The computer 130 may process the sample container 304 in the method 600 and analyze it through the classification algorithm 138. The AI ​​and / or deep learning neural network in the classification algorithm 138 may have been trained to identify sample containers 104 that are aligned and misaligned relative to the gripper 340. Based on the analysis, the computer 130 may determine that the sample container 304 is misaligned relative to the gripper 340. The computer 130 may then notify the user of the misalignment, such as by sending a notification via the workstation 139. In some embodiments, the computer 130 may execute one or more programs, such as the robotic controller 136, to repair the misaligned sample container 304.

[0062] Additional references Figure 8 , which illustrates another example of the robot 228 mishandling the sample container 304. Figure 8In some embodiments, the gripper 340 has grasped the sample container 304 at a position that is too high. The AI ​​and / or deep learning neural network of the classification algorithm 138 may be trained to recognize conditions where the sample container 304 is too low relative to the gripper 340. In other embodiments, the AI ​​and / or network may be trained to recognize sample containers that are properly aligned relative to the gripper 340. If the sample container 304 is not found to be properly aligned, the computer 130 may assume that the improper alignment is caused by the gripper 340. A notification of the misalignment may then be sent to a user, for example, via the workstation 139.

[0063] In addition to the above, the imaging device 226 may also capture images of other items or locations in the sample processing machine 106. One or more cameras in the imaging device 226 may be configured to capture images at one or more vantage points that enable monitoring of a large portion of the sample processing machine 106. For example, the imaging device 226 may be raised in the z direction so that the second camera 438 ( Figure 4 ) can capture images of a majority of the sample processing machine 106, including, for example, a majority of the tray 214. By analyzing these images, such as by using the classification algorithm 138, the robotic controller 136 can generate instructions to direct the robot 228 and imaging device 226 to specific areas for detailed sample container characterization or incident verification and recovery when necessary.

[0064] In the event of extreme misalignment of the sample container of the gripper 340, or in the event that the gripper 340 drops a sample container, the imaging device 226 in conjunction with the classification algorithm 138 can detect these conditions. The workstation 139 can then notify the user. A dropped sample container or encountering other sample processing anomalies may spill a biohazardous liquid in the sample processor 106, on the track 114, or on one of the sample carriers 112, which may cause the biohazardous liquid to spread throughout the laboratory system 100.

[0065] Additional references Fig. 9 , which illustrates sample handler 106 with first spill 910 and second spill 912. In some embodiments, imaging device 226 may capture images of first spill 910 and second spill 912. By imaging a large area of ​​sample handler 106, imaging device 226 may capture images of suspected spills. Classification algorithm 138 may be trained to recognize spilled liquids, such as first spill 910 and second spill 912, in image data. Robot controller 136 may generate instructions to move imaging device 226 closer to the suspected spill to capture more images to verify that a spill occurred and identify the exact location of the spill. The user may then be notified of the spill.

[0066] exist Fig. 9In some embodiments, the first spill 910 is located on the tray 914. When the imaging device 226 is located above the first spill 910, the second camera 438 can capture an image of the first spill 910. The robotic controller 136 can then generate instructions to move the imaging device 226 to the vicinity of the first spill 910 so that the imaging device 226 can capture additional close-up images of the first spill 910. In some embodiments, the classification algorithm 138 can use AI, such as a model or CNN, to determine the liquid in the first spill 910. The second spill 912 is located on the carrier and may spread throughout the laboratory system 100 if the second spill 912 is unattended. The second spill 912 can be identified and / or classified using a process similar to the process used to identify the first spill 910.

[0067] In addition to the above, the imaging device 226 in conjunction with the computer 130 may be used to determine whether the slide 212 is properly closed. Fig. 9 As shown in FIG. 1 , the third slide 212C is partially open, which in conventional sample processing machines may cause the gripper 340 ( Figure 3 ) improperly gripping the sample container 104. If the third slide 212C is properly closed, the imaging device 226 can be moved to the position where the holding position 210 is expected to be located. In some embodiments, the classification algorithm 138 can identify the holding position 210 in the captured image and determine whether the holding position 210 is in a predetermined position. If the holding position 210 is not in the predetermined position, the computer 130 can determine that the third slide 212C is not properly closed. The user can be notified by means of the workstation 139 that the third slide 212C is open. In other embodiments, the computer 130 can use the position of the holding position 210 to calibrate the robot controller 136 using the actual position of the holding position 210.

[0068] In some embodiments, the classification algorithm 138 can be trained to identify a dropped sample container. A dropped sample container may be presented horizontally in the image and can be identified (e.g., classified) by the classification algorithm 138. If a horizontal sample container is identified, the computer 130 can start one or more algorithms configured to determine whether there is also spillage near the horizontal sample container. The horizontal sample container may block one or more holding positions 210 near the horizontal sample container. In response, the robot controller 136 can cause the robot 228 to bypass the horizontal sample container. The user can also be notified of the dropped sample container.

[0069] refer to Fig.10, which is a flow chart illustrating a method 1000 of operating a sample processing machine (e.g., sample processing machine 106) of a diagnostic laboratory system (e.g., laboratory system 100). The method 1000 includes providing a plurality of holding positions (e.g., holding positions 210) within the sample processing machine at process block 1102, each of the plurality of holding positions being configured to receive a sample container (e.g., sample container 104). The method 1000 includes providing a robot (e.g., robot 228) at process block 1004 having a gripper (e.g., gripper 340) configured to grip a sample container and move the sample container into and out of the plurality of holding positions. The method 1000 includes transporting an imaging device (e.g., imaging device 226) within the sample processing machine at process block 1006. The method 1000 includes capturing an image of one or more sample containers at process block 1008. The method 1000 includes, at process block 1010 , classifying an image using a classification algorithm implemented in computer code (eg, the classification algorithm 138 ) that includes a trained model configured to identify a sample container.

[0070] Although the present disclosure is susceptible to various modifications and alternative forms, specific method and device embodiments have been shown by way of example in the drawings and described in detail herein. However, it should be understood that the specific methods and devices disclosed herein are not intended to limit the present disclosure, but on the contrary, are intended to cover all modifications, equivalent forms and alternatives that fall within the scope of the claims.

Claims

1. A sample processing machine for a diagnostic laboratory system, comprising: a plurality of holding positions configured to receive sample containers; an imaging device movable within the sample processing machine, the imaging device configured to capture an image of the holding location and generate image data representative of the image; a controller configured to generate instructions that cause the imaging device to move within the sample processing machine and to cause the imaging device to capture an image; as well as A classification algorithm implemented in computer code, the classification algorithm comprising a trained model configured to classify objects in the image.

2. The sample processing machine according to claim 1, wherein: The classification algorithm is configured to identify the sample container as at least a capped, a capless, and a tube top sample cup.

3. The sample processing machine according to claim 1, wherein: The classification algorithm is configured to classify at least one of: The color of the cap of the sample container; the shape of the cap of the sample container; and Identification mark on the sample container.

4. The sample processing machine according to claim 1, wherein: The classification algorithm is configured to identify a position of a sample container relative to at least one of: Maintain position; a robotic gripper configured to move the sample container within the sample processing machine; and Sample carrier.

5. The sample processing machine of claim 1 , further comprising a sensor configured to detect movement of one or more of the holding positions and to generate a signal in response to the movement, wherein The controller is configured to generate instructions to move the imaging device within the sample processing machine and capture an image in response to the signal.

6. The sample processing machine of claim 1, further comprising an illumination source movable within the sample processing machine and configured to illuminate an object.

7. The sample processing machine according to claim 6, wherein: The controller is configured to control at least one of lighting intensity and lighting spectrum.

8. The sample processing machine according to claim 1, wherein: The controller is configured to: generating instructions that cause the imaging device to move within the sample processing machine and cause the imaging device to capture an image of a sample container; identifying the location of one or more sample containers by analyzing the captured images; as well as The imaging device is moved to the location of the one or more sample containers.

9. The sample processing machine according to claim 1, wherein: The classification algorithm is configured to identify misplaced sample containers.

10. The sample processing machine according to claim 1, wherein: The classification algorithm is configured to identify spilled liquid in the sample processing machine.

11. The sample processing machine of claim 1 , further comprising a robot configured to move within the sample processing machine, wherein: The imaging device is secured to the robot, and wherein the controller generates instructions to move the robot within the sample processing machine.

12. The sample processing machine of claim 11, further comprising a fixed camera at a fixed position within the sample processing machine, wherein: The controller is configured to: capturing an image of the holding position using the fixed camera; analyzing the image to identify the location of the holding position; and Instructions are generated that cause the robot to move to a holding position within the sample processing machine in response to identifying the position of the holding position.

13. The sample processing machine according to claim 11, wherein: The robot includes a gripper configured to grip the sample container.

14. The sample processing machine according to claim 13, wherein: The imaging device is configured to capture an image of the gripper gripping a sample container.

15. The sample processing machine according to claim 14, wherein: The classification algorithm is configured to identify one or more anomalies in the gripping of the sample container.

16. A sample processing machine for a diagnostic laboratory system, comprising: a plurality of holding positions configured to receive sample containers; a robot capable of moving within the sample processing machine, the robot comprising a gripper configured to grip the sample container to move the sample container into and out of the holding position; an imaging device secured to the robot, the imaging device configured to capture an image of the sample container and generate image data representative of the image; a controller configured to generate instructions that cause the robot to move within the sample processing machine and capture images using the imaging device; as well as A classification algorithm implemented in computer code, the classification algorithm comprising a training model configured to identify the sample container.

17. The sample processing machine according to claim 16, wherein: The imaging device is configured to capture images of spilled liquid in the sample processing machine, and wherein the classification algorithm is trained to identify the spilled liquid.

18. The sample processing machine according to claim 16, wherein: The imaging device is configured to capture images of the sample container while being gripped by the gripper, and wherein the classification algorithm is trained to identify anomalies between the gripper and the sample container.

19. The sample processing machine according to claim 16, wherein: The classification algorithm is trained to identify misplaced sample containers.

20. A method of operating a sample processor of a diagnostic laboratory system, the method comprising: providing a plurality of holding positions within the sample processing machine, each of the plurality of holding positions being configured to receive a sample container; providing a robot having a gripper configured to grip a sample container and move the sample container into and out of the plurality of holding positions; transporting an imaging device within the sample processing machine; capturing an image of one or more of the sample containers; and The image is classified using a classification algorithm implemented in computer code, the classification algorithm comprising a trained model configured to identify the sample container.

21. The method according to claim 20, wherein: The classification includes identifying a spilled liquid in the sample processing machine.

22. The method according to claim 20, wherein: The classification includes identifying an anomaly in the grip between the gripper and the sample container.

23. The method according to claim 20, wherein: The classification includes identifying misplaced sample containers.

Images