Sample Handler for a Diagnostic Laboratory Analyzer and Method of Use

Abstract

The sample handler of the diagnostic examination room system includes a plurality of holding positions configured to receive sample containers. The imaging device is movable within the sample handler and is configured to capture images of the holding positions and the sample containers received therein. The controller is configured to generate instructions to move the imaging device within the sample handler to capture images. The classification algorithm includes a trained model implemented in computer code and configured to classify objects in the captured images. Other methods of handling sample handlers and sample containers are disclosed.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit of U.S. Provisional Patent Application No. 63 / 364,911, filed May 18, 2022, entitled "SAMPLE HANDLERS OF DIAGNOSTIC LABORATORY ANALYZERS AND METHODS OF USE", the entire disclosure of which is hereby incorporated by reference herein for all purposes.

[0002] Embodiments of the present disclosure relate to sample handlers for diagnostic laboratory analyzers and methods of using the sample handlers.

Background Art

[0003] Clinical diagnostic laboratory systems process patient samples such as blood, urine, or body tissue to test for various analytes. After a sample is obtained from a patient and placed in a sample container, the sample container is delivered to a laboratory housing the diagnostic system. The laboratory system includes a sample handler that receives the sample container. After the sample container is placed in a tray, the tray is loaded into the sample handler. A robot transfers the sample container between carriers that transport the sample container between instruments and other components within the laboratory system.

Summary of the Invention

Problems to be Solved by the Invention

[0004] To accurately access and process a sample container, it is necessary to identify the position and type of the sample container in the tray so that a robot can locate and transport the sample container throughout the laboratory system. The shorter the processing time of the laboratory system, the greater the need for faster identification and location of the sample container. For this reason, there is a need for a sample handler that quickly identifies and locates a sample container and a method of handling the sample container.

Means for Solving the Problems

[0005] According to a first aspect, a sample handler for a diagnostic examination room system is provided. The sample handler includes a plurality of holding positions configured to receive sample containers; an imaging device movable within the sample handler and configured to capture an image of a holding position and generate image data representing the image; a controller configured to move the imaging device within the sample handler and generate an instruction to cause the imaging device to capture an image; and a classification algorithm including a trained model implemented in computer code and configured to classify an object in the image.

[0006] In another aspect, another sample handler for a diagnostic examination room system is provided. The sample handler includes a plurality of holding positions configured to receive sample containers; a robot including a gripping portion movable within the sample handler and configured to grip a sample container and move the sample container into and out of the holding position; an imaging device fixed to the robot and configured to capture an image of the sample container and generate image data representing the image; a controller configured to move the robot within the sample handler and generate an instruction to cause the imaging device to capture an image; and a classification algorithm including a trained model implemented in computer code and configured to identify the sample container.

[0007] In another aspect, a method of operating a sample handler for a diagnostic examination room system is provided. The method includes providing a plurality of holding positions within the sample handler, each configured to receive a sample container; providing a robot having a gripping portion configured to grip a sample container and move the sample container into and out of the plurality of holding positions; transporting an imaging device within the sample handler; capturing an image of one or more of the sample containers; and classifying the image using a classification algorithm including a trained model implemented in computer code and configured to identify the sample container.

[0008] From the following description and illustrations of numerous exemplary embodiments, including the best mode contemplated for carrying out the present disclosure, further aspects, configurations, and advantages of the present disclosure will become readily apparent. Also, the present disclosure is capable of other different embodiments and its several details can be modified in various respects without departing from the scope of the present disclosure. The present disclosure is intended to cover all improvements, equivalents, and variations included within the scope of the claims and their respective equivalents.

[0009] The drawings described below are for illustrative purposes only and are not necessarily to scale. Accordingly, the drawings and description are to be regarded as illustrative in nature and not restrictive. The drawings are not intended to limit the scope of the present disclosure in any way.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

DETAILED DESCRIPTION OF THE INVENTION

[0011] The diagnostic examination room system performs clinical chemistry and / or chemical analysis to identify analytes or other components in biological samples such as serum, plasma, urine, interstitial fluid, cerebrospinal fluid, etc. These samples are collected in sample containers and then delivered to the diagnostic examination room system. Thereafter, after the sample container is loaded into the tray, the tray is loaded into the sample handler of the examination room system.

[0012] The robot in the sample handler is configured to grip the sample container and transfer the sample container to a sample carrier that delivers the sample container to a specific location in the examination room system, such as a specific processing or analysis instrument. The robot or the controller of the robot needs to know the position of the sample container in the tray in order to grip the correct sample container. Also, the examination room system is considered to need to determine the type of sample container stored in a specific position in the tray. For example, by identification, it can be determined whether the sample container has a cap, no cap, or is a tube-top sample cup. Also, by identification, it can be determined who the manufacturer of the sample container is and whether any chemical substances used during the test are included in the sample container.

[0013] It is considered that a great deal of time is required to accurately identify a sample container. For example, some sample handlers include at least one fixed imaging device in a fixed position that captures an image of the sample container while the sample container is placed in the sample handler. These fixed cameras have a limited field of view and are considered unable to capture an image of the sample container sufficient to accurately identify the sample container. Some sample handlers have overcome some of these problems with multiple fixed cameras. However, multiple fixed cameras increase the cost of the sample handler and increase the processing resources of the sample handler.

[0014] The sample handler described herein includes an imaging device (e.g., a camera) that is movable within the sample handler. In some embodiments, the imaging device is attached to a robot that is movable within the sample handler. In some embodiments, the robot is configured to move a sample container within the sample handler. In other embodiments, there is another dedicated robot that moves the imaging device throughout the sample handler. When the imaging device moves throughout the sample handler, it can capture images of the sample container and other objects within the sample handler. These images are used for identification, localization, and / or classification of the sample container and / or other objects.

[0015] The robot can include a gripping portion configured to grip the sample container. The imaging device is fixed to the gripping portion to image the sample container. In other embodiments, the imaging device is fixed to the side of the robot and can capture an image of the sample container. In classification, in addition to identifying the sample container, it can be determined whether the robot is properly gripping the sample container. In some embodiments, the imaging device is oriented to capture images downward to capture the top or cap of the sample container. Also, with this orientation, the imaging device can capture images of spills and other objects within the sample handler. In the classification described herein, spills and other objects can be identified.

[0016] A method of handling sample containers in the above and other sample handlers and laboratory systems will be described in more detail with reference to FIGS. 1-10.

[0017] Referring now to FIG. 1, which is a block diagram of one embodiment of a diagnostic laboratory system 100. The laboratory system 100 can include a plurality of instruments 102 configured to process a sample container 104 (only a portion shown) and perform chemical analysis or tests on the sample disposed in the sample container 104. The laboratory system 100 can have a first instrument 102A and a second instrument 102B. In other embodiments of the laboratory system 100, more or fewer instruments can be included.

[0018] Examples of samples disposed in the sample container 104 can include various biological specimens collected from an individual such as a patient under evaluation by a medical professional. These samples are collected from the patient and placed directly into the sample container 104. The sample container 104 is then delivered to a laboratory or facility housing the laboratory system 100. As will be described in more detail below, the sample container 104 is loaded into a sample handler 106, which can be considered an instrument of the laboratory system 100. The sample container 104 is transferred from the sample handler 106 to a sample carrier 112 (only a portion shown), which transports the sample container 104 throughout the laboratory system 100 to the instruments 102, etc. by a track 114.

[0019] Track 114 is configured to enable the sample carrier 112 to move throughout the laboratory system 100, including in and out of the sample handler 106. For example, as shown in FIG. 1, the track 114 can extend into the vicinity or surrounding area of at least some of the instruments 102 and the sample handler 106. The instruments 102 and the sample handler 106 can have devices such as a robot (not shown in FIG. 1) that transfers the sample container 104 to and from the sample carrier 112. The track 114 can include a plurality of interconnected segments 120 (only some are shown). The carrier 112 can move along the dashed line 122 as shown in the segment 120. In some embodiments, some of the segments 120 can be integrated with one or more of the instruments 102.

[0020] Components such as the sample handler 106 and the instruments 102 of the laboratory system 100 can be provided or connected to a computer 130 configured to execute one or more programs that control the laboratory system 100, including the components of the sample handler 106. The computer 130 is configured to communicate with the instruments 102, the sample handler 106, and other components of the laboratory system 100. The computer 130 can include a processor 132 configured to execute a program including programs other than those described herein. The program is implemented in computer code.

[0021] Computer 130 can also include, or can have, access to a memory 134 that can store one or more programs and / or data described herein. The memory 134 and / or the program stored therein is referred to as a non-transitory computer-readable medium. The program can be computer code executable on or by a processor 132. The memory 134 can include a robot controller 136 configured to generate instructions for controlling robots and / or similar devices in the instrument 102 and the sample handler 106. As described herein, the instructions generated by the robot controller 136 can respond to data such as image data received from the sample handler 106.

[0022] Further, the memory 134 can store a classification algorithm 138 configured to identify and / or classify other elements in the sample container 104 and / or the sample handler 106. In some embodiments, the classification algorithm 138 classifies objects in the image data. The classification algorithm 138 can include a trained model such as one or more neural networks. For example, the classification algorithm 138 can include a convolutional neural network (CNN) trained to identify objects in the image data. The trained model is implemented using artificial intelligence (AI). Thus, the trained model can learn to classify objects. Note that the classification algorithm 138 is not a look-up table.

[0023] Computer 130 is connected to a workstation 139 configured to enable a user to collaborate with the examination room system 100. The workstation 139 can include a display 140, a keyboard 142, and other peripheral devices (not shown). Data generated by the computer 130 can be made displayable on the display 140. The data can include warnings of anomalies detected by the classification algorithm 138. Also, the user can input data into the computer 130 by means of the workstation 139. Examples of data input by the user can be instructions to cause the robot controller 136 or the classification algorithm 138 to perform specific operations.

[0024] Referring now additionally to FIG. 2, which is a top view of the interior of the sample handler 106 according to one or more embodiments. The sample handler 106 is configured to capture an image of the sample container 104 and to move the sample container 104 between a holding position 210 (only part shown) and the sample carrier 112. In the embodiment of FIG. 2, the holding position 210 is disposed within the tray 124, as will be described in more detail below. The sample handler 106 can include a plurality of slides 212 configured to hold a tray 214. In some embodiments, the sample handler 106 can include four slides 212 individually designated as a first slide 212A, a second slide 212B, a third slide 212C, and a fourth slide 212D. The third slide 212C is shown as a partial removal from the sample handler 106 that may occur during the replacement of the tray 214. Other embodiments of the sample handler 106 can include fewer slides than shown in FIG. 2 or can include more slides.

[0025] Each of the slides 212 is configured to hold one or more trays 214. In the embodiment of FIG. 2, the slide 212 can include a receiver 216 configured to receive the tray 214. Each of the trays 214 can include a plurality of holding positions 210, and each holding position 210 is configured to receive one of the sample containers 104. In the embodiment of FIG. 2, the size of the tray can vary to include a large tray having 24 holding positions 210 and a small tray having 8 holding positions 210. In other configurations of the tray, different numbers of holding positions 210 can be included.

[0026] In some embodiments, the sample handler 106 can include one or more slide sensors 220 configured to detect the movement of one or more of the slides 212. The slide sensor 220 can generate a signal indicative of the movement of the slide, where these signals can be received and / or processed by the robot controller 136 as described herein. In the embodiment of FIG. 2, the sample handler 106 includes four slide sensors 220 arranged such that each of the slides 212 is associated with one. A first slide sensor 220A detects the movement of the first slide 212A, a second slide sensor 220B detects the movement of the second slide 212B, a third slide sensor 220C detects the movement of the third slide 212C, and a fourth slide sensor 220D detects the movement of the fourth slide 212D. Various techniques for detecting the movement of the slide 212 are employed by the slide sensor 220. In some embodiments, the slide sensor 220 can include a mechanical switch that switches when the slide 212 moves. This switching generates a signal indicating that the slide has moved. In other embodiments, the slide sensor 220 can generate an optical signal in response to the movement of the slide 212. In yet other embodiments, the slide sensor 220 can be an imaging device that generates image data when the slide 212 moves.

[0027] The sample handler 106 includes an imaging device 226 that is movable throughout the sample handler 106. In the embodiment of FIG. 2, the imaging device is fixed to a robot 228 that is movable along an x-axis (e.g., x-direction) and a y-axis (e.g., y-direction) throughout the sample handler 106. In some embodiments, the imaging device 226 can be integrated with the robot 228. In some embodiments, the robot 228 is considered to be movable along a z-axis (e.g., z-direction) that enters and exits the plane of the paper. In other embodiments, the robot 228 can include one or more components (not shown in FIG. 2) that move the imaging device 226 in the z-direction. In some embodiments, the robot 228 can receive movement instructions generated by a robot controller 136 (FIG. 1). These instructions can include data indicating the x and y positions to which the robot 228 should move. In other embodiments, these instructions can include electrical signals that move the robot 228 in the x-direction and the y-direction. The robot controller 136 can generate instructions to move the robot 228 in response to one or more of slide sensors 220 that detect movement of one or more of slides 212. These instructions can move the robot 228 while the imaging device 226 is capturing an image of a newly added sample container.

[0028] The imaging device 226 includes one or more cameras that capture images, where capturing an image generates image data representing the image. The image data is transmitted to the computer 130 and processed by the classification algorithm 138 as described herein. The one or more cameras are configured to capture images of the sample container 104 and / or other locations or objects in the sample handler 106. Images may include the top and / or sides of the sample container 104. In some embodiments, the robot 228 may be a gripping robot that grips the sample container 104 and moves the sample container 104 between the holding position 210 and the sample carrier 112. As described herein, an image is captured while the robot 228 is gripping the sample container 104.

[0029] Referring additionally to FIG. 3, which is a perspective view of one embodiment of the robot 228 coupled to a gantry 330 configured to move the robot 228 in the x, y, and z directions. The gantry 330 can include two y-slides 332 that enable movement of the robot 228 in the y direction, an x-slide 334 that enables movement of the robot 228 in the x direction, and a z-slide 336 that enables movement of the robot 228 in the z direction. Movement in the three directions can be performed simultaneously and is controlled by the robot controller 136. For example, the robot controller 136 can generate commands to move the slides to a motor (not shown) coupled to the gantry 330 to move the robot 228 and the imaging device 226 to a predetermined position.

[0030] Robot 228 can include a gripping portion 340 (e.g., an end effector) configured to grip the sample container 304. As an example of the sample container 304, the sample container 104 can be considered. After the robot 228 moves to a position above the holding position, it moves in the z direction to take out the sample container 304 from the holding position. When the gripping portion 340 opens and the robot 228 moves downward in the z direction, the gripping portion 340 extends above the sample container 304. When the gripping portion 340 closes to grip the sample container 304, the robot 228 moves upward in the z direction to extract the sample container 304 from the holding position. As shown in FIG. 3, the imaging device 226 is fixed to the robot 228. The imaging device 226 includes at least one camera configured to capture an image, where the captured image is converted into image data for processing by a classification algorithm 138 or the like.

[0031] Referring additionally to FIG. 4, which is a side view of one embodiment of the robot 228 gripping the sample container 340 with the gripping portion 340 while the sample container 304 is being imaged by the imaging device 226. The imaging device 226 shown in FIG. 4 can include a first camera 436 and a second camera 438. In other embodiments of the imaging device 226, it can include a single camera or three or more cameras. The first camera 436 has a field of view 439 that extends at least partially in the y direction and is configured to capture an image of the sample container (e.g., the sample container 304) being gripped by the gripping portion 340. A lighting source 440 can illuminate an object in the field of view 439. In some embodiments, the spectrum and intensity of the light emitted by the lighting source 440 are controlled by the classification algorithm 138. In other embodiments, the robot controller 136 (FIG. 1) is configured to control at least one of the intensity of the lighting source 440 and the spectrum of the light emitted by the lighting source 440.

[0032] As described herein, the image captured by the first camera 436 is analyzed by a classification algorithm 138 to determine the characteristics of the sample container 304, the robot 228, and / or other components of the sample handler 106. For example, the classification algorithm 138 can classify or identify the type of the sample container 304. Also, the classification algorithm 138 can determine whether the sample container 304 is properly gripped by the gripping portion 340. Also, as described herein, the classification algorithm 138 can also determine whether there is an abnormality in the sample handler 106. Examples of abnormalities include a sample overflowing from one of the sample containers 104 (FIG. 2), a sample container 104 in the wrong position, a slide 212 that is accidentally closed, and other problems.

[0033] The second camera 438 can have a field of view 442 extending in the z direction and can capture images of the tray 214, the sample containers 104 disposed on the tray 214, and other objects in the sample handler 106. The illumination source 444 can illuminate the objects in the field of view 442. In some embodiments, the spectrum and intensity of the light emitted by the illumination source 444 are controlled by the classification algorithm 138. In other embodiments, the robot controller 136 (FIG. 1) is configured to control at least one of the intensity of the illumination source 444 and the spectrum of the light emitted by the illumination source 444. The field of view 442 enables the capture of an image of the upper portion of the sample container 104 as shown in FIG. 2. The captured image is analyzed by the classification algorithm 138 (FIG. 1) for classification or identification of the sample container 104 and / or determination of whether there is an abnormality in the sample handler 106. In some embodiments, the imaging device 226 can have a single camera having a field of view that can capture at least a portion of at least one of the sample handler 106 and the tray 214.

[0034] During operation, a healthcare provider can instruct the execution of a specific test on a sample collected from a patient. The collected sample is placed in a sample container 104. The sample container 104 is received at a facility such as a laboratory where one or more of the trays 214 are located outside the sample handler 106. Then, a laboratory technician (e.g., a user) places the sample container 104 in the holding position 210 of the tray 214.

[0035] Referring additionally to FIG. 5 here, this is a flowchart of a method 500 for operating a robot 228 and capturing images using an imaging device 226 according to one or more embodiments. If one or more of the holding positions 210 in one or more of the trays 214 are filled with the sample container 104, the tray is placed on one of the slides 212 and the slide is inserted into the sample handler 106. Thereafter, the process proceeds to tray placement detection 502 where the reception of the slide in the sample handler 106 is detected. In this example, the above-described tray can be placed on the third slide 212C. When the third slide 212C slides into the sample handler 106, the third slide sensor 220C detects the movement of the third slide 212C and sends a signal to the computer 130. The robot controller 136 (FIG. 1) and / or the classification algorithm 138 can receive this signal.

[0036] In response to the signal, the robot controller 136 can generate commands to move the robot to one or more positions within the sample handler 106 so that the imaging device 226 can capture images of one or more of the newly added sample containers. Thus, in some embodiments, the robot controller 136 is configured to generate commands to move the imaging device 226 within the sample handler 106 to capture one or more images in response to the signal. These commands can move the robot 228 in the z direction away from the third slide 212C so that the imaging device 226 can capture wide-angle images of the newly added plurality of sample containers. The captured images are analyzed in the image analysis 504. Based on this analysis, the computer 130 can determine the holding positions 210 that include the sample containers.

[0037] As will be described in more detail herein, the robot controller 136 can move the imaging device 226 to a specific position relative to the third slide 212C in the image control 506. For example, the robot controller 136 can move the robot 228 to the holding positions 210 that include these sample containers so that the imaging device 226 can capture an image of the sample container 104 and the classification algorithm 138 can classify or identify the sample container 104. Thus, the robot controller 136 can generate commands to move the robot 228 within the sample handler 106 to the holding position 210 in response to the identification that the sample container has been placed in the holding position 210.

[0038] Based on the image analysis 504, in the image control 508, subsequent images can be captured in the image capture 512 by setting the illumination with the illumination 510. In some embodiments, the intensity of the illumination is adjusted in the illumination 510. For example, if the image is dark, in the image control 508, the illumination 510 can be instructed to increase the intensity during one or more subsequent image captures. Also, in the image control 508, the illumination 510 can be instructed to set a specific spectrum of the illumination. The subsequently captured images are analyzed by the image analysis 504, thereby generating other image control and robot control commands.

[0039] A plurality of other embodiments for controlling the robot 228 and the imaging device 226 will be described below. Note that in some embodiments, the imaging device 226 can be moved throughout the sample handler 106 by a transport system independent of the robot 228. Thus, in these embodiments, the imaging device 226 is not fixed to the robot 228. In other embodiments, the imaging device 226 is fixed to a dedicated robot (not shown) that moves the imaging device 226 throughout the sample handler 106.

[0040] In some embodiments, one or more of the trays 214 can be dedicated to sample containers that require a high priority, which can be referred to as "stat". For example, a tray 214 (e.g., a small one of the trays 214) having specific designated content such as an imageable identification mark or a specific size can be dedicated to the stat sample container. In other embodiments, a tray loaded on a specific slide, such as the fourth slide 212D, is designated as the stat sample container. As described herein, the stat sample container is placed in the stat queue for priority classification by the classification algorithm 136.

[0041] One of the methods for characterizing the sample container 104 newly loaded into the sample handler 106 is called opportunistic scanning, which can minimize the impact of scanning on the cycle time of the sample handler 106. For example, opportunistic scanning can minimize the impact on the ability of the robot 228 to transfer the sample container 104 to or from the sample handler 106. In opportunistic scanning, the laboratory system 100 can use a dual-queue first-in first-out (FIFO) approach for scanning to process (e.g., image) the sample container 104. In this case, all sample containers in the stat queue have priority over sample containers in the normal, i.e., non-stat queue. Therefore, newly added sample containers are considered to be time-constrained (e.g., stat) only if: (1) there are no sample containers in the stat queue and only trays containing stat sample containers are loaded, or (2) no sample containers of any kind (stat or normal) have been loaded in the past. In the opportunistic scanning algorithm, the newly added sample containers and / or trays can be scanned only if the sample handler 106 has no other tasks to perform or if one of conditions (1) or (2) is met.

[0042] The daylight scan is further optimized when the holding position 210 occupied by the sample container 104 is known. Determining the occupied holding position 210 is achieved by using a stationary wide - field camera attached at a distant and visible location, coarsely scanning the newly inserted tray at high speed, or placing the imaging device 226 at a high position to obtain a large field of view. Depending on the field of view of the imaging device 226 and the distribution of sample containers in the tray 214, the robot controller 136 (FIG. 1) can guide the robot 228 to determine an optimal path to image the sample container 104 or other objects. The stationary wide - field imaging device is implemented in one or more of the slide sensors 220. The high - speed and coarse scanning of the newly inserted tray is performed as described above upon detection of the insertion or movement of each slide 212 by one of the slide sensors 220.

[0043] Another scanning method, called an improved reliability scanning algorithm, can eliminate inconsistent characteristic evaluations. For example, the classification algorithm 138 can determine that one or more of the characteristic evaluations of the sample container 104 or other objects (e.g., spillage) are incorrect or that the classification reliability is low. This algorithm can schedule an additional scanning path by the imaging device 226 to capture additional images of the sample container 104 with low classification reliability determined by the classification algorithm 138. In these additional images, the intensity or spectrum of the illumination can be changed by the illumination 510 (FIG. 1) etc. Also, these additional images are captured using different positions of the robot 228 and / or the imaging device 226. In other embodiments, to improve the robustness of the characteristic evaluation of the sample container, the scanning speed of the imaging device 226 during image capture is changed (e.g., reduced). This algorithm is implemented by a closed - loop system triggered by another vision system that does not match the characteristic evaluation of the sample container.

[0044] Refer additionally to FIG. 6, which is a flowchart showing image analysis 504 in conjunction with classification algorithm 138. Image data can be received at operation block 602, where additional processing is performed after preprocessing such as sharpening, gamma correction, and radial distortion correction. The preprocessing executed at operation block 602 is carried out in cooperation with, or using, the algorithms in image analysis 504 of FIG. 5. The image data is captured using one or both of the first camera 436 and the second camera 438. For this reason, these images can include the sample container 104 and / or the upper part of the sample container 304 being gripped by the gripping portion 340.

[0045] The process proceeds to the positioning and classification of the sample container at operation block 604, where positioning and classification can be performed for the image of the sample container 104. Positioning can include surrounding the image of the sample container or other object with a virtual box (e.g., a bounding box) to isolate and classify the sample container 104 and other objects. Classification is performed using a data-driven machine learning-based approach such as a convolutional neural network (CNN). The CNN is enhanced using YOLOv4 or other image identification networks or models.

[0046] YOLOv4 is a real-time object detection model that operates by dividing the object detection task into two parts, identifying the placement of the object via a bounding box using regression, and determining the class of the object using classification. In positioning, a bounding box is provided for each detected sample container or object. In classification, high-level characteristics of the sample container are determined, such as whether the sample container is present at the holding position 210 of the tray 214. Also, the high-level characteristics can include, in addition to the classification confidence, a determination of whether the sample container 104 has a cap, no cap, or is a tube-top sample cup (TTSC).

[0047] An example of high-level characteristics is shown in FIG. 2. As shown, some of the holding positions 210 are displayed as circular, triangular, square, or empty. The circle can represent a sample container without a cap, the square can represent a sample container with a cap, and the triangle can represent a tube-top sample cup. A holding position that is neither circular, square, nor triangular represents an empty holding position.

[0048] The process proceeds to tracking the sample containers in operation block 606, where, for each newly detected sample container, a computer 130 (e.g., a robot controller 136 or a classification algorithm 138) can assign a new tracklet identifier to each sample container. Alternatively, the computer 130 can attempt to associate the detected sample container with an existing tracklet established in a past image based on the overlap area between the detected bounding box and a predicted bounding box established based on the motion trajectory, classification confidence, and other configurations derived from the appearance of the image of the sample container. In situations where detections may be missed and tracking may be disrupted, a more sophisticated data association algorithm, such as the Hungarian algorithm, is utilized to ensure the robustness of the tracking.

[0049] If a tracklet contains sufficient observations collected over multiple images (e.g., frames), the classification algorithm 138 can start estimating more detailed characteristics in operation block 608. These characteristics include, but are not limited to, the height and diameter of the sample container, the color of the cap, the shape of the cap, and the reading of the barcode if the barcode is present in the field of view of the imaging device 226. Since the sample containers 104 do not change their respective positions within the tray 214, each tracklet is mapped to a virtual tray position in operation block 610 while maintaining its relative position with respect to other tracklets. In operation block 612, each tracklet is associated with a physical position in the tray 214 based on the placement information and motion profile obtained by the robot controller 136.

[0050] In some embodiments, in the processing in the sample handler 106, other operations of the sample handler 106 can be executed using the sample container characteristic evaluation information and the image information. For example, the imaging device 226 is movable and can monitor each sample container present in the field of view 439 (FIG. 4) of the first camera 436 and / or the field of view 442 of the second camera 438. In such a situation, in order to confirm the lifting and placement operations of the sample container 104 when the gripping portion 340 (FIG. 4) interacts with the sample container 104, the image data is processed by the computer 130.

[0051] Referring additionally to FIG. 7, this shows a state where the robot 228 is inappropriately gripping the sample container 304. As shown in FIG. 7, the sample container 304 is tilted with respect to the gripping portion 340. The imaging device 226 such as the first camera 436 captures an image of the sample container 304 after being illuminated by the illumination source 440. The image data is processed by the method 600 of FIG. 6 and analyzed by the classification algorithm 138. The AI and / or deep learning neural network of the classification algorithm 138 is trained to recognize the sample container 104 aligned with the gripping portion 340 and the misaligned sample container 104. Based on the analysis, the computer 130 can determine that the sample container 304 is misaligned with respect to the gripping portion 340. Thereafter, the computer 130 can notify the user of the misalignment, for example, by sending a notification via the workstation 139. In some embodiments, the computer 130 can execute one or more programs such as the robot controller 136 to correct the misaligned sample container 304.

[0052] With additional reference to FIG. 8, which shows another example where the robot 228 mishandles the sample container 304. In the embodiment of FIG. 8, the gripping portion 340 holds the sample container 304 at a position that is too high. The AI and / or deep learning neural network of the classification algorithm 138 is trained to recognize situations where the sample container 304 is too low relative to the gripping portion 340. In other embodiments, the AI and / or network is trained to recognize a sample container that is properly aligned with the gripping portion 340. If it is found that the sample container 304 is not properly aligned, the computer 130 can assume that the cause of the misalignment lies with the gripping portion 340. Thereafter, a misalignment notification is sent to the user via, for example, the workstation 139 or the like.

[0053] In addition to the above, the imaging device 226 can capture images of other elements or positions within the sample handler 106. One or more of the cameras in the imaging device 226 are configured to capture images at one or more visible locations that enable monitoring of a large portion of the sample handler 106. For example, the imaging device 226 is lifted high in the z-direction so that a second camera 438 (FIG. 4) can capture an image of a large portion of the sample handler 106, including a large portion of the tray 214. By analyzing the images using, for example, the classification algorithm 138, the robot controller 136 can generate commands to guide the robot 228 and the imaging device 226 to specific areas for detailed characterization of the sample container or for confirmation and dissemination of incidents as needed.

[0054] If the displacement of the sample container by the gripping portion 340 is extreme or the gripping portion 340 drops the sample container, the imaging device 226 can cooperate with the classification algorithm 138 to detect these situations. Then, the workstation 139 can notify the user. If the sample container drops or there are abnormalities in other sample handling, it is conceivable that a biohazardous liquid overflows in the sample handler 106, on the track 114, or on one of the sample carriers 112, and the biohazardous liquid spreads throughout the laboratory system 100.

[0055] Referring additionally to FIG. 9, which shows a sample handler 106 including a first spill 910 and a second spill 912. In some embodiments, the imaging device 226 can capture images of the first spill 910 and the second spill 912. By imaging a large area of the sample handler 106, the imaging device 226 can capture images suspected of having a spill. The classification algorithm 138 is trained to identify spilled liquids in the image data, such as the first spill 910 and the second spill 912. The robot controller 136 can move the imaging device 226 near the suspected spill area and capture more images to confirm that a spill has occurred and generate an instruction to identify the exact location of the spill. Then, the user is informed of the spill.

[0056] In the embodiment of FIG. 9, the first spill 910 is on the tray 914. When the imaging device 226 is disposed above the first spill 910, the second camera 438 can capture an image of the first spill 910. Thereafter, the robot controller 136 can generate an instruction to move the imaging device 226 near the first spill 910 so that the imaging device 226 can capture an additional enlarged image of the first spill 910. In some embodiments, the classification algorithm 138 can use AI such as a model or CNN to determine the liquid in the first spill 910. Since the second spill 912 is on the carrier, if left unattended, it may spread throughout the inspection chamber system 100. The second spill 912 is identified and / or classified by a process similar to the process used for the identification of the first spill 910.

[0057] In addition to the above, the imaging device 226 is used in cooperation with the computer 130 to determine whether the slide 212 is properly closed. As shown in FIG. 9, although the third slide 212C is partially open, in a conventional sample handler, it is considered that the gripping portion 340 (FIG. 3) causes improper gripping of the sample container 104. The imaging device 226 can move to the position of the holding position 210 expected when the third slide 212C is properly closed. In some embodiments, the classification algorithm 138 can identify the holding position 210 in the captured image and determine whether the holding position 210 is at a predetermined position. If the holding position 210 is not at the predetermined position, the computer 130 can determine that the third slide 212C is not properly closed. The user is informed by the workstation 139 that the third slide 212C is open. In other embodiments, the computer 130 can use the position of the holding position 210 to calibrate the robot controller 136 according to the actual position of the holding position 210.

[0058] In some embodiments, the classification algorithm 138 is trained to identify a dropped sample container. Since the dropped sample container appears horizontal in the image, it is thus identified (e.g., classified) as such by the classification algorithm 138. When a horizontal sample container is identified, the computer 130 can initiate one or more algorithms configured to determine whether there is also an overflow in the vicinity of the horizontal sample container. The horizontal sample container is considered to block access to one or more of the holding positions 210 in the vicinity of the horizontal sample container. In response, the robot controller 136 can pivot the robot 228 around the horizontal sample container. Also, the user is informed of the dropping of the sample container.

[0059] Referring to FIG. 10, this is a flowchart showing a method 1000 of operating a sample handler (e.g., sample handler 106) of a diagnostic examination room system (e.g., examination room system 100). The method 1000 includes, in method block 1002, providing a plurality of holding positions (e.g., holding positions 210) within the sample handler, each configured to receive a sample container (e.g., sample container 104). The method 1000 includes, in method block 1004, providing a robot (e.g., robot 228) having a gripping portion (e.g., gripping portion 340) configured to grip the sample container and move the sample container in and out relative to the plurality of holding positions. The method 1000 includes, in method block 1006, transporting an imaging device (e.g., imaging device 226) within the sample handler. The method 1000 includes, in method block 1008, capturing an image of one or more of the sample containers. The method 1000 includes, in method block 1010, classifying the image using a classification algorithm (e.g., classification algorithm 138) including a trained model implemented in computer code and configured to identify the sample container.

[0060] The present disclosure is subject to various improvements and alternative forms, and as an example, specific embodiments of methods and apparatuses are shown in the drawings and described in detail herein. However, it is understood that the specific methods and apparatuses disclosed herein are not intended to limit the present disclosure, but on the contrary, are intended to cover all improvements, equivalents, and modifications included within the scope of the claims.

Claims

1. A sample handler for a diagnostic examination room system, comprising: a plurality of holding positions configured to receive sample containers; an imaging device movable within the sample handler and configured to capture an image of a holding position and generate image data representing the image; a controller configured to move the imaging device within the sample handler and generate an instruction to cause the imaging device to capture an image; a classification algorithm implemented in computer code and including a trained model configured to classify an object in the image; the sample handler including the above.

2. The sample handler according to claim 1, wherein the classification algorithm is configured to identify that the sample container is at least one of having a cap, not having a cap, and a tube-top sample cup.

3. The classification algorithm is:[[]] the color of the cap of the sample container; the shape of the cap of the sample container; the identification mark on the sample container, and is configured to classify at least one of them. The sample handler according to claim 1.

4. The classification algorithm is:[[]] the holding position; the gripping part of a robot configured to move the sample container in the sample handler; the sample carrier, and is configured to identify the position of the sample container relative to at least one of them. The sample handler according to claim 1.

5. The sample handler according to claim 1, further comprising a sensor configured to detect movement of one or more of the holding positions and generate a signal in response to the movement, wherein the controller is configured to move the imaging device within the sample handler and generate an instruction to capture an image in response to the signal.

6. The sample handler according to claim 1, further comprising an illumination source movable within the sample handler and configured to illuminate an object.

7. The sample handler according to claim 6, wherein the controller is configured to control at least one of the intensity of the illumination and the spectrum of the illumination.

8. The controller:[[]] moves the imaging device within the sample handler and generates an instruction to cause the imaging device to capture an image of the sample container; identifies the position of one or more sample containers by analyzing the captured image; moves the imaging device to the position of one or more sample containers The sample handler according to claim 1, which is configured to perform

9. The sample handler according to claim 1, wherein the classification algorithm is configured to identify a sample container at a wrong position.

10. The sample handler according to claim 1, wherein the classification algorithm is configured to identify a liquid overflowed in the sample handler.

11. The sample handler according to claim 1, further comprising a robot configured to move within the sample handler, wherein the imaging device is fixed to the robot, and wherein the controller generates an instruction to move the robot within the sample handler.

12. The sample handler according to claim 11, further comprising a camera fixed at a fixed position within the sample handler, wherein the controller: uses the fixed camera to capture an image of the holding position; analyzes the image to identify the position of the holding position; generates an instruction to move the robot within the sample handler to the holding position in response to the identification of the position of the holding position.

13. The sample handler according to claim 11, wherein the robot includes a gripping portion configured to grip a sample container.

14. The sample handler according to claim 13, wherein the imaging device is configured to capture an image of a gripping portion that grips a sample container.

15. The sample handler according to claim 14, wherein the classification algorithm is configured to identify one or more abnormalities in the gripping of a sample container.

16. A sample handler of a diagnostic examination room system, comprising: a plurality of holding positions configured to receive sample containers; a robot that is movable within the sample handler and includes a gripping portion configured to grip a sample container and move the sample container into and out of the holding position; an imaging device fixed to the robot and configured to capture an image of the sample container and generate image data representing the image; a controller configured to generate an instruction to move the robot within the sample handler and cause the imaging device to capture an image; a classification algorithm implemented in computer code and including a trained model configured to identify a sample container;

17. The imaging device is configured to capture an image of the liquid overflowed in the sample handler, where the classification algorithm is trained to identify the overflowed liquid, the sample handler according to claim 16.

18. The imaging device is configured to capture an image of the sample container while being held by the gripping part, where the classification algorithm is trained to identify an abnormality between the gripping part and the sample container, the sample handler according to claim 16.

19. The classification algorithm is trained to identify a sample container at the wrong position, the sample handler according to claim 16.

20. A method for operating a sample handler of a diagnostic examination room system, comprising: providing a plurality of holding positions in the sample handler, each configured to receive a sample container; providing a robot having a gripping part configured to grip the sample container and move the sample container in and out with respect to the plurality of holding positions; transporting the imaging device within the sample handler; capturing an image of one or more of the sample containers; classifying the image using a classification algorithm including a trained model implemented in computer code and configured to identify the sample container; the method as described above.

21. Classifying includes identifying the liquid overflowed in the sample handler, the method according to claim 20.

22. Classifying includes identifying an abnormality in the gripping between the gripping part and the sample container, the method according to claim 20.

23. Classifying includes identifying a sample container at the wrong position, the method according to claim 20.

Images