Apparatus and method for training sample container identification network in diagnostic laboratory system

Abstract

A method of training a sample container recognition network of a diagnostic laboratory system includes obtaining a plurality of data subsets, where each data subset is smaller than a complete training data set for training the sample container recognition network and includes a plurality of images of one or more sample containers. A sample container identification network is trained on each of the plurality of data subsets to generate a plurality of trained sample container identification networks. Each trained sample container identification network is tested using test data including a test image of a sample container, wherein the testing includes identifying a sample container in the test image. A core dataset is selected from one of the plurality of data subsets based on the testing, the core dataset being used to train the deployed sample container identification network. Other methods and systems are disclosed.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 63 / 374,889, filed on September 7, 2022, entitled “DEVICES AND METHODS FOR TRAINING SAMPLE CONTAINER IDENTIFICATION NETWORKS IN DIAGNOSTIC LABORATORY SYSTEMS,” the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] Embodiments of the present disclosure relate to apparatus and methods for training a sample container recognition network in a diagnostic laboratory system. Background Art

[0003] Diagnostic laboratory systems perform clinical chemistry or assays to identify and / or quantify analytes or other components in biological samples, such as serum, plasma, urine, interstitial fluid, cerebrospinal fluid, etc. Samples may be received and / or transported through such laboratory systems in sample containers. Such laboratory systems may process a large number of sample containers and the samples contained therein.

[0004] Some laboratory systems use machine vision and machine learning to facilitate sample processing and sample container identification, which can be based on characterization and / or classification of sample containers. For example, vision-based machine learning models (e.g., artificial intelligence (AI) networks) have been adapted to provide a fast and non-invasive method for sample container identification. However, the training cost of adding new types of sample containers to the machine learning model may be prohibitive because a large amount of training data may be required to retrain or adapt the machine learning model to recognize the new type of sample container.

[0005] Therefore, there is a need for laboratory systems and methods for improving the training of machine vision systems in diagnostic laboratory systems. Summary of the invention

[0006] According to a first aspect, a method for training a sample container recognition network of a diagnostic laboratory system is provided. The method includes obtaining a plurality of data subsets, wherein each data subset is smaller than a complete training data set for training the sample container recognition network and includes a plurality of images of one or more sample containers, training a sample container recognition network on each of the plurality of data subsets to generate a plurality of trained sample container recognition networks, testing each of the trained sample container recognition networks using test data including test images of the sample containers, wherein the testing includes recognizing the sample containers in the test images, and selecting a core data set from one of the plurality of data subsets based on the testing, the core data set being used to train a deployed sample container recognition network.

[0007] According to another aspect, a method for retraining a deployed sample container recognition network of a diagnostic laboratory system is provided. The method includes capturing a raw image of a sample container using an imaging device within the diagnostic laboratory system, attempting to identify the sample container using a deployed sample container recognition network to analyze the raw image, the deployed sample container recognition network being trained on a full training data set, allowing the raw image to be added to a core data set if the deployed sample container recognition network fails to identify the sample container, and retraining the deployed sample container recognition network using the core data set, wherein the core data set is smaller than the full training data set.

[0008] According to another aspect, a diagnostic laboratory system is provided. The diagnostic laboratory system includes a track, a sample carrier movable on the track and configured to receive a sample container including a sample, an imaging device configured to capture an image of the sample container, a memory including a sample container recognition network, the sample container recognition network trained on a complete training data set, a computer coupled to the imaging device and the memory, and computer program code, when executed by the computer, causes the computer to: use the imaging device to capture an image of a sample container within the diagnostic laboratory system, attempt to identify the sample container by analyzing the captured image using the sample container recognition network, if the sample container recognition network fails to identify the sample container, add the captured image to a core data set, wherein the core data set is smaller than the complete training data set, and allow the sample container recognition network to be retrained using the core data set.

[0009] Other aspects, features and advantages of the present invention will become apparent from the following description and illustration of multiple exemplary embodiments, including the best mode of implementing the present invention. The present disclosure may also have other and different embodiments, and its several details may be modified in various aspects, all of which do not depart from the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described below are provided for illustrative purposes and are not necessarily drawn to scale. Therefore, the drawings and descriptions are to be regarded as illustrative rather than restrictive in nature. The drawings are not intended to limit the scope of the present disclosure in any way.

[0011] Figure 1 A block diagram of a diagnostic laboratory system including a sample handler is shown in accordance with one or more embodiments.

[0012] Figure 2 A top plan view of the interior of a sample handler of a diagnostic laboratory system is shown in accordance with one or more embodiments.

[0013] Figures 3A-3C Different types of sample containers that may be used within a diagnostic laboratory system are shown, including a test tube with an attached cap, in accordance with one or more embodiments.

[0014] Figures 4A-4C Different types of sample containers without caps that may be used within a diagnostic laboratory system are shown in accordance with one or more embodiments.

[0015] Figure 5 A perspective view of a robot of a sample handler of a diagnostic laboratory system including a gantry configured to move the sample handler and attached imaging device along x, y, and z axes is shown in accordance with one or more embodiments.

[0016] Figure 6 According to one or more embodiments, Figure 5 A side elevation view of a robot with an imaging device configured and operated to capture an image of a sample container.

[0017] Figures 7A-7E Shown are original and enhanced images of sample containers for training a sample container recognition network of a diagnostic laboratory system in accordance with one or more embodiments.

[0018] Figures 8A-8E Another example of original and enhanced images of sample containers for training a sample container recognition network of a diagnostic laboratory system is shown in accordance with one or more embodiments.

[0019] Fig. 9 A workflow is shown that illustrates contrastive learning for training a sample container recognition network for a diagnostic laboratory system in accordance with one or more embodiments.

[0020] Fig.10 A training network according to one or more embodiments is shown. The training network can be Figure 1 The sample container recognition network is implemented in accordance with Fig. 9Workflow for training a sample container recognition network.

[0021] Fig.11 A diagram depicting a method configured to select a core data set from a plurality of data subsets to train a sample container recognition network of a diagnostic laboratory system in accordance with one or more embodiments is shown.

[0022] Fig.12 A workflow of a method for determining whether to add an image of a sample container to a core dataset configured to train a sample container recognition network of a diagnostic laboratory system according to one or more embodiments is shown.

[0023] Fig.13 A flow chart is shown of a method of training a sample container recognition network of a diagnostic laboratory system in accordance with one or more embodiments.

[0024] Fig.14 A flow chart illustrating a method of retraining a deployed sample container recognition network of a diagnostic laboratory system in accordance with one or more embodiments. DETAILED DESCRIPTION

[0025] As described herein, the diagnostic laboratory system performs clinical chemistry and / or assays to identify analytes or other components in biological samples (hereinafter referred to as "samples"), such as serum, plasma, urine, interstitial fluid, cerebrospinal fluid, etc. The sample is collected in a sample container and then transported to the diagnostic laboratory system. The sample container is then loaded into a sample handler of the diagnostic laboratory system and then transferred to a sample carrier by a suitable robot. The sample carrier transports the sample container to the instruments, analyzers and components of the diagnostic laboratory system, where the sample is processed and / or analyzed.

[0026] The diagnostic laboratory system described herein uses a vision system that captures an image of a sample container and / or contents (e.g., a sample) contained in the sample container. The captured image is then used to identify the sample container and / or the contents of the sample container. For example, the diagnostic laboratory system may include a vision-based artificial intelligence (AI) model and / or network that is configured to provide a fast and non-invasive method for sample container identification.

[0027] The diagnostic laboratory system may include a sample handler, which may be an entrance for sample containers to enter the diagnostic laboratory system. Many diagnostic laboratory systems include machine vision located in or operating with the sample handler for identifying sample container characteristics, such as geometry, cap condition, tube color, cap color, and other container and / or cap characteristics. Based on these characteristics, the trained sample container recognition network identifies the sample container type. Other instruments within the diagnostic laboratory system may also capture images of the sample container, and the sample container recognition network may identify the sample container type after identifying the sample container characteristics.

[0028] The sample container recognition network can be trained on images of sample containers captured under controlled settings (such as under ideal lighting conditions). However, images captured under these controlled settings do not capture all changes in the appearance of the sample containers that may occur when the images are actually used in a sample handler or other instrument / analyzer within a diagnostic laboratory system. For example, the appearance of a sample container taken out of a refrigerator is different from the appearance of the sample container when it has been stored at room temperature for a period of time. In addition, the appearance of the sample container may change due to handling and transportation. These changes may include slight dents or indentations, slight changes in the color of the lid, and changes in the appearance of the label. The sample container recognition network may not be trained to recognize sample containers based on these real-world appearances that do not exist under the imaging conditions used to initially train the sample container recognition network.

[0029] In addition to the foregoing, the sample container recognition network in the diagnostic laboratory system may not be trained to recognize newly added sample container types. As new sample container types are introduced into the diagnostic laboratory system, the sample container recognition network employed must be updated or "retrained" to be able to recognize the new sample container types. Retraining the AI ​​network in a conventional diagnostic laboratory system is expensive and time-consuming because a very large number of different (new) sample container types need to be imaged and manually annotated in order to retrain the AI ​​network.

[0030] Embodiments of the systems and methods described herein overcome the problem of retraining a sample container recognition network. A diagnostic laboratory system is equipped with a deployed trained sample container recognition network that is trained using an initial dataset of images of sample containers that may become outdated over time. Specifically, the deployed recognition network is retrained using a core dataset that includes images of sample containers or a dataset of sample container types that reflect the current conditions of sample containers in the diagnostic laboratory system. The retrained recognition network is then able to recognize sample containers under the current conditions encountered in the diagnostic laboratory system, such as new sample container types that have never been encountered before.

[0031] The core data set may have sufficient variation in images so that when the sample container recognition network is trained or retrained using the core data set, the recognition network can recognize the sample container with good confidence (e.g., high confidence). In some embodiments, the core data set is smaller than the initial data set used to train the deployed sample container recognition network, but has sufficient variation so that the retrained recognition network can recognize the sample container under the current conditions present in the laboratory system. For example, the core data set may have at most half the number of images of sample containers or sample container types that can be recognized by the deployed sample container recognition network.

[0032] In some embodiments, new sample container images can be added to the core data set based on defined heuristics. For example, if a sample container fails to be recognized by the recognition network, it can be determined whether the sample container image should be added to the core data set. If the sample container image is added to the core data set, the recognition network is retrained on the revised core data set, which enables the recognition network to recognize previously unrecognizable sample containers. In some embodiments, the core data set may include images of sample containers captured by imaging devices from multiple different diagnostic laboratory systems or multiple instruments in a single diagnostic laboratory system. Therefore, the revised core data set may not be site-specific.

[0033] Reference below Figure 1-14 These and other systems and methods are described in further detail.

[0034] Reference now Figure 1 , which shows a block diagram of an example embodiment of a diagnostic laboratory system 100. The laboratory system 100 may include a plurality of instruments 102 configured to process sample containers 104 (some labeled) and perform assays or tests on samples located in the sample containers 104. The diagnostic laboratory system 100 may have a first instrument 102A and a second instrument 102B. Other embodiments of the laboratory system 100 may include more or fewer instruments.

[0035] The sample located in the sample container 104 can be a variety of biological samples (e.g., specimens) collected from individuals (such as patients evaluated by medical professionals). The sample can be collected from the patient and placed directly into the sample container 104. The sample container 104 can then be transported to the diagnostic laboratory system 100. The sample container 104 can be loaded into a sample handler 106, which can be an instrument or component of the diagnostic laboratory system 100. The sample container 104 can be transported from the sample handler 106 to a sample carrier 112 (some are labeled), which transports the sample container 104 throughout the diagnostic laboratory system 100, such as to the instrument 102, via a track 114. The track 114 is configured to enable the sample carrier 112 to move throughout the diagnostic laboratory system 100, including to and from the sample handler 106.

[0036] Components such as the sample handler 106 and the instrument 102 of the diagnostic laboratory system 100 may include or be coupled to a computer 130 configured to execute one or more programs that control the diagnostic laboratory system 100. The computer 130 may be configured to communicate with the instrument 102, the sample handler 106, and other components of the diagnostic laboratory system 100. The computer 130 may include a processor 132 configured to execute programs, including programs other than those described herein. The programs may be implemented in computer program code.

[0037] The computer 130 may include or have access to a memory 134, which may store one or more programs and / or data sets described herein. The memory 134, the data sets, and the programs stored therein may be referred to as non-transitory computer-readable media. The program may be a computer program code that may be executed on or by the processor 132. The memory 134 may store a core data set 136, which may be a set of images (e.g., image data) representing sample containers for training or retraining the sample container recognition network 138. As described herein, the core data set 136 may be revised or updated when certain conditions are met. The memory 134 may also include one or more data subsets that include sample container images that may or may not be included in the core data set 136. As described herein, one or more data subsets may be selected as the core data set 136.

[0038] The memory 134 may store a sample container recognition network 138 (sometimes referred to herein simply as recognition network 138) configured to recognize the sample container 104. The recognition network 138 may be implemented as computer code executable on the processor 132 and may include an AI model, such as one or more neural networks. The recognition network 138 has a first state of a deployed sample container recognition network 138A, or simply deployed recognition network 138A, and a retrained sample container recognition network 138B, or simply retrained recognition network 138B.

[0039] The deployed recognition network 138A is the state of the recognition network 138 that initially exists or is deployed in the diagnostic laboratory system 100. The deployed recognition network 138A is trained on a complete training data set of images (e.g., a data set that may be large and difficult to use as a retraining source due to, for example, its size, particularly within a recognition network), which may or may not be stored in the memory 134. The deployed recognition network 138A is retrained using data in the core data set 136 to produce a retrained recognition network 138B. In some embodiments, the recognition network 138 may be repeatedly retrained as the core data set 136 is repeatedly updated.

[0040] In some embodiments, the recognition network 138 may include a convolutional neural network (CNN) trained to recognize the sample container 104 by analyzing image data representing the sample container 104. As described herein, the recognition network 138 is implemented using artificial intelligence (AI) that is configured to recognize different types and / or configurations of sample containers 104. The recognition network 138 is not a lookup table, but rather a supervised or unsupervised model or network trained to recognize various types and / or configurations of sample containers 104.

[0041] The recognition network 138 recognizes the at least one imaging device ( Figure 1 Not shown; see e.g. Figure 2 In some embodiments, the image of the sample container 104 may be captured by an imaging device 214 (e.g., an imaging device 214) of the sample container 104. In some embodiments, during imaging, there may be relative motion between the imaging device and the sample container 104. Thus, the image may be a video image or video data. The image may be captured within the sample handler 106, the instrument 102, or within other areas of the diagnostic laboratory system 100. In some embodiments, a robot located at one or more of the instrument 102 and / or the sample handler 106 may be configured to move the imaging device 214 relative to the sample container 104 to capture an image of the sample container 104. In addition, the robot may be configured to move the sample container 104 relative to the imaging device 214 to capture an image, wherein the imaging device 214 may be disposed in a fixed position.

[0042] The imaging controller 140 may be implemented within the computer 130. The imaging controller 140 may be a computer program code stored in the memory 134 and executed by the processor 132. The imaging controller 140 may be configured to control the imaging device ( Figure 1 Not shown; see e.g. Figure 2 Imaging device 214) and illumination source ( Figure 1 Not shown; see, for example, Figure 6 The imaging controller 140 may generate instructions to control the settings of the imaging device (e.g., a camera), such as setting a predetermined frame rate and exposure time during imaging. The imaging controller 140 may also generate instructions that set the illumination intensity and one or more spectrums that illuminate the sample container 104 during imaging. In some embodiments, the imaging controller 140 may also enable changing the image color (s) of the captured image, and may enable changing the brightness of the captured image. In other embodiments, the imaging controller 140 may enable cropping of the captured image.

[0043] The computer 130 may be coupled to a workstation 142 that is configured to enable a user to interact with the diagnostic laboratory system 100. The workstation 142 may include a display 144, a keyboard 146, and other peripherals (not shown). Data generated by the computer 130 may be displayed on the display 144. In some embodiments, the data may include alerts of anomalies detected by the recognition network 138. Anomalies may include notification that something in the sample container 104 cannot be recognized.

[0044] The user may input data into the computer 130 via the workstation 142. The data input by the user may be instructions that cause the core data set 136, the recognition network 138, or the imaging controller 140 to perform certain operations, such as capturing and / or analyzing images of the sample container 104 and retraining the recognition network 138. Other data input by the user may be decisions regarding whether certain captured images of the sample container 104 may be added to the core data set 136. The user may also use the workstation 142 to manually enhance images of the sample container and select a viewpoint for images captured by the imaging device.

[0045] Now refer to Figure 2, which shows a top plan view of the interior of a sample handler 106 according to one or more embodiments. The sample handler 106 is a component of the diagnostic laboratory system 100 that receives a sample container 104. An imaging device within the sample handler 106 can be configured to capture an image of the sample container 104. A robot within the sample handler 106 is configured to transfer the sample container 104 between holding locations 200 (some are labeled) and a sample carrier 112 on a track 114.

[0046] exist Figure 2 In some embodiments, the holding position 200 can be a receptacle located within a tray 202, and the tray 202 can be removable from the sample handler 106. The sample handler 106 can be configured to include a plurality of slides 204 that hold the tray 202. In some embodiments, the sample handler 106 can include four slides 204, which are referred to as a first slide 204A, a second slide 204B, a third slide 204C, and a fourth slide 204D. The third slide 204C is shown as being partially removed (e.g., slid out) from the sample handler 106, which may occur during replacement of the tray 202. Other embodiments of the sample handler 106 may include more than one slide 204. Figure 2 Sliders for less or more as shown.

[0047] Each slide 204 can be configured to hold one or more trays 202. Figure 2 In the embodiment of the present invention, the slide 204 can include a receiver 208 configured to receive the tray 202. The receiver 208 can take any form (e.g., a pocket or groove) that allows the corresponding tray 202 to be substantially fixed in an XY position relative to the slide 204 that receives it. Each tray 202 can include a plurality of holding positions 200, wherein each holding position 200 can be configured to receive one of the sample containers 104. Figure 2 In the embodiment of the present invention, for example, the tray 202 can vary in size and can include a large tray having twenty-four holding positions 200 and a small tray having eight holding positions 200. Other configurations of the tray 202 can include different numbers of holding positions 200 and holding positions configured to hold more than one sample container.

[0048] In some embodiments, the sample handler 106 may include one or more slider sensors 210 configured to sense movement of one or more sliders 204. The slider sensors 210 may generate signals indicative of movement of the corresponding sliders 204, wherein the signals may be received and / or processed by the computer 130 as described herein. Figure 2In an embodiment of the present invention, the sample handler 106 includes four slider sensors 210, which are arranged so that each slider 204 is associated with one of the slider sensors 210. The first slider sensor 210A senses the movement of the first slider 204A, the second slider sensor 210B senses the movement of the second slider 204B, the third slider sensor 210C senses the movement of the third slider 204C, and the fourth slider sensor 210D senses the movement of the fourth slider 204D. The slider sensors 210 can use various technologies to sense the movement of the sliders 204. In some embodiments, the slider sensor 210 can include a mechanical switch that is triggered (toggle) when the slider 204 is moved, wherein the triggering generates a signal indicating that the slider has moved. The slider sensor 210 can be configured to determine whether the slider is slid out (open) or slid in (closed).

[0049] In some embodiments, the slider sensor 210 may be an imaging device that generates image data representing a top view of the sample container 104. For example, the slider sensor 210 may generate image data as the sample container 104 is moved (slid) into the sample handler 106. Thus, the image data may be video data captured as the slider 204 moves relative to the slider sensor 210. The image data may be processed by the recognition network 138 to identify individual sample containers in the sample container 104. Additionally, the image data may be added to the core data set 136 as described herein.

[0050] The sample handler 106 can receive many different types of sample containers 104. A first type of sample container 104 is marked with a triangle, a second type of sample container 104 is marked with a square, a third type of sample container 104 is marked with a circle, and a fourth type of sample container is marked with a cross. Some of the plurality of holding positions 200 may be empty. The identification network 138 is configured to identify the sample container 104 so that the sample container 104 can be easily identified by the computer 130 ( Figure 1 ) Identification. As described herein, the identification network 138 can also identify new types of sample containers 104.

[0051] Now refer to Figures 3A-3C , which shows different types of example sample containers 104 that may be present within the diagnostic laboratory system 100. Other types of sample containers 104 may exist. In some embodiments, the sample container 104 may include a test tube with or without a cap attached to the test tube. The sample container 104 may also include a sample or other contents (e.g., a liquid) located in the sample container. Figures 4A-4C , which shows the Figures 3A-3CAs shown, all sample containers may have different configurations or geometries. For example, the lids and tubes of different sample container types may each have different structural or color features, such as different tube and lid geometries and / or colors. As described herein, the unique features of sample container 104 may be identified by recognition network 138 ( Figure 1 ) identification.

[0052] Figure 3A The first sample container 104A shown in FIG. 1 includes a cover 300 that is white with a red stripe and has a smaller extended vertical portion than a base that is coupled to a test tube 302. The cover 300 can be mounted over or in the test tube 302. The first sample container 104A has a height H31. Figure 4A A test tube 302 is shown without a cap 300. Test tube 302 has a test tube geometry including a height H41 and a width W41. Test tube 302 may also have features such as test tube color, test tube material, and / or test tube surface properties (e.g., reflectivity). These dimensions, size ratios, and other material or color properties may be referred to as features and may be used by recognition network 138 to identify first sample container 104A.

[0053] Figure 3B The second sample container 104B shown in FIG. 1 includes a lid 306 that is blue with a dome-shaped top and can fit over or in a test tube 308. The second sample container 104B has a height H32. Figure 4B The test tube 308 is shown without the cap 306. The test tube 308 can have a test tube geometry including a height H42 and a width W42. The test tube 308 can also have a test tube color, a test tube material, and / or test tube surface properties. These dimensions, dimension ratios, and other properties can be referred to as features and can be used by the recognition network 138 to identify the second sample container 104B.

[0054] Figure 3C The third sample container 104C is shown to include a lid 310 that is red and gray with a flat top and can fit over or in a test tube 312. The third sample container 104C has a height H33. Figure 4C The test tube 312 is shown without the cap 310. The test tube 312 may also have a test tube geometry including a height H43 and a width W43. The test tube 312 may have a test tube color, a test tube material, and / or test tube surface properties. These dimensions, dimension ratios, and other properties may be referred to as features and may be used by the recognition network 138 to identify the third sample container 104C.

[0055] Test tube 302 may have identifying indicia thereon in the form of bar code 314. Likewise, test tube 312 may have identifying indicia thereon in the form of bar code 316. Identification network 138 may analyze the images of bar code 314 and bar code 316 to help identify first sample container 104A and third sample container 104C, or any other sample container 104 having a bar code thereon.

[0056] Different types of sample containers 104 ( Figure 1 ) may have different characteristics, such as different sizes, different surface properties, different covers and / or different chemical additives therein, e.g. Figures 3A-3C For example, some sample container types are chemically active, meaning that the sample container 104 may contain one or more added chemicals that are used to change or maintain the state of the sample stored therein, or otherwise assist the instrument 102 ( Figure 1 ) for sample processing. In some embodiments, the inner wall of the test tube can be coated with one or more additives, or the additives can be disposed elsewhere in the sample container 104. In some embodiments, the type of additive contained in the test tube can be a serum separator, a coagulant such as thrombin, an anticoagulant such as EDTA or sodium citrate, an anti-glucose additive, or other additives for changing or maintaining one or more characteristics of the sample. For example, the sample container manufacturer can associate the color of the cap on the test tube and / or the shape of the test tube or cap with a specific type of chemical additive contained in the sample container 104.

[0057] Different manufacturers may have their own standards for associating the attributes of sample containers 104, such as lid color, lid shape (e.g., lid geometry), and test tube shape, with specific properties of the sample container. For example, the attributes may be associated with the contents of the sample container 104, or may be associated with whether the sample container 104 is provided with vacuum capability. In some embodiments, the manufacturer may associate all sample containers 104 with gray lids with test tubes including potassium oxalate and sodium fluoride configured to test glucose and lactate. Sample containers with green lids may contain heparin for stabilizing electrolytes such as sodium, potassium, chloride, and bicarbonate. Sample containers with lavender lids may identify test tubes containing EDTA (ethylenediaminetetraacetic acid-an anticoagulant) configured to detect CBC with differential, HgBA1c, and parathyroid hormone. Other lid colors such as red, yellow, light blue, royal blue, pink, orange, and black may be used to represent other additives or no additives. In other embodiments, a combination of cap colors may be used, such as yellow and lavender to indicate a combination of EDTA and a separator gel, or green and yellow to indicate lithium heparin and a separator gel.

[0058] Since the sample container 104 ( Figure 1 ) may be chemically active, so it is important to associate a specific test that can be performed on a sample with a specific sample container type. Thus, the diagnostic laboratory system 100 can be used to identify the sample container using the identification network 138 ( Figure 1 ) Identify the type of sample container 104 to confirm that the test performed on the sample in the sample container 104 is correct.

[0059] Reference again Figure 2 In some embodiments, the sample handler 106 may include an imaging device 214 that may be movable relative to and / or throughout the sample handler 106. Figure 2 In some embodiments, the imaging device 214 can be fixed to the robot 216, which can move along the x-axis (e.g., in the x-direction) and the y-axis (e.g., in the y-direction) relative to the sample handler 106. In some embodiments, the imaging device 214 can be integrated with the robot 216. In one or more embodiments, the robot 216 can additionally be movable along the z-axis (e.g., in the z-direction), which enters and exits the page. The robot 216 can be attached to the sample handler 106 or coupled to another structure located near the sample handler 106.

[0060] The robot 216 may receive the image from the imaging controller 140 ( Figure 1 ) to generate movement instructions. The instructions may be data indicating the x, y, and z positions to which the robot 216 should move. In other embodiments, the instructions may be electrical signals that cause the robot 216 to move in the x-direction, the y-direction, and the z-direction. The imaging controller 140 may also generate instructions to move the robot 216 in response to one or more slider sensors 210 detecting movement of one or more sliders 204. For example, upon detecting movement of one of the sliders 204, the robot 216 may move to grip a sample container or move the imaging device 214 close to one of the trays 202 or one or more of the sample containers 104. In some embodiments, the instructions may cause the robot 216 to move while the imaging device 214 captures an image of the sample container 104.

[0061] Imaging device 214 may include one or more cameras ( Figure 2 Not shown; see, for example, Figure 6A first camera 600 is provided, wherein capturing an image generates image data representing the image. A camera as used herein is any imaging device capable of capturing an image (e.g., a digital image) that can be analyzed, such as a digital camera, a digital sensor, such as a charge coupled device (CCD), a complementary metal oxide semiconductor (CMOS) sensor, a metal oxide semiconductor (MOS) sensor, an electron multiplying charge coupled device (EMCCD), etc.

[0062] Image data may be transmitted to computer 130 ( Figure 1 ) for processing by recognition network 138, as described herein. Imaging device 214 is configured to capture images of sample container 104 and / or other locations or objects in sample handler 106. For example, the image may be of the top and / or side of sample container 104. In some embodiments, for example, robot 216 may be a gripper-type robot that includes a gripper (e.g., Figure 5 The robot 216 may be used to hold the sample container 104 and to transport the sample container 104 between the holding position 200 and the sample carrier 112, or vice versa. In some embodiments, images may be captured while the robot 216 is holding the sample container 104. Movement of the imaging device 214 enables image capture of the sample container 104 from a first viewpoint, followed by image capture of the sample container from one or more different viewpoints. The one or more different viewpoints may be enhancements of an image as described herein. An enhancement of an image is a subsequent image that is captured under different imaging conditions or otherwise modified compared to the original image. In some embodiments, a user may input instructions to the workstation 142 that cause the imaging device 214 to capture an image from a specific viewpoint.

[0063] Additional references Figure 5 , Figure 5 5 is a perspective view of an embodiment of a robot 216 including a gantry 500 configured to move a gripper 510 of the robot 216 in the x-direction, the y-direction, and the z-direction. The gantry 500 may include two y-slides 502 that enable movement in the y-direction, an x-slide 504 that enables movement in the x-direction, and a z-slide 506 that enables movement in the z-direction. In some embodiments, movement in the three directions may be simultaneous and may be controlled by the imaging controller 140 ( Figure 1 ) generated by the imaging controller 140. For example, the imaging controller 140 may generate instructions that cause a motor (not shown) coupled to the stage 500 to move the slider to move the gripper 510 and the imaging device 214 to one or more predetermined positions or in one or more predetermined directions.

[0064] In some embodiments, the gripper 510 (eg, an end effector) can be configured to grip the sample container 104 ( Figure 2 ). The sample container 104 is shown as being held by a holder 510. The sample container 104 may be, for example Figures 3A-4C The gripper 510 moves to a position above the holding position 200 and then moves in the z direction to retrieve the sample container 104 from the holding position 200. The gripper 510 opens, and the robot 216 moves downward in the z direction so that the gripper 510 extends above the sample container 104. The gripper 510 closes to grip the sample container 104, and the robot 216 moves upward in the z direction to extract the sample container 104 from the holding position 200.

[0065] like Figure 5 As shown, the imaging device 214 can be fixed to a portion of the robot 216 so that the imaging device 214 can move with the robot 216 and capture the sample container 104 and other sample containers 104 located in the sample handler 106 ( Figure 2 ). The imaging device 214 includes at least one camera configured to capture images, where the captured images are converted into image data for processing, such as by the recognition network 138. As described herein, the image data can also be used to update the core data set 136 ( Figure 1 ).

[0066] Additional references Figure 6 , Figure 6 FIG. 2 is a side elevational view of an embodiment of the robot 216 holding the sample container 104D with the gripper 510 while the sample container 104 is being imaged by the imaging device 214. Figure 6 The imaging device 214 shown may include a first camera 600 and a second camera 602. Other embodiments of the imaging device 214 may include a single camera or more than two cameras. For example, additional cameras may be provided and aimed in the x-direction or y-direction opposite to the first camera 600. The first camera 600 has a field of view 606 extending at least partially in the y-direction and may be configured to capture an image of a sample container 104 held by the holder 510. The first illumination source 610 may illuminate the sample container 104 in the field of view 606 through an illumination field 612. In some embodiments, the imaging controller 140 may be configured to control at least one of the intensity of the light emitted by the first illumination source 610 and one or more desired spectra of the light emitted by the first illumination source 610.

[0067] The second camera 602 may have a field of view 616 extending in the z-direction and may capture the tray 202 ( Figure 2 ), the sample container 104 located in the tray 202 ( Figure 2 ) and other objects in the sample handler 106. The second illumination source 618 may illuminate objects in the field of view 616 via the illumination field 620. In some embodiments, one or more spectra and / or intensities of light emitted by the second illumination source 618 may be controlled by the imaging controller 140 ( Figure 1 )control.

[0068] Field of view 606 and field of view 616 enable images of the top (e.g., lid) and / or sides of sample container 104 to be captured. For example, when sample container 104 is in one of holding positions 200 ( Figure 2 ), the top of the sample container 104 can be captured. The captured image can be recognized by the network 138 ( Figure 1 ) analysis to identify the sample container 104. In some embodiments, the imaging device 214 can have a single camera with a field of view that can capture at least a portion of the sample handler 106 and one or more holding locations 200 with or without a sample container 104 located therein. The field of view 606 enables images of the top (e.g., lid) and / or sides of the gripper 510 gripping the sample container 104 to be captured. These images can also be used to update the core data set 136, as described herein.

[0069] Reference again Figure 2 and Figure 5 , the sample handler 106 may include one or more fixed imaging devices 220. Figure 2 and 5 In an embodiment of the present invention, the sample handler 106 includes a fixed imaging device 220. The fixed imaging device 220 can capture an image of a sample container 104 located at a holding position 200 or a sample container 104 held by a gripper 510 of the robot 216. For example, the robot 216 can move the sample container 104 within the field of view of the fixed imaging device 220 so that the fixed imaging device 220 can capture an image of the sample container 104.

[0070] Fixed imaging device 220 may include camera 514 and illumination source 516. Camera 514 may be similar to first camera 600 ( Figure 6 ) and the second camera 602 ( Figure 6 ), and operates in a similar manner. The illumination source 516 may be similar to the first illumination source 610 ( Figure 6 ) and a second illumination source 618 ( Figure 6 ), and operate in a similar manner. Thus, the camera 514 and the illumination source 516 may be operated by the imaging controller 140, and the images captured by the camera 514 may be used to identify the sample container 104 and / or update the core data set 136.

[0071] Reference again Figure 1 , the diagnostic laboratory system 100 may include other cameras and lighting sources. All cameras and lighting sources may be controlled by an imaging controller 140. The imaging controller 140 may set one or more imaging conditions for these devices during imaging as described herein. For example, the imaging controller 140 may generate instructions to set exposure time, frame rate, lighting intensity, and / or one or more lighting spectra during image capture. In some embodiments, the recognition network 138 may determine the imaging conditions. The image data generated by the camera may represent a video and / or a still image.

[0072] Imaging conditions may vary from one image to another, which is referred to as an enhanced image. The first image may be referred to as an original image, and subsequent images captured under different imaging conditions may be referred to as enhanced images. Many different imaging conditions may be used to enhance the original image, such as changing the lighting conditions, which may include changing the brightness of the illumination of the sample container and / or one or more spectra of the illumination during imaging. Enhancement may also include changing the image quality or using a different imaging device relative to the original image. In some embodiments, an enhanced image may include capturing an image of the sample container from a different viewpoint than that used to capture the original image. However, in other embodiments, an enhanced image may include cropping the image relative to the original image. In some embodiments, a user of the diagnostic laboratory system 100 may set imaging conditions for enhancement.

[0073] Reference again Figure 1 , the operation of the diagnostic laboratory system 100 and the method of updating the core data set 136 and retraining the recognition network 138 into the retrained recognition network 138B will now be described. During the operation of the diagnostic laboratory system 100, a medical professional can order tests to be performed on biological samples collected from one or more patients. A technician collects the sample and places the sample in a sample container 104. The sample container 104 containing the sample is then transported to the diagnostic laboratory system 100. Electronic instructions (e.g., computer code) detailing the test to be performed on the sample can be transmitted from the medical professional or hospital information system (HIS) to the diagnostic laboratory system 100, such as transmitted to the computer 130 via an intranet system.

[0074] See also Figure 2, a laboratory user receives a sample container 104 and loads the sample container 104 into a holding position 200 in a tray 202. The tray 202 is then placed on a slide 204, and the slide 204 slides into the sample handler 106. As the tray 202 slides into the sample handler 106, the slide sensor 210 can capture an image of the sample container 104. The computer 130 can use the captured image to identify the holding position 200 occupied by the sample container. The recognition network 138 can also use the captured image to identify the sample container 104 and / or retrain the recognition network 138, as described herein. For example, the image can be added to a core data set 136, which is used to retrain the recognition network 138 into a retrained recognition network 138B.

[0075] After the sample container 104 is received in the sample handler 106, other images of the sample container 104 may be captured. Figure 2 ) can employ a robot 216 to move the imaging device 214 to predetermined positions in the sample handler 106. When the imaging device 214 is in these predetermined positions, the imaging controller 140 can generate instructions that cause the second illumination source 618 ( Figure 6 ) emits light having a predetermined frequency and / or intensity. The imaging controller 140 may also generate instructions that cause the second camera 602 to capture an image under predetermined imaging conditions. The predetermined imaging conditions may include lighting conditions and / or camera settings, such as exposure time, etc.

[0076] A first camera 600 ( Figure 6 ) can capture images of the sample container 104 while the sample container 104 is gripped by the robot 216. For example, the robot 216 can move to a predetermined position and remove a particular sample container from one of the holding positions 200. The imaging controller 140 can then generate instructions as described herein that cause the first illumination source 610 to illuminate and the first camera 600 to capture images of the sample container under one or more predetermined imaging conditions.

[0077] Including fixed imaging device 220 ( Figure 2 ) can capture an image of the sample container 104 when the sample container 104 is in the field of view of the fixed imaging device 220. The imaging controller 140 can generate instructions to communicate with the second illumination source 618 ( Figure 6 ) and the second camera 602 ( Figure 6) and camera 514 are operated in a similar manner. In some embodiments, fixed imaging device 220 may be located within sample handler 106 and may be configured to capture images of the tops of sample containers 104. In other embodiments, robot 216 may transport certain of sample containers 104 into the field of view of fixed imaging device 220 to capture images of sample containers 104.

[0078] Some embodiments of the diagnostic laboratory system 100 may include imaging devices in other instruments and locations. For example, one or more instruments 102 may include one or more imaging devices, which may be controlled by an imaging controller 140, as described herein. Image data generated by these imaging devices may be used by a recognition network 138 to identify sample containers 104. The image data may also be raw images and enhanced images, and may be used to update a core data set 136, as described herein.

[0079] The images used to train the deployed recognition network 138A may have been captured in a controlled setting outside the diagnostic laboratory system 100. It is generally not possible to capture all possible changes in the appearance of the sample container 104 in these controlled settings. For example, the appearance of the sample container 104 may change between when the sample container 104 is removed from refrigeration and when the sample container 104 has been at ambient temperature for a period of time. The appearance of the sample container 104 may also change during the handling and transportation of the sample container 104. For example, the sample container 104 may receive slight dents and indentations, the color of the lid may change due to frost or moisture, and the identification markings may change. When the deployed recognition network 138A is trained on images captured in a controlled setting, the recognition network 138 may not be able to recognize sample containers 104 with such changes.

[0080] The diagnostic laboratory system 100 described herein is configured to scale (e.g., update or retrain) the recognition network 138 to be able to recognize new sample container types that the recognition network 138 cannot recognize and container changes of known (e.g., previously recognized) sample container types. The updating of the recognition network 138 can be performed using self-supervised learning or a combination of self-supervised and supervised learning. The updating also includes updating the core data set 136 using one or more sample container images. Updating the core data set 136 can also be performed by self-supervised learning or a combination of self-supervised and supervised learning. The recognition network 138 is then retrained to the state of the retrained recognition network 138B using the updated core data set 136.

[0081] The core dataset 136 is a dataset of images of sample container types that are sufficiently varied so that when the recognition network 138 is retrained using the core dataset 136, the retrained recognition network 138B operates similarly to or better than the deployed recognition network 138A. The deployed recognition network 138A may have been trained on more data (e.g., images) than in the core dataset 136. Thus, the core dataset 136 may be smaller than the training dataset used to train the deployed recognition network 138A or a previous version of the recognition network 138. As described herein, when the recognition network 138 fails to recognize a particular sample container 104, the images of the sample container 104 that could not be recognized may be used to update the core dataset 136, as described herein.

[0082] In some embodiments, a determination may be made as to whether a newly acquired image of a sample container 104 that cannot be identified is to be added to the core data set 136. For example, an image of a sample container 104 that cannot be identified may be displayed on a display 144. The user may then decide whether the image should be used to update the core data set 136 and input the decision into the workstation 142. For example, images of sample containers 104 that are not continuous or that may not be used frequently may not be added to the core data set 136 in order to keep the core data set 136 small.

[0083] In some embodiments, the sample container 104 that cannot be recognized may have been previously recognized under different imaging conditions. The failure to recognize the sample container may be due to the imaging conditions during the imaging of the sample container 104 used to train the deployed recognition network 138A being different from the imaging conditions within the sample handler 106. In some embodiments, the different imaging conditions may be different lighting conditions. For example, the image used to train the deployed recognition network 138A may have been captured at a different brightness than the current brightness in the sample handler 106. The difference in lighting conditions may be caused by the illumination source (e.g., the second illumination source 618- Figure 2 ) is caused by emitting one or more spectra or intensities of light that are different from one or more spectra or intensities of light used to capture an image of the sample container 104 used to train the deployed recognition network 138A.

[0084] In some embodiments, the quality of the images captured in the sample handler 106 may differ from the quality of the images used to train the deployed recognition network 138A. The difference in image quality may prevent the recognition network 138 from recognizing the sample container 104. For example, the camera and / or lighting source in the sample handler 106 may have become dirty, which changes the image quality relative to the images used to train the deployed recognition network 138A. In other embodiments, the characteristics of the camera and / or lighting equipment may age, which may change the image quality.

[0085] In order to simulate different conditions that may exist within the diagnostic laboratory system 100 and make the recognition network 138 more accurate, enhanced images of unrecognized sample containers 104 may be used to update the core data set 136. Figures 7A-7E and 8A-8E, which illustrate training images of sample containers that may be used in the core dataset 136 to retrain the recognition network 138.

[0086] Figures 7A-7E 700, which may be a first type of sample container 104 ( Figure 1 ). Figures 8A-8E is a top view of a second sample container 800 , which may be a second type of sample container 104 . Fig. 7A is the original image 702A of the first sample container 700, and Fig. 8A is an original image 802A of the second sample container 800 . Figure 7B-7E and Figures 8B-8E The views φ and φ are additional images, which are enhanced images of the original image 702A and the original image 802A, respectively. Figure 7B-7E and Figures 8B-8E The view of is generated by enhancements such as color jittering, changing image brightness, zooming, changing lighting color, cropping, reorienting the viewpoint, and other changes relative to the original image 702A and the original image 802A. In some embodiments, random Figure 7B-7E and Figures 8B-8E The enhanced image and the original image can be used for comparative learning to retrain the recognition network 138, as described herein.

[0087] In some embodiments, image 702B may be enhanced by changes in brightness and color relative to original image 702A. In other cases, image 702C may be enhanced by changes in imaging angle (e.g., viewpoint) or pose and lighting color or spectrum relative to original image 702A. Image 702D is enhanced by a change in imaging angle and is also cropped and enlarged relative to original image 702A. Image 702E is enhanced by a change in color and is cropped and enlarged relative to original image 702A.

[0088] Image 802B is enhanced by a change in color relative to the original image 802A. Image 802C is enhanced by a change in color and blur (image quality), and is cropped and enlarged relative to the original image 802A. Image 802D is enhanced by a change in color and viewpoint, and is cropped and enlarged relative to the original image 802A. Image 802E is enhanced by a change in imaging angle, blur, and color, and is cropped and enlarged relative to the original image 802A.

[0089] In some embodiments, the recognition network 138 can be trained or retrained into a retrained recognition network 138B using a combination of a classification loss function and a contrast loss function using an enhanced image. The goal of the classification loss function is to find the appropriate division of images into groups representing the correct sample container type classification. The classification loss function can be performed by minimizing the entropy between the output of the recognition network 138 and the target category, which has the side effect of clustering objects from the same category together. The target category is a class of similar images. In contrastive learning, a network is created that embeds data into a vector space. A loss function is used that attempts to map similar images to similar vectors and dissimilar images to dissimilar vectors. Once trained, the retrained network has learned how to embed images into a vector space that encodes information about image similarity. The trained network can be trained for other tasks in a shorter time and / or with less data.

[0090] As described herein, the contrastive loss network attracts similar images and repels dissimilar images. There are different contrastive loss networks or models that can perform the functions of attracting similar images and repelling dissimilar images. The operation of repelling dissimilar images is optional in some networks. The attraction / repulsion function can be optimized by the recognition network 138 through loss, which means that the way the images are attracted / repelled is loss-dependent. In some embodiments, the attraction / repulsion can be performed based on a triplet loss, which minimizes the distance between an anchor image and a positive image, which have the same identity. The triplet loss can also maximize the distance between an anchor image and a negative image with different identities. In the triplet loss, the anchor image is the original image, such as the original unenhanced image. The positive image is close to (e.g., similar to) the anchor image, and the negative image is far away from (e.g., not similar to) the anchor image. The triplet loss forces dissimilar image pairs to be at least a certain margin value (loss value L) away from any similar image pairs, and can be defined by equation (1): L = max(d(a, p) - d(a, n) + m, 0) Equation (1) in: a - anchor image, p - a positive image with the same label as the anchor image a (a label can be a vector of sample containers to be identified), n - negative images with different labels from the anchor image a, d - a function that measures the distance between three images (anchor image a, positive image p and negative image n), m - margin value to keep negative images away from each other.

[0091] In some embodiments, the contrastive loss can be calculated by InfoNCE loss (Information Noise Contrastive Estimation), which can be referred to as NT-Xent (Normalized Temperature Scaled Cross Entropy Loss). (For example, see Grill et al., "Bootstrap Your Own Latent A New Approach to Self-Supervised Learning", Arxiv, arXiv:2006.07733, September 10, 2020, https: / / arxiv.org / abs / 2006.07733). Applying the InfoNCE loss can involve randomly sampling a batch of N images and defining a contrastive prediction task on augmented image pairs derived from the batch, which results in 2N data points. Examples of augmented images include Figure 7B-7E and Figures 8B-8E In some embodiments, a contrastive loss function can be defined for the contrastive prediction task. (See, for example, Chen et al., “A Simple Framework for Contrastive Learning of Visual Representation,” Arxiv, arXiv:2002.05709, July 1, 2020, https: / / arxiv.org / abs / 2002.05709?context=stat.ML). For example, given a pair of positive images and Image collection Compare the prediction functions for a given Identification In Negative images may not be explicitly sampled. Conversely, given a positive image pair such as and The other 2(N-1) enhanced images in a batch are considered as negative images. This method sets sim(u,v)=u T v / ||u||||v||, which represents the loss function Normalized dot product between u and v (i.e., cosine similarity). Based on the above, the loss function of the positive image pair (Xi, Xj) It is defined by equation (2) as follows: in: is an indicator function that evaluates to 1 if and only if k≠i, and τ represents a temperature parameter; for example, the final loss can be calculated on all positive pairs (i, j) and (j, i) in a batch; (z) is a vector representation of the images Xi and Xj after being processed by the recognition network 138. A suitable temperature parameter can help the model learn from difficult negative values. In addition, the optimal temperature is different for different batch sizes and numbers of training epochs. Based on the loss function, the recognition network 138 can be trained to recognize images that are closely adjacent to similar images.

[0092] During the training phase, the cosine similarity between all images in a given batch can be calculated. In some embodiments, similar pairs of images consist of different enhancements of the original image, and the negative images are other images in the batch. The similarity between similar images is maximized with respect to noise, where the noise is dissimilar images. This process can be equivalent to maximizing the mutual information (MI) between similar images while minimizing the MI between dissimilar images. In some embodiments, the loss function can be similar to the cross entropy loss (classification loss), where each image in the batch has a different label between zero and the batch size. The difference from the classification loss is that the recognition network 138 can: (1) control what it means to have similar images, and (2) have a better chance of extracting richer features in the images because similar sample container types are closer to each other, regardless of the sample container type.

[0093] Additional references Fig. 9 , Fig. 9 is a diagram illustrating an embodiment of contrastive learning according to one or more embodiments. Fig. 9 702A of the first sample container 700 and the original image 802A of the second sample container 800. Fig. 9 In the embodiment of the present invention, enhancements of original image 702A and original image 802A are performed. An enhancement is an additional image that can be captured or otherwise generated. Images 702B, 702C, 802D, and 802E are examples of enhancements of original image 702A and original image 802A, respectively. Other enhancements may be included, but are not described in detail. Fig. 9 Each of the enhanced images 702B, 702C, 802D, and 802E may be encoded by the encoder 900 into a representation of each enhanced image. In some embodiments, the original images 702A and 802A may also be encoded.

[0094] The encoder function may be in the computer 130 ( Figure 1), such as in recognition network 138. A transformed image or feature map may be processed, where the transformed image may be obtained from the output of the neural network before the classification layer processes the image. The transformed image contains the features extracted by the neural network. The transformed image may then be transformed or encoded into a single vector by conventional processes.

[0095] The representation output from encoder 900 can be shown as an array of values. Each element of the array can be an encoded item from an image, and the value of the element represents the item or a condition of the item. For example, one element can be a color, and the value can be the average color. Another element can be a lid configuration (e.g., lidded, uncovered, or a test tube top sample cup), and the value can indicate the state of the lid. For example, a value of one can indicate an uncovered sample container, and a value of two can indicate a covered sample container. Other elements can be related to geometric features of the sample container, and the value can indicate the geometric features. Fig. 9 The description of the representation in an array is for illustrative purposes, and the output of encoder 900 may be in other forms.

[0096] A multi-layer perceptron (MLP) processes the representations and determines which images are similar and which images are dissimilar based on the values ​​from the encoder 900 described above. For example, the values ​​in the array can be compared to each other to determine similar and dissimilar images. Fig. 9 In the example of , similar images attract each other, and dissimilar images repel each other. This attraction and repulsion is used by the contrast learning routine or network to train the recognition network 138. Similar images can be used to train the recognition network 138 to learn different variations of the sample container 104.

[0097] Contrastive learning may be used during training of the recognition network 138 . Fig. 9 The output of the MLP in can be a vector of size (batch size × dimension), where batch size is the number of images used in a batch during training, and dimension is the number of dimensions of the representation vector. The dimension of the representation vector can be any value (e.g., 1024). The representation vector corresponds to (z) in equation (2). Because the loss function maximizes the mutual information (MI) between similar images and minimizes the MI between dissimilar images, the loss function can be performed as follows Fig. 9 Attract / repel functionality shown.

[0098] use Fig. 9 The contrastive learning described in enables control of the recognition network 138 ( Figure 1) can focus on features to determine whether sample container images are similar. For example, the control can include weighting the elements of the array to determine which images are similar or dissimilar. For example, the weighting can determine how close the values ​​of different elements in the array need to be relative to each other in order for images corresponding to different images to be considered similar. Sample container image similarities can include sample containers from the same manufacturer, sample containers with the same or similar colors, and sample containers with the same or similar shapes or geometric features. In addition, the recognition network 138 can learn image representations introduced by extension (e.g., changes to the sample containers), and learn how to recognize different imaging conditions, such as blur, camera conditions, lighting conditions, dents, indentations, and new tube colors.

[0099] In some embodiments, the recognition network 138 can be configured to operate with multiple different imaging devices, where images captured with different imaging devices can include enhancements relative to the original image. The imaging devices can be configured to operate with different modalities, where images captured with different modalities can be enhancements. The modalities can include color, exposure time, lighting intensity, one or more lighting spectra, and other imaging conditions.

[0100] Fig. 9 The workflow described in can operate in at least two different configurations. In the first configuration, each of the different modalities, viewpoints, and / or other imaging conditions is considered an enhancement of the original image 702A, 802A. In some embodiments, images captured from different imaging conditions can undergo additional enhancement. Fig. 9 Contrastive learning attempts to bring similar sample container images of different viewpoints, modalities, and / or other imaging conditions closer to each other, while rejecting images of different sample containers. In a second configuration, images with other imaging conditions of different viewpoints, modalities, and / or instances are first augmented, and then the images are grouped into one instance (e.g., by stitching). The resulting image has a higher dimensionality and is a new augmented image.

[0101] Contrastive learning will then attempt to bring higher dimensional images closer to each other while rejecting enhancements of other images. Other types of self-supervised losses or contrastive learning, such as methods that do not rely on rejecting dissimilar images, can be used to train the recognition network 138.

[0102] When a color camera is used to capture the sample container 104 ( Figure 1), the resulting image can have three color dimensions, which are color channels such as red, green, and blue (RGB). The image can then be described as a matrix of shape, for example, including three color dimensions in addition to height and width. If more modalities are used, the image can have higher (e.g., more) dimensions. For example, if the image includes RGB data, depth data, and infrared (IR) data, the dimensions are RGB+depth+IR, height, and width. Therefore, the resulting dimensions can be referred to as a matrix of (5, height, width), which can be referred to as a five-dimensional image representation.

[0103] In some embodiments, training can include capturing three images of different views of the sample container 104, each image having three color channels (e.g., RGB). The images can be concatenated along their RGB color channels. The resulting image has a matrix (3+3+3, height, width), which is a matrix (9, height, width). Other methods of clustering images into matrices can be used. In other embodiments, a three-dimensional (3D) image can be created, which can have a four-dimensional matrix (3, number of images, height, width).

[0104] Additional references Fig.10 , which broadly illustrates that can be implemented in the recognition network 138 to follow Fig. 9 An example of a workflow for training a network for recognizing network 138. An input image is received at backbone 1000, which in some embodiments may be a convolutional neural network (CNN). In some embodiments, backbone 1000 may be an efficientNetV2 network. The EfficientNetV2 network is a series of CNNs that have faster training speed and better parameter efficiency than other CNNs. The EfficientNetV2 network includes a combination of training-aware neural architecture search and scaling to jointly optimize training speed and parameter efficiency. The final classification layer of backbone 1000 may be removed. Other types of backbones may be used to generate representations of the input image. Examples of other backbones include deep networks, transformers, and principal component analysis (PCA).

[0105] During training, the image representation is then fed to both the classification head 1002 and the comparison head 1004. In some embodiments, one or both of the classification head 1002 and the comparison head 1004 can be a network with a set of fully connected layers. In other embodiments, more complex networks can be used, such as by including a Siamese branch in the comparison head 1004. The comparison head 1004 outputs data indicating whether the input image is similar to other images and which images the input image is similar to. The similarity is used Fig. 9The classification head 1002 outputs a probability or confidence level that the input image is similar to other images that the backbone 1000 was originally trained on. In some embodiments, the confidence level is high (e.g., >95%), indicating a certainty that the image has been correctly classified or identified. The outputs from the classification head 1002 and the comparison head 1004 enable heuristic data of confidence level and similarity to be used to determine when to add an image to the core dataset 136 ( Figure 1 ), as described in this article.

[0106] The classification head may include one or more linear layers and may output a probability or likelihood that the sample container is properly identified. The similarity between images may be determined by K-nearest neighbors. After the classification head 1002 or the comparison head 1004 is processed, a distance may be applied or used that is the likelihood of proper sample container identification. In such an embodiment, only calculating any number of nearest neighbors and distances may be applied. In other embodiments, when the recognition network 138 is trained to optimize similarity in images, cosine similarity as described above may be employed.

[0107] The classification head 1002 and the comparison head 1004 can operate by processing the enhanced image, which can be input into the same recognition network 138. The image corresponding to the enhanced image is Fig.10 The image representation after processing by the backbone 1000 of . The contrast head 1004 can have a projection layer and a prediction layer. The projection layer can include three linear layers, each followed by a batch norm layer and an activation layer (except the last layer, which may be just a linear layer). The prediction layer can be a linear layer, followed by a batch norm layer, an activation layer, and another linear layer. In some embodiments, the outputs of the projection and prediction layers are applied to train the recognition network 138 as in MocoV3.

[0108] The training method described here uses the core dataset 136 ( Figure 1 ) to update or retrain the recognition network 138 ( Figure 1 ). For example, training updates the recognition network 138 from the deployed recognition network 138A to the retrained recognition network 138B. The training method can also repeatedly retrain or update the retrained recognition network 138B. The core data set 136 can be updated to include images that were not used in the previous training of the recognition network 138. The deployed recognition network 138A may have been trained using a combination of images, and can be retrained to a retrained recognition network 138B using a different combination of images.

[0109] The same core data set 136 can be deployed with each individual lab system. For example, the core data set 136 can be located locally on the lab system 100, or remotely from the lab system 100 and accessible by the lab system 100 over a data network. When an image is to be saved for training, the image can be sent to an additional database (local or remote) or added to the core data set 136, which can be local or remote. Updating the core data set 136 to create a new core data set 136 can be done periodically to ensure that the core data set 136 has the best representation of the image.

[0110] In some embodiments, images of new sample containers or images of sample containers previously used to update the core data set 136 are not deleted from the core data set 136 unless the user deletes the image. Updating the core data set 136 with images of sample containers 104 enables the recognition network 138 to be checked to determine whether the recognition network 138 is operating correctly after having been trained with the new images. Therefore, the images can be used to perform a benchmark test of the recognition network 138. In addition, the laboratory system 100 may be able to revert to a previous version of the core data set 136, such as an initially deployed core data set. If the laboratory system 100 or one or more instruments 102 were set at a previous location and moved to a new location, reverting to the previous core data set can be performed. Reverting can also be performed if one of the instruments 102 becomes dedicated to a given new type of sample container and needs to work completely with the new type of sample container.

[0111] In some embodiments, the laboratory system 100 can save memory 134 ( Figure 1 ) in space. The recognition network 138 can recreate the core data set 136 from the remote image data. In some embodiments, the user can be given the option to delete images of sample container types that are not used to update the core data set 136.

[0112] The retraining may be self-supervised and may use augmented images of sample containers, as described with reference to FIGS. 7 and 8 . Retraining with a large number of augmented images improves the performance and scalability of the recognition network 138 , but it may be difficult to recognize sample containers 104 with completely new designs and / or sample container images captured under very different imaging conditions. The recognition difficulties are overcome with the systems and methods described herein. The core data set 136 may include a subset of sample container images used during the original or subsequent training of the recognition network 138 . For example, the core data set 136 may include a subset of sample container images used to train the deployed recognition network 138A. The subset of sample container images may have sufficient variation so that when the core data set 136 retrains the recognition network 138 , the retrained recognition network 138B is able to recognize the sample container 104 with at least a predetermined confidence level. For example, the retrained recognition network 138B is able to recognize the sample container 104 at least with a confidence level greater than 0.95 (greater than 95%).

[0113] Additional references Fig.11 , Fig.11 is a method for selecting a core data set 136 ( Figure 1 ), the data subsets may be referred to as training data subsets. Training data subsets are different data sets that include different sets or combinations of images that may further include enhanced images as described herein. In some embodiments, data subsets may be generated (e.g., obtained) from at least a portion of a complete training data set, each data subset including a different combination of sample container images obtained from the complete training data set. In some embodiments, the methods described herein may be configured to generate multiple training data subsets. In summary, the method 1100 selects one of the multiple training data subsets as the core data set 136. As described herein, the method 1100 tests the training data subsets against a benchmark data set, and the training data subset with the best test results may be selected as the core data set 136.

[0114] Different methods of selecting a subset of training data can be used. In some embodiments, the subset of training data can be based on the granularity of sample container identification. Different levels of granularity can be defined according to the type of annotation used for each sample container type. The level of granularity depends on how the data in the image is sampled and how the sample container type is defined, which itself may affect how the sample is tested. The granularity can be coarse, such as related to the type of sample container. Examples of coarse granularity include determining whether the sample container in the image is uncovered, covered, sealed, etc. Examples of fine granularity include determining whether the sample container has a cover and the type of test performed in the sample in the sample container based on the image. Examples of even finer granularity include determining whether the sample container has a cover, the type of test performed in the sample in the sample container, and the sample container manufacturer from the image.

[0115] The granularity can determine how to select different training data subsets of the sample container images. For each training data subset, a certain number of images can be retained for training, and the remaining images can be used for testing. One method of selecting a training data subset is through a subtraction method. The subtraction method includes generating multiple possible training data subsets, training a recognition network on the training data subsets, testing the test data, and at least saving the metric of interest. The method also includes removing one or more subsets, training a new network on the remaining subsets, and testing the test data. If the metric of interest is within a threshold, the process is repeated using at least the subset. When the performance is above the threshold and the number of data subsets in the core data set 136 is greater than the target number of subsets, the process continues. The process can also be repeated at the image level rather than the image subset level to reduce the number of images required for processing.

[0116] Another method of selecting one of the subsets of training data is the additive method. One or more subsets are selected for training and testing. The performance is then checked. If the performance is better than the previous performance test, but below the acceptance threshold, and the number of subsets is below the number of subsets needed, one or more subsets are added. The performance is checked again until acceptable performance is measured. The additive method can avoid having to completely retrain the recognition network 138. Instead, the classification layer can be reset by possibly adding some network adjustments. Therefore, the additive method may be faster than the subtractive method.

[0117] In the first process of method 1100, different subsets of data are used to train the network, thereby generating a trained network. Fig.11In the embodiment of the present invention, the first data subset 1 trains the network to form the first training network 1, the second data subset 2 trains the network to form the second training network 2, and the Kth data subset K trains the network to form the kth training network K. The second process of the method 1100 operates to test each trained network using test data. The test data may include as many sample container types as possible, but it may not include all sample container types from the current core data set 136. The test data may include, for example, images of sample containers 104 that cannot be recognized by the recognition network 138. The results of the test can be compared with the benchmark, and the data subset that generates the network that provides the best results can be selected as the core data set 136 ( Figure 1 ) or data used to revise core data set 136. Core data set 136 is then used to train or retrain recognition network 138, as described herein.

[0118] In some embodiments, some images of sample containers 104 that cannot be recognized by the recognition network 138 are not added to the core data set 136. For example, sample containers 104 that cannot be recognized may be ready to be discontinued from use in the diagnostic laboratory system 100, and therefore, there is no need to use these sample containers 104 in the core data set 136. Other sample containers 104 may be rarely used and may not be added to the core data set 136.

[0119] Additional references Fig.12 , which shows a description of determining whether an image of a sample container 104 is to be added to Fig.11 Flowchart of method 1200 for a core data set 136 or data subset as described in . Fig.12 The method may provide a heuristic determination as to whether an image of a sample container 104 is to be added to the core data set 136. The method 1200 described in the flowchart may be performed by the computer 130 ( Figure 1 ) implemented in computer code. The recognition network 138 uses the AI ​​described herein to recognize the sample container 104, such as the Figure 5 The comparison head 1004 can determine the similarity of the image to other images, such as relative Fig.10 The classification head 1002 calculates a confidence level for the image recognition performed by the classification head 1002. In some embodiments, the comparison head 1004 may be used to retrieve the nearest neighbors of the image being processed from the core data set 136. For example, the comparison head 1004 ( Fig.10 ) can be discarded and the nearest neighbor can be applied to the image (e.g., image representation). In some cases, the results may be better by performing this process using the comparison head 1004 because the comparison head 1004 may have been optimized using the cosine similarity between similar images as described above.

[0120] Nearest neighbor data can be used to enable a decision as to whether an image is similar to a stored image. In some embodiments, the contrastive loss optimizes the cosine similarity between different images. The cosine similarity metric for image representations of similar images is close to 1.0, while the cosine similarity metric for dissimilar images is close to -1.0.

[0121] Other measures or distances can be used to calculate similarity between images. For example, information related to the image representation can be generated. The image representation can be obtained from Fig. 9 The image shown is generated as an output of the classification head 1002 or the comparison head 1004. When the similarity between images is to be calculated, the cosine similarity between the above image representations can be calculated to determine the similarity. In some embodiments, the Euclidean distance can be used to determine the distance between image representations.

[0122] The data generated by both the classification head 1002 and the comparison head 1004 are provided as input to a heuristic processing block 1201 that performs automatic heuristic-based decisions and / or a feedback processing block 1202 that performs feedback-based decisions. Data (e.g., image data) in the original or previous core data set 136 is also input to both the heuristic processing block 1201 and the feedback processing block 1202. For example, as described above, the heuristic processing block 1201 can use the nearest neighbor results to determine how close the image is to other images.

[0123] In some embodiments, the feedback processing block 1202 defines more complex heuristic decisions based on multiple nearest neighbors generated by the nearest neighbor routine. The feedback processing block 1202 can also use the nearest neighbor routine to provide data to the user to retrieve feedback about the proximity of the image to other images. The image can be displayed on the display 144 ( Figure 1 ), and the user can use the workstation 142 to input information about the image similarity of the sample container 104 to the computer 130. In some embodiments, similar to Fig. 9 With contrastive learning, users are able to generate a controllable latent space of nearest neighbor routines by defining which sample bin images should be closer to other sample bins.

[0124] According to method 1200, a decision block 1204 may be provided that determines whether to retain an image of a sample container 104 based on the output of one or both of a heuristic processing block 1201 and a feedback processing block 1202. For example, the decision may be based on a nearest neighbor routine and / or user input. The heuristic processing block 1201 may bring the confidence from the classification head 1002 into the decision making process. If the confidence is below a threshold, the heuristic processing block 1201 may query the core data set 136 to find the nearest neighbor. The user may be notified of the nearest neighbor result immediately, or the user may be asked to check the log at a specific time, such as at the end of the day. The decision in the heuristic processing block 1201 may analyze the sample container image that triggered the heuristic analysis, and compare a given sample container image with a similar sample container image. The determination described herein may determine whether the sample container image should be added to the core data set 136.

[0125] If the decision at decision block 1204 is negative (No), the method 1200 proceeds to processing block 1206, where the image of the sample container 104 is ignored and not added to the core data set 136 or the data subset. If the decision at decision block 1204 is positive (Yes), the method 1200 proceeds to processing block 1210, where the image of the sample container 104 and / or the enhanced image of the sample container 104 is added to the core data set 136 or the data subset. Fig.11 Core data set 136 or a subset of the data. Figure 7B-7E Examples of enhanced sample container images are provided in and 8B-8E.

[0126] After the core data set 136 is updated, the method 1200 may proceed to a trigger block 1212 where the retraining of the recognition network 138 is again triggered. The retraining may be performed continuously, at a predetermined time, or based on user input. In some embodiments, the user may provide the computer 130 ( Figure 1 ) input information, which enables the recognition network 138 to be retrained using the data in the core data set 136. The computer 130 can output a signal, such as a signal displayed on the display 144, indicating that there is sufficient data in the core data set 136 to be used to retrain the recognition network 138. In other embodiments, the recognition network 138 can be retrained on a schedule, such as weekly or when the diagnostic laboratory system 100 is idle. In some embodiments, the core data set 136 can be transmitted to other diagnostic laboratory systems to update their recognition networks. In other embodiments, the core data set 136 can be retrieved from a source external to the diagnostic laboratory system 100, such as from a server or another diagnostic laboratory system.

[0127] The core data set 136 may be a central core data set that receives updates or images from a plurality of different diagnostic laboratory systems. The central core data set may be deployed in the diagnostic laboratory system 100 to retrain the recognition network 138. The central core data set may include images of a large number of sample containers 104, so the recognition network 138 may then be retrained to recognize a large number of sample containers 104.

[0128] In some embodiments, the core data set 136 may be smaller than the available data for all sample container types that can be recognized by the deployed recognition network 138A. "Less than" means that the core data set 136 contains fewer images than the number of images used to train the deployed recognition network 138A. For example, the core data set 136 may include at most half of the total number of images of the sample container type on which the deployed recognition network 138A is trained. The enhancement of the images of the sample containers 104 enables a small number of images of each sample container type to be used during training and retraining. The enhancement can also keep the core data set 136 small because the images of the sample containers 104 may undergo significant enhancement to produce large changes in the new sample container images.

[0129] During training, the sample container images may undergo additional augmentation. By augmenting the images during training, only a few sets of each sample container image are needed, since new sample container images can be generated by augmenting the images. One of the challenges of augmentation is to simulate the light and reflection conditions of different sample containers. By saving images of the actual sample containers that cause these problems, challenging conditions in lighting conditions are saved. Data augmentation can then create new colors in the images and manipulate the images with perspective, deformation, etc. in the augmented images.

[0130] Reference now Fig.13, which shows a flow chart of a method 1300 for training a sample container recognition network (e.g., recognition network 138) of a diagnostic laboratory system (e.g., diagnostic laboratory system 100). The method 1300 includes obtaining a plurality of data subsets at block 1302, wherein each data subset is smaller than a complete training data set for the sample container recognition network 138 and includes a plurality of images of one or more sample containers (e.g., sample container 104). The method 1300 includes, at block 1304, training a sample container recognition network on each of the plurality of data subsets to generate a plurality of trained sample container recognition networks. The method 1300 includes, at block 1306, testing each trained sample container recognition network using test data including a test image of the sample container 104, wherein the testing includes recognizing the sample container 104 in the test image. The method 1300 includes, at block 1308, selecting a core data set (e.g., core data set 136) from one of the plurality of data subsets based on the testing, the core data set used to train the deployed sample container recognition network.

[0131] refer to Fig.14 , which shows a flow chart of a method 1400 for retraining a deployed sample container recognition network (e.g., recognition network 138) of a diagnostic laboratory system (e.g., laboratory system 100). The method 1400 includes, in block 1402, capturing a raw image of a sample container (e.g., sample container 104D) using a camera (e.g., imaging device 214) within the diagnostic laboratory system. The method 1400 includes, in block 1404, attempting to identify the sample container 104 by analyzing the raw image using a deployed sample container recognition network, the deployed sample container recognition network being trained on a full training data set. The method 1400 includes, in block 1406, allowing the raw image to be added to a core data set (e.g., core data set 136) if the deployed sample container recognition network fails to recognize the sample container. The method 1400 includes, in block 1408, retraining the deployed sample container recognition network using the core data set, wherein the core data set is smaller than the full training data set.

[0132] Although the present disclosure is susceptible to various modifications and alternative forms, specific method and device embodiments have been shown by way of example in the accompanying drawings and are described in detail herein. However, it should be understood that the specific methods and devices disclosed herein are not intended to limit the present disclosure, but rather to cover all modifications, equivalents, and alternatives falling within the scope of the claims.

Claims

1. A method for training a sample container recognition network for a diagnostic laboratory system, the method comprising: obtaining a plurality of data subsets, wherein each data subset is smaller than a complete training data set for training a sample container recognition network and includes a plurality of images of one or more sample containers; training a sample container recognition network on each of a plurality of data subsets to generate a plurality of trained sample container recognition networks; testing each trained sample container recognition network using test data including test images of sample containers, wherein the testing includes recognizing the sample containers in the test images; as well as A core data set is selected from one of the plurality of data subsets based on testing, the core data set being used to train the deployed sample container recognition network.

2. The method according to claim 1 further comprises using the core dataset to train a deployed sample container recognition network.

3. The method of claim 1 , wherein obtaining a plurality of data subsets comprises: Obtaining a complete training data set for a sample container recognition network, wherein the complete training data set includes multiple images of sample containers; as well as A plurality of data subsets are generated from at least a portion of the full training data set, each data subset comprising a different combination of sample container images obtained from the full training data set. 4 . The method according to claim 1 , further comprising retraining the deployed sample container recognition network using the core dataset. 5 . The method of claim 1 , wherein obtaining a plurality of data subsets comprises capturing images of sample containers in the diagnostic laboratory system.

6. The method of claim 1 , wherein obtaining a plurality of data subsets comprises: capturing a raw image of the sample container; enhancing an original image of the sample container to generate one or more enhanced images; as well as At least one of the original image and the one or more enhanced images is used in at least one data subset of the plurality of data subsets.

7. The method of claim 6, wherein enhancing the original image comprises capturing the image of the sample container under lighting conditions that are different from lighting conditions used to capture the original image.

8. The method of claim 7, wherein the lighting conditions include brightness of illumination of the sample container or one or more spectra of illumination.

9. The method of claim 6, wherein enhancing the original image comprises capturing an image of the sample container having an image quality different from an image quality used to capture the original image.

10. The method of claim 6, wherein enhancing the original image comprises capturing the image of the sample container using an imaging device different from an imaging device used to capture the original image.

11. The method of claim 6, wherein enhancing the original image comprises at least one of capturing an image of the sample container from a viewpoint different from a viewpoint used to capture the original image and cropping the image relative to the original image.

12. A method for retraining a deployed sample container recognition network of a diagnostic laboratory system, the method comprising: capturing a raw image of the specimen container using an imaging device within a diagnostic laboratory system; Try to use the deployed sample container recognition network to analyze the original image to identify the sample container. The deployed sample container recognition network is trained on the complete training dataset; If the deployed sample container recognition network fails to recognize a sample container, allowing the original image to be added to the core dataset; and The deployed sample container recognition network is retrained using the core dataset, wherein the core dataset is smaller than the complete training dataset.

13. The method according to claim 12, further comprising: determining a confidence level of the sample container identified by the deployed sample container identification network; as well as A determination is made whether to include the captured image of the sample container in the core data set based on the confidence level.

14. The method of claim 12, further comprising adding one or more images from a deployed sample container recognition network of a second diagnostic laboratory system to the core data set.

15. The method according to claim 12, further comprising: generating an additional image of the sample container by enhancing the original image of the sample container to generate an enhanced image; as well as Added augmented images of sample containers to the core dataset.

16. The method of claim 15, wherein generating additional images comprises allowing a user to determine how to enhance the captured image of the sample container.

17. The method of claim 15, wherein generating the additional image comprises automatically enhancing the original image to generate the enhanced image.

18. The method of claim 15, wherein enhancing the original image comprises one or more of changing image brightness, changing image quality, changing one or more illumination spectra, changing image color relative to the original image, and cropping the original image.

19. The method of claim 15, wherein the original image is captured from a first viewpoint, and wherein enhancing the original image comprises capturing an image of the sample container from a viewpoint different from the first viewpoint.

20. The method of claim 19, further comprising allowing a user to determine a viewpoint of the sample container.

21. The method of claim 12, wherein retraining the deployed sample container recognition network comprises employing a combination of a classification loss function and a contrastive loss function during retraining.

22. The method of claim 12, wherein retraining the deployed sample container recognition network comprises automatically retraining the deployed sample container recognition network.

23. A diagnostic laboratory system comprising: track; a sample carrier movable on the track and configured to receive a sample container including a sample; an imaging device configured to capture an image of the sample container; a memory including a sample container recognition network, the sample container recognition network being trained on a complete training data set; a computer coupled to the imaging device and the memory; and Computer program code which, when executed by a computer, causes the computer to: employing an imaging device to capture an image of a sample container within a diagnostic laboratory system; attempting to identify the sample container by analyzing the captured image using a sample container identification network; If the sample container recognition network fails to recognize the sample container, adding the captured image to a core dataset, wherein the core dataset is smaller than the full training dataset; as well as Allows retraining of sample container recognition networks using the core dataset.

24. The system of claim 23, wherein the core data set is stored in a memory.

25. The system of claim 23, wherein the core data set is stored on a computer remote from the diagnostic laboratory system.

26. The system of claim 23, further comprising computer program code which, when executed by a computer, causes the computer to: determining a confidence level that the sample container is identified by the sample container identification network; and A determination is made whether to include the captured image of the sample container in the core data set based on the confidence level.

27. The system of claim 23, further comprising computer program code that, when executed by the computer, causes the computer to add one or more images from another sample container identification network of a second diagnostic laboratory system to the core data set.

28. The system of claim 23, further comprising computer program code which, when executed by a computer, causes the computer to: generating an additional image of the sample container by enhancing the image of the sample container; and Added augmented images of sample containers to the core dataset.

29. The system of claim 28, wherein the image is captured from a first viewpoint, and wherein the augmentation comprises capturing an image of the sample container from a viewpoint different from the first viewpoint.

30. The system of claim 29, further comprising computer program code which, when executed by a computer, causes the computer to allow a user to determine a viewpoint of a sample container.

Images