Systems and methods for developing vision system models and algorithms for sample management within diagnostic laboratories

Abstract

In some embodiments, a method includes obtaining design information for a diagnostic laboratory sample management (DLSM) system design; creating a digital-twin DLSM system based on the design information; generating a library of synthetic images using one or more cameras of the digital-twin DLSM system; and one or more of: training at least one ML model using synthetic images from the library of synthetic images; and developing at least one non-ML vision system algorithm using synthetic images from the library of synthetic images. Numerous other aspects are provided.

Description

SYSTEMS AND METHODS FOR DEVELOPING VISION SYSTEM MODELS AND ALGORITHMS FOR SAMPLE MANAGEMENT WITHIN DIAGNOSTIC LABORATORIESCROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims benefit under 35 USC § 119(e) of US Provisional Application No. 63 / 771,943, filed March 14, 2025. The entire contents of the abovereferenced patent application(s) are hereby expressly incorporated herein by reference.FIELD

[0002] The present application relates to medical testing and more particularly to systems and methods for developing vision system models and algorithms for sample management within diagnostic laboratories.BACKGROUND

[0003] Vision-based machine-learning (ML) models and non-ML vision system algorithms may be employed to assist with the identification and handling of blood or other biological samples within a diagnostic laboratory. However, training such models and algorithms requires a large library of training images that covers the full range of variability of visual appearances of sample containers (e.g., sample tubes) in real world applications. To obtain such images, a prototype sample handling system may be developed and used to capture images of sample containers. For successful ML model training, thousands of images of sample containers must be captured and labeled. Visual image variations that may be observed in real world applications should be included such as different sample container heights, widths (e.g., diameters), and tilts, different cap colors and shapes, different lighting conditions, different sample properties, etc.

[0004] Capturing and curating real world images is expensive and time consuming.Additionally, capturing images that reflect real-world conditions requires handling thousands of sample containers with fluids such as blood, serum, plasma, or their surrogates.Randomization of sample containers for image capturing to cover an entire design space should be performed.

[0005] Once such images are captured, the images must be annotated to provide labelled data for supervised training of the ML model being developed. Annotating is an expensive, time consuming, and labor-intensive manual process that must be repeated whenever a hardware change is made (making system updates expensive and possibly cost-prohibitive). Likewise, adding new sample container types requires capturing images of the new sample container types under relevant real-world conditions and annotating such images, makingsample container changes difficult and expensive. Because developing non-ML vision system algorithms also requires a large number of images, it is similarly time consuming and expensive.

[0006] As such, a need exists for improved systems and methods for developing vision system models and algorithms for sample management within diagnostic laboratories.SUMMARY

[0007] In some embodiments, a method includes obtaining design information for a diagnostic laboratory sample management (DLSM) system design; creating a digital-twin DLSM system based on the design information; generating a library of synthetic images using one or more cameras of the digital-twin DLSM system; and one or more of: training at least one ML model using synthetic images from the library of synthetic images; and developing at least one non-ML vision system algorithm using synthetic images from the library of synthetic images.

[0008] In some embodiments, a method includes obtaining computer-aided design information for a diagnostic laboratory sample management (DLSM) system design; creating a digital-twin DLSM system based on the computer-aided design information; generating a library of synthetic images using one or more cameras of the digital-twin DLSM system, the library of synthetic images including: top view synthetic images of sample containers within one or more sample trays of the digital-twin DLSM system; and side view synthetic images of sample containers. The method further includes training a cap detection ML model using the top view synthetic images, the cap detection ML model trained to detect caps of sample containers; developing a geometry detection algorithm using the top view synthetic images; and developing a barcode reading algorithm using the side view synthetic images.

[0009] In some embodiments, a method includes obtaining design information for a diagnostic laboratory sample management (DLSM) system design; creating a digital-twin DLSM system based on the design information; identifying components of the digital-twin DLSM system relevant to generation of synthetic images; modelling noise and variability in the identified components; and programmatically controlling the modelled noise and variability in the identified components during synthetic image generation with the digital-twin DLSM system to generate a library of synthetic images representative of the noise and variability of the identified components.

[0010] Other features and aspects of the present invention will become more fully apparent from the following detailed description, the appended claims, and the accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0011] FIG. 1A illustrates a flow diagram of a system for developing sample management vision system models and algorithms for use within diagnostic laboratories according to embodiments provided herein.

[0012] FIG. 1B illustrates an example image generation and vision system model / algorithm development system according to embodiments provided herein.

[0013] FIG. 2 illustrates a flow diagram of a method of employing the system of FIG. 1A and / or FIG. 1B to develop vision-based ML models and non-ML vision system algorithms for a DLSM system in accordance with one or more embodiments.

[0014] FIG. 3 is a top view of an example embodiment of the real-world DLSM system of FIG. 1 A according to one or more embodiments.

[0015] FIG. 4A is an example top view image of sample containers captured by one of the cameras of the real-world DLSM system of FIG. 3 in accordance with embodiments provided herein.

[0016] FIG. 4B is an example side view image of a sample container captured by one of the cameras of the real-world DLSM system of FIG. 3 in accordance with embodiments provided herein.

[0017] FIG. 5 is a flowchart of an example process of training a vision system model or developing a vision system algorithm for use in a diagnostic laboratory sample management system in accordance with embodiments provided herein.

[0018] FIG. 6 is a flowchart of an example process of training a vision system model and developing one or more vision system algorithms for use in a diagnostic laboratory sample management system in accordance with embodiments provided herein.

[0019] FIG. 7 is a flowchart of an example process of developing training images with a digital-twin diagnostic laboratory sample management system in accordance with embodiments provided herein.DETAILED DESCRIPTION

[0020] Independent of the grammatical term usage, individuals with male, female or other gender identities are included within the term.

[0021] Systems and methods provided herein allow for the generation of synthetic images fortraining vision-based ML models and developing non-ML vision-system algorithms used to assist with the identification and handling of blood or other biological samples within a diagnostic laboratory. Training images may be created and annotated quickly and inexpensively without requiring use of a real-world sample management system or the handling of blood or other biological samples. Through use of the systems and methodsdescribed herein, the significant time and expense associated with capturing and annotating training images may be substantially reduced, allowing hardware changes or new sample container types to be accommodated with minimal effort and expense (e.g., as training images for any hardware or sample container changes may be created rapidly and economically).

[0022] Embodiments described herein may include creation of a digital-twin diagnostic laboratory sample management (DLSM) system based on design information (e.g., computer aided design (CAD) information) fora DLSM system design (e.g., regardless of whether a real-world DLSM system is merely in development and has not been built or is ever built). The digital-twin DLSM system allows for the generation of an extensive library of synthetic images of sample containers (e.g., test tubes) covering an entire design space for a DLSM system. In some embodiments, the digital-twin DLSM system may include sufficient fidelity to the visual aspects of a real-world DLSM system to aid in the development and / or maintenance of a vision system for the DLSM system. The term ‘aid in the development’ may include, but is not limited to, the synthetic generation of images to train and / or test machine vision models and develop other vision system algorithms, the synthetic generation of images to assist in the evaluation of design tradeoffs related to optics and / or lighting, the analytic evaluation of physics-based models to assist in the evaluation of design tradeoffs related to optics and / or lighting, and / or the synthetic generation of images to aid in the industrial design and / or user-experience design of vision systems.

[0023] The synthetic images may be annotated for ML model training and also used to develop non-ML vision system algorithms.

[0024] These and other embodiments are described below with reference to FIGS. 1 A-7.

[0025] FIG. 1A illustrates a flow diagram of a system 100 for developing sample management vision system models and algorithms for use within diagnostic laboratories according to embodiments provided herein. With reference to FIG. 1A, design information 102 (e.g., computer-aided design (CAD) information) may be obtained (e.g., developed) fora diagnostic laboratory sample management (DLSM) system. A DLSM system may be used to process sample containers (e.g., test tubes) supplied to the DLSM system in racks or trays. For example, sample containers of different sizes and types, with different samples and caps, may be loaded into racks or trays. The sample containers may be imaged using one or more cameras to identify and classify each sample container, characterize sample container caps, and read barcode label information, among other things, using trained vision-based ML models and / or non-ML vision-based algorithms. These models and / or algorithms may also be used to control pick and place operations by robotics within the DLSM system based on images of the sample containers.

[0026] Example design information may include CAD information that may be obtained for a DLSM system may include geometric data such as two-dimensional (2D) and three-dimensional (3D) models for components within the DLSM system, material specifications, dimensions and tolerances, electrical, mechanical, and structural layers, assembly details such as part relationships, animations, etc. Material specifications may include (but are not limited to) optical specifications such as transparency, reflectiveness, gloss, color, etc. CAD information may be generated using any suitable CAD software such as Siemens NX CAD / CAM software available from Siemens Digital Industries Software of Plano, Texas, AutoCad available from AutoDesk, Inc. of Mill Valley, California, or the like.

[0027] The CAD information (e.g., a CAD file) or other design information is provided to a 3D simulation tool 104 such as NVIDIA Omniverse™ (available from NVIDIA Corporation of Santa Clara, CA), Blender (available from the Blender Institute of Amsterdam, The Netherlands), Unity (available from Unity Software, Inc. of San Francisco, CA), or another similar simulation tool (e.g., executing on one or more processors (not shown)). The 3D simulation tool 104 may be employed to construct a digital-twin DLSM system 106 based on design information 102. Using 3D simulation tool 104, operation of digital-twin DLSM system 106 may be simulated as described further below (e.g., through raytracing, physic simulation, etc.).

[0028] To accurately simulate DLSM system operation, “photosimilar” simulated material properties 108 are developed for sample fluids, foils, sample container caps, barcode labels, and the like, and are fed to 3D simulation tool 104 for enhancing simulated operation of digital-twin DLSM system 106. As used herein, “photosimilar” means similar enough to a real-world image for the intended purpose (e.g., close enough to a real-world image to allow training of vision-based ML models such as cap detection ML models, barcode region-of-interest (ROI) detection ML models, etc., and / or development of non-ML vision system algorithms such as geometry detection algorithms, barcode reading algorithms, computer-vision-based robotics algorithms, etc.). Photosimilar may be equal to or less than the accuracy of photorealistic.

[0029] Once developed, digital-twin DLSM system 106 may be employed to generate a library 110 of synthetic images 112 using one or more cameras 114 of digital-twin DLSM system 106. In some embodiments, synthetic images 112 may include synthetic images of sample containers (e.g., test tubes) covering an entire design space forthe DLSM system. The synthetic images may be annotated and used to train one or more ML vision system models 116. Synthetic images 112 may also be used to develop one or more non-ML vision system algorithms 118.

[0030] Once developed, ML vision system model(s) 116 and / or non-ML vision systemalgorithm(s) 118 may be deployed within a real-world DLSM system 120 of a diagnostic laboratory 122 (e.g., as deployed vision system model(s) and algorithm(s) 124).

[0031] FIG. 1B illustrates an example image generation and vision system model / algorithm development system 150 according to embodiments provided herein. With reference to FIG.1B, system 150 includes a first computer system 152 configured to generate synthetic images 112 using digital-twin DLSM system 106 (FIG. 1A) simulated with 3D simulation tool 104. System 150 also includes a second computer system 154 configured to employ synthetic images 112 to develop one or more ML vision system model(s) 116 and / or non-ML vision system algorithm(s) 118. In some embodiments, a single computer system or more than two computer systems may be employed.

[0032] First computer system 152 includes one or more processors 156, one or more GPUs 158, and a memory 160 that includes 3D simulation tool 104. Second computer system 154 includes one or more processors 162, one or more GPUs 164, and a memory 166. Memory 166 may include computer program instructions (e.g., one or more programs 168) for developing ML vision system model(s) 116 and / or non-ML vision system algorithm(s) 118 employing synthetic images 112, processors) 162, and GPUs 164 (as described further below). Other computer system configurations may be employed.

[0033] Processors 156 and / or 162 may be computational resources such as, but not limited to, microprocessors, microcontrollers, embedded microcontrollers, digital signal processors (DSPs), field-programmable gate arrays configured to perform as microcontrollers, or the like. In one or more embodiments, memory 160 and / or 166 may be non-transitory memories (e.g., hard drives, solid-state drives, flash-drives, etc.). Computer program instructions stored in memory 160 may include computer code that, when executed by processor(s) 156, causes processors) 156 to control operation of 3D simulation tool 104 in accordance with one or more of the methods described herein. Computer program instructions stored in memory 166 may include computer code that, when executed by processor 162, causes processor 162 to employ synthetic images 112 to train ML vision system model(s) 116 and / or develop non-ML vision system algorithm(s) 118 in accordance with one or more of the methods described herein.

[0034] Memory 160 and / or memory 166 may be any suitable types of memory, such as, but not limited to, one or more of a volatile memory and / or a non-volatile memory. Memory 160 may be located within computer system 152 and / or processor(s) 156 or a part or all of memory 160 may be located outside of computer system 152 (e.g., remote from computer system 152 such as in cloud storage). Likewise, memory 166 may be located within computer system 154 and / or processor(s) 162 ora part or all of memory 166 may be located outside of computer system 154 (e.g., remote from computer system 154 such as in cloudstorage). Memory 160 and / or 166 may include multiple memory units that may or may not be proximate to one another.

[0035] Memory 160 may have a plurality of instructions (e.g., 3D simulation tool 104) stored therein that, when executed by processor(s) 156, cause processor(s) 156 to perform various actions specified by one or more of the stored instructions. These computer program instructions may be provided to processor(s) 156 to perform operation acts in accordance with the present systems and methods specified in the flowchart(s) and / or block diagram blocks herein. Each processor 156 so configured becomes a special purpose machine particularly suited for performing in accordance with the present systems and methods. Computer program instructions may be stored in a computer readable medium, such as memory 160, that can direct the processor(s) 156 to function in a particular manner.

[0036] Similarly, memory 166 may have a plurality of instructions (e.g., programs 168) stored therein that, when executed by processor(s) 162, cause processors) 162 to perform various actions specified by one or more of the stored instructions. These computer program instructions may be provided to processor(s) 162 to perform operation acts in accordance with the present systems and methods specified in the flowchart(s) and / or block diagram blocks herein. Each processor 162 so configured becomes a special purpose machine particularly suited for performing in accordance with the present systems and methods. Computer program instructions may be stored in a computer readable medium, such as memory 166, that can direct the processor(s) 162 to function in a particular manner.

[0037] FIG. 2 illustrates a flow diagram of a method 200 of employing system 100 of FIG. 1A to develop vision-based ML models and non-ML vision system algorithms for a DLSM system in accordance with one or more embodiments. With reference to FIG. 2, in block 202, 3D simulation tool 104 (FIG. 1A) is used to create digital-twin DLSM system 106. For example, in block 204, a single CAD model of the entire DLSM system to be simulated may be created. In other embodiments, multiple CAD models may be employed. Note that the CAD model may be from a real-world DLSM system (e.g., a DLSM system that has been built and / or is in use) or merely a proposed real-world DLSM system design ora real-world DLSM system being developed.

[0038] In some embodiments, the complexity of components within digital-twin DLSM system 106 may be optimized to reduce the computation load during rendering. For example, a designer may identify components of digital-twin DLSM system 106 that do not affect properties of synthetic images captured by the one or more cameras (e.g., camera(s) 114) of the digital-twin DLSM system and reduce a complexity of these components (e.g., by reducing mesh / vertex density of the components). For example, components that are not visible within a field-of-view of camera(s) 114 of digital-twin DLSM system 106 may havetheir complexities reduced (e.g., robotics or other components that support camera equipment, frames or other structural elements, the underside or inside of components, objects outside the depth of field / focus of the camera(s), etc.). For example, in some embodiments, mesh / vertex density may be reduced by 70% to 90%. Other mesh / vertex density reductions may be employed. Additionally, objects that are within the field of view of the camera(s) but which are not considered by vision-based ML models or non-ML vision system algorithms may also be simplified and / or removed. For example, tray retention bars within a DLSM system may be imaged due to their direct contact with the edges of sample trays, but the tray retention bars add insignificant amounts of signal or noise to the relevant vision models / algorithms. Note that realistic robot movement at speed is maintained. In some embodiments, components that are critical for synthetic image generation may have their complexities increase (e.g., sample containers, barcodes, trays, tray drawers, reflectors, cameras, shrouds, etc.). In some embodiments, this may include employing the original CAD resolution.

[0039] Further, in some embodiments, components of the digital-twin DLSM system may be split into multiple entities such as physics attributes, geometry meshes, semantic data, collider meshes, etc. By employing multiple entities for components, one or more aspects of a component may be changed without requiring the entire component model to be reworked. For example, during development it may be desirable to improve the geometry, physics, or colliders of every component within digital-twin DLSM system 106 (e.g., thousands of components), the geometry (or physics or colliders) of all objects may be updated at once. Additionally, having multiple entities may allow multiple team members to develop different aspects of the sample component (e.g., at the same time or at different times).

[0040] In block 206, photosimilar simulated material properties are provided to 3D simulation tool 104. For example, a library of photosimilar simulation material properties may be created to accurately model the visual properties of components that will be imaged by camera(s) 114 of digital-twin DLSM system 106. Example material properties that may be simulated include the visual properties of fluids (e.g., patient samples, control fluids, calibration fluids, etc.), foils and caps used to cover sample containers, sample containers, barcode labels, etc. In particular, photosimilar simulated material properties for the sample containers may be created and applied to the sample containers within digital-twin DLSM system 106. Similarly, photosimilar simulated material properties for other components such as sample trays may be created and applied to the respective components within digital-twin DLSM system 106. More generally, material definitions may be added for components relevant to (e.g., viewable in) any synthetic images to be generated. For example, material properties may be simulated by selecting and adjusting parameters, such as surfaceroughness, reflectivity, transparency, actual color, etc. For deterministic objects such as tubes, tube caps, foils, etc., their properties may be captured / measured from real world objects. For less deterministic objects such as dust, dirt, splashes, spills, and fingerprints, their properties may be iteratively adjusted to match observed real world appearances.

[0041] Because of the wide variation in barcode label conditions that may be present on sample containers, in some embodiments, a graphical pipeline may be employed to simulate barcode label conditions such as deformations (e.g., tears, crinkles, etc.), peeling, or other barcode label condition variations. The simulated barcode label conditions such as deformations, peeling, etc., may be programmatically variable so as to allow for fully programmatic simulation and randomization for synthetic data generation. Barcode cloth simulation and / or photogrammetry may also be employed.

[0042] Photosimilar simulated material properties may also be created to simulate one or more imperfections that may be present during operation of digital-twin DLSM system 106. For example, imperfections on sample containers and fluid residue on exterior surfaces of sample containers or sample trays may be simulated such as, but not limited to, one or more of fingerprints, dirt, dust, particles, spills including spilled liquid that is still wet or spilled liquid that has dried and left a residue, detritus like barcode paper pulp and / or shards of plastic or glass tubes, wear, noise, other environmental noise factors, etc. As with barcode label properties, the simulated imperfection conditions may be programmatically variable so as to allow for fully programmatic simulation and randomization for synthetic data generation.

[0043] In block 208, simulated optics and lighting within digital-twin DLSM system 106 may be provided. For example, simulated camera properties forcamera(s) 114 of digital-twin DLSM system 106 may include F-Stop, aperture, working distance, etc. Lighting conditions or properties that may be simulated may include lighting geometries (e.g., ring or bar light), lighting spectral properties, lighting intensities, and / or homogeneity.

[0044] In block 210, physically accurate pick and place robotics may be simulated within 3D simulation tool 104. For example, 3D simulation tool 104 may be used to simulate pick and place robotics used to pick and place sample containers within digital-twin DLSM system 106. In some embodiments, this may include adding joints or motion profiles to robotic components such as a gantry, gripper, wrist, arm(s), or the like. Any suitable rigging techniques may be used to add motion or collider meshes to components (e.g., robots, grippers, etc.).

[0045] In block 212, digital-twin DLSM system 106 (and camera(s) 114) may be employed to generate a library of synthetic images (e.g., library 110 of synthetic images 112). Through use of digital-twin DLSM system 106, all synthetic data generation may be controlled via software and / or be fully automated across all areas of interest. For example, top view, sideview, or other viewpoint synthetic images may be generated with camera(s) 114 of digitaltwin DLSM system 106. Robotic pick and place operations may be simulated for image capture when employing a robot mounted camera.

[0046] Synthetic images of sample containers (e.g., test tubes) covering an entire design space for the DLSM system may be generated. For example, in block 214, different sample container properties may be applied during image generation and, in block 216, different optical and lighting properties may be applied during image generation. Note that blocks 214 and 216 may be performed together or at least partially overlapping in time. In some embodiments, different sample types, sample container types or sizes, sample container caps or foils, barcode labels or label conditions, sample and / or sample container imperfections, sample container tilts, etc., may be employed as may different camera settings, camera positions, camera noise, lighting conditions, lighting geometries, etc.Further, 3D simulation tool 104 may be employed to annotate the synthetic images (e.g., generate bounding boxes around sample containers, sample container caps, barcode labels, etc.). In some embodiments, a dictionary file may be employed to define ranges of parameters to randomize and parameters may be programmatically varied to generate (e.g., automatically) the desired library of synthetic images.

[0047] In some embodiments, library 110 of synthetic images 112 may include top view synthetic images of sample containers within one or more sample trays of the digital-twin DLSM system and / or side view synthetic images of sample containers (e.g., within a sample carrier, held by a gripper of a robot, etc.). For example, top view images may be helpful during training of cap identification ML models and tube height / width estimation algorithms, whereas side view images may be helpful during training of barcode region-of-interest (ROI) detection ML models and barcode reading algorithms (and / or tube height / width estimation algorithms).

[0048] In block 218, vision system models and algorithms may be developed for managing sample containers within a DLSM system. For example, synthetic images 112 may be employed to train one or more vision-based ML models (block 220) and to develop one or more vision system algorithms (block 222). Example vision-based ML models that may be trained include a cap detection ML model trained to detect caps of sample containers (e.g., based on top view images of the sample containers), a barcode ROI detection ML model (e.g., a semantic segmentation model), or the like. A sample container cap characterization ML model may also be trained using top and / or side view synthetic images of sample containers. Other vision-based ML models may be trained. Example frameworks for the ML models may include neural networks, convolutional neural networks, deep neural networks, object detection networks (e.g., you only look once (YOLO) neural networks), segmentationneural networks, deformable part models, edge detection models, or the like.

[0049] In some embodiments, an ML model may be trained using only synthetic images generated by digital-twin DLSM system 106 (e.g., synthetic images 112 from library 110). However, in other embodiments both synthetic images and real-world DLSM system images (e.g., legacy images and / or images from new systems after active labelling) may be used to train an ML model. For example, approximately 3% to 10% of the training images may be real-world images although other numbers of real-world images may be employed. In some embodiments, the amount and / or percentage of real-world images may follow a flat rule of thumb (e.g., a given number, range, percentage, etc., of training images may be real-world images). In other embodiments, the gap between real-world images and synthetically generated images may be analyzed and / or characterized to determine which models and / or algorithms may most benefit from real-world image data. The percentage and / or amount of real-world images may be a computed value based on statistical analysis of the performance difference of real-world images versus synthetic images. In other cases, a strategic decision may be made to rely more heavily on real-world images for edge cases that are time and / or resource intensive to generate synthetically. In some embodiments, a single model may be trained on both top and side view images while in other embodiments, separate models may be trained for each viewpoint.

[0050] Example non-ML vision system algorithms that may be developed based on synthetic images from digital-twin DLSM system 106 include geometry detection algorithms (e.g., for estimating the height, width, centerline, and / or tilt of sample containers), barcode reading algorithms for reading barcode labels on the side of sample containers, computer-vision-based robotics algorithms (e.g., to control pick and place robotics), or the like.

[0051] Note that computer vision algorithms may be developed and tested before any hardware is available. Synthetic images generated through use of digital-twin DLSM system 106 may cover a wider range of visual variability of sample containers than is practical with a real-world DLSM system. Synthetic image generation allows development of vision-based robotics applications faster, cheaper, and with more robust outcomes, and with a more open system design that may be modified during product life cycle.

[0052] Note that development of geometry detection algorithms may require a large number of images (e.g., 250 for development and more than 10,000 for robust testing). Thus, use of synthetic images from digital-twin DLSM system 106 may significantly reduce geometry detection development and testing costs.

[0053] Other non-ML computer vision algorithms that may be developed from synthetic images generated by digital-twin DLSM 106 include tray slot identification algorithms (e.g., when a sample container needs to be assigned to a specific slot in a tray for a pickoperation), algorithms for solving correspondence for DLSM robotic systems, etc.

[0054] The 3D simulation tool 104 may employ physically accurate simulations to simulate pick and place robotics of a DLSM system. By applying ranges of visual sample container properties and ranges of optical system properties within digital-twin DLSM 106, a large library of synthetic images may be generated. This allows for computer vision training at a fraction of the cost compared to using a real-world DLSM system for image generation. Further, sample container detection model performance testing, validation, verification, and improvement may be performed in a closed loop in software in digital-twin DLSM 106.

[0055] For any changes in a hardware configuration (e.g., changing camera distance to the image plane), an updated synthetic image library may be regenerated quickly and at low costs in software within digital-twin DLSM 106. Additionally, new sample container support may be rapidly performed in digital-twin DLSM 106 on simulated images.

[0056] In some embodiments, library 110 of synthetic images 112 may include top view images for pick and place processes and side view images for barcode detection. Other synthetic image viewpoints may be employed.

[0057] In block 224, the vision system model(s) and algorithm(s) developed in blocks 220 and 222, respectively, may be deployed within a real-world DLSM system of a diagnostic laboratory (e.g., real-world DLSM system 120 of diagnostic laboratory 122 as deployed vision system model(s) and algorithm(s) 124).

[0058] In yet further embodiments, photosimilar representations of internal features of digital-twin DLSM system 106 may be provided to an operator of real-world DLSM system 120 fortraining, troubleshooting, or illustration (e.g., so internal components can be viewed without having to remove covers, safety shields, etc., of the real-world DLSM system 120). Graphical animations of operations of digital-twin DLSM system 106 similarly may be displayed on a real-world DLSM system (e.g., via a display of the real-world DLSM system).

[0059] In some embodiments, both noise and variability in components relevant to synthetic image generation (e.g., one or more components that appear in the synthetic images) may be modelled. For example, after creation of a digital-twin DLSM system based on computer-aided design information, a designer may identify components of the digital-twin DLSM system relevant to generation of synthetic images. Both noise and variability of the identified components may be modelled (e.g., and provided to the 3D simulation tool). The modelling of noise and variability may depend on the type of component and / or noise considered. As an example, for barcode labels, the crinkling and / or folding of the label material (e.g., paper) is a noise factor that may be introduced by an operator at random. Real-world examples may be studied and programmatically controllable parameters may be developed forrandomization.

[0060] Thereafter, the modelled noise and variability in the identified components may be programmatically controlled during synthetic image generation with the digital-twin DLSM system to generate a library of synthetic images representative of the noise and variability of the identified components. For example, objects regarded as noise, such as fingerprints, spills, smudges, and dust, may be randomly distributed on tubes, caps, foils, and trays with varying levels of intensity. In some embodiments, the geometries and sizes of tubes and caps may be adjusted to account for manufacturing and specification variability.

[0061] FIG. 3 is a top view of an example embodiment of the real-world DLSM system 120 of FIG. 1A according to one or more embodiments. In some embodiments, digital-twin DLSM 106 may be a digital twin of the real-world DLSM 120 of FIG. 3.

[0062] Real-world DLSM system 120 may be configured to receive one or more sample container holders 302A, 302B (e.g., sample trays) each having sample containers 304 (some labelled) held therein. Real-world DLSM system 120 may also include one or more cameras (e.g., cameras C1-C3 in FIG. 3, although fewer or more cameras may be employed as may other camera locations). In some embodiments, at least one camera (e.g., camera C2) may be moveable with a robot 306.

[0063] Robot 306 may be directed to grasp each sample container 304 and move it from sample container holder 302A or 302B to an empty sample carrier 308B received at real-world DLSM system 120 via a track segment 310B. Cameras C1-C3 may be imaging sensors configured to capture digital images of top views, side views, or other views of sample containers 304 to allow deployed vision system models 124 to characterize the sample containers (e.g., held in sample container holders 302A, 302B) and to facilitate operation of robot 306 based on that characterization.

[0064] As shown in FIG. 3, a sample carrier 308A loaded with a sample container 304 from real-world DLSM system 120 may exit real-world DLSM system 120 via track segment 310A. Note that in some embodiments, sample carriers 308 are not limited to the directions of travel as described herein for sample carriers 308A and 308B.

[0065] A controller 312 may be provided that includes a processor 314 coupled to a memory 316. Memory 316 may include computer program instructions (e.g., one or more computer programs) for controlling operation of real-world DLSM system 120. For example, memory 316 may include a robot controller 318 configured to control operation of robot 306 and / or deployed vision system models 124.

[0066] Processor 314 may be a computational resource such as, but not limited to, a microprocessor, a microcontroller, an embedded microcontroller, a digital signal processor (DSP), an FPGA configured to perform as a microcontroller, or the like. Processor 314 may include one or more processors.

[0067] In one or more embodiments, memory 316 may be a non-transitory memory (e.g., a hard drive, a solid-state drive, a flash-drive, etc.). Computer program instructions stored in memory 316 may include computer code that, when executed by processor 314, causes processor 314 to control operation of real-world DLSM system 120 in accordance with one or more of the methods described herein.

[0068] Memory 316 may be any suitable type of memory, such as, but not limited to, one or more of a volatile memory and / or a non-volatile memory. Memory 316 may be located within controller 312 and / or processor 314 or a part or all of memory 316 may be located outside of controller 312 (e.g., remote from controller 312 such as in cloud storage). Memory 316 may include multiple memory units that may or may not be proximate one another.

[0069] Memory 316 may have a plurality of instructions (e.g., one or more programs for robot controller 318 and computer program instructions for employing deployed vision system model(s) and algorithm(s) 124) stored therein that, when executed by processor 314, cause processor 314 to perform various actions specified by one or more of the stored instructions. These computer program instructions may be provided to processor 314 to perform operation acts in accordance with the present systems and methods specified in the flowchart(s) and / or block diagram blocks herein. Processor 314 so configured becomes a special purpose machine particularly suited for performing in accordance with the present systems and methods. Computer program instructions may be stored in a computer readable medium, such as memory 316, that can direct the processor 314 to function in a particular manner.

[0070] FIG. 4A is an example top view image 400 of sample containers 304 captured by one of the cameras C1-C3 of real-world DLSM system 120 in accordance with embodiments provided herein. Deployed vision system models 124 may be employed to identify and characterize sample containers 304, identify and characterize sample container caps (e.g., caps 402 shown with bounding boxes 404), or the like, from top view image 400.

[0071] FIG. 4B is an example side view image 410 of a sample container 304 captured by one of the cameras C1-C3 of real-world DLSM system 120 in accordance with embodiments provided herein. Side view image 410 allows viewing of a barcode label 412 and sample fluid 414 of sample container 304. Sample container 304 is shown supported in a sample carrier 416. However, side view images may be captured while sample container 304 is in other locations (e.g., a sample tray, supported by a robot gripper, etc.). Deployed vision system models 124 may be employed to identify and characterize the sample container 304, identify and characterize sample container cap 402, identify and characterize barcode label 412, identify and characterize sample fluid 414, or the like, from side view image 410.

[0072] Digital-twin DLSM system 106 (FIG. 1A) may be employed to generate imagessimilar to those of FIGS. 4A and 4B (e.g., as synthetic images) such as by using camera(s) 114 of digital-twin DLSM system 106.

[0073] In some embodiments, digital-twin DLSM system 106 may be employed to test the robustness of at least one of a vision-based ML model and a non-ML vision system algorithm in a closed loop. This may include, for example, testing for degradation of image quality (e.g., due to noise, motion blur, wear and tear of DLSM system components, spills of fluids, dirt, dust, and / or the like).

[0074] In one or more embodiments, digital-twin DLSM system 106 may be employed to perform at least one of DLSM design improvements, iterations, and trade-off investigations using the digital-twin DLSM system. A digital-twin diagnostic laboratory sample management system may be used to virtually model and simulate various operational scenarios, allowing designers to test modifications and assess their impact without disrupting real-world processes. One example is optimizing object detection accuracy and precision by considering parameters such as robot speed, light intensity, and camera exposure time. Additionally, cost optimization of selected hardware components can be achieved by modeling hardware degradation while continuously monitoring system performance. By leveraging real-time data and virtual experimentation, stakeholders can efficiently evaluate multiple design alternatives, conduct rapid iterations, and perform trade-off analyses to optimize system performance, cost, and reliability.

[0075] FIG. 5 is a flowchart of an example process 500 of training a vision system model or developing a vision system algorithm for use in a diagnostic laboratory sample management system in accordance with embodiments provided herein. In some implementations, one or more process blocks of FIG. 5 may be performed by 3D simulation tool 104 (FIGS. 1A and 1B), processor(s) 156 and / or GPU(s) 158 of first computer system 152 (FIG. 1B), program(s) 168, processor(s) 162, and / or GPU(s) 164 of second computer system 154, etc.

[0076] As shown in FIG. 5, process 500 may include obtaining design information for a diagnostic laboratory sample management (DLSM) system design (block 502). For example, processor(s) 156 of first computer system 152 may obtain design information (e.g., computer-aided design information within DLSM system design information 102) fora diagnostic laboratory sample management (DLSM) system design, as described above.

[0077] As also shown in FIG. 5, process 500 may include creating a digital-twin DLSM system based on the design information (block 504). For example, processor(s) 156 (FIG.1 B) of first computer system 152 may employ 3D simulation tool 104 and GPU(s) 158 to create digital-twin DLSM system 106 (FIG. 1A) based on the design information, as described above.

[0078] As further shown in FIG. 5, process 500 may include generating a library of syntheticimages using one or more cameras of the digital-twin DLSM system (block 506). For example, processor(s) 156 of first computer system 152 may employ 3D simulation tool 104 and GPU(s) 158 to generate library 110 of synthetic images 112 using one or more cameras 114 of digital-twin DLSM system 106, as described above. In some embodiments, synthetic images 112 may be annotated (e.g., automatically by processor 156 and / or processor 162 and / or 3D simulation tool 104) or manually by one or more users.

[0079] As also shown in FIG. 5, process 500 may include one or more of: training at least one ML model using synthetic images from the library of synthetic images; and developing at least one non-ML vision system algorithm using synthetic images from the library of synthetic images (block 508). For example, processor(s) 162 executing program(s) 168 may employ GPU(s) 164 to train at least one ML model using synthetic images 112 from library 110 of synthetic images and / or to develop at least one non-ML vision system algorithm using synthetic images 112 from library 110 of synthetic images, as described above.

[0080] Although FIG. 5 shows example blocks of process 500, in some implementations, process 500 may include additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted in FIG. 5. Additionally, or alternatively, two or more of the blocks of process 500 may be performed in parallel.

[0081] FIG. 6 is a flowchart of an example process 600 of training a vision system model and developing one or more vision system algorithms for use in a diagnostic laboratory sample management system in accordance with embodiments provided herein. In some implementations, one or more process blocks of FIG. 6 may be performed by 3D simulation tool 104 (FIGS. 1A and 1B), processor(s) 156 and / or GPU(s) 158 of first computer system 152 (FIG. 1B), program(s) 168, processor(s) 162, and / or GPU(s) 164 of second computer system 154, etc.

[0082] As shown in FIG. 6, process 600 may include obtaining computer-aided design information for a diagnostic laboratory sample management (DLSM) system design (block 602). For example, processor(s) 156 of first computer system 152 may obtain computer-aided design information (e.g., within DLSM system design information 102) for a diagnostic laboratory sample management (DLSM) system design, as described above.

[0083] As also shown in FIG. 6, process 600 may include creating a digital-twin DLSM system based on the computer-aided design information (block 604). For example, processor(s) 156 of first computer system 152 may employ 3D simulation tool 104 and GPU(s) 158 to create digital-twin DLSM system 106 based on the computer-aided design information, as described above.

[0084] As further shown in FIG. 6, process 600 may include generating a library of synthetic images using one or more cameras of the digital-twin DLSM system, the library of syntheticimages including: top view synthetic images of sample containers within one or more sample trays of the digital-twin DLSM system; and side view synthetic images of sample containers (block 606). For example, processor(s) 156 of first computer system 152 may employ 3D simulation tool 104 and GPU(s) 158 to generate library 110 of synthetic images 112 using one or more cameras 114 of digital-twin DLSM system 106, library 110 of synthetic images 112 including: top view synthetic images of sample containers within one or more sample trays of digital-twin DLSM system 106; and side view synthetic images of sample containers, as described above.

[0085] As also shown in FIG. 6, process 600 may include training a cap detection ML model using the top view synthetic images, the cap detection ML model trained to detect caps of sample containers (block 608). For example, processor(s) 162 executing program(s) 168 may employ GPU(s) 164 to train a cap detection ML model using the top view synthetic images, the cap detection ML model trained to detect caps of sample containers, as described above.

[0086] As further shown in FIG. 6, process 600 may include developing a geometry detection algorithm using the top view synthetic images (block 610). For example, processor(s) 162 executing program(s) 168 may employ GPU(s) 164 to develop a geometry detection algorithm using the top view synthetic images, as described above.

[0087] As also shown in FIG. 6, process 600 may include developing a barcode reading algorithm using the side view synthetic images (block 612). For example, processor(s) 162 executing program(s) 168 may employ GPU(s) 164 to develop a barcode reading algorithm using the side view synthetic images, as described above. Side view synthetic images may include numerous side view synthetic images of sample containers with barcode labels.

[0088] Although FIG. 6 shows example blocks of process 600, in some implementations, process 600 may include additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted in FIG. 6. Additionally, or alternatively, two or more of the blocks of process 600 may be performed in parallel.

[0089] FIG. 7 is a flowchart of an example process 700 of developing training images with a digital-twin diagnostic laboratory sample management system in accordance with embodiments provided herein. In some implementations, one or more process blocks of FIG.7 may be performed by 3D simulation tool 104 (FIGS. 1A and 1B), processor(s) 156 and / or GPU(s) 158 of first computer system 152 (FIG. 1B).

[0090] As shown in FIG. 7, process 700 may include obtaining design information for a diagnostic laboratory sample management (DLSM) system design (block 702). For example, processor(s) 156 of first computer system 152 may obtain design information fora diagnostic laboratory sample management (DLSM) system design, as described above.

[0091] As also shown in FIG. 7, process 700 may include creating a digital-twin DLSM system based on the design information (block 704). For example, processor(s) 156 of first computer system 152 may employ 3D simulation tool 104 and GPU(s) 158 to create digitaltwin DLSM system 106 based on the design information, as described above.

[0092] As further shown in FIG. 7, process 700 may include identifying components of the digital-twin DLSM system relevant to generation of synthetic images (block 706). For example, a designer may identify components of digital-twin DLSM system 106 relevant to generation of synthetic images such as sample containers, barcodes, trays, tray drawers, reflectors, cameras, shrouds, etc., as described above.

[0093] As also shown in FIG. 7, process 700 may include modelling noise and variability in the identified components (block 708). For example, processors) 156 (via one or more programs (not shown)) of first computer system 152 may model noise and variability in the identified components, as described above. Noise and variability of the identified components may be modelled and provided to 3D simulation tool 104. The modelling of noise and variability may depend on the type of component and / or noise considered. As an example, for barcode labels, the crinkling and / orfolding of the label material (e.g., paper) is a noise factor that may be introduced by an operator at random.

[0094] As further shown in FIG. 7, process 700 may include programmatically controlling the modelled noise and variability in the identified components during synthetic image generation with the digital-twin DLSM system to generate a library of synthetic images representative of the noise and variability of the identified components (block 710). For example, processor(s) 156 (via one or more programs (not shown)) of first computer system 152 may programmatically control (e.g., programmatically adjust such as increase and / or decrease) the modelled noise and variability in the identified components during synthetic image generation with digital-twin DLSM system 106 to generate library 110 of synthetic images 112 representative of the noise and variability of the identified components, as described above.

[0095] Although FIG. 7 shows example blocks of process 700, in some implementations, process 700 may include additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted in FIG. 7. Additionally, or alternatively, two or more of the blocks of process 700 may be performed in parallel.Illustrative Embodiments:

[0096] The following is a list of non-limiting illustrative embodiments disclosed herein:

[0097] An illustrative method, comprising obtaining design information fora diagnostic laboratory sample management (DLSM) system design; creating a digital-twin DLSM systembased on the design information; generating a library of synthetic images using one or more cameras of the digital-twin DLSM system; and one or more of: training at least one ML model using synthetic images from the library of synthetic images; and developing at least one non-ML vision system algorithm using synthetic images from the library of synthetic images.

[0098] The illustrative method of any one of the proceeding illustrative embodiments, further comprising deploying one or more of the at least one ML model and the at least one non-ML vision system algorithm within a real-world DLSM system of a diagnostic laboratory.

[0099] The illustrative method of any one of the proceeding illustrative embodiments, wherein creating a digital-twin DLSM system based on the design information comprises creating the digital-twin DLSM system using a three-dimensional simulation tool and the design information.

[0100] The illustrative method of any one of the proceeding illustrative embodiments, wherein the design information includes computer-aided design information.

[0101] The illustrative method of any one of the proceeding illustrative embodiments, further comprising: determining components of the digital-twin DLSM system that do not affect properties of synthetic images captured by the one or more cameras of the digital-twin DLSM system; and reducing a complexity of the components.

[0102] The illustrative method of any one of the proceeding illustrative embodiments, further comprising splitting components of the digital-twin DLSM system into multiple entities.

[0103] The illustrative method of any one of the proceeding illustrative embodiments, wherein the multiple entities include two or more of physics attributes, geometry meshes, semantic data, and collider meshes.

[0104] The illustrative method of any one of the proceeding illustrative embodiments, further comprising: creating photosimilar simulated material properties for sample containers; and applying the photosimilar simulated material properties to sample containers within the digital-twin DLSM system.

[0105] The illustrative method of any one of the proceeding illustrative embodiments, wherein the photosimilar simulated material properties simulate one or more of fluids, foils, sample container caps, and barcode labels.

[0106] The illustrative method of any one of the proceeding illustrative embodiments, wherein the photosimilar simulated material properties include simulated deformations or peeling of barcode labels.

[0107] The illustrative method of any one of the proceeding illustrative embodiments, wherein the simulated deformations or peeling of barcode labels is programmatically variable.

[0108] The illustrative method of any one of the proceeding illustrative embodiments, wherein the photosimilar simulated material properties include one or more of imperfections on sample containers and fluid residue on exterior surfaces of sample containers or sample trays.

[0109] The illustrative method of any one of the proceeding illustrative embodiments, wherein the imperfections include at least one of fingerprints, dirt, dust, particles, spills, detritus, and noise.

[0110] The illustrative method of any one of the proceeding illustrative embodiments, further comprising simulating optics and lighting within the digital-twin DLSM system.

[0111] The illustrative method of any one of the proceeding illustrative embodiments, wherein simulating optics and lighting within the digital-twin DLSM system comprises: simulating one or more of F-Stop, aperture, and working distance of the one or more cameras of the digital-twin DLSM system; and simulating one or more of different lighting geometries, lighting spectral properties, and lighting intensities.

[0112] The illustrative method of any one of the proceeding illustrative embodiments, further comprising simulating pick and place robotics used to pick and place sample containers within the DLSM system.

[0113] The illustrative method of any one of the proceeding illustrative embodiments, wherein generating a library of synthetic images using one or more cameras of the digitaltwin DLSM system comprises generating a library of annotated synthetic images.

[0114] The illustrative method of any one of the proceeding illustrative embodiments, wherein training at least one ML model using synthetic images from the library of synthetic images comprises: training at least one ML model using only synthetic images from the library of synthetic images; or training at least one ML model using both synthetic images from the library of synthetic images and a plurality of real-world images of sample containers.

[0115] The illustrative method of any one of the proceeding illustrative embodiments, wherein training at least one ML model using synthetic images from the library of synthetic images comprises training a cap detection ML model using top view synthetic images, the cap detection ML model trained to detect caps of sample containers based on top view images of the sample containers.

[0116] The illustrative method of any one of the proceeding illustrative embodiments, wherein training at least one ML model using synthetic images from the library of synthetic images comprises training an ML model for barcode detection.

[0117] The illustrative method of any one of the proceeding illustrative embodiments, wherein developing at least one non-ML vision system algorithm using synthetic imagesfrom the library of synthetic images comprises: developing a geometry detection algorithm using top view synthetic images; and developing a barcode reading algorithm using side view synthetic images.

[0118] The illustrative method of any one of the proceeding illustrative embodiments, wherein developing a geometry detection algorithm using top view synthetic images comprises developing a geometry detection algorithm for estimating tube height and tube width from top view synthetic images.

[0119] The illustrative method of any one of the proceeding illustrative embodiments, further comprising: adding a new sample container type to the digital-twin DLSM system; generating synthetic images of the new sample container type with the digital-twin DLSM system; and at least one of training a vision-based ML model and a non-ML vision system algorithm for a real-world DLSM system using only synthetic images from the digital-twin DLSM system including the synthetic images of the new sample container type.

[0120] The illustrative method of any one of the proceeding illustrative embodiments, further comprising performing at least one of DLSM design improvements, iterations, and trade-off investigations using the digital-twin DLSM system.

[0121] The illustrative method of any one of the proceeding illustrative embodiments, further comprising testing robustness of at least one of a vision-based ML model and a non-ML vision system algorithm in a closed loop in the digital-twin DLSM system.

[0122] An illustrative method, comprising: obtaining computer-aided design information fora diagnostic laboratory sample management (DLSM) system design; creating a digital-twin DLSM system based on the computer-aided design information; generating a library of synthetic images using one or more cameras of the digital-twin DLSM system, the library of synthetic images including: top view synthetic images of sample containers within one or more sample trays of the digital-twin DLSM system and side view synthetic images of sample containers; training a cap detection ML model using the top view synthetic images, the cap detection ML model trained to detect caps of sample containers; developing a geometry detection algorithm using the top view synthetic images; and developing a barcode reading algorithm using the side view synthetic images.

[0123] The illustrative method of any one of the proceeding illustrative embodiments, further comprising deploying the cap detection ML model, geometry detection algorithm, and barcode reading algorithm within a real-world DLSM system of a diagnostic laboratory.

[0124] An illustrative method, comprising: obtaining design information for a diagnostic laboratory sample management (DLSM) system design; creating a digital-twin DLSM system based on the design information; identifying components of the digital-twin DLSM system relevant to generation of synthetic images; modelling noise and variability in the identifiedcomponents; and programmatically controlling the modelled noise and variability in the identified components during synthetic image generation with the digital-twin DLSM system to generate a library of synthetic images representative of the noise and variability of the identified components.

[0125] The illustrative method of any one of the proceeding illustrative embodiments, further comprising training at least one ML model based on the library of synthetic images for use in a real-world DLSM system.

[0126] The foregoing description discloses only example embodiments of the invention. Modifications of the above disclosed apparatus and methods which fall within the scope of the invention will be readily apparent to those of ordinary skill in the art.

[0127] Accordingly, while the present invention has been disclosed in connection with example embodiments thereof, it should be understood that other embodiments may fall within the spirit and scope of the invention, as defined by the following claims.

Claims

WHAT IS CLAIMED IS:

1. A method, comprising:obtaining design information for a diagnostic laboratory sample management (DLSM) system design;creating a digital-twin DLSM system based on the design information; generating a library of synthetic images using one or more cameras of the digital-twin DLSM system; andone or more of:training at least one ML model using synthetic images from the library of synthetic images; anddeveloping at least one non-ML vision system algorithm using synthetic images from the library of synthetic images.

2. The method of claim 1 , further comprising deploying one or more of the at least one ML model and the at least one non-ML vision system algorithm within a real-world DLSM system of a diagnostic laboratory.

3. The method of claim 1 wherein creating a digital-twin DLSM system based on the design information comprises creating the digital-twin DLSM system using a three-dimensional simulation tool and the design information.

4. The method of claim 3, wherein the design information includes computer-aided design information.

5. The method of claim 3 further comprising:determining components of the digital-twin DLSM system that do not affect properties of synthetic images captured by the one or more cameras of the digital-twin DLSM system; andreducing a complexity of the components.

6. The method of claim 3 further comprising splitting components of the digital-twin DLSM system into multiple entities.

7. The method of claim 6 wherein the multiple entities include two or more of physics attributes, geometry meshes, semantic data, and collider meshes.

8. The method of claim 3 further comprising:creating photosimilar simulated material properties for sample containers; and applying the photosimilar simulated material properties to sample containers within the digital-twin DLSM system.

9. The method of claim 8 wherein the photosimilar simulated material properties simulate one or more of fluids, foils, sample container caps, and barcode labels.

10. The method of claim 8 wherein the photosimilar simulated material properties include simulated deformations or peeling of barcode labels.

11. The method of claim 10 wherein the simulated deformations or peeling of barcode labels is programmatically variable.

12. The method of claim 8 wherein the photosimilar simulated material properties include one or more of imperfections on sample containers and fluid residue on exterior surfaces of sample containers or sample trays.

13. The method of claim 12 wherein the imperfections include at least one of fingerprints, dirt, dust, particles, spills, detritus, and noise.

14. The method of claim 3 further comprising simulating optics and lighting within the digital-twin DLSM system.

15. The method of claim 14 wherein simulating optics and lighting within the digital-twin DLSM system comprises:simulating one or more of F-Stop, aperture, and working distance of the one or more cameras of the digital-twin DLSM system; andsimulating one or more of different lighting geometries, lighting spectral properties, and lighting intensities.

16. The method of claim 3 further comprising simulating pick and place robotics used to pick and place sample containers within the DLSM system.

17. The method of claim 1 wherein generating a library of synthetic images using one or more cameras of the digital-twin DLSM system comprises generating a library of annotated synthetic images.

18. The method of claim 1 wherein training at least one ML model using synthetic images from the library of synthetic images comprises:training at least one ML model using only synthetic images from the library of synthetic images; ortraining at least one ML model using both synthetic images from the library of synthetic images and a plurality of real-world images of sample containers.

19. The method of claim 1 wherein training at least one ML model using synthetic images from the library of synthetic images comprises training a cap detection ML model using top view synthetic images, the cap detection ML model trained to detect caps of sample containers based on top view images of the sample containers.

20. The method of claim 1 wherein training at least one ML model using synthetic images from the library of synthetic images comprises training an ML model for barcode detection.

21. The method of claim 1 wherein developing at least one non-ML vision system algorithm using synthetic images from the library of synthetic images comprises:developing a geometry detection algorithm using top view synthetic images; and developing a barcode reading algorithm using side view synthetic images.

22. The method of claim 21 wherein developing a geometry detection algorithm using top view synthetic images comprises developing a geometry detection algorithm for estimating tube height and tube width from top view synthetic images.

23. The method of claim 1 , further comprising:adding a new sample container type to the digital-twin DLSM system; generating synthetic images of the new sample container type with the digital-twin DLSM system; andat least one of training a vision-based ML model and a non-ML vision system algorithm for a real-world DLSM system using only synthetic images from the digital-twin DLSM system including the synthetic images of the new sample container type.

24. The method of claim 1 , further comprising performing at least one of DLSM design improvements, iterations, and trade-off investigations using the digital-twin DLSM system.

25. The method of claim 1 , further comprising testing robustness of at least one of a vision-based ML model and a non-ML vision system algorithm in a closed loop in the digitaltwin DLSM system.

26. A method, comprising:obtaining computer-aided design information fora diagnostic laboratory sample management (DLSM) system design;creating a digital-twin DLSM system based on the computer-aided design information;generating a library of synthetic images using one or more cameras of the digital-twin DLSM system, the library of synthetic images including:top view synthetic images of sample containers within one or more sample trays of the digital-twin DLSM system; and side view synthetic images of sample containers;training a cap detection ML model using the top view synthetic images, the cap detection ML model trained to detect caps of sample containers;developing a geometry detection algorithm using the top view synthetic images; and developing a barcode reading algorithm using the side view synthetic images.

27. The method of claim 26, further comprising deploying the cap detection ML model, geometry detection algorithm, and barcode reading algorithm within a real-world DLSM system of a diagnostic laboratory.

28. A method, comprising:obtaining design information for a diagnostic laboratory sample management (DLSM) system design;creating a digital-twin DLSM system based on the design information; identifying components of the digital-twin DLSM system relevant to generation of synthetic images;modelling noise and variability in the identified components; and programmatically controlling the modelled noise and variability in the identified components during synthetic image generation with the digital-twin DLSM system togenerate a library of synthetic images representative of the noise and variability of the identified components.

29. The method of claim 28, further comprising training at least one ML model based on the library of synthetic images for use in a real-world DLSM system.

Images