Method and apparatus for protecting patient information during characterization of a patient in an automated diagnostic analysis system

Abstract

A method for characterizing a specimen and specimen container to be analyzed in an automated diagnostic analysis system. The method can provide a segmentation determination and / or HILN determination (hemolysis, icteric, hyperlipidemia, or normolysis) for the specimen while protecting patient information. The method includes: capturing an image of the specimen container via an image capture device; identifying a label attached to the specimen container in the captured image via an anonymization network; and editing the captured image via the anonymization network to mask some or all information present in the label, thereby removing the information from the captured image. A quality inspection module and system configured to perform this method are also described as further aspects.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 62 / 929,062, filed on October 31, 2019, entitled “METHODS AND APPARATUS FOR PROTECTING PATIENT INFORMATION DURING CHARACTERIZATION OF A SPECIMEN IN ANAUTOMATED DIAGNOSTIC ANALYSIS SYSTEM,” the disclosure of which is incorporated herein by reference in its entirety for all purposes. Technical Field

[0003] The present disclosure relates to methods and apparatus for characterizing a specimen in an automated diagnostic analysis system. Background Art

[0004] Automated diagnostic analysis systems can analyze samples, such as urine, serum, plasma, interstitial fluid, cerebrospinal fluid, etc., to identify analytes or other components in the samples. Such samples are typically contained in sample containers (e.g., sample collection tubes), which can be transported via automated tracks to various pre-processing, pre-screening (including digital imaging), and analyzer stations within the automated diagnostic analysis system.

[0005] As part of the analysis, the sample can be treated with one or more reagents and possibly other materials added thereto, and then analyzed at one or more analyzer stations. Following the reaction, analytical measurements can be performed on the treated sample via photometric or fluorescence readings, by using a beam of interrogating radiation, by reading fluorescence or luminescence emissions, etc. Analytical measurements allow the amount of an analyte or other component in the sample to be determined using well-known techniques.

[0006] In some instances, the presence of interfering substances in a sample (e.g., hemolysis, icterus, and / or lipemia), which may be caused by a patient's condition or sample pretreatment, may adversely affect the results of analyte or component measurements obtained from one or more analyzers. For example, the presence of hemolysis (H) in a sample—which may not be related to the patient's disease state—may result in a different interpretation of the patient's disease state. Similarly, the presence of icterus (I) and / or lipemia (L) in a sample may also result in a different interpretation of the patient's disease state.

[0007] Thus, a pre-screening process for characterizing a sample can be performed in an automated diagnostic analysis system. Pre-screening characterization of the sample can include determining the presence and, in some cases, the extent of interfering substances (such as H, I, and / or L) in the sample to be analyzed. Sample characterization can also include segmentation determination, which can identify various regions of the sample container and sample. This pre-screening process can be based on one or more images (e.g., digital images) of the sample in the sample container captured at one or more imaging stations (otherwise referred to as "quality control stations") of the automated diagnostic analysis system. The images can be stored in computer memory.

[0008] However, along with the specimen and specimen container, the image may also include an image of one or more labels attached to the specimen container. The one or more labels may contain printed or barcoded information that may include sensitive patient information (e.g., name, date of birth, address, patient number, and / or other personal information), along with other information such as the test to be performed, the time and date the specimen was obtained, medical facility information, tracking and routing information, etc. Summary of the Invention

[0009] According to a first aspect, a method for characterizing a specimen in an automated diagnostic analysis system is provided. The method comprises: capturing an image of a specimen container using an image capture device; identifying a label attached to the specimen container in the image using an anonymization network; and editing the image using the anonymization network to mask the identified label, such that information present in the label is removed from the image to produce an edited image.

[0010] According to a second aspect, a method for characterizing a specimen container in an automated diagnostic analysis system is provided. The method comprises: capturing an image of the specimen container; identifying a label attached to the specimen container in the image using an anonymization network of the automated diagnostic analysis system; and editing the image using the anonymization network to mask the label such that some or all information present on the label is removed from the image.

[0011] According to another aspect, a quality inspection module for an automated diagnostic analysis system is provided. The quality inspection module includes: a plurality of image capture devices arranged around an imaging location, the plurality of image capture devices configured to capture a plurality of images of a sample container from a plurality of viewpoints; and a computer coupled to the plurality of image capture devices. The computer is configured and operable via programming instructions to input a captured image obtained by one of the plurality of image capture devices into an anonymization network executed on the computer, wherein the captured image depicts at least the sample container and a label attached to the sample container. The computer is further configured and operable via programming instructions to identify the label attached to the sample container in the captured image via the anonymization network, and to edit the captured image via the anonymization network to mask the identified label, thereby removing some or all information present in the label, and particularly sensitive patient information, from the captured image. The edited image can be output for use in performing segmentation or interference determination (e.g., HILN—hemolytic, icteric, hyperlipidemic, or normotensive) of a sample in the sample container via a HILN network executed on the computer.

[0012] Still other aspects, features, and advantages of the present disclosure may readily become apparent from the following description and illustrations, including a plurality of exemplary embodiments and implementations contemplated for implementing the best mode of the present invention. The present disclosure may also have other and different embodiments, and several of its details may be modified in various respects, all without departing from the scope of the present disclosure. The present disclosure is intended to cover all modifications, equivalents, and alternatives falling within the scope of the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings described below are for illustrative purposes and are not necessarily drawn to scale. Therefore, the accompanying drawings and description are to be regarded as illustrative rather than restrictive in nature. The accompanying drawings are not intended to limit the scope of the present invention in any way.

[0014] Figure 1 Illustrated is a top-down schematic diagram of an automated diagnostic analysis system including one or more quality check modules configured to perform imaging and label anonymization, as well as segmentation and / or HILN (hemolysis, icterus, hyperlipidemia, normal) determination methods according to one or more embodiments.

[0015] Figure 2 Illustrated is a side view of a sample container comprising a separated sample having a serum or plasma portion that may contain interfering substances and further comprising a label comprising patient information and / or a bar code thereon.

[0016] Figure 3 Pictured Figure 2A side view of a sample container held in a vertical orientation in a holder that can be Figure 1 The equipment is transported on tracks within an automated diagnostic analysis system.

[0017] Figure 4A Illustrated is a schematic top view of a quality check module (with the top removed) according to one or more embodiments, the quality check module including multiple viewpoints (e.g., viewpoints 1-3) and configured to capture and analyze multiple images for label anonymization and segmentation and / or HILN determination.

[0018] Figure 4B Illustrated is a diagram illustrating a method for Figure 4A The section line 4B-4B is taken Figure 4A A schematic side view of a quality inspection module with the front enclosure wall removed.

[0019] Figure 5 A functional block diagram of a HILN network according to one or more embodiments is illustrated, the HILN network being operable to output segmentation data and interferent determination and classification indices for a sample in a sample container, and the block diagram further including an anonymization network (AN) being operable to edit a captured image to mask some or all information on a label.

[0020] Figure 6 Illustrated is an edited image of a sample in a sample container output from an anonymization network, wherein information present on a label attached to the sample container (e.g., patient information and a barcode) has been masked and removed, according to one or more embodiments.

[0021] Figure 7 is a flow chart of a method for characterizing a specimen in an automated diagnostic analysis system including label anonymization according to one or more embodiments. DETAILED DESCRIPTION

[0022] Having patient information included in stored specimen container images can potentially compromise patient privacy. Therefore, there is an unmet need for a method, system, and apparatus that allows characterization of a specimen in a specimen container while protecting patient information that may be present in one or more labels attached to the specimen container in an automated diagnostic analysis system.

[0023] Pre-screening of a sample contained in a sample container can be automatically performed at a quality inspection module of an automated diagnostic analysis system. Pre-screening, according to one or more embodiments, can include: capturing one or more images (e.g., one or more digital images) of the sample contained in the sample container at an imaging station (e.g., an imaging station of the quality inspection module); editing the captured images to mask patient information that may be present in one or more labels attached to the sample container without affecting the imaged fluid characteristics of the sample; and storing the edited images in a computer memory or database (where the original captured images depicting the label information are not stored). Using the edited images, pre-screening can then automatically perform a segmentation determination and / or a HILN determination. The segmentation determination can identify various regions of the sample container and sample, such as, for example, a serum or plasma portion, a sedimented blood portion, a gel separator (if used), an air region, one or more label regions, the type of sample container (indicating, for example, height and width / diameter), and / or the type and / or color of the sample container cap. The HILN determination can determine the presence of an interferent in the serum or plasma portion of a blood sample and, in some embodiments, the extent of the interferent, or whether the sample is normal (N), which indicates that the sample either contains an acceptably low amount of the interferent or contains no interferent at all.

[0024] Interferors can be hemolysis (H), icterus (I), or hyperlipidemia (L). Hemolysis can be defined as a condition in the serum or plasma fraction in which red blood cells are disrupted during pretreatment, resulting in the release of hemoglobin from the red blood cells into the serum or plasma fraction, giving the serum or plasma fraction a reddish hue. The extent of hemolysis can be quantified by assigning a hemolysis index (e.g., H0-H6 in some embodiments, and more or less in other embodiments). Icterus can be defined as a condition in which the serum or plasma fraction is a faded, dark yellow color, caused by the accumulation of the bile pigment (bilirubin). The extent of icterus can be quantified by assigning an icterus index (e.g., I0-I6 in some embodiments, and more or less in other embodiments). Hyperlipidemia can be defined as the presence of abnormally high concentrations of emulsified fat in the blood, giving the serum or plasma fraction a whitish or milky appearance. The extent of hyperlipidemia can be quantified by assigning a hyperlipidemia index (e.g., L0-L4 in some embodiments, and more or less in other embodiments). In some embodiments, the pre-screening process may include determining an uncentrifuged (U) classification for the serum or plasma portion of the sample that has not been centrifuged.

[0025] A quality inspection module of an automated diagnostic analysis system configured to perform a pre-screening characterization method may include an anonymization network and a HILN network, both of which are implemented via programming instructions executable in a computer of the quality inspection module. The anonymization network may be a Generic Adversarial Network (GAN) or a Variational Autoencoder (VAE). Alternatively, the anonymization network may be any suitable machine learning algorithm or method capable of performing image inpainting to mask some or all information on a label. In particular, patent information may be inpainted. An example anonymization network that may be suitable is described in Guilin Liu et al., “Image Inpainting For Irregular Holes Using Partial Convolutions,” NVIDIA Corporation, Version 2, last revised December 15, 2018, which is incorporated herein by reference.

[0026] According to one or more embodiments, an anonymization network can be configured to patch one or more labels in a captured image of a specimen and specimen container so as to mask (remove, replace, or erase) all barcodes and printed text on the one or more labels while preserving the imaged fluid properties of the specimen so that the edited image will not affect subsequent specimen segmentation and / or HILN determination. The one or more patched labels in the edited image may appear as pure white labels in the edited image, although other colors are possible. In some embodiments, the anonymization network is configured to reconstruct / generate an image of the specimen and specimen container without any labels. In some embodiments, the anonymization network can be configured to report an error if the specimen container is not depicted in the captured image and / or the captured image lacks a barcode and / or label. In some embodiments, the anonymization network can be configured to remove some noise in the captured image caused by, for example, variations in machine / camera settings and / or calibration.

[0027] The quality inspection module of the automated diagnostic analysis system may also include a HILN network, which may be, for example, a segmentation convolutional neural network (SCNN), which receives as input one or more edited images from the anonymization network. In some embodiments, the SCNN may include more than 100 operational layers, including, for example, BatchNorm, ReLU activation, convolution (e.g., 2D), dropout, and deconvolution (e.g., 2D) layers, to extract features such as simple edges, textures, and serum or plasma fractions and labeled regions. The top layer (e.g., a fully convolutional layer) may be used to provide correlations between fractions. The output of this layer may be fed into a SoftMax layer, which generates a per-pixel (or per-patch, comprising n×n pixels) output indicating whether each pixel or patch contains a HILN. In some embodiments, the SCNN may only provide the HILN output. In other embodiments, the SCNN output may include HILNs for more than 20 categories, so that for each interferent present, an estimate of the interferent's level (index) is also obtained. The SCNN can also include a front-end container segmentation network (CSN) to determine container type and container boundaries. More specifically, the CSN can classify (or "segment") various regions of the sample container and sample, such as the serum or plasma fraction, the sedimented blood fraction, the gel separator (if used), the air region, one or more label regions, the type of sample container (indicating, for example, height and width / diameter), and / or the type and / or color of the sample container cap. The sample container holder or background can also be classified. Alternatively, other types of HILN networks can be used.

[0028] If a sample is found to contain one or more of H, I, and L, an appropriate notification may be provided to the operator and / or the sample container may be taken off line (1) to perform remediation to correct one or more of H, I, or L, (2) to redraw the sample, or (3) to perform other processing. Thus, the ability to pre-screen for HILN prior to analysis by one or more analyzers may advantageously (a) minimize the time wasted analyzing samples that have qualities unsuitable for analysis; (b) avoid or minimize erroneous test results; (c) minimize delays in patient test results; and / or (d) avoid waste of patient samples, all while protecting patient information. Incorrect / low confidence sample samples may be stored in a local database or cloud-based system.

[0029] This article will refer to Figure 1-7The characterization method of the present invention, a quality inspection module configured to perform the characterization method of the present invention, and an automated diagnostic analysis system including one or more quality inspection modules are further described in detail below.

[0030] Figure 1 The diagram shows an example of a method for automatically processing a sample 212 (see Figure 2 ) is an automated diagnostic analysis system 100 that stores a plurality of sample containers 102. The sample containers 102 can be provided in one or more racks 104 at a loading area 105 before being transported to and analyzed by one or more analyzers (e.g., a first analyzer 106, a second analyzer 108, and / or a third analyzer 110) arranged around the automated diagnostic analysis system 100. A greater or fewer number of analyzers can be present in the system. The analyzers can be any number of clinical chemistry analyzers, laboratory instruments, or the like, or a combination thereof. The sample containers 102 can be any suitable transparent or translucent container, such as a blood collection tube, a test tube, a sample cup, a cuvette, or other transparent or opaque glass or plastic container capable of containing a sample 212 and allowing imaging of the sample 212 contained therein. The sample containers 102 can vary in size.

[0031] Sample 212 (see Figure 2 ) can be provided to the automated diagnostic analysis system 100 in a sample container 102, which can be covered with a cap 214. The cap 214 can have different types and / or colors (e.g., red, royal blue, light blue, green, gray, tan, yellow, or a combination of colors), which can have implications for the test being used for the sample container 102, the type of additives included therein, whether the container includes a gel separation agent, etc. Other colors can also be used. In one embodiment, the cap type can be determined using the characterization methods described herein.

[0032] Each sample container 102 may be provided with one or more labels 218, which may include identification information 218 (i.e., a label) thereon, such as a bar code, alphabetic characters, numeric characters, or a combination thereof. The identification information 218i may include, for example, patient information (e.g., name, date of birth, address, and / or other personal information), the test to be performed, the time and date the sample was obtained, medical facility information, tracking and routing information, and the like. Other information may also be included. The identification information 218i may be machine readable at various locations about the automated diagnostic analysis system 100. The machine-readable information may be darker (e.g., black) than the label material (e.g., white) so that it may be easily imaged. The identification information 218i may indicate the patient's identification and the test to be performed on the sample 212, or otherwise be associated therewith, via a laboratory information system (LIS) 147. Such identification information 218i may be provided on a label 218, which may be adhered to or otherwise provided on the outer surface of the tube 215. As Figure 2 As shown in , the label 218 may not extend all the way around the sample container 102, or may not extend all the way along the length of the sample container 102, such that from the particular front viewpoint shown, a majority of the serum or plasma portion 212SP is visible (the portion shown in dashed lines) and is not obscured by the label 218.

[0033] The sample 212 may include a serum or plasma portion 212SP and a sedimented blood portion 212SB contained within a tube 215. Air 216 may be provided above the serum and plasma portions 212SP, and the boundary therebetween is defined as the liquid-air interface (LA). The boundary between the serum or plasma portion 212SP and the sedimented blood portion 212SB is defined as the serum-blood interface (SB). The interface between the air 216 and the cap 214 is defined as the tube-cap interface (TC). The tube height (HT) is defined as the height from the bottommost portion of the tube 215 to the bottom of the cap 214 and can be used to determine tube size. The height of the serum or plasma portion 212SP is HSP, defined as the height from the top of the serum or plasma portion 212SP to the top of the sedimented blood portion 212SB. The height of the sedimented blood portion 212SB is HSB, defined as the height from the bottom of the sedimented blood portion 212SB to the top of the sedimented blood portion 212SB at SB. HTOT is the total height of the specimen 212 and is equal to HSP plus HSB.

[0034] In more detail, the automated diagnostic analysis system 100 may include a base 120 ( Figure 1) (e.g., a frame, floor, or other structure), a track 121 can be mounted on the base 120. The track 121 can be a track with guide rails (e.g., a single track or multiple tracks), a collection of conveyor belts, a conveyor chain, a movable platform, or any other suitable type of conveying mechanism. The track 121 can be circular or any other suitable shape, and in some embodiments can be a closed track (e.g., a circular track). In operation, the track 121 can transport individual sample container carriers 122 in the sample container 102 to various locations spaced apart about the track 121.

[0035] The carrier 122 can be a passive, non-motor driven puck that can be configured to carry a single specimen container 102 on the track 121, or the carrier 122 can alternatively be an automated carrier that includes an onboard drive motor, such as a linear motor that is programmed to move about the track 121 and stop at a pre-programmed position. Other configurations of the carrier 122 can be used. The carriers 122 can each include a holder 122H (see Figure 3 ), the holder 122H is configured to hold the sample container 102 in a defined vertical position and orientation (as shown). The holder 122H may include a plurality of fingers or leaf springs that secure the sample container 102 to the carrier 122, but some of the fingers or leaf springs are movable or flexible to accommodate sample containers 102 of different sizes. In some embodiments, the carrier 122 can be removed from the loading area 105 after being unloaded from one or more racks 104. The loading area 105 can serve a dual function, namely, also allowing the sample container 102 to be reloaded from the carrier 122 to the loading area 105 after pre-screening and / or analysis is completed.

[0036] A robot 124 may be provided at the loading area 105 and may be configured to grab sample containers 102 from one or more racks 104 and load the sample containers 102 onto a carrier 122, such as onto an input lane of a track 121. The robot 124 may also be configured to reload sample containers 102 from the carrier 122 onto one or more racks 104. The robot 124 may include one or more (e.g., at least two) robotic arms or assemblies capable of X (lateral) and Z (vertical—as viewed from the page, as shown), Y and Z, X, Y and Z, or r (radial) and theta (rotational) motion. The robot 124 may be a gantry robot, an articulated robot, an R-theta robot, or other suitable robot, wherein the robot 124 may be equipped with robotic gripper fingers oriented, sized, and configured to pick up and place the sample containers 102.

[0037] Upon being loaded onto the track 121, the sample containers 102 carried by the carrier 122 can proceed to a first pre-processing station 125. For example, the first pre-processing station 125 can be an automated centrifuge configured to perform fractionation of the sample 212. The carrier 122 carrying the sample containers 102 can be transferred to the first pre-processing station 125 via an inflow channel or other suitable robotics. After being centrifuged, the sample containers 102 can exit on an outflow channel or be otherwise removed by the robotics and continue along the track 121. In the depicted embodiment, the sample containers 102 in the carrier 122 can then be transported to a quality inspection module 130 to perform pre-screening, as will be further described herein.

[0038] The quality inspection module 130 is configured to perform a pre-screening and execute the characterization method described herein to automatically determine the presence of H, I, and / or L contained in the sample 212 and optionally determine its extent or degree, or whether the sample is normal (N). If it is found to contain a significantly low amount of H, I, and / or L and is therefore considered normal (N), the sample 212 can continue on track 121 and can then be analyzed by one or more analyzers (e.g., the first, second, and / or third analyzers 106, 108, and / or 110) based on the test order and corresponding test menu of the analyzers 106, 108, and / or 110. Thereafter, the sample container 102 can be returned to the loading area 105 for reloading to one or more racks 104.

[0039] In some embodiments, in addition to HILN detection, segmentation of the sample container 102 and sample 212 may also be performed. Based on the segmented data, post-processing can be used to quantify the sample 212 (i.e., determining the HSP, HSB, HTOT, and determining the location of the SB or SG and LA). In some embodiments, characterization of the physical properties of the sample container 102 (e.g., size—height and width / diameter) may be performed at the quality inspection module 130. This characterization may include determining HT and W, and possibly TC and / or Wi. Based on this characterization, the size of the sample container 102 may be extracted. Furthermore, in some embodiments, the quality inspection module 130 may also determine the cap type, which can serve as a safety check and can detect if the wrong tube type has been used for an ordered test. The color and / or shape of the cap 214 can indicate the type of chemical additive (anticoagulant, etc.) contained in the sample container 102 for a particular ordered test.

[0040] In some embodiments, a remote station 132 may be provided on the automated diagnostic analysis system 100 that is not directly linked to the track 121. For example, a separate robot 133 (shown in dashed lines) may carry sample containers 102 containing samples 212 to the remote station 132 and return them after testing / pre-processing. Alternatively, the sample containers 102 may be removed and returned manually. For example, the remote station 132 may be used to test for certain components (such as hemolysis levels) or may be used for further processing, such as to reduce hyperlipidemia levels through one or more additions and / or additional processing, or to remove clots, bubbles, or foam. Other pre-screening using the HILN detection methods described herein may be performed at the remote station 132.

[0041] Additional station(s) may be provided at one or more locations on or along the track 121. The additional station(s) may include a decapping station, an aliquoting station, one or more additional quality inspection modules 130, and the like.

[0042] The automated diagnostic analysis system 100 may include a plurality of sensors 116 at one or more locations around the track 121. The sensors 116 may be used to detect the position of the sample container 102 on the track 121 by reading identification information 218i or similar information (not shown) provided on each carrier 122. Any suitable means for tracking position may be used, such as proximity sensors. All sensors 116 may be interfaced with a computer 143 so that the position of each sample container 102 is known at all times.

[0043] The pre-processing stations and analyzers 106 , 108 , and 110 may be equipped with robotic mechanisms and / or inflow channels configured to remove carriers 122 from the track 121 , and equipped with robotic mechanisms and / or outflow channels configured to re-enter the track 121 .

[0044] The automated diagnostic analysis system 100 can be controlled by a computer 143, which can be a microprocessor-based central processing unit (CPU) with appropriate memory, regulation electronics, and drivers for operating the various system components. The computer 143 can be housed as part of the base 120 of the automated diagnostic analysis system 100 or separately therefrom. The computer 143 can be operable to control the movement of the carrier 122 to and from the loading area 105, its movement about the track 121, its movement to and from the first pre-processing station 125 and the operation of the first pre-processing station 125 (e.g., a centrifuge), its movement to and from the quality inspection module 130 and its operation, and the movement to and from each of the analyzers 106, 108, 110. The operation of each analyzer 106, 108, 110 for performing various types of tests (e.g., assays or clinical chemistry) can be provided locally by the analyzer's workstation and / or computer, which can communicate with the computer 143.

[0045] For all modules except the quality check module 130, the computer 143 can control the automated diagnostic analysis system 100 according to software, firmware, and / or hardware commands or circuits such as those used on the Dimension® clinical chemistry analyzer sold by Siemens Healthcare Diagnostics of Tarrytown, New York, and such control is typical for those skilled in the art of computer-based electromechanical control programming and will not be described further herein. Other suitable systems for controlling the automated diagnostic analysis system 100 may be used. Control of the quality check module 130 may also be provided by the computer 143, but according to the characterization method of the present invention as described in detail herein.

[0046] The computer 143 used for image processing and performing the characterization methods described herein may include a CPU or GPU, sufficient processing power and RAM, and suitable storage. In one example, the computer 143 may be a multi-processor PC with one or more GPUs, 8 GB RAM or more, and terabytes or more of storage. In another example, the computer 143 may be a PC equipped with a GPU, or alternatively, a PC equipped with a CPU operating in parallelized mode. MKL, 8 GB RAM or more, and suitable storage may also be used.

[0047] Embodiments of the present disclosure may be implemented using a computer interface module (CIM) 145, which allows a user to easily and quickly access various control and status displays. These control and status displays can display and enable control of some or all aspects of the plurality of interrelated automated devices used to prepare and analyze specimens 212. CIM 145 can be employed to provide information regarding the operational status of the plurality of interrelated automated devices, as well as information describing the location of any specimen 212 and the status of tests to be performed or currently being performed on the specimen 212. Thus, CIM 145 is adapted to facilitate operator interaction with the automated diagnostic analysis system 100. CIM 145 may include a display operable to display a menu comprising icons, scroll bars, boxes, and buttons, through which an operator can interface with the automated diagnostic analysis system 100. The menu may include a plurality of functional elements programmed to display and / or operate functional aspects of the automated diagnostic analysis system 100.

[0048] Figure 4A and Figure 4B An embodiment of a quality check module 130 is shown that is configured to perform the characterization method as shown and described herein. The quality check module 130 can be configured with programming instructions to pre-screen a sample 212 (e.g., in a serum or plasma portion 212SP thereof) for the presence and extent of interfering substances (e.g., H, I, and / or L) prior to analysis by one or more analyzers 106, 108, 110. Pre-screening in this manner allows the sample 212 to be subjected to additional processing, additional quantification or characterization, and / or discarded and / or re-extracted without wasting valuable analyzer resources or potentially causing the presence of interfering substances to affect the authenticity of the test results.

[0049] In addition to the interferor detection methods described herein, other detection methods may be performed on the sample 212 contained in the sample container 102 at the quality inspection module 130. For example, a method may be performed at the quality inspection module 130 to provide segmentation data. The segmentation data may be used in a post-processing step to quantify the sample 212, for example, to determine certain physical dimensional characteristics of the sample 212, such as LA and SB, and / or to determine HSP, HSB, and / or HTOT. Quantification may also involve estimating, for example, the volume of the serum or plasma fraction (VSP) and / or the volume of the sedimented blood fraction (VSB). Furthermore, the quality inspection module 130 may be used to quantify the geometry of the sample container 102, i.e., to quantify certain physical dimensional characteristics of the sample container 102, such as the location of the TC, HT, and / or W or Wi of the sample container 102. Other quantifiable geometric features may also be determined.

[0050] The quality inspection module 130 may include a housing 446 that at least partially surrounds or covers the track 121 to minimize external lighting effects. During an image acquisition sequence, the sample container 102 may be located within the housing 446. The housing 446 may include one or more doors 446D to allow the carrier 122 to enter and / or exit the housing 446. In some embodiments, the ceiling may include an opening 446O to allow a robot including movable robotic fingers to load or remove the sample container 102 from the carrier 122 from above.

[0051] like Figure 4A and 4B As shown in FIG, the quality inspection module 130 may include multiple image capture devices 440A-440C configured to capture lateral images of the sample container 102 and sample 212 at an imaging location 432 from multiple viewpoints (labeled viewpoints 1, 2, and 3). While three image capture devices 440A-440C are shown and are preferred, two, four, or more may alternatively be used. Viewpoints 1-3 may be arranged so that they are approximately equally spaced from one another, such as approximately 120° apart, as shown. Images may be acquired in a round-robin fashion, for example, where one or more images from viewpoint 1 may be acquired sequentially from viewpoints 2 and 3. Other image acquisition sequences may be used. Light sources 444A-444C may backlight the sample container 102 (as shown). Multiple viewpoints are advantageous because one or more images taken from viewpoints 1 - 3 may be partially or completely obscured by one or more labels 218 (ie, no clear view of the serum or plasma portion 212SP).

[0052] As depicted, image capture devices 440A, 440B, and 440C can be arranged around track 121. Other arrangements of multiple image capture devices 440A, 440B, and 440C can be used. In this manner, an image of the specimen 212 in the specimen container 102 can be acquired while the specimen container 102 resides in carrier 122 at imaging position 432. The fields of view of the multiple images acquired by image capture devices 440A, 440B, and 440C can overlap slightly within a circumferential extent.

[0053] Image capture devices 440A-440C may be any suitable device for capturing well-defined digital images, such as a conventional digital camera capable of capturing pixelated images, a charge-coupled device (CCD), a photodetector array, one or more CMOS sensors, and the like. The captured image size may be, for example, approximately 2560×694 pixels. In another embodiment, image capture devices 440A, 440B, 440C may capture an image size that may be, for example, approximately 1280×387 pixels. Other image sizes and pixel densities may be used.

[0054] Each image may be triggered and captured at the quality inspection module 130 in response to receiving a trigger signal provided in the communication lines 443A, 443B, and 443C from the computer 143. According to one or more embodiments, each captured image may be processed by the computer 143. In one particularly effective approach, high dynamic range (HDR) processing may be used to capture and process image data from the captured images.

[0055] Figure 5 5 shows a functional HILN network architecture 500 configured to perform the characterization method and label anonymization described herein. The functional architecture 500 can be implemented in the quality inspection module 130 and executed by the computer 143 via suitable programming instructions. As discussed above, the imaging location 432 ( Figure 4A and Figure 4B ), as indicated at functional block 502. Multiple viewpoint images of the sample container 102 (e.g., from viewpoints 1, 2, and 3) may be captured by a plurality of image capture devices 440A-440C, as indicated at functional block 504. Optionally, the image data for each of the multiple viewpoint images may be initially processed to provide a plurality of optimally exposed and normalized image datasets (hereinafter, "image datasets"), as indicated at functional block 506. According to one or more embodiments, the image data (i.e., pixel data) of the captured images of the sample 212 (and the sample container 102) may be provided as input to an anonymization network 508. In some embodiments, the various digital images from the multiple viewpoints may be digitally joined side-by-side to provide a 360-degree image including one or more labels.

[0056] An anonymization network 508, which may be part of the quality inspection module 130, receives an image dataset of captured images of a specimen container and the specimen therein and identifies one or more labels that may be attached to the specimen container in the captured images. Specimen containers are known to have different label configurations, including varying numbers of labels, degrees of overlap, and positioning (vertically and circumferentially). The anonymization network 508 may have been previously trained to identify one or more labels 218 in the captured images via any label identification method. For example, a label configuration database may be used that includes label data for hundreds or even thousands of training images of specimen containers 102 with various configurations of labels 218 captured during the training phase of the anonymization network 508. Image comparison techniques may be used to compare and select the closest image. The image comparison may be a pixel-by-pixel comparison, where the color of each pixel in the captured image is compared to the equivalent pixel in the training image. If all pixel colors match, the two images are identical. The image comparison tool may have parameters to adjust, such as a pixel / color tolerance, which can be set to the number of pixels allowed to differ between the two images. Therefore, some differences between the training images and the captured images can be tolerated. Optionally, the anonymization network 508 can include a trained convolutional neural network (CNN) that can be used to identify the labels 218. Other suitable methods for defining regions or pixels in the image as labels 218 can be used. The training phase of the anonymization network 508 can be performed in the quality inspection module 130 before any pre-screening (characterization).

[0057] Later during characterization, once the label has been identified, the anonymization network 508 can edit the captured image. This editing can occur via inpainting to mask the identified label, removing or otherwise replacing some or all information present in the label from the captured image. For example, inpainting can be performed by matching the background color of the identified label 218, utilizing the same color as the label's background (white). Alternatively, the entire identified region of the label 218 can be inpainted. Other masking techniques can be used to remove information present in the label, such as redacting the label 218 in the image with a pattern (e.g., one or more broad lines) or utilizing a color other than the label's 218 background color. Inpainting can be used to change the color of pixels identified as labels that previously contained patient information, masking (removing) the previous patent information. Furthermore, the anonymization network 508 preserves the imaged fluid properties of the specimen in the edited image, such that subsequent specimen segmentation and / or HILN determination is unaffected by the masking. The edited image 509 from the anonymization network 508 can be stored in the computer memory or database and / or other storage location(s) of the computer 143 and provided as input to the HILN network 535, which can perform segmentation and / or perform HILN determination. The captured image (represented at function block 504) and image data (represented at optional function block 506) containing the labels with patient information are not permanently stored in any ROM memory or database and are not accessible to the HILN network 535 to protect patient information. The anonymization network 508 can be a Generative Adversarial Network (GAN), sometimes also referred to as a Generative Adversarial Network. GAN technology learns to generate new data using the same statistics as the training set. For example, a GAN trained on a photograph including labels can generate new images that look like the label 218. Alternatively, a variational autoencoder (VAE) or any other suitable machine learning algorithm capable of performing image inpainting or other suitable masking on image data can be used.

[0058] Figure 6 A display of an edited image 609 that may be output from the anonymization network 508 is shown. The edited image 609 depicts the sample container 602 with a patched (e.g., solid white) label 618 in which some or all of the printed information and barcode information has been obscured by being removed, replaced, patched, or redacted. For example, Figure 2 and 3The specimen container 102 may have been the source object of a captured image that was provided as input to the anonymization network 508 that produced the edited image 609. In some embodiments, the edited image 609 may include images from multiple viewpoints that have been digitally stitched together (e.g., joined) to provide an enhanced image because the label is viewed from multiple viewpoints that include a portion of the label.

[0059] return Figure 5 The HILN network 535, which may be a segmentation convolutional neural network (SCNN) (other types of HILN networks may be employed to provide segmentation and / or HILN determination), can provide a detailed characterization of the edited image of a sample container (such as, for example, sample container 602). This can include, for example, separating the sample container from its background and understanding the content of the serum or plasma portion. These tasks can be accomplished using the HILN network 535, which is capable of performing pixel-level classification. Given the edited input image 509 (i.e., pixel data) from the anonymization network 508, the HILN network 535 is operable to assign a classification index to each pixel of the image based on its local appearance as indicated by the pixel data values ​​of the image. The HILN network 535 can further process the extracted pixel index information to determine a final HILN classification index for the serum or plasma portion 212SP. In some embodiments, the classification index may include 21 serum classes, including an uncentrifuged class, a normal class, and 19 HIL classes / subclasses, as described in more detail below.

[0060] Challenges in determining the HILN classification index may arise from the small appearance differences within each subcategory of the H, I, and L categories. That is, pixel data values ​​of adjacent subcategories are very similar. To overcome these challenges, the HILN network 535 may include a very deep semantic segmentation network (DSSN) 538, which in some embodiments includes more than 100 operational layers.

[0061] To overcome appearance differences that may be caused by variations in sample container type (e.g., size and / or shape), the HILN network 535 may also include a container segmentation network (CSN) 536 at the front end of the DSSN 538. The CSN 536 is configured and operable to determine and output container type and boundary segmentation information 539, which may include, for example, the location, size, area, and / or volume of the serum or plasma portion 212SP, the sedimented blood portion 212SB, the gel separator (if used), the air region 216, one or more label regions 218, the type of sample container (indicating, for example, height and width / diameter), and / or the type and / or color of the sample container cap 214. In some embodiments, the CSN 536 may have a network structure similar to that of the DSSN 538, but shallower (i.e., having far fewer layers).

[0062] like Figure 5 As shown in , the output of the HILN network 535 can be a classification index 540, which in some embodiments can include an uncentrifuged class 540U, a normal class 540N, a hemolysis class 540H, an icteric class 540I, and a hyperlipidemia class 540L. In some embodiments, the hemolysis class 540H can include subclasses H0, H1, H2, H3, H4, H5, and H6. The icteric class 540I can include subclasses I0, I1, I2, I3, I4, I5, and I6. And the hyperlipidemia class 540L can include subclasses L0, L1, L2, L3, and L4. In other embodiments, each of the hemolysis class 540H, the icteric class 540I, and / or the hyperlipidemia class 540L can have other numbers of fine-grained subclasses.

[0063] Figure 7 A flow chart of a characterization method 700 according to one or more embodiments is illustrated. As described herein, the characterization method 700 may be performed by the quality inspection module 130 (in conjunction with the computer 143 executing programming instructions) and may include, at process block 702, capturing an image of a sample container via an image capture device. For example, the image may be captured by ( Figure 4A and Figure 4B The images may be captured by one or more of the image capture devices 440A-440C, wherein each image may be a digital, pixelated image. The captured images may be processed individually or digitally joined as one captured image and processed.

[0064] The characterization method 700 may further include: in process block 704, via an anonymization network of the quality check module (such as, Figure 5 The anonymization network 508) identifies the label attached to the sample container in the captured image.

[0065] At process block 706, the characterization method 700 may include editing the captured image via the anonymization network (e.g., anonymization network 508) to mask the identified tag (e.g., tag 218) such that some or all information present in the tag is removed from the edited image. Figure 5 and Figure 6 The edited image may be edited image 609 in which the imaged label 618 has been masked so that the label 218 that may have been present in the specimen container 102 (see Figure 2 and Figure 3 ) is removed from the edited image. Removal can include patching or otherwise replacing pixels containing the print information and any barcode information 218i. In some instances, removal can include redaction.

[0066] At process block 708, the characterization method 700 may include outputting the edited image for use in performing segmentation and / or HILN (hemolytic, icteric, hyperlipidemic, normal) determination of the sample in the sample container via the HILN network of the quality inspection module. Figure 5 As shown in , the output of the anonymization network 508 (e.g., the edited image 509) can be provided to the HILN 535 for segmentation and / or HILN determination. The HILN determination can be a classification index 540 (e.g., 540U, 540N, 540H, 540I, or 540L), and in some embodiments, can also include subcategories such as 540H (H0, H1, H2, H3, H4, H5, H6), 540I (I0, I1, I2, I3, I4, I5, I6), and 540L (L0, L1, L2, L3, and L4).

[0067] Thus, based on the foregoing, it should be apparent that an improved characterization method 700 is provided that protects patient information during automated segmentation and / or HILN determination.

[0068] It should also be appreciated that the characterization method described above can be performed using a quality check module (e.g., quality check module 130) that includes a plurality of image capture devices (e.g., image capture devices) 440A-440C arranged around an imaging location (e.g., imaging location 432) and configured to capture images from one or more viewpoints (e.g., Figure 4AThe quality inspection module may further include a computer (e.g., computer 143) coupled to the plurality of image capture devices and configured to process pixel data of the one or more images. The computer (e.g., computer 143) may also be configured and operable to mask any information present in a label attached to the specimen container depicted in the one or more captured images of the specimen, and provide a HILN determination based on the one or more edited images of the specimen in which the label information is masked. In some embodiments, the images including the information present in the label 218 are not permanently stored and are only temporarily stored as part of the identification and editing process of blocks 704 and 706.

[0069] While the present disclosure is susceptible to various modifications and alternative forms, specific method and apparatus embodiments have been shown by way of example in the drawings and described in detail herein. However, it should be understood that the specific methods and apparatus disclosed herein are not intended to limit the present disclosure, but on the contrary, to cover all modifications, equivalents, and alternatives falling within the scope of the claims.

Claims

1. A method for characterizing a sample in an automated diagnostic analysis system, comprising: capturing an image of the sample container using an image capture device; identifying, in the image, a label attached to the sample container using an anonymization network; as well as editing the image using the anonymization network to mask the label such that information present in the label is removed from the image to produce an edited image, The anonymized network: - is part of a quality control module, or - including general adversarial networks, or - including variational autoencoders, or - configured to repair said label in said image of the specimen and specimen container so as to erase all barcodes and printed text on said label and form the repaired label in the edited image, or - is configured to patch the label so as to mask all barcodes on the label, or - configured to patch the label so as to mask all printed text on the label.

2. The method according to claim 1, comprising: The edited image is output for use in performing segmentation or interference determination on the sample in the sample container using a hemolytic, icteric, hyperlipidemia, or normal HILN network of a quality inspection module, wherein the HILN network is configured to perform segmentation or interference determination on the sample in the sample container based on the edited image. The method of claim 1 , wherein the repaired label appears as a white label in the edited image.

4. The method of claim 1 , wherein a segmentation convolutional neural network (SCNN) receives as input one or more edited images from the anonymization network. The method of claim 1 , wherein the anonymization network is trained in a training phase before any characterization.

6. The method of claim 1, wherein editing of the image using the anonymization network preserves the imaged fluid properties of the specimen in the edited image.

7. The method of claim 1, wherein the captured image including the label with the patient information is not permanently stored to protect the patient information. The method of claim 1 , wherein the information is patient information.

9. The method of claim 1, wherein the editing of the image using the anonymization network removes all information present on the label.

10. The method according to claim 1, comprising: Capture images of the specimen container from other viewpoints.

11. A method for characterizing a sample container in an automated diagnostic analysis system, comprising: capturing an image of the sample container; Identify labels attached to sample containers; editing the image using an anonymizing network to mask the label such that some or all printed information present on the label is removed from the image to form an edited image; as well as Save the edited image, The anonymized network: - is part of a quality control module, or - including general adversarial networks, or - including variational autoencoders, or - configured to repair said label in said image of the specimen and specimen container so as to erase all barcodes and printed text on said label and form the repaired label in the edited image, or - is configured to patch the label so as to mask all barcodes on the label, or - configured to patch the label so as to mask all printed text on the label.

12. A quality inspection module for an automated diagnostic analysis system, comprising: a plurality of image capture devices arranged about the imaging location, the plurality of image capture devices configured to capture a plurality of images of the specimen container from a plurality of viewpoints; as well as a computer coupled to the plurality of image capture devices, the computer being configured and operable via programming instructions to: inputting a captured image taken by one of the plurality of image capture devices into an anonymization network executing on the computer, the captured image depicting at least a specimen container and a label attached to the specimen container, identifying, via the anonymization network, in the captured image a label attached to the specimen container, and editing the captured image via the anonymization network to produce an edited image to mask the identified tag such that information present in the tag is removed from the captured image, The anonymized network: - is part of a quality control module, or - including general adversarial networks, or - including variational autoencoders, or - configured to repair said label in said image of the specimen and specimen container so as to erase all barcodes and printed text on said label and form the repaired label in the edited image, or - is configured to patch the label so as to mask all barcodes on the label, or - configured to patch the label so as to mask all printed text on the label.

13. The quality inspection module of claim 12, wherein the computer is further configured and operable via programming instructions to: The edited image is output for use in performing segmentation or interferent determination on the sample in the sample container via a hemolytic, icteric, hyperlipidemic, or normal HILN network executed on the computer, wherein the HILN network is configured to perform segmentation or interferent determination on the sample in the sample container based on the edited image.

14. The quality inspection module of claim 13, wherein the computer is further configured via programming instructions and is operable to display an edited image in which the information on the label is removed from the captured image.

Images